Fluent Bit

Configuring Fluent Bit to send logs, metrics and traces to Berserk over OTLP/gRPC

Fluent Bit sends to Berserk with its opentelemetry output plugin, over OTLP/gRPC, the recommended protocol. Fluent Bit compresses only on its HTTP path (Compression), and drops the oldest buffered data when its buffer is full (Durability).

The ingest endpoint is its own host

Send telemetry to your cluster's ingest host on 4317, not to the gateway your CLI and UI use. On Managed Berserk it is ingest.<your-domain>:4317. Settings → Endpoints shows your installation's address; if a self-hosted install lists none, its operator publishes it: Ingest Endpoints.

Minimal Configuration

The plugin defaults to OTLP/HTTP on port 80. Set grpc: on, port: 4317, and a header carrying your ingest token:

pipeline:
  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Bearer is case-sensitive; the header name is not. Ingest rejects a value without the literal prefix Bearer . Write the name lowercase (authorization), as HTTP/2 does on the wire.
tls: on for a public endpoint. Leave it off (the default) only for a plaintext in-cluster address such as ingest:4317. tls.verify: off accepts any certificate.
grpc: on also turns on HTTP/2, h2c included, so http2 needs no setting. The plugin uses the standard OTLP gRPC method paths (APIs).

Recommended Configuration

This adds a filesystem buffer, unlimited retries and bounded backoff:

service:
  flush: 1
  log_level: info
  # Turns on filesystem buffering; storage.type below needs it.
  storage.path: /var/log/flb-storage/
  storage.sync: normal
  storage.checksum: off
  # Chunks held in memory; the rest stay on disk.
  storage.max_chunks_up: 128
  # Retry backoff bounds, in seconds.
  scheduler.base: 2
  scheduler.cap: 60

pipeline:
  inputs:
    - name: tail
      path: /var/log/containers/*.log
      tag: kube.*
      # Read offsets survive a restart.
      db: /var/log/flb-storage/tail.db
      # This input uses the disk buffer.
      storage.type: filesystem

  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

      # The default is one retry.
      retry_limit: no_limits

      # This output's disk queue: peak rate × tolerated outage.
      # The oldest chunks are dropped past it.
      storage.total_limit_size: 2G

      # Records per flush.
      batch_size: 5000

      workers: 2

Why These Settings

storage.type: filesystem + storage.path: Fluent Bit buffers in memory by default, and a restart or OOM loses the buffer. Filesystem storage keeps each chunk on disk. The three storage settings have different scopes: storage.path is global (service), storage.type is per input (or global with storage.inherit: on), and storage.total_limit_size is per output.
storage.total_limit_size: 2G: this output's disk queue, which bounds the outage it can buffer. When it is full, Fluent Bit drops the oldest chunk. Unset, the queue is bounded only by the disk under storage.path.
retry_limit: no_limits: the default is one retry, after which the chunk is dropped. Berserk returns retryable errors for backpressure (429 / UNAVAILABLE with RetryInfo), transient S3 failures and stream recovery.
scheduler.base / scheduler.cap: the bounds, in seconds, of Fluent Bit's exponential backoff with jitter. With the defaults, 5 and 2000, the interval grows past half an hour; with a cap of 60, Fluent Bit retries within a minute of an outage ending. Both apply to every output.
Timeouts: a request takes seconds, because Ingest batches for up to 2 seconds and answers after the S3 upload. Fluent Bit sets no response deadline by default (net.io_timeout: 0s); keep it unset. A short net.io_timeout turns Berserk's throttling responses into connection errors. net.connect_timeout (default 10s) covers only the connection and TLS handshake.
No compress: it applies only to the OTLP/HTTP path and is ignored with grpc: on (Compression).
batch_size: 5000: records per flush for logs. Fewer, larger requests mean less waiting on Ingest and less throttling. Over gRPC, requests are uncompressed, so this setting also keeps them under the 16 MiB limit.
db: on the tail input: persists read offsets. Without it, a restart can re-read or skip file content.
workers: 2: parallel flush threads for this output. Berserk accepts a limited number of concurrent requests, so keep workers low across your fleet and raise batch_size instead.

Compression

The plugin's compress option (gzip / zstd) applies only to its OTLP/HTTP path. With grpc: on it is ignored without a warning, and requests go out uncompressed. Berserk accepts gzip and zstd on both transports.

Choice	Result
gRPC on `4317` (recommended)	Uncompressed protobuf. The 16 MiB limit applies to the raw bytes.
HTTP on `4318` + `compress: gzip`	Requests typically 3–5× smaller.
Fluent Bit → OpenTelemetry Collector → Berserk	The collector sends compressed OTLP over gRPC and keeps the durable queue.

Compressed OTLP/HTTP

Drop grpc: on, use port 4318, and add compress. The buffer, retry and token settings are those of the recommended configuration:

pipeline:
  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4318
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

      # gzip or zstd; honored only on this path.
      compress: gzip

      retry_limit: no_limits
      storage.total_limit_size: 2G
      batch_size: 5000
      workers: 2

No grpc: on and no http2. With both off, the plugin uses HTTP/1.1, where compress applies. Requests carry content-encoding: gzip, which Ingest decompresses.
The signal paths match. logs_uri, metrics_uri and traces_uri default to /v1/logs, /v1/metrics and /v1/traces, the paths Ingest serves.
tls: on for any https:// endpoint. Port 4318 does not imply plaintext.
Retries. On this path the plugin retries 429, 502, 503 and 504 and treats every other non-2xx as permanent, as in Delivery Semantics. It ignores Retry-After; the wait comes from scheduler.base / scheduler.cap.
Size. The 16 MiB limit applies to the compressed body, typically 40–80 MiB of telemetry, so batch_size can be higher than over gRPC.

Batching and Request Size

A request over 16 MiB on the wire gets InvalidArgument (gRPC) or 413 (HTTP). Both are permanent: the chunk is dropped. Over gRPC the limit applies to uncompressed protobuf.

batch_size (records per flush) sets the request size. Chunk size and storage.max_chunks_up decide how much accumulates before a flush, so check actual request sizes after a change. If a batch_size low enough for the limit costs too much throughput, use compressed OTLP/HTTP.

Durability: Fluent Bit Is Not the Collector

Property	OpenTelemetry Collector	Fluent Bit
Buffer on disk	`file_storage` extension on the exporter's `sending_queue`	`storage.type: filesystem`, per input
Retry budget	`max_elapsed_time: 0` retries forever	`retry_limit: no_limits` retries forever; the default is one retry
When full	Pushes back on receivers via `memory_limiter`	Drops the oldest chunks once `storage.total_limit_size` is reached
Retry pacing	Follows Berserk's `Retry-After`	Its own exponential backoff with jitter (`scheduler.base` / `scheduler.cap`)

For data that must not have gaps, point Fluent Bit's opentelemetry output at an OpenTelemetry Collector's OTLP receiver and let the collector send to Berserk.

The plugin's gRPC retry classification matches Delivery Semantics: it retries UNAVAILABLE, RESOURCE_EXHAUSTED, INTERNAL, ABORTED, CANCELLED and DEADLINE_EXCEEDED, and treats INVALID_ARGUMENT and UNAUTHENTICATED as permanent.

Fluent Bit's dead letter queue keeps the chunks it discards.

Signals Other Than Logs

The opentelemetry output sends whatever the pipeline routes to it, with the OTLP service for each signal:

pipeline:
  inputs:
    - name: node_exporter_metrics
      tag: node_metrics
      scrape_interval: 10

  outputs:
    - name: opentelemetry
      match: node_metrics
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Berserk does not accept OTLP profiles; do not route them to this output.

Mapping Log Records

Fluent Bit records are key/value maps; an OTLP log record has a body, a severity and trace context. Without a mapping, the whole record becomes the body. Map known keys to the OTLP fields, which land in the matching Berserk columns:

      logs_body_key: $message
      logs_severity_text_message_key: level
      logs_trace_id_message_key: trace_id
      logs_span_id_message_key: span_id
      # Keep the remaining record fields as log attributes.
      logs_body_key_attributes: true

Without logs_body_key_attributes: true, selecting a body key drops the record's other fields.

Troubleshooting

log_response_payload: true logs Berserk's responses:

      log_response_payload: true

Symptom	Cause
`UNAUTHENTICATED` / 401	Missing, revoked or malformed token. Check the literal `Bearer` prefix and that the value has no extra quotes.
Connection refused, or a hang then a timeout	OTLP/HTTP sent to `4317` (`grpc` off), or plaintext sent to a TLS endpoint (`tls` off).
`INVALID_ARGUMENT` mentioning size	Over the 16 MiB limit. Lower `batch_size`, or use OTLP/HTTP with `compress: gzip` (Compression).
Data accepted but not in the table you query	The token routes that signal to a different table: Verifying Ingestion.
Chunks dropped during an outage	`retry_limit` at its default of `1`, or `storage.total_limit_size` reached.

Then check end to end:

bzrk search "<your table> | take 10" --since "5m ago"

Minimal ConfigurationCopy Anchor Link

Recommended ConfigurationCopy Anchor Link

Why These SettingsCopy Anchor Link

CompressionCopy Anchor Link

Compressed OTLP/HTTPCopy Anchor Link

Batching and Request SizeCopy Anchor Link

Durability: Fluent Bit Is Not the CollectorCopy Anchor Link

Signals Other Than LogsCopy Anchor Link

Mapping Log RecordsCopy Anchor Link

TroubleshootingCopy Anchor Link

On this page