Fluent Bit

Fluent Bit ships to Berserk through its opentelemetry output plugin, which speaks the same OTLP that Berserk's ingest endpoint accepts. Use it when Fluent Bit is already your log collector, or when you want a smaller footprint than the OpenTelemetry Collector on an edge node.

For the ingest contract — tokens, ports, response codes, the wire cap — see Ingestion. Read the durability caveat below before you make Fluent Bit the only thing between your logs and Berserk, and Compression if egress or request size matters to you — Fluent Bit compresses only on its HTTP path.

Minimal Configuration

Two things make this work: grpc: on (the plugin defaults to OTLP/HTTP, and to port 80) and a header carrying your ingest token.

pipeline:
  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[OUTPUT]
  Name   opentelemetry
  Match  *
  Host   <your-ingest-host>
  Port   4317
  Grpc   On
  Tls    On
  Header authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Three details that are easy to get wrong:

• port defaults to 80, not 4317. Set it explicitly.
• Bearer is case-sensitive; the header name is not. Ingest looks for the literal prefix Bearer and rejects anything else as unauthenticated. Writing the name lowercase (authorization) is the safe form, since gRPC rides on HTTP/2 where header names are lowercase on the wire.
• tls: on for a public endpoint. Leave it off (the default) only for a plaintext in-cluster address such as ingest:4317. Do not set tls.verify: off outside of local debugging — it accepts any certificate.

Setting grpc: on also enables HTTP/2, including h2c, so you don't need to set http2 yourself. The plugin then addresses each signal by its standard OTLP gRPC method path, all of which Ingest serves — see Protocols.

Recommended Configuration

The minimal config works but leaves data on the floor when Berserk is briefly unavailable. This one adds a filesystem buffer, unbounded retries, and bounded backoff:

service:
  flush: 1
  log_level: info
  # Enables filesystem buffering; without it storage.type below is inert.
  storage.path: /var/log/flb-storage/
  storage.sync: normal
  storage.checksum: off
  # Global ceiling on chunks held in memory; the rest stay on disk only.
  storage.max_chunks_up: 128
  # Exponential backoff bounds for retries (seconds).
  scheduler.base: 2
  scheduler.cap: 60

pipeline:
  inputs:
    - name: tail
      path: /var/log/containers/*.log
      tag: kube.*
      # Persist read offsets so a restart neither re-reads nor skips.
      db: /var/log/flb-storage/tail.db
      # Opt this input into the disk buffer.
      storage.type: filesystem

  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

      # Retry forever rather than dropping after the default single attempt.
      retry_limit: no_limits

      # Outage budget for this output's on-disk queue:
      # peak_rate x tolerated_outage. Oldest chunks drop past this.
      storage.total_limit_size: 2G

      # Fewer, larger requests: Berserk pays per request, not per record.
      batch_size: 5000

      workers: 2

Why These Settings

storage.type: filesystem + storage.path — Berserk's ingest service has no local durability, so your shipper is the durability layer. Fluent Bit buffers in memory by default; a restart or an OOM loses everything buffered. Filesystem storage keeps a copy of each chunk on disk. Note the three storage settings have three different scopes, and all three are needed: storage.path is global (service), storage.type is opted into per input (or globally with storage.inherit: on), and storage.total_limit_size is per output.

storage.total_limit_size: 2G — The on-disk queue for this output, and therefore your outage budget: size it as peak_rate × tolerated_outage. When the queue is full Fluent Bit discards the oldest chunk to make room — it drops rather than pushing back, so this number is the difference between a gap in your data and no gap. There is no default; left unset, the queue is bounded only by the disk behind storage.path.

retry_limit: no_limits — The default is a single retry, after which the chunk is discarded. Berserk returns retryable errors for backpressure (429 / UNAVAILABLE with RetryInfo), transient S3 failures, and stream recovery; one retry is nowhere near enough to ride out a real outage. With a disk-backed buffer, retry indefinitely.

scheduler.base / scheduler.cap — Fluent Bit's retry delay is exponential backoff with jitter between these bounds, in seconds. The defaults (5 and 2000) let the interval grow past half an hour, far longer than any Berserk backpressure window; a 60-second cap recovers promptly after a transient outage without hammering the endpoint. Both are service-level and apply to every output.

Timeouts — Berserk batches for up to 2 seconds before uploading to S3 and holds the request open until the upload completes, so a request routinely takes seconds. Fluent Bit imposes no response deadline by default (net.io_timeout is 0s, unlimited), which is what you want here — leave it alone. net.connect_timeout (default 10s) covers only connection establishment and the TLS handshake and is fine as-is. Do not set a short net.io_timeout: it would convert Berserk's deliberate throttles into generic connection errors.

No compress — Deliberately absent: the plugin's compress option applies only to its OTLP/HTTP path, and is silently ignored when grpc: on. See Compression for what to do about it.

batch_size: 5000 — Records per flush for log payloads. Berserk holds each request open until the data is durable, so fewer, larger requests mean less total waiting and less throttling. Because gRPC requests go out uncompressed, this is also the knob that keeps you under the wire cap — raise it while watching actual request sizes.

db: on the tail input — Offset persistence. An output-side buffer only protects data that was already read; without the offset database a restart re-reads (duplicates) or skips (loss) file content regardless of how good your buffering is.

workers: 2 — Parallel flush threads for this output. Berserk accepts a limited number of concurrent requests, so keep this modest across your fleet and prefer larger batches over more workers.

Compression

The plugin's compress option (gzip / zstd) applies only to its OTLP/HTTP path. With grpc: on it is ignored — no error, no warning, requests just go out uncompressed. Berserk accepts gzip and zstd on both transports, so this is a Fluent Bit limitation, not an ingest one.

That leaves a real trade-off:

Compressed OTLP/HTTP

Drop grpc: on, move to port 4318, and add compress. Everything else — the buffer, the retry budget, the token — is identical to the recommended configuration above:

pipeline:
  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-ingest-host>
      port: 4318
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

      # gzip or zstd. Effective here, unlike on the gRPC path.
      compress: gzip

      retry_limit: no_limits
      storage.total_limit_size: 2G
      batch_size: 5000
      workers: 2

[OUTPUT]
  Name                     opentelemetry
  Match                    *
  Host                     <your-ingest-host>
  Port                     4318
  Tls                      On
  Header                   authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  Compress                 gzip
  Retry_Limit              no_limits
  Storage.Total_Limit_Size 2G
  Batch_Size               5000
  Workers                  2

Notes specific to this transport:

• No grpc: on, and no http2 either. Leaving both off selects the plugin's HTTP/1.1 path, which is where compress is honored. The request carries content-encoding: gzip, which Ingest decompresses.
• The signal URIs already match. logs_uri, metrics_uri, and traces_uri default to /v1/logs, /v1/metrics, and /v1/traces — exactly what Ingest serves, so leave them unset.
• tls: on is still port-independent. 4318 is not implicitly plaintext; set tls: on for any https:// endpoint, off only for an in-cluster address.
• Retry classification stays correct. On this path the plugin retries 429, 502, 503, and 504 and treats every other non-2xx as permanent — so Berserk's 429 backpressure and 503s are retried, while 400/401/413 are not, matching Delivery Semantics. It does not read the Retry-After header, so the wait comes from scheduler.base / scheduler.cap rather than Berserk's own estimate. Keep scheduler.cap in the tens of seconds so a throttle clears promptly.
• Sizing gets easier, not free. The 16 MiB cap applies to the compressed body, so a gzipped batch typically carries 40–80 MiB of telemetry. You can raise batch_size well past what gRPC tolerates.

Batching and Request Size

Requests over 16 MiB on the wire come back as InvalidArgument (gRPC) or 413 (HTTP). Both are permanent, so the chunk is dropped rather than retried — and over gRPC that ceiling applies to uncompressed protobuf, so it is closer than it looks.

batch_size (records per flush) is the direct lever. Fluent Bit's chunk size and storage.max_chunks_up govern how much accumulates before a flush, so watch actual request sizes after any change rather than reasoning from record counts. If lowering batch_size enough to clear the cap costs you too much throughput, switch to compressed OTLP/HTTP instead — the cap then applies to the compressed body.

Durability: Fluent Bit Is Not the Collector

Worth being explicit, because it decides whether Fluent Bit is the right choice for a given signal:

The practical consequence: Fluent Bit's retry pacing is coarser than the collector's, and its overflow behavior is drop-oldest rather than push-back. For high-volume logs where a gap during a long outage is acceptable, that is a fine trade for the smaller footprint. For audit-grade data, put an OpenTelemetry Collector between Fluent Bit and Berserk — point Fluent Bit's opentelemetry output at the collector's OTLP receiver and let the collector own durability.

One thing does line up cleanly: the plugin's gRPC retry classification matches Berserk's. It retries UNAVAILABLE, RESOURCE_EXHAUSTED, INTERNAL, ABORTED, CANCELLED, and DEADLINE_EXCEEDED, and treats INVALID_ARGUMENT and UNAUTHENTICATED as permanent — exactly the split described in Delivery Semantics.

If you do run Fluent Bit standalone and want a record of what it gave up on, enable its dead letter queue — discarded chunks are written there instead of vanishing.

Signals Other Than Logs

The opentelemetry output carries whatever the pipeline routes to it, so metrics and traces work the same way — the plugin selects the right OTLP service path per signal:

pipeline:
  inputs:
    - name: node_exporter_metrics
      tag: node_metrics
      scrape_interval: 10

  outputs:
    - name: opentelemetry
      match: node_metrics
      host: <your-ingest-host>
      port: 4317
      grpc: on
      tls: on
      header: authorization Bearer ing_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

OTLP profiles are not accepted by Berserk; don't route the profiles signal to this output.

Mapping Log Records

Fluent Bit records are key/value maps; OTLP log records have a distinguished body, severity, and trace context. With no mapping configured, the record is sent as the log body as a whole. If your records already carry those fields under known keys, map them explicitly so they land in the right OTLP slots — and therefore in the right Berserk columns — instead of being buried in the body:

      logs_body_key: $message
      logs_severity_text_message_key: level
      logs_trace_id_message_key: trace_id
      logs_span_id_message_key: span_id
      # Keep the remaining record fields as log attributes.
      logs_body_key_attributes: true

logs_body_key_attributes: true is what carries the record's remaining fields across as log attributes; without it, selecting a body key leaves them behind.

Troubleshooting

Turn on response logging to see what Berserk actually said:

      log_response_payload: true

Then confirm end to end:

bzrk search "<your table> | take 10" --since "5m ago"