OpenTelemetry Collector

The OpenTelemetry Collector sends all three signals over OTLP, and its persistent queue holds data until Berserk acknowledges it (Ingestion). The configuration below exports over gRPC, the recommended protocol.

Recommended ConfigurationCopy Anchor Link

extensions:
  # Disk-backed queue: buffered data survives collector restarts.
  file_storage/queue:
    directory: /var/lib/otel/queue

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  # The otlp exporter sends OTLP over gRPC.
  otlp/berserk:
    endpoint: "<your-ingest-endpoint>:4317"
    # Plaintext endpoints only, such as an in-cluster ingest:4317:
    #   tls:
    #     insecure: true
    headers:
      authorization: "Bearer <your-ingest-token>"

    compression: gzip

    # Above ingest's 25 s acknowledgement timeout.
    timeout: 30s

    sending_queue:
      storage: file_storage/queue

      # A ceiling on parallel export workers.
      num_consumers: 4

      # Outage buffer: peak rate × tolerated outage × headroom.
      queue_size: 1073741824
      sizer: bytes

      batch:
        sizer: bytes
        min_size: 4194304     # 4 MiB
        flush_timeout: 1s     # about 3 s end to end; ingest adds up to 2 s
        max_size: 10485760    # under the 16 MiB request limit

    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_interval: 60s
      # 0 retries without a limit; the default drops data after 5 minutes.
      max_elapsed_time: 0
      multiplier: 2

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 256
    spike_limit_mib: 64

service:
  extensions: [file_storage/queue]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp/berserk]
    logs:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp/berserk]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp/berserk]

What Each Setting DoesCopy Anchor Link

• timeout: 30s: Ingest holds a request for up to 2 seconds of batching plus the S3 upload, so the collector's default 5 s timeout fails requests in normal operation. During a sustained S3 outage, Ingest answers within its 25 s acknowledgement timeout with 429 and a Retry-After of 30–45 s. A collector timeout under 25 s turns that 429 into a timeout.
• file_storage/queue: Ingest does not buffer, so the collector's queue holds unacknowledged data. An in-memory queue loses it on restart; file_storage keeps it on disk.
• max_elapsed_time: 0: retries without a time limit. Ingest returns retryable errors for backpressure (429), transient S3 failures (503) and stream recovery (503). With the default limit, the exporter drops data after 5 minutes of retries.
• num_consumers: 4: a ceiling on parallel export workers. Concurrency is min(num_consumers, batches ready): about 1 under light load, more only with a backlog, which also builds when Berserk is slow. Berserk accepts a limited number of concurrent requests, so keep num_consumers × signals, summed across your collectors, low. Larger batches raise throughput without more requests.
• sending_queue.batch: the collector batches, so Berserk receives fewer, larger requests. min_size fills requests, flush_timeout: 1s caps the added latency, and max_size keeps requests under the 16 MiB limit.
• No batch processor: the processor holds data before the persistent queue, so a restart loses it. It is deprecated upstream in favor of the exporter's sending_queue.batch.
• memory_limiter: pushes back on receivers as the collector nears its memory limit, so a filling queue does not run it out of memory.
• compression: gzip: OTLP payloads typically compress 3–5×. Compression cuts egress and request time, and lets larger batches fit under the 16 MiB limit.

Queue and Retry SettingsCopy Anchor Link

Queue storage and the retry limit decide what a collector can lose:

• A retrying batch holds its queue consumer for the whole wait. With max_elapsed_time: 0, up to num_consumers batches stay in memory during an outage while the backlog builds in the queue, so set num_consumers, memory_limiter and queue_size together.
• The exporter queue protects data only after it is enqueued. With the filelog receiver, also set its storage:, so file offsets survive a restart; without it, a restart can re-read or skip file content.

Staying Under the Wire CapCopy Anchor Link

Ingest accepts requests up to 16 MiB on the wire. Two exporter settings keep requests under it:

• compression: gzip, in the configuration above.
• max_size on sending_queue.batch.

A request over the limit gets 413 (HTTP) or InvalidArgument (gRPC); the exporter treats both as permanent and drops the batch.