Life of a Row

A row — one log line, span, or metric point — passes through several Berserk components on its way in, and is repackaged a few times after that. This page follows a single row end to end: where its bytes live at each stage, when it becomes visible to queries, and what breaks (and doesn't) when a component is down.

The solid path is the row itself, moving from one keeper to the next. The dotted lines show who answers a query while the row is at that stage.

1. Accepted

Your collector sends the row to ingest in an OTLP batch. The ingest token on the request decides which table it belongs to (a bzrk.table resource attribute can override this per record).

Ingest buffers for up to 2 seconds or 16 MiB, then writes the batch into object storage and only then answers the collector. A success response means the row is durably in S3 — there is no window where an acknowledged row exists only in memory. Until then it is your collector's responsibility: unacknowledged data must sit in the collector's queue.

At this point the row is durable but not yet queryable. See Ingestion for the response codes and backpressure behavior, and OpenTelemetry Collector for collector configuration.

2. Queryable

A nursery follows the stream in object storage in order and turns newly arrived data into baby segments — small files it holds locally and answers queries from directly. The row becomes queryable here, typically within a second or two of the acknowledgment; 1–10 seconds end to end from the collector's send is the normal range.

One upload can carry rows for several tables; the nursery splits them out, so from this point on the row is only ever grouped with rows of its own table.

Nothing else has to happen for the row to show up in the UI, the CLI, or Grafana. Merging, compaction, and caching are all optimizations that happen after it is already visible.

3. Published

In the background the nursery merges its baby segments into a segment: an immutable, time-ordered file, uploaded to object storage and registered with meta. Registration is the moment the row moves from the nursery's keeping to the cluster's: meta advances the published point, and from then on the row is served by the query nodes rather than by the nursery.

Registration also pushes a copy of the new segment into the cache of the query node that owns it, so the first query to touch it doesn't have to fetch it back from S3. See Segment Placement for how that owner is chosen.

4. Compacted

Meta schedules merge work and the janitor carries it out: it reads several segments, writes one larger segment covering their combined time range, registers it, and marks the inputs as replaced. The row's content doesn't change — only its packaging. Fewer, larger segments mean fewer objects to open per query and better compression and index efficiency.

How wide a segment is allowed to grow in time is capped per table, derived from the table's retention and overridable with bzrk table merge-span set <table> <duration>. A row may be rewritten into a larger segment several times over its life.

Replaced segments are not deleted at once: they linger for about ten minutes so queries that already resolved them can finish reading, after which the janitor removes the objects from S3.

5. Read

When a query runs, the coordinator asks meta which segments of the table overlap the query's time range — everything outside it is skipped without being opened. The matching segments are grouped by owning query node, and each node scans its own from local cache, falling back to S3 for anything not resident. In parallel, the nurseries contribute the tail of data not yet published. The coordinator combines the partial results and streams them back as they arrive.

The row is read from exactly one place at any moment: the nursery before publication, its owning query node afterwards.

6. Ages out

Retention is configured per table and off by default — with no retention set, data is kept indefinitely:

bzrk table retention set my-table 30d
bzrk table retention clear my-table

Meta sweeps about once a minute and marks every segment whose end is older than the cutoff as expired; the janitor then deletes those objects from S3, again after a short grace period. Retention works on whole segments, so a row survives until the last row of its segment is past the cutoff.

Deleting a table is a separate path: bzrk table delete <table> tombstones it immediately (its data stops being visible) but leaves it recoverable with bzrk table resurrect <table> for a 7-day grace window, after which the table and its segments are removed for good. --immediate skips the grace window.

Where the row lives

Operating notes

• Freshness comes from the nursery, throughput from ingest, scan capacity from query nodes. If new data is slow to appear, look at the nurseries; if collectors are being throttled, look at ingest; if queries over older data are slow, look at query-node count and cache size. Hardware Sizing has the numbers for each.
• A nursery going down costs freshness, not data. Stream data is already durable in S3 before any nursery touches it. Meta reassigns the stream to another nursery, which resumes from the first unpublished offset — recent rows briefly stop appearing, then catch up.
• Meta being unavailable stops publication and new queries, not ingestion. Rows keep landing durably in object storage and are picked up once meta is back.
• Losing a query node costs cache warmth, not data. Published segments live in S3; another node takes over that share and re-reads what it needs.
• Compaction is a background cost you can observe. Merge activity shows up in the janitor's metrics; if segments stay small and numerous, queries do more object opens than they should.