Berserk Docs

Tracing

Find traces by structural pattern with trace-find — no KQL required

The Tracing tab finds whole traces matching a structural pattern and lets you drill into any of them as a span waterfall. It composes a trace-find query for you, so you can express "a span in api with a descendant error in db" without writing KQL.

Table Selection

Choose a traces table from the dropdown at the top of the tab. This sets the source for the outlier heatmap, the query builder, and the trace list.

Outlier Heatmap

Above the builder, a heatmap shows the trace-duration distribution over the selected time range — one cell per (time × duration) bucket, counting the traces that match the builder's pattern, with error-containing traces highlighted in red. It shares the builder's matched set, so it lines up with the trace list's latency percentiles. Use it to spot latency or error hot-spots before narrowing in with the builder.

Query Builder

The builder expresses a chain of span-sets joined by relations:

  • A span-set { … } is a set of span predicates (service, attributes, free-text search) combined with and. Build each one with the same filter bar used elsewhere in the UI. An empty span-set matches any span.

  • A relation connects two span-sets by how their spans relate within a trace:

    RelationMeaning
    has descendant (>>)a span anywhere below in the trace tree
    has child (>)a direct child span
    has ancestor (<<)a span anywhere above
    has parent (<)a direct parent span
    has sibling (~)a span sharing the same parent
    has correlated log (::)a log record correlated to the span
    and / orboth / either pattern in the same trace

Use + Add step to extend the chain and the on a step to remove it. The optional within field bounds how far apart related spans may be in time, and summarize rolls matching traces up into aggregates (e.g. count() by services[0]). A live preview shows the exact KQL being run.

The query runs automatically once any span-set has a filter; an empty builder stays idle.

Views: Table, Groups, Triage

Below the heatmap, a switcher chooses how the matching traces are shown — all three share the same filters, time range, and matched set:

  • Table (default) — one row per trace (newest first, up to 1000): operation, services, span count, duration, and start time. The flat, unaggregated view.
  • Groups — traces rolled up by root operation, with trace count, p50/p95/p99 latency, and the representative (slowest) trace's service flow. A group-by selector is planned; for now the only grouping is by operation.
  • Triage — reserved for guided triage (coming soon).

Drill-down

Selecting a row in Table or Groups opens that trace as a waterfall on the right, the same span timeline used in the Explore tab's Trace view. Click any span to inspect its attributes, resource, and status in the side panel.

On this page