Berserk Docs

Running Queries

Inspect what the cluster is executing right now with bzrk ps, tell a slow query from a stuck one, see where each query came from, and stop one

bzrk ps lists the queries your cluster is executing at this moment, and bzrk kill stops one. Between them they answer the two questions an operator has when a cluster is busy: what is running, and should it be.

The listing is cluster-wide. A query runs on exactly one query pod, so the gateway asks every pod and merges the answers — a pod that cannot be reached is left out of the result rather than failing it, which shows up as that pod's absence from the POD column.

Reading a row

bzrk ps
QUERY ID                               POD                      ELAPSED   QUEUED      IDLE    SPAN  PROGRESS CHUNKS scan/need           ROWS
0199f3a2-7c41-7db3-9f0e-2a1b4c6d8e10   10.0.1.5:9510              1m2s        -     200ms      7d     42.0%        1200/3000        1204331
  who:    alice@example.com (session) from 203.0.113.7
  claims: bzrk-cli 1.0.19 · cli
  window: 7d  2026-08-27T09:12:00Z → 2026-09-03T09:12:00Z
  search "error" | summarize count() by bin(timestamp, 1m)
ColumnMeaning
QUERY IDServer-side id, and the handle bzrk kill takes
PODWhich query pod is running it
ELAPSEDHow long since the query arrived
QUEUEDHow long it waited for an execution slot; - when it never waited
IDLESince the last frame reached the client; - before the first one
SPANHow much time the query covers — see The window a query scans
PROGRESSShare of planned work done; - while still planning
CHUNKS scan/needChunks scanned against chunks still to scan
ROWSRows processed so far

ELAPSED, QUEUED and IDLE are all durations of the execution; SPAN is the duration the query asks about. A row reading 1m2s elapsed over 30d of data is doing a lot of work; the same elapsed time over 10m of data is not.

The column that earns its place is IDLE. Elapsed time alone cannot tell a slow query from a stuck one: a query four minutes in whose last frame was 200 ms ago is working, and one whose last frame was three minutes ago is not. QUEUED makes the same distinction at the other end — a query a minute old that has not started is waiting for a slot, not running slowly, and it has no progress to report because planning has not happened yet.

Buffered surfaces show no idle time

IDLE reads - for the whole life of a query on a buffered surface (the Kusto REST and Loki endpoints), which delivers its result once at the end. Nothing is wrong; there are simply no frames to be idle since.

The window a query scans

SPAN is how much time the query covers, and it is often the difference between an expensive query and a mistake. Nothing else in the row distinguishes a ten-minute lookback from a thirty-day one: both can sit at the same elapsed time, the same progress and the same chunk counts for a while.

The window: line under the row gives the bounds behind that span, because a 30-day window ending last week is not the same query as one ending now. A query bounding the ingest-time axis gets an ingest: line on the same shape, and the column marks an ingest-only span as i30d so it is not read as an event window.

The window is the server's own resolution of what the request carried — relative bounds like 24h ago are already resolved against the now captured when the request arrived — so it is not a client claim and is not grouped with one.

A self-bounded query reports no span

SPAN reads - when the request imposed no window and the query carries its own bounds in KQL (| where timestamp > ago(1h)). The coordinator derives that window while planning, after the listing entry is made, so the pod answering bzrk ps genuinely does not know it. Passing --since (or the equivalent on your surface) is what makes the span visible.

--full widens the table with every execution statistic the engine reports — slices, chunk coverage, skip reasons by kind, bytes scanned and skipped, fetch and snapshot time, state-cache reuse — the same set bzrk search --stats prints for a finished query.

Where a query came from

Under each row are up to three lines: who asked, what the client says it is, and the query itself.

  who:    grafana-dev (service_token) from 203.0.113.7
  claims: grafana-plugin 1.4.0 · dashboard · Ingest health › p99 latency · user alice
  search "error" | summarize count() by bin(timestamp, 1m)

who: is what the server established. The gateway authenticated the caller, resolved the credential, and injected the result on its way to the query pod. A client cannot assert any of it: the header namespace these travel in is stripped from every inbound request at the public edge, so a caller that sends them has them removed before anything reads them.

Field
principalEmail for a user, principal name for a service
authn methodsession, cli_token, service_token or trusted_proxy
client addressLeft-most X-Forwarded-For from the trusted edge proxy, else the TCP peer

claims: is what the client said about itself, and nothing verifies it. It is worth reading anyway — a dashboard title is usually the fastest answer to "why is this running" — but it is a self-report, which is why the listing labels it rather than mixing it into the line above.

Field
app + versionThe program that issued the query
surfacecli, dashboard, explore, alerting, …
dashboard › panelTitle when the client sent one, uid or panel number otherwise
userThe human behind a shared credential
agentThe agent driving the client, when one is

The user row is what makes Grafana legible. The Berserk datasource plugin authenticates the whole organization with one service principal, so who: says grafana-dev for every panel in the company; claims: … user alice is the only thing distinguishing one person's dashboard from another's. The plugin takes the login from Grafana itself rather than from anything the browser can set, but it is still the plugin's word, not the server's — hence the label.

Either line is omitted when there is nothing to say. A service calling a query pod directly, inside the cluster, passes no gateway and so has no who: at all; a client that reports nothing about itself has no claims:.

The listing is not private

Any authenticated caller sees every running query, including its full text and the identity that submitted it. Treat bzrk ps output the way you would treat the query log.

Structured output

--json keeps the two halves in separate objects, so nothing consuming the output can mistake a self-reported user for a verified one by reaching for the wrong key:

{
  "query_id": "0199f3a2-7c41-7db3-9f0e-2a1b4c6d8e10",
  "pod": "10.0.1.5:9510",
  "query": "search \"error\" | summarize count() by bin(timestamp, 1m)",
  "attested": {
    "principal_type": "service",
    "principal_name": "grafana-dev",
    "authn_method": "service_token",
    "client_ip": "203.0.113.7"
  },
  "claimed": {
    "app": "grafana-plugin",
    "app_version": "1.4.0",
    "surface": "dashboard",
    "end_user": "alice",
    "dashboard_uid": "abc123",
    "dashboard_title": "Ingest health",
    "panel_title": "p99 latency"
  },
  "window": {
    "span_ms": 604800000,
    "since": "2026-08-27T09:12:00Z",
    "until": "2026-09-03T09:12:00Z",
    "ingest_span_ms": null,
    "ingest_since": null,
    "ingest_until": null,
    "now": "2026-09-03T09:12:00Z",
    "bypass_state_cache": false
  },
  "elapsed_ms": 62000,
  "idle_ms": 200
}

--csv and --tsv carry the same fields, flattened to attested.principal_name and claimed.dashboard_title — the prefix survives, so a spreadsheet keeps the distinction too.

The query text is stored truncated (with a trailing ) so that one pathological client cannot make the listing unreadable, and the table collapses it to a single line. --json gives you what the pod holds.

Attributing your own client

Every field under claims: comes from one optional message on the query request, so any client can fill it in — and none has to. Berserk's own clients already do: the CLI reports itself, its version and its driving agent, and the Grafana plugin reports the dashboard, panel and Grafana login on both its streaming and its one-shot paths.

Two things are worth knowing before you add your own:

  • It travels on the request, not in a header. The gateway drops the entire x-bzrk-* header namespace on inbound so that nothing can forge the identity headers it injects itself, which means a header is not a channel a client can use. The Loki- and Kusto-compatible HTTP endpoints are the exception: a stock Grafana datasource pointed at them is attributed automatically from the dashboard headers Grafana already sends.
  • Nothing branches on it. Claims are display strings. They are capped in length and stripped of control characters on arrival, and no planning, routing or authorization decision reads them.

Stopping a query

bzrk kill 0199f3a2-7c41-7db3-9f0e-2a1b4c6d8e10

Cancellation is cooperative. The command returns as soon as the signal is delivered — reported as signalled, not "stopped" — and the query stops starting new work at its next boundary rather than being interrupted mid-chunk. The client running it sees its stream end as cancelled rather than completed, so it can report the query as killed rather than done.

not_found means no pod is running that id, which usually means the query finished on its own between your bzrk ps and your bzrk kill.

After the fact

bzrk ps only sees queries in flight. The same provenance is recorded on each query's completion event, so the historical question — which dashboard is generating the load, which principal is behind the expensive queries — is answerable from your telemetry rather than by watching the listing. See Observability for where those events land, and Query Usage Reporting for the anonymized per-query record, which deliberately carries none of this.

On this page