Best Practices

Most Microsoft Kusto best practices carry over to Berserk, but Berserk's indexing changes which ones matter. A large class of patterns that are "slow, avoid them" in Azure Data Explorer (ADX) — contains, case-insensitive matching, predicates on dynamic fields, free-text and wildcard search — are cheap in Berserk because they reduce to index probes that prune data before any rows are scanned. This page is the ADX best-practices list, re-mapped to Berserk.

How Berserk prunes

Berserk skips data at three levels before reading rows. Knowing them explains every recommendation below:

• Shard index — segments are sharded by a set of fields configured per table (for example resource['service.name'] and metric_name). An == or in (…) filter on a shard field eliminates whole segments. Sharding is recorded per segment file at ingestion and never re-indexed, so changing the shard fields affects only newly-ingested data — the schema can evolve over time without rewriting history.
• Range index — datetime and numeric columns carry per-chunk min/max bounds, so the intrinsic timestamp filter and numeric == / < / > / between prune chunks cheaply.
• Token bloom — every string and dynamic field has a per-chunk, case-folded token bloom, so string search and equality prune chunks without reading them.

A predicate engages these indexes most reliably when applied to a bare column or path. Wrapping it in a function generally forces per-row evaluation — but the planner can see through a number of common wrappers and projections, so treat this as a strong default rather than an absolute.

In short

Why so many ADX rules don't apply

The ADX best-practices page is largely a catalogue of ways to avoid scans: prefer has over contains, avoid =~, don't search *, pre-filter dynamic lookups, materialize extracted columns at ingestion. Each exists because, in ADX, those operations fall back to scanning column data.

In Berserk the chunk-level bloom filter is case-folded and token-based, so the operations ADX warns about all become the same cheap primitive — a token probe that prunes chunks before any row is read:

• has, contains, startswith, … — one token bloom each, equal cost.
• =~ / case-insensitive — the bloom is already folded, so it prunes like ==.
• where dynamic.path == v — the path is a bloom key; prunes directly.
• search "term" / * / * matches regex "…" — whole-row token bloom (regex mines its literal anchor), then a SIMD scan of survivors.

The result: pick operators by semantics, not by a performance cheat sheet. The two rules that still carry real weight are the universal ones — filter early (so pruning runs first) and prefer bare-column predicates, keeping in mind the see-through cases below.

Index see-through

"Filter on a bare column" is a strong default, but the planner recognizes several wrappers and traces a predicate back to an indexed source, so these still prune:

• Case folding — tolower(col) == "x" / toupper(col) == "X" reuse the case-folded bloom (the same one =~ uses).

• Safe unwrapping — a safe tostring(col) == "x" and identity numeric to*() comparisons are unwrapped back to the bare column.

• Projection & parse lineage — a literal compared against a column derived from a string source is traced back to that source. For example:

  __json_logs
  | extend evt = parse_json(body)
  | where evt.action == "exception"

  filters on the computed evt, yet the planner emits a * has "exception" chunk bloom on the source body and prunes before parsing; the evt.action == "exception" check then runs only on surviving rows. The same lineage tracking applies through project and the parse operator.

The literal still has to be a clean token to be bloom-safe: a literal containing spaces or punctuation (e.g. "GET /api") isn't a single token, so the bloom leaf is dropped and that predicate falls back to a scan. This is the same reason a regex needs a clear literal anchor.

When in doubt, look at the physical plan (the --physical plan view): a Bloom:, Range:, or shard line for your predicate means the index is engaged; its absence means you're scanning.

Not available in Berserk

A few ADX best-practice rows have no Berserk equivalent:

• materialize() — no explicit named-result materialization. Map-reduce state caching covers some of the repeated-computation benefit automatically.
• Materialized views / materialized_view() — not implemented.
• Update policies — no ingestion-time field extraction. Rely on bloom indexes and asXXX coercion at query time instead.
• Join tuning — hint.shufflekey, hint.strategy=broadcast, and the lookup operator are not implemented; join distribution is automatic.
• in~ — use in (case-sensitive).

See Compared to Microsoft KQL and Missing functions for the full compatibility surface.