Berserk Docs
Tabular OperatorsLogging Operators

fieldstats

Analyzes dynamic column values to discover field paths and their statistics, returning a table with AttributePath, Type, Cardinality, Frequency, Hint, and HintExhaustive columns.

HintExhaustive is true when the Hint values represent the complete set of distinct values.

Useful for exploring JSON-like data to understand what fields exist across records. Supports analyzing multiple columns in a single call. If no columns are specified, analyzes all input columns (or the _ column if present).

IMPORTANT: This is an expensive operation. Use the with limit=N parameter to limit samples (default 1000), or use | take N before fieldstats.

Use with depth=N to control how deep to descend into nested dynamic structures. When the depth limit is reached, nested objects and arrays are reported as dynamic type instead of recursing into their contents. depth=1 means only top-level fields.

Use with field="*glob*" and/or with value="*glob*" to look for specific fields: only leaves whose AttributePath (and, for value, at least one value) matches the case-insensitive * glob are reported, and only rows containing such a leaf count toward limit. This finds rare fields a plain sample would miss. A pattern that matches nothing scans the whole time range and returns no rows.

Syntax

fieldstats [columns] [with limit=N depth=M hint=K field="glob" value="glob"]

Analyze fields in columns (all columns if none specified)

Parameters

NameDescription
columnsColumns to analyze (all columns if omitted) (optional)

with limit=

Maximum rows to sample (default 1000)

ValueDescription
NSample at most N rows.

with depth=

Nesting depth for dynamic columns

ValueDescription
NRecurse at most N levels (N must be > 0). depth=1 means top-level fields only.

with hint=

Distinct values tracked per field for the Hint column (default 20)

ValueDescription
NTrack at most N distinct values (N must be > 0).

with field=

Case-insensitive * glob the AttributePath must match

ValueDescription
"*glob*"Report only paths matching the glob, and sample only rows containing one.

with value=

Case-insensitive * glob a leaf value must match

ValueDescription
"*glob*"Report only leaves with a matching value; Hint holds only matching values.

Examples

Example 1 — Discover fields in a dynamic column

datatable(log:dynamic)[
  dynamic({"warrior":"Ragnar", "weapon":"axe", "voyages":42}),
  dynamic({"warrior":"Bjorn", "weapon":"sword", "voyages":31}),
  dynamic({"warrior":"Ivar", "weapon":"bow", "voyages":35})
]
| fieldstats log
AttributePath (string)Type (string)Cardinality (long)Frequency (real)Hint (dynamic)HintExhaustive (bool)
log.voyageslong31.0["31","35","42"]true
log.warriorstring31.0["Bjorn","Ivar","Ragnar"]true
log.weaponstring31.0["axe","bow","sword"]true

Example 2 — Analyze top-level fields only

datatable(log:dynamic)[
  dynamic({"ship":{"name":"Naglfar", "crew":80}, "destination":"Lindisfarne"}),
  dynamic({"ship":{"name":"Wave Rider", "crew":45}, "destination":"York"})
]
| fieldstats log with depth = 1
AttributePath (string)Type (string)Cardinality (long)Frequency (real)Hint (dynamic)HintExhaustive (bool)
log.destinationstring21.0["Lindisfarne","York"]true
log.shipdynamic01.0nulltrue

Example 3 — Find fields whose path contains "name"

datatable(log:dynamic)[
  dynamic({"warrior":"Ragnar", "ship":{"name":"Naglfar"}}),
  dynamic({"warrior":"Bjorn", "ship":{"name":"Wave Rider"}})
]
| fieldstats log with field = "*name*"
AttributePath (string)Type (string)Cardinality (long)Frequency (real)Hint (dynamic)HintExhaustive (bool)
log.ship.namestring21.0["Naglfar","Wave Rider"]true

On this page