Tamper Evidence
How Berserk detects stored telemetry that was changed or removed, and the read-time verification setting
Berserk detects telemetry that was changed or removed after it was written. Every stored segment carries hashes of its own contents, and the metadata database keeps both each segment's digest and an append-only, hash-chained record of what was written, merged and deleted.
What is recorded
- In each segment. When a segment is written to object storage, each of its chunks is hashed with a cryptographic hash function, and the hashes are stored in the segment along with hashes of its header and chunk index. The hash of that record, the segment's digest, covers every byte of the file.
- In the catalog. The digest is stored with the segment's catalog entry when the segment is registered, and cannot be changed afterwards.
- In the audit chain. Creating, merging or deleting a segment adds an event to an append-only chain kept per table. Each event includes the hash of the one before it, so an edited, removed or reordered event breaks the chain.
Each event carries the hash of the one before it, so changing or removing an event breaks the link to the next. The daily check re-derives every link.
When it is checked
| When | What is checked |
|---|---|
| A query opens a segment | The file's digest against the catalog; its header and chunk index against their hashes |
| A query reads a chunk | The chunk's bytes against its hash |
| A segment is merged | Every input and the merged output, in full |
| Daily | The audit chain, event by event, then the catalog against the chain |
A query never returns data that fails these checks. A segment that fails at open is reported as a partial failure. A chunk that fails is first fetched again from object storage, in case the local cache was damaged; if it still fails, it is reported as a warning on the result.
The daily chain check
The daily check attests what it can still prove. If the chain itself has been altered, it withholds the attestation and logs an error. If a segment disagrees with the chain — missing, present without a recorded creation, or with a different digest — it logs that segment and goes on attesting the rest. A missing or unrecorded segment is also written into the chain as an event of its own once the disagreement persists. One damaged or tampered segment does not make the rest of your data unverifiable.
Read-time verification
The query checks above are controlled by audtVerify, which is true by
default.
audtVerify: true (default) | audtVerify: false | |
|---|---|---|
| A query opens a segment | Digest checked against the catalog | Not checked |
| A query reads a chunk | Hashed and compared with its recorded hash | Checked by its CRC32 checksum only |
| Merges and the daily chain check | Full verification | Full verification |
With false, queries do not pay for hashing the data they read. The checksum
still catches accidental corruption, but not a deliberate change that also
rewrites the checksum.
Set it on both charts, because the nursery also answers queries:
query:
config:
audtVerify: false
nursery:
config:
audtVerify: falseLimits
The audit chain lives in the metadata database. Someone who can write to both the object store and the metadata database could make the two agree again. Keep them under separate credentials, so that no single credential can rewrite both. Copying the chain to write-once storage outside the cluster, which closes this gap, is available on request.
Tamper evidence covers data after it is written. It cannot tell whether a compromised Berserk service wrote the wrong data in the first place.