Network Policies
Securing inter-service communication with Kubernetes NetworkPolicies
The chart's optional Kubernetes NetworkPolicies restrict each service's ingress and egress to known paths.
Two services take traffic from outside the cluster. The gateway authenticates requests and passes them on (UI over HTTP, the Kusto REST endpoints to query, gRPC to meta, query and ingest). Ingest accepts telemetry over the OTLP, Loki Push and Datadog APIs. Every other service accepts connections only from its in-cluster peers.
NetworkPolicy enforcement requires a CNI plugin. On EKS, use the VPC CNI
v1.14+ with ENABLE_NETWORK_POLICY=true on the aws-node DaemonSet, or
install Calico. Without a compatible CNI, the NetworkPolicy resources are
created but have no effect.
Enabling
Network policies are off by default:
global:
networkPolicy:
enabled: trueService Communication Map
With network policies on, only the gateway and ingest accept traffic from outside the cluster; every other service accepts it only from its peers:
| Service | Accepts ingress from (peers) | Connects out to (peers) | External ingress |
|---|---|---|---|
| gateway | public edge | ui, permissions, meta, query, ingest | Yes |
| ingest | external telemetry sources | meta | Yes |
| meta | gateway, query, ingest, janitor, ui, nursery | -- | No |
| query | gateway, ui | meta, nursery, gateway | No |
| nursery | query | meta, query | No |
| janitor | -- | meta, query | No |
| ui | gateway | gateway, query, meta, permissions | No |
| permissions | gateway, ui | -- | No |
query answers only requests the gateway signed. A pod or namespace that is
allowed to reach query:9510 still gets 401 (UNAUTHENTICATED over gRPC)
unless its request came through the gateway, so point every client at the
gateway.
Beyond the peers above, every service also gets:
- DNS egress (port 53 UDP/TCP), always.
- External egress to the CIDR allowlist (S3 on 443, PostgreSQL on 5432 by
default) when
allowExternalEgressis set, the default for every service. See External Egress. - OTLP egress to
ingest:4317when observability is enabled. See OTLP Telemetry.
Extra in-cluster rules:
querypods connect to each other on port 9510, for a cache of partial query results they share, and on port 9520, the QWS worker-sibling lane.nurseryandjanitorconnect toqueryon port 9520 only, to place each merged segment in the cache of the query pod that owns it.metaadditionally accepts connections from one-offsetup/adminoperator pods (e.g.kubectl run bzrk-cli) used for token and table creation.
External Egress
Services reach external endpoints (S3, PostgreSQL) through a CIDR allowlist. By default, HTTPS (443) and PostgreSQL (5432) are open to all addresses. To narrow them:
global:
networkPolicy:
enabled: true
externalEgress:
# S3 VPC endpoint
- cidr: 10.0.0.0/16
ports:
- port: 443
protocol: TCP
# RDS PostgreSQL
- cidr: 10.1.0.0/24
ports:
- port: 5432
protocol: TCPOTLP Telemetry
With global.observability.otlpEnabled: true, every service gets egress to the
OTLP endpoint:
Default endpoint (ingest:4317): egress to the ingest pod, by pod selector,
and an ingest rule that accepts telemetry from every pod in the namespace on
port 4317.
External collector: for an otlpEndpoint outside the namespace, add CIDR
rules under global.networkPolicy.otlpEgress:
global:
observability:
otlpEnabled: true
otlpEndpoint: "otel-collector.monitoring:4317"
networkPolicy:
enabled: true
otlpEgress:
- cidr: 10.2.0.0/16
ports:
- port: 4317
protocol: TCPHelm prints a warning during install/upgrade if OTLP is enabled with a
non-default endpoint but no otlpEgress rules are configured.
Custom Rules
Extra ingress or egress rules go in the global values, for every service, or in a service's own values.
Global:
global:
networkPolicy:
additionalIngress:
- from:
- namespaceSelector:
matchLabels:
name: monitoring
ports:
- port: 9090
protocol: TCPPer service, here a monitoring namespace reaching query's operator port,
which serves /metrics and the profiling endpoints:
query:
networkPolicy:
additionalIngress:
- from:
- namespaceSelector:
matchLabels:
name: monitoring
ports:
- port: 9511
protocol: TCPPer-Service Configuration
Each service supports these networkPolicy values:
| Value | Type | Description |
|---|---|---|
allowExternal | bool | Allow ingress from any source on service ports |
allowExternalEgress | bool | Allow egress to global.networkPolicy.externalEgress CIDRs |
ingressFrom | list | Berserk peers allowed to connect ([{app, ports}]) |
egressTo | list | Berserk peers this service connects to ([{app, ports}]) |
additionalIngress | list | Extra raw NetworkPolicy ingress rules |
additionalEgress | list | Extra raw NetworkPolicy egress rules |
Guide: Single Sign-on with Google Workspace
The Google Cloud console pages that give Berserk an OpenID Connect client for your Workspace, and what to enter in Berserk
Tamper Evidence
How Berserk detects stored telemetry that was changed or removed, the read-time verification setting, and the row structure check