Cilium Tetragon and Falco — runtime security with eBPF
TL;DR: eBPF lets you observe and enforce policy inside the Linux kernel without modifying it. Falco is the original CNCF eBPF/syscall runtime security tool (detection/alerting). Cilium Tetragon adds kernel-level enforcement (block syscalls per identity). Together: deep workload visibility + selective blocking, without sidecars or host agent invasiveness.
What it is
Three closely related tools:
| Tool | Mode | Strength |
|---|---|---|
| Falco | Detection (alerts) | Rule-based, mature, broad coverage |
| Cilium Tetragon | Detection + enforcement (kernel-level) | Pod-identity-aware, selective blocking |
| Cilium (network) | Network policy via eBPF | L3-L7 network enforcement |
All three deploy as DaemonSets on Kubernetes nodes. All use eBPF for low overhead.
Preconditions / where it applies
- Kubernetes 1.20+ on modern Linux kernels (5.4+ recommended for full feature set)
- Privileged DaemonSet allowed (these tools need kernel access)
- Existing SIEM / SOC for alert ingestion
Falco tradecraft
Install
1
2
3
4
5
6
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
--namespace falco --create-namespace \
--set tty=true \
--set falcosidekick.enabled=true \
--set falcosidekick.config.slack.webhookurl=$SLACK_HOOK
Default rule set
Falco ships with falco_rules.yaml covering common detection patterns:
- Container shell spawn
- Privileged container creation
- Sensitive file read (passwords, SSH keys)
- Network connection from container to unusual destination
- Cryptocurrency mining indicators
- Privilege escalation
- Container drift (running binary not in image)
Custom rule
1
2
3
4
5
6
7
8
9
10
11
- rule: Suspicious package install in container
desc: Detect package manager invocation in running container
condition: >
spawned_process and container and
(proc.name in (apt, apt-get, dnf, yum, apk, pip, gem, npm))
and not container.name in (allowed_pkg_install_containers)
output: >
Package install in container (user=%user.name container=%container.name
cmd=%proc.cmdline)
priority: WARNING
tags: [container, supply_chain]
Falco events → SIEM
Falcosidekick fans events out to:
- Slack / Teams / Discord webhooks
- Splunk / Elastic / Datadog / Loki
- Kafka / NATS
- AWS SNS / SQS / SecurityHub / EventBridge
- GCP Pub/Sub / Cloud Logging
- Azure Event Hub / Sentinel
- PagerDuty / OpsGenie
- 50+ outputs total
Performance
Falco using modern eBPF probe: ~1-3% CPU on busy nodes. Use engine.kind: modern_ebpf over the legacy kernel module.
Cilium Tetragon tradecraft
Install
1
2
helm repo add cilium https://helm.cilium.io
helm install tetragon cilium/tetragon -n kube-system
Tracing policy — observation only
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata: {name: file-monitoring-sensitive}
spec:
kprobes:
- call: "fd_install"
syscall: false
args:
- index: 0
type: "int"
- index: 1
type: "file"
selectors:
- matchArgs:
- index: 1
operator: "Prefix"
values: ["/etc/shadow", "/etc/passwd", "/root/.ssh/"]
Logs every open of these paths with pod context.
Enforcement policy — kill / signal on match
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata: {name: block-shell-in-prod}
spec:
podSelector:
matchLabels: {env: production}
kprobes:
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
- matchArgs:
- index: 0
operator: "Equal"
values: ["/bin/bash", "/bin/sh"]
matchActions:
- action: Sigkill
Blocks shell execution in production pods. Real, kernel-level enforcement.
Tetragon vs Falco strengths
| Feature | Falco | Tetragon |
|---|---|---|
| Detection (alerts) | ✅ Mature | ✅ Newer |
| Blocking (Sigkill) | ❌ | ✅ |
| Pod identity awareness | ✅ via plugins | ✅ Native |
| eBPF only | ✅ Modern path | ✅ Always |
| K8s-native policy CRDs | Partial | ✅ TracingPolicy CRDs |
| Maturity | High (since 2018) | Growing (2022+) |
| Community / docs | Largest | Growing |
Most orgs run Falco for breadth; add Tetragon when blocking is required.
Cilium (network)
Separate but companion to Tetragon. CNI replacing kube-proxy with eBPF:
- L3-L7 network policy (HTTP method/path filtering, gRPC, Kafka)
- Service mesh without sidecars (Cilium Service Mesh)
- Encrypted node-to-node traffic (WireGuard or IPsec)
- Cluster Mesh (multi-cluster connectivity)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata: {name: allow-api-l7}
spec:
endpointSelector: {matchLabels: {app: api}}
ingress:
- fromEndpoints: [{matchLabels: {app: frontend}}]
toPorts:
- ports: [{port: "8080", protocol: TCP}]
rules:
http:
- method: GET
path: "/v1/users/.*"
- method: POST
path: "/v1/users"
L7-aware policy. Standard k8s NetworkPolicy is L3/L4 only.
What runtime security catches that admission doesn’t
Admission policy (policy-as-code-opa-kyverno-defender) gates manifest at apply time. Runtime catches:
- Container drift — running binary not in image (e.g., curl downloaded post-launch)
- Privilege escalation post-deployment
- Reverse-shell from compromised pod
- Cryptocurrency mining workloads
- Data exfil over network from pods that shouldn’t egress
- Lateral movement via cluster API server abuse
Defense in depth: admission for static, runtime for behaviour.
Detection patterns to deploy first
- Shell in container — most pods shouldn’t spawn interactive shells
- Sensitive file access —
/etc/shadow,/var/run/secrets/kubernetes.io/serviceaccount/token,/.aws/credentials - Outbound to unusual destination — Tor, cryptocurrency, attacker C2 ranges
- Privilege escalation syscalls —
setuid,capset - Kernel module load —
init_module,finit_module - Mount syscall — pod mounting host filesystem post-deployment
- Crypto miner indicators — xmrig, monero, kawpow strings
- Anomalous process tree —
kubectl execspawning grep then curl then sh - Container escape attempts — chroot manipulation, nsenter from unprivileged
- Modifications to /proc, /sys beyond normal kubelet patterns
Common implementation pitfalls
- Falco with default rules + no tuning — noisy in real clusters; tune per workload
- Tetragon enforcement without observation — block legitimate workloads, downtime
- Missing sidekick / output config — events generated but no one sees them
- Skipping eBPF kernel requirements check — older kernels lose features (LSM hooks need 5.7+)
- Privileged DaemonSet not scoped — Falco/Tetragon need host access; restrict via PSA exception only for them
- Cilium replaces kube-proxy mid-prod — major networking change; pilot on non-prod first
Performance considerations
- Falco modern eBPF: 1-3% CPU
- Tetragon eBPF: similar
- Cilium full eBPF datapath: lower than iptables-based kube-proxy at scale
- Combined: 3-5% CPU overhead; node memory: ~200MB
Acceptable for the visibility / enforcement gain on most workloads.
OPSEC for blue team
- Audit TracingPolicy / Falco rule changes — equivalent to detection bypass
- Alert on Falco DaemonSet pods being deleted / restarted unexpectedly
- Hunt for kernel-version-specific feature gaps (Tetragon LSM enforcement needs 5.7+)
- Validate enforcement actually fires via Atomic Red Team tests
- For SOC: high-volume Falco alerts get deduped + correlated upstream; don’t page on every event
eBPF security considerations
eBPF itself is a Linux security surface:
- CVE-2022-23222 (eBPF verifier bug → LPE)
- CVE-2021-3490 (eBPF JIT → arbitrary R/W)
- Kernel updates matter; eBPF tools need current kernels
eBPF tooling adds value but also widens kernel attack surface. Patch promptly.
References
- Falco docs
- Falcosidekick outputs
- Tetragon docs
- Cilium docs
- eBPF.io — broader eBPF ecosystem
- Isovalent — eBPF security blog series
- Sysdig — runtime detection patterns
See also: ebpf-offensive-and-defensive, policy-as-code-opa-kyverno-defender, k8s-admission-controllers, k8s-rbac-abuse, k8s-privileged-pod, k8s-host-mount-escape, container-runtime-escapes-modern, sigma-rules-detection-as-code, velociraptor-threat-hunting, service-mesh-security-deep, edr-hooks-and-unhooking