Linux kernel architecture
TL;DR: The kernel is one monolithic address space entered through syscalls, ioctls, netlink, and eBPF; bugs in any of those edges yield ring-0 primitives that exploit dev must understand structurally before fuzzing.
What it is
A short orientation map for kernel researchers: how user→kernel transitions work on x86_64, what subsystems present the largest attack surface, and which mitigations gate exploitation. This is scaffolding for the kernel-bug notes — not a replacement for source-tree reading.
Preconditions / where it applies
- Modern x86_64 Linux (5.x/6.x); ARM64 has analogous structure with a different syscall ABI.
- Researcher access: source tree (
kernel.org), ftrace, kprobes, KASAN/KFENCE/KMSAN sanitisers, syzkaller for coverage feedback.
Technique
Entry surfaces worth mapping first:
- syscalls —
arch/x86/entry/syscalls/syscall_64.tblenumerates them; handlers live acrossfs/,net/,kernel/,mm/. - ioctl — character/block device drivers;
unlocked_ioctlhandler is the routing function; lookup withgit grep "\.unlocked_ioctl". - netlink —
AF_NETLINKsockets;genetlinkfamilies parse attributes per policy table — historically rich in bugs. - eBPF — verifier in
kernel/bpf/verifier.cplus JIT; many CVEs (e.g. CVE-2021-3490, CVE-2022-23222) abuse verifier confusion. - filesystems —
io_uring,fuse,overlayfs,ntfs3are recent high-yield surfaces. - drm/gpu, USB, Bluetooth (HCI), WiFi (mac80211) — peripheral parsers with attacker-controlled buffers.
Memory model:
- Per-CPU stacks (~16 KiB), separate IRQ stack; kernel heap is SLAB/SLUB with per-CPU freelists; allocations bucketed by size into
kmalloc-Ncaches. vmallocfor large non-contiguous;module_alloclands within ±2 GiB of kernel text for short RIP-relative calls.
Mitigations gating exploitation:
- KASLR — text + module base randomised; defeat with leak primitives.
- SMEP/SMAP — see smep-smap-overview for the Windows mirror; Linux uses CR4 bits identically.
- KPTI — page-table isolation against Meltdown; affects syscall entry path.
- CFI —
CONFIG_CFI_CLANGenforces type-tagged indirect calls (kernel CFI similar in spirit to xfg). - STACKLEAK, KFENCE, RANDSTRUCT, slab freelist hardening — kill classic spray techniques and uninit reads.
Privilege escalation pattern: corrupt cred struct, overwrite modprobe_path, hijack core_pattern, or call commit_creds(prepare_kernel_cred(NULL)) from a controlled gadget.
Detection and defence
- Production hardening: lockdown, ftrace restrictions, unprivileged BPF off, KASLR + KPTI, module signing, SELinux/AppArmor profiles.
- Detection:
auditdoninit_module,bpf(), suspicioussetresuid(0,0,0)from non-root parents; kernel oops with KASAN messages indicate found bugs. - Vendor kernels add Yama ptrace scoping, restricted dmesg, kptr_restrict.
References
- Linux Kernel Defence Map — visual mitigation matrix
- syzkaller internals — coverage-guided syscall fuzzer
- LWN Kernel Index — running glossary of subsystem write-ups
See also: ret2csu, ret2libc, heap-exploitation-linux.