Whistler

Whistler is a Common Lisp compiler that produces eBPF bytecode. You write s-expressions, the compiler emits valid ELF object files, and your kernel loads them directly. There is no LLVM, no kernel headers, no CFFI. The entire compiler is self-contained SBCL code.

(defmap pkt-count :type :array
  :key-size 4 :value-size 8 :max-entries 1)

(defprog count-packets (:type :xdp :license "GPL")
  (incf (getmap pkt-count 0))
  XDP_PASS)

This compiles to 11 eBPF instructions and a valid BPF ELF object file.

Why Whistler

eBPF programs are small, verifier-constrained, and pattern-driven. They parse headers, validate bounds, look up state in maps, and emit events. These recurring patterns make eBPF a natural fit for a high-level language with real metaprogramming -- not string concatenation or preprocessor macros, but a compiler that understands the domain.

Real metaprogramming

Whistler programs are Common Lisp. CL macros are hygienic, composable, and operate on the AST at compile time. A with-tcp macro that parses Ethernet, IP, and TCP headers with bounds checks is not a preprocessor trick -- it is a function that generates verified code and participates in the compiler pipeline.

No C toolchain

The compiler is approximately 7,000 lines of SBCL. It includes its own ELF writer, BTF encoder, and peephole optimizer. You do not need clang, LLVM, libelf, or kernel headers installed.

Compiler-aware abstractions

Struct accessors, protocol helpers, and map operations are part of the language. The compiler optimizes them intentionally rather than recovering patterns after C lowering. (incf (getmap m k)) compiles to an atomic add on the map value -- the compiler knows the idiom.

Automatic CO-RE

Struct identity is preserved through the compilation pipeline. When you use import-kernel-struct to access kernel data structures, the compiler emits CO-RE relocations automatically. No manual BPF_CORE_READ calls.

Interactive REPL development

Compile, load, attach, inspect maps, iterate -- all from one Lisp image. Change a BPF program, recompile, and reload without leaving your REPL session.

Comparison

Whistler matches clang -O2 instruction counts on real programs. On the Cilium nodeport-lb4 load balancer (a complex production BPF program), Whistler produces 76 instructions to clang's 75.

The userspace loader

whistler/loader is a pure Common Lisp BPF loader. No libbpf, no CFFI. It parses .bpf.o files, creates maps via bpf() syscalls, loads programs into the kernel, attaches them (kprobe, uprobe, tracepoint, XDP, TC, cgroup), and consumes ring buffers -- all from SBCL.

You can also use inline BPF sessions that compile and load in the same form:

(whistler/loader:with-bpf-session ()
  ;; Kernel side -- compiled to eBPF at macroexpand time
  (bpf:map counter :type :hash
    :key-size 4 :value-size 8 :max-entries 1024)
  (bpf:prog trace (:type :kprobe
                    :section "kprobe/__x64_sys_execve"
                    :license "GPL")
    (incf (getmap counter 0))
    0)

  ;; Userspace side -- normal CL at runtime
  (bpf:attach trace "__x64_sys_execve")
  (loop (sleep 1)
        (format t "count: ~d~%" (bpf:map-ref counter 0))))

One file, one language, no intermediate artifacts.

Installation

Requirements

• SBCL (Steel Bank Common Lisp) 2.0 or later. Install from sbcl.org or your distribution's package manager (dnf install sbcl, apt install sbcl).
• Linux kernel 5.3+ for bounded loop support in the BPF verifier. Kernel 5.8+ is recommended for ring buffer maps and BTF support.
• FiveAM is required only for running the test suite.

Loading the compiler

Clone the repository and load the system with ASDF:

git clone https://github.com/atgreen/whistler.git
cd whistler

From an SBCL REPL:

(require :asdf)
(push #p"/path/to/whistler/" asdf:*central-registry*)
(asdf:load-system "whistler")
(in-package #:whistler-user)

Or use the Makefile to start a REPL with Whistler already loaded:

make repl

This drops you into the whistler-user package, ready to define maps and programs.

Loading the userspace loader

The loader is a separate ASDF system that depends on the compiler:

(asdf:load-system "whistler/loader")
(in-package #:whistler-loader-user)

This gives you with-bpf-session, with-bpf-object, ring buffer consumers, and map accessors -- all in pure Common Lisp.

Or use the Makefile:

make repl-loader

This is the most convenient day-to-day workflow for Lisp development: compiler, loader, and REPL in one image.

Building the CLI binary

Whistler can be built as a standalone command-line binary using ASDF's program-op:

make

This produces a whistler executable in the repository root. The Makefile runs:

sbcl --noinform --non-interactive \
  --eval '(require :asdf)' \
  --eval '(push #p"./" asdf:*central-registry*)' \
  --eval '(asdf:make "whistler")'

The whistler.asd system definition specifies :build-operation "program-op" and :entry-point "whistler:main", so asdf:make produces a self-contained binary.

Use it to compile BPF source files from the shell:

./whistler compile examples/count-xdp.lisp -o count.bpf.o

Check your local environment before trying to load programs:

./whistler doctor

This reports the local kernel version, tool availability, tracefs/BTF readability, and any obvious missing capabilities.

Running the test suite

Tests require FiveAM:

make test

Or from a REPL:

(asdf:test-system "whistler")

The test suite covers ALU operations, memory access, branching, control flow, protocol parsing, map operations, register allocation, and end-to-end compilation.

Hello eBPF

This chapter walks through the simplest useful Whistler program: an XDP packet counter.

The program

Create a file count-xdp.lisp:

(in-package #:whistler)

(defmap pkt-count :type :array
  :key-size 4 :value-size 8 :max-entries 1)

(defprog count-packets (:type :xdp :section "xdp" :license "GPL")
  (incf (getmap pkt-count 0))
  XDP_PASS)

This defines:

• A BPF map called pkt-count. It is an array with a single 64-bit entry, keyed by a 32-bit index.
• A BPF program called count-packets. It is an XDP program (attached to a network interface), placed in the ELF section "xdp", and licensed as GPL (required for BPF programs that call GPL-only kernel helpers).

The program body does two things:

1. (incf (getmap pkt-count 0)) -- look up key 0 in the map and atomically increment the value.
2. XDP_PASS -- return the XDP verdict that passes the packet through unchanged.

Compiling

From the REPL

(require :asdf)
(push #p"/path/to/whistler/" asdf:*central-registry*)
(asdf:load-system "whistler")
(in-package #:whistler-user)

(load "count-xdp.lisp")
(compile-to-elf "count.bpf.o")
;; => Compiled 1 program (11 instructions total), 1 map -> count.bpf.o

From the command line

./whistler compile count-xdp.lisp -o count.bpf.o

Either way, you get a standard BPF ELF object file that any BPF loader can process.

Loading and attaching

With bpftool / ip

Attach the XDP program to a network interface:

sudo ip link set dev eth0 xdp obj count.bpf.o sec xdp

Read the counter:

sudo bpftool map dump name pkt_count

Detach when done:

sudo ip link set dev eth0 xdp off

With the Whistler loader

You can skip the external tools entirely and do everything from the REPL using whistler/loader:

(asdf:load-system "whistler/loader")
(in-package #:whistler-loader-user)

(with-bpf-object (obj "count.bpf.o")
  (attach-obj-xdp obj "count_packets" "eth0")
  (let ((counter (bpf-object-map obj "pkt_count")))
  (loop repeat 5
        do (sleep 1)
           (format t "packets: ~d~%"
                   (or (map-lookup-int counter 0) 0)))))

Inline session (no intermediate file)

The most Lisp-native approach compiles and loads in one form:

(in-package #:whistler-loader-user)

(with-bpf-session ()
  (bpf:map pkt-count :type :array
    :key-size 4 :value-size 8 :max-entries 1)
  (bpf:prog count-packets (:type :xdp :section "xdp" :license "GPL")
    (incf (getmap pkt-count 0))
    XDP_PASS)

  (bpf:attach count-packets "eth0")
  (loop repeat 5
        do (sleep 1)
           (format t "packets: ~d~%" (bpf:map-ref pkt-count 0))))

The bpf: forms compile to eBPF at macroexpand time. The rest is ordinary Common Lisp that runs at load time.

What the compiler produces

The 11-instruction output for this program:

1. Store key 0 to the stack.
2. Load the map file descriptor.
3. Call bpf_map_lookup_elem.
4. Check for null (verifier requires this).
5. Load the current value.
6. Add 1.
7. Store the new value (atomic).
8. Set return value to XDP_PASS (2).
9. Exit.

This matches the instruction count of the equivalent C program compiled with clang -O2.

Permissions

Loading and attaching BPF programs requires elevated privileges. You do not need full root access -- Linux capabilities are sufficient.

Required capabilities

Setting capabilities on SBCL

Grant the capabilities to the SBCL binary:

sudo setcap cap_bpf,cap_perfmon+ep /usr/bin/sbcl

Verify:

getcap /usr/bin/sbcl
# /usr/bin/sbcl cap_bpf,cap_perfmon=ep

After this, any SBCL process can load BPF programs and attach probes.

If you built a standalone whistler binary, set capabilities on that instead:

sudo setcap cap_bpf,cap_perfmon+ep ./whistler

Tracepoint format files

deftracepoint reads tracepoint format definitions from tracefs at macroexpand time. These files are often root-readable only by default:

ls -l /sys/kernel/tracing/events/sched/sched_switch/format
# -r--r----- 1 root root ...

Make them world-readable so the compiler can parse them without root:

# Single tracepoint
sudo chmod a+r /sys/kernel/tracing/events/sched/sched_switch/format

# All tracepoints in a category
sudo chmod -R a+r /sys/kernel/tracing/events/sched/

# All tracepoints
sudo find /sys/kernel/tracing/events -name format -exec chmod a+r {} +

This only affects the format metadata files. It does not grant access to the trace ring buffer or enable tracing.

vmlinux BTF

import-kernel-struct reads BTF type information from /sys/kernel/btf/vmlinux. On most distributions this file is already world-readable:

ls -l /sys/kernel/btf/vmlinux
# -r--r--r-- 1 root root ...

If it is not, make it readable:

sudo chmod a+r /sys/kernel/btf/vmlinux

XDP and TC attachment

Attaching XDP and TC programs to network interfaces requires CAP_NET_ADMIN in addition to CAP_BPF:

sudo setcap cap_bpf,cap_perfmon,cap_net_admin+ep /usr/bin/sbcl

Cgroup attachment

Attaching cgroup BPF programs (cgroup_skb, cgroup/sock, etc.) requires write access to the cgroup directory and CAP_BPF. Typically this means:

sudo setcap cap_bpf,cap_perfmon,cap_net_admin+ep /usr/bin/sbcl

Running as root

If capabilities are not an option, sudo works:

sudo sbcl --noinform \
  --eval '(require :asdf)' \
  --eval '(push #p"./" asdf:*central-registry*)' \
  --eval '(asdf:load-system "whistler/loader")' \
  --eval '(in-package #:whistler)'

This grants all capabilities but is less precise than setting individual caps.

Summary

Top-Level Declarations

A Whistler source file consists of top-level declarations: defmap, defprog, defstruct, and defunion. Every other construct lives inside the body of a defprog.

defmap

defmap declares a BPF map -- a kernel-side data structure shared between BPF programs and userspace. You choose a map type (hash table, array, ring buffer, etc.), specify key and value sizes, and set capacity. The compiler emits the corresponding ELF map section and generates accessor forms you can use inside defprog bodies.

defprog

defprog defines a BPF program. Each defprog produces one program section in the output ELF. You specify the program type (XDP, kprobe, tracepoint, and so on), write the body using Whistler forms, and the compiler handles register allocation, stack layout, and bytecode emission. Multiple defprog forms in the same source file compile into a single ELF object.

defstruct

defstruct declares a C-compatible struct layout used on both the BPF side and the Common Lisp userspace side. On the BPF side the compiler generates constructors, field accessors, and sizeof. On the CL side it generates a corresponding CL struct with encode and decode functions, so you can pack and unpack data exchanged through maps or ring buffers without manual byte wrangling.

defunion

defunion declares a union of existing struct types. It allocates the size of the largest member; the returned pointer can be used with any member's field accessors (all members share offset 0). This is useful for packet parsing where the same stack buffer is reused for different header types:

(defstruct ip-hdr  ...)
(defstruct udp-hdr ...)
(defunion packet-buf ip-hdr udp-hdr)

(let ((pkt (make-packet-buf)))
  (skb-load-bytes (ctx-ptr) 0 pkt (sizeof ip-hdr))
  (ip-hdr-protocol pkt))    ; access through any member's accessors

defmap

defmap declares a BPF map that can be shared between BPF programs and userspace.

Syntax

(defmap name :type TYPE
  [:key-size N] [:value-size N]
  :max-entries N
  [:map-flags FLAGS])

Required arguments are :type and :max-entries. The :key-size and :value-size arguments are required for most map types but omitted for ring buffers. :map-flags is optional and defaults to 0.

Map Types

Whistler supports the following map types:

Examples

Hash map

(defmap connection-table :type :hash
  :key-size 16
  :value-size 8
  :max-entries 1024)

Array map

(defmap counters :type :array
  :key-size 4
  :value-size 8
  :max-entries 256)

Per-CPU hash

(defmap per-cpu-cache :type :percpu-hash
  :key-size 4
  :value-size 64
  :max-entries 512)

Per-CPU array

(defmap per-cpu-stats :type :percpu-array
  :key-size 4
  :value-size 32
  :max-entries 16)

Ring buffer

Ring buffer maps only require :type and :max-entries. The :max-entries value must be a power of two and specifies the buffer size in bytes.

(defmap events :type :ringbuf
  :max-entries (* 256 1024))

Program array (for tail calls)

(defmap dispatch :type :prog-array
  :key-size 4
  :value-size 4
  :max-entries 8)

LPM trie

LPM trie maps require the BPF_F_NO_PREALLOC flag (value 1).

(defmap routes :type :lpm-trie
  :key-size 8
  :value-size 4
  :max-entries 1024
  :map-flags 1)

LRU hash

(defmap recent-flows :type :lru-hash
  :key-size 16
  :value-size 8
  :max-entries 4096)

defprog

defprog defines a BPF program. Each defprog form compiles into one program section in the output ELF object.

Syntax

(defprog name (:type :xdp :section nil :license "GPL")
  body...)

Program Types

Return Value

The last expression in the body is implicitly returned as the program's return code. There is no explicit return form. For XDP programs, this is typically an XDP action constant:

(defprog drop-all (:type :xdp)
  XDP_DROP)

Section Names

The ELF section name defaults to the lowercase string form of the program type. For example, a :kprobe program gets the section name "kprobe". To override this -- for instance, to attach to a specific kernel function -- pass :section explicitly:

(defprog trace-exec (:type :kprobe :section "kprobe/sys_execve")
  0)

License

The :license parameter controls the license string embedded in the ELF. It defaults to "GPL". Some BPF helper functions are restricted to GPL-licensed programs. If you set a non-GPL license, calls to GPL-only helpers will be rejected by the kernel verifier at load time.

Multiple Programs

Multiple defprog forms in the same source file compile into a single ELF object, each in its own section. This is useful for tail call dispatch or for bundling related programs:

(defmap dispatch :type :prog-array
  :key-size 4
  :value-size 4
  :max-entries 4)

(defprog handler-a (:type :xdp :section "xdp/handler_a")
  ;; Handle protocol A
  XDP_PASS)

(defprog handler-b (:type :xdp :section "xdp/handler_b")
  ;; Handle protocol B
  XDP_DROP)

(defprog main (:type :xdp)
  ;; Dispatch to sub-programs via tail call
  (tail-call dispatch 0)
  XDP_PASS)

Full Example

A minimal tracepoint program that records events to a ring buffer:

(defstruct event
  (pid u32)
  (ts u64))

(defmap events :type :ringbuf
  :max-entries (* 256 1024))

(defprog trace-sched (:type :tracepoint
                      :section "tracepoint/sched/sched_process_exec")
  (with-ringbuf (e events (sizeof event))
    (setf (event-pid e) (get-current-pid-tgid)
          (event-ts e) (ktime-get-ns)))
  0)

defstruct

defstruct declares a C-compatible struct layout. The compiler generates accessors for both the BPF side (inside defprog) and the Common Lisp userspace side (for packing and unpacking data exchanged through maps and ring buffers).

Syntax

(defstruct name
  (field-name type)
  ...)

Field Types

Layout and Alignment

Fields are laid out with C-compatible natural alignment. Each field is aligned to its own size: u16 fields align to 2 bytes, u32 to 4 bytes, u64 to 8 bytes. The compiler inserts padding bytes between fields as needed. The overall struct size is padded to a multiple of the largest field alignment.

For example:

(defstruct sample
  (flags u8)
  (id u32)
  (value u64))

This produces a 16-byte struct: 1 byte for flags, 3 bytes of padding, 4 bytes for id, and 8 bytes for value.

BPF-Side API

Inside a defprog body, defstruct generates the following forms:

Constructor

(make-NAME)

Allocates the struct on the BPF stack with all fields zeroed.

(let ((e (make-sample)))
  (setf (sample-id e) 42)
  ...)

Field Accessor

(NAME-FIELD ptr)

Reads a field value from a struct pointer.

(let ((pid (sample-id e)))
  ...)

Field Setter

(setf (NAME-FIELD ptr) value)

Writes a value to a field.

(setf (sample-value e) 100)

Field Pointer

(NAME-FIELD-PTR ptr)

Returns a pointer to the field within the struct. This is useful for passing field addresses to BPF helpers like bpf_probe_read or bpf_get_current_comm:

(get-current-comm (my-event-comm-ptr e) 16)

Sizeof

(sizeof NAME)

Returns the total size of the struct in bytes, including padding. This is typically used with ringbuf-reserve:

(ringbuf-reserve events (sizeof my-event))

CL-Side API

On the Common Lisp userspace side, defstruct generates:

Record Struct

NAME-RECORD

A standard CL record type with a slot for each field. Array fields become CL vectors.

Decoder

(decode-NAME byte-vector &optional offset)

Parses a byte vector (or a subrange starting at offset) into a NAME-RECORD instance. This is used when reading data from maps or ring buffers.

Encoder

(encode-NAME record)

Serializes a NAME instance into a byte vector suitable for writing to a map.

Full Example

Consider a struct for reporting process events:

(defstruct my-event
  (pid u32)
  (comm (array u8 16))
  (data (array u8 64)))

This defines a struct with a 32-bit PID, a 16-byte command name buffer, and a 64-byte data buffer. The total size is 84 bytes (4 + 16 + 64, no padding needed since the largest field alignment is 4).

BPF side

(defmap events :type :ringbuf
  :max-entries (* 256 1024))

(defprog trace-exec (:type :tracepoint
                     :section "tracepoint/sched/sched_process_exec")
  (with-ringbuf (e events (sizeof my-event))
    (setf (my-event-pid e) (get-current-pid-tgid))
    (get-current-comm (my-event-comm-ptr e) 16)
    (probe-read (my-event-data-ptr e) 64 some-source))
  0)

CL side

;; Reading events from the ring buffer
(with-ringbuf-consumer (buf events)
  (lambda (data size)
    (let ((evt (decode-my-event data)))
      (format t "pid=~A comm=~A~%"
              (my-event-record-pid evt)
              (map 'string #'code-char
                   (my-event-record-comm evt))))))

;; Creating and encoding a record manually
(let ((rec (make-my-event-record :pid 1234
                                 :comm (make-array 16 :element-type '(unsigned-byte 8))
                                 :data (make-array 64 :element-type '(unsigned-byte 8)))))
  (encode-my-event rec))

Forms

The body of a defprog consists of Whistler forms that the compiler translates to eBPF bytecode. These forms look like Common Lisp but target a restricted execution model: 64-bit registers, no heap, no closures, no general recursion.

Standard CL is fully available at compile time. You can write defmacro, defconstant, helper functions, and arbitrary Lisp code that runs during compilation. The compiler expands all macros before lowering to eBPF. Only the primitive Whistler forms survive into the final bytecode.

;; CL macro -- runs at compile time, gone before bytecode emission
(defmacro drop-if (test)
  `(when ,test (return XDP_DROP)))

(defprog my-filter (:type :xdp :section "xdp" :license "GPL")
  ;; These are Whistler forms compiled to eBPF:
  (let ((data (xdp-data))
        (data-end (xdp-data-end)))
    (when (> (+ data 34) data-end)
      (return XDP_PASS))
    (drop-if (= (eth-type data) +ethertype-ipv4+)))
  XDP_PASS)

The following chapters document each category of form:

• Variables and Types -- let, let*, setf, type inference
• Control Flow -- if, when, unless, cond, case, and, or
• Arithmetic and Bitwise -- math, shifts, comparisons, casts, byte order
• Memory Access -- load, store, ctx, stack-addr, atomic-add, memset, memcpy
• Map Operations -- low-level and high-level map access
• Ring Buffers -- ringbuf-reserve, with-ringbuf
• BPF Helpers -- kernel helper function calls
• Loops -- dotimes, do-user-ptrs, do-user-array
• Tail Calls -- program chaining via tail-call
• Inline Assembly -- raw BPF instruction emission

Variables and Types

let and let*

let binds variables with parallel semantics -- all initializers are evaluated before any variable becomes visible. let* binds sequentially, so each initializer can reference variables bound earlier in the same form.

;; Parallel: b does not see the new a
(let ((a 10)
      (b (+ a 1)))   ; a here refers to an outer binding
  ...)

;; Sequential: b sees the new a
(let* ((a 10)
       (b (+ a 1)))  ; a is 10
  ...)

Each binding has one of these shapes:

(var init)            ; type inferred from init
(var type init)       ; explicit type
(var type)            ; explicit type, zero-initialized

Type inference

Variables default to u64. The compiler infers narrower types from certain initializer forms:

Type declarations

Use declare after the binding list to explicitly narrow variables. This follows the CL convention:

(let ((proto (ipv4-protocol ip)))
  (declare (type u32 proto))
  (tail-call jt proto))

Multiple variables can share a declaration:

(let ((a (load u32 ptr 0))
      (b (load u32 ptr 4)))
  (declare (type u32 a b))
  (+ a b))

setf

setf assigns to a variable or a struct field accessor:

(setf count (+ count 1))

Multi-pair setf assigns several places in sequence:

(setf (conn-event-src-addr event) (ipv4-src-addr ip)
      (conn-event-dst-addr event) (ipv4-dst-addr ip)
      (conn-event-dst-port event) (tcp-dst-port tcp))

Struct field setf works because defstruct generates defsetf expanders. The compiler expands multi-pair setf into a progn of single assignments.

Control Flow

if

Two-way conditional. Returns the value of the taken branch.

(if (> data-end (+ data 34))
    (process-packet data)
    XDP_DROP)

When the test is a comparison form (=, >, <, etc.), the compiler emits a direct conditional jump instead of materializing a 0/1 value.

when, unless

One-armed conditionals with implicit progn body. Return 0 when the body is skipped.

(when (= proto +ip-proto-tcp+)
  (incf (getmap tcp-count 0))
  (process-tcp data))

(unless (> (+ data 34) data-end)
  (return XDP_PASS))

when-let

Bind variables and execute the body only if all bound values are non-zero. If any initializer evaluates to 0, the remaining bindings and the body are skipped. Returns 0 when skipped.

;; Null-check a map lookup and use the pointer
(when-let ((p (map-lookup my-map key)))
  (atomic-add p 0 1))

;; Typed binding
(when-let ((p u64 (map-lookup my-map key)))
  (load u32 p 0))

;; Multiple bindings -- all must be non-zero
(when-let ((a (map-lookup m1 k1))
           (b (map-lookup m2 k2)))
  (+ (load u64 a 0) (load u64 b 0)))

if-let

Bind a variable and branch on its value. If the initializer is non-zero, execute the then branch with the variable in scope. Otherwise execute the else branch.

(if-let (p (map-lookup my-map key))
  (load u64 p 0)   ; then: p is bound and non-zero
  0)                ; else: key not found

;; Typed binding
(if-let (p u64 (map-lookup my-map key))
  (load u32 p 0)
  0)

cond

Multi-way conditional. Clauses are tested in order; the body of the first matching clause executes. A final t clause is the default.

(cond
  ((= proto +ip-proto-tcp+)  (handle-tcp data))
  ((= proto +ip-proto-udp+)  (handle-udp data))
  (t                          XDP_PASS))

case

Multi-way dispatch on a value. Shadows CL's case. Each clause is (value body...) or ((v1 v2 ...) body...). The final clause may use t or otherwise as a catch-all. Compiles to a cond chain.

(case (ipv4-protocol ip)
  (+ip-proto-tcp+  (handle-tcp))
  (+ip-proto-udp+  (handle-udp))
  ((41 47)         (handle-tunnel))   ; match multiple values
  (t               XDP_PASS))

and, or

Short-circuit logical operators. and returns 0 as soon as any operand is zero; or returns the first non-zero operand. These are compiler primitives, not CL macros.

(when (and (= proto +ip-proto-tcp+)
           (logand flags +tcp-syn+)
           (not (logand flags +tcp-ack+)))
  (log-syn-packet data))

(let ((port (or (tcp-dst-port tcp) (tcp-src-port tcp))))
  ...)

progn

Evaluate forms in sequence, return the value of the last one.

(progn
  (incf (getmap pkt-count 0))
  XDP_PASS)

return

Exit the BPF program early with a return value. If no value is given, returns 0.

(when (> (+ data 34) data-end)
  (return XDP_PASS))

Arithmetic and Bitwise

All arithmetic operates on 64-bit registers. The optimizer narrows to 32-bit instructions when it can prove the operands fit.

Arithmetic operators

Division or modulo by a compile-time zero is a compile error. incf and decf are macros that expand to setf for variables and to atomic-increment for map places (see Map Operations).

(let ((total (+ a b c)))       ; n-ary addition
  (setf total (* total 2))
  (incf total)                 ; total = total + 1
  total)

Bitwise operators

Shift amounts must be 0--63; shifting by 64 or more is a compile error.

;; Extract PID from tgid (upper 32 bits)
(let ((pid (cast u32 (>> (get-current-pid-tgid) 32))))
  ...)

;; Check TCP SYN flag
(when (logand flags +tcp-syn+)
  ...)

Comparison operators

Comparisons return 1 (true) or 0 (false). When used directly as the test of if, when, or unless, the compiler emits a conditional jump without materializing the value.

Unsigned (default)

Signed

(when (s< offset 0)
  (return XDP_DROP))

Logic

(not expr)   ; returns 1 if expr is 0, else 0

Type cast

Truncate or zero-extend a value to a specific width:

(cast u8 val)    ; keep low 8 bits
(cast u16 val)   ; keep low 16 bits
(cast u32 val)   ; keep low 32 bits
(cast u64 val)   ; no-op (identity)

Byte-order conversion

Network-to-host and host-to-network byte swaps:

The return type reflects the width: ntohs returns u16, ntohl returns u32, ntohll returns u64.

When a byte-swapped value is compared against a compile-time constant, the compiler folds the swap into the constant rather than emitting a runtime swap instruction.

;; Compiler folds: instead of swapping at runtime, it compares
;; against the byte-swapped constant directly
(when (= (ntohs (load u16 data 12)) +ethertype-ipv4+)
  ...)

Memory Access

load

Read a value of a given type from a pointer at a byte offset:

(load type ptr offset)
(load type ptr)          ; offset defaults to 0

Types: u8, u16, u32, u64. The result type matches the load type.

(let ((ethertype (load u16 data 12))
      (protocol  (load u8  ip   9)))
  ...)

store

Write a value to memory:

(store type ptr offset value)

(store u32 event 0 src-addr)
(store u8  event 10 proto)

ctx

Read from or write to the BPF program context (the ctx pointer passed by the kernel). The context structure varies by program type -- for XDP it contains data, data_end, and data_meta; for cgroup-sock-addr it contains user_ip4, user_port, etc.

ctx is a setf-able place. The preferred form uses field names, which the compiler resolves from the program type's context struct:

;; Read a context field by name
(ctx field-name)

;; Write a context field
(setf (ctx field-name) value)

;; Array fields require an index
(ctx user-ip6 0)                  ; first u32 of IPv6 address
(setf (ctx user-ip6 2) value)     ; write third element

;; XDP: read packet bounds
(defprog my-xdp (:type :xdp ...)
  (let ((data     (ctx data))
        (data-end (ctx data-end)))
    ...))

;; cgroup/connect4: redirect destination
(defprog connect4 (:type :cgroup-sock-addr ...)
  (let ((ip (ctx user-ip4)))
    (setf (ctx user-ip4) +localhost-nbo+)
    (setf (ctx user-port) (htons 8080))))

The compiler knows which context struct each program type uses (xdp_md, __sk_buff, bpf_sock_addr, bpf_sock_ops) and resolves field names to types and offsets at compile time.

Legacy syntax: (ctx type offset) with explicit type and numeric offset still works for backward compatibility:

(ctx u32 4)              ; equivalent to (ctx user-ip4) in cgroup-sock-addr
(setf (ctx u32 4) val)   ; equivalent to (setf (ctx user-ip4) val)

Note: ctx-load and ctx-store are deprecated. Use (ctx ...) and (setf (ctx ...) ...) instead.

stack-addr

Take the address of a stack-allocated variable (analogous to &var in C). Useful for passing pointers to BPF helpers that expect pointer arguments.

(let ((key u32 0))
  (map-lookup my-map (stack-addr key)))

atomic-add

Atomically add a value to memory at a given offset:

(atomic-add ptr offset value)
(atomic-add ptr offset value type)   ; type defaults to u64

(when-let ((p (map-lookup counters key)))
  (atomic-add p 0 1))

The offset must be aligned to the type width.

memset

Fill a region of memory with a byte value. Both offset and nbytes must be compile-time constants. When the value is a compile-time constant, the compiler emits widened stores (u64/u32/u16) for efficiency.

(memset ptr offset value nbytes)

;; Zero 32 bytes starting at offset 0
(memset buf 0 0 32)

;; Fill 16 bytes with 0xFF
(memset key 16 #xFF 16)

memcpy

Copy bytes between memory regions. All offsets and nbytes must be compile-time constants. The compiler uses the widest possible loads/stores (u64, then u32, u16, u8) for efficiency.

(memcpy dst dst-offset src src-offset nbytes)

(memcpy event 0 data 14 20)   ; copy 20 bytes of IP header

pt_regs access

For kprobe and uprobe programs, function arguments are available through pt_regs accessors. These are compile-time macros that expand to ctx-load with architecture-specific offsets (x86-64 and aarch64 supported).

(defprog trace-open (:type :kprobe :section "kprobe/do_sys_open" :license "GPL")
  (let ((filename-ptr (pt-regs-parm2)))
    (probe-read-user-str buf 256 filename-ptr)
    ...))

Map Operations

Whistler provides both low-level primitives and high-level macros for BPF map access. The high-level macros mirror CL's gethash pattern.

Low-level primitives

These compile directly to BPF helper calls with the map file descriptor in R1 and a pointer to the key on the stack in R2.

map-lookup

(map-lookup map-name key)

Returns a pointer to the value, or 0 (null) if the key is not found. You must null-check this pointer before dereferencing it -- the BPF verifier enforces this.

(when-let ((p (map-lookup counters key)))
  (load u64 p 0))

map-update

(map-update map-name key value flags)

Flags: BPF_ANY (0), BPF_NOEXIST (1), BPF_EXIST (2).

map-delete

(map-delete map-name key)

Pointer-key variants

When a map has a struct key (key-size > 8 bytes), use the -ptr variants. These take pointers to data on the stack instead of scalar values:

(map-lookup-ptr map-name key-ptr)
(map-update-ptr map-name key-ptr val-ptr flags)
(map-delete-ptr map-name key-ptr)

The high-level macros auto-select the -ptr variants when the map's key-size exceeds 8 bytes.

High-level macros

getmap

Look up a map value. For scalar values (value-size <= 8), dereferences the pointer and returns the value directly. For struct values, returns the pointer. Returns 0 if the key is not found.

(getmap map-name key)

(let ((count (getmap pkt-count 0)))
  (when (> count 1000)
    (return XDP_DROP)))

(setf (getmap ...))

Update a map entry:

(setf (getmap my-map key) new-value)

remmap

Delete a map entry (mirrors CL's remhash):

(remmap my-map key)

(incf (getmap ...))

Atomically increment a map value. For array maps, this is a lookup followed by atomic-add. For hash maps, it initializes the entry to the delta if the key does not exist.

(incf (getmap pkt-count 0))       ; increment by 1
(incf (getmap pkt-count 0) 5)     ; increment by 5

Struct keys

When a map is declared with a key-size greater than 8 bytes, the high-level macros (getmap, setf, remmap, incf) automatically switch to the -ptr variants, passing a pointer to the struct key on the stack. No user code changes needed.

(defstruct stats-key
  (comm (array u8 16))
  (nargs u16) (rtype u8) (abi u8) (pad u32))

(defmap stats :type :hash :key-size 40 :value-size 8 :max-entries 10240)

;; incf auto-uses map-lookup-ptr / map-update-ptr for the 40-byte key
(incf (getmap stats key))

Ring Buffers

Ring buffers send variable-sized records from BPF programs to userspace without per-event syscalls. Whistler provides both raw primitives and a convenience macro.

Primitives

ringbuf-reserve

Reserve space in a ring buffer map. Returns a pointer to the reserved region, or 0 if the buffer is full.

(ringbuf-reserve map-name size flags)

size should be a compile-time constant or (sizeof struct-name). flags is typically 0.

ringbuf-submit

Submit a previously reserved record to the ring buffer, making it visible to the userspace consumer.

(ringbuf-submit ptr flags)

ringbuf-discard

Discard a reserved record without submitting it.

(ringbuf-discard ptr flags)

ringbuf-output

Copy a stack-allocated struct directly into the ring buffer. This is a single helper call -- more compact than the reserve/submit pattern when you build the entire record before sending.

(ringbuf-output map-name data-ptr size flags)

data-ptr must point to a stack-allocated struct (e.g., from make-event). The BPF verifier requires the pointer to be fp-derived.

(let ((evt (make-conn-event)))
  (setf (conn-event-src-addr evt) src
        (conn-event-dst-addr evt) dst
        (conn-event-dst-port evt) port
        (conn-event-proto evt)    proto)
  (ringbuf-output events evt (sizeof conn-event) 0))

Use ringbuf-output when filling all fields before sending. Use with-ringbuf (below) when you need conditional field logic or want to avoid the stack copy.

with-ringbuf

The with-ringbuf macro handles reserve, body execution, and submit in one form. If ringbuf-reserve returns null, the body is skipped entirely.

(with-ringbuf (var map-name size [:flags 0])
  body...)

The variable is bound to the reserved pointer inside the body. On normal exit, ringbuf-submit is called automatically. The buffer is not zeroed -- set fields explicitly or use memset.

(defstruct conn-event
  (src-addr u32)
  (dst-addr u32)
  (dst-port u16)
  (proto    u8)
  (pad      u8))

(defmap events :type :ringbuf :max-entries 4096)

(with-ringbuf (event events (sizeof conn-event))
  (setf (conn-event-src-addr event) (ipv4-src-addr ip)
        (conn-event-dst-addr event) (ipv4-dst-addr ip)
        (conn-event-dst-port event) (tcp-dst-port tcp)
        (conn-event-proto event)    proto))

If the body executes (return ...), the reservation is not auto-submitted. Call (ringbuf-discard var 0) before returning if needed.

fill-process-info

A convenience macro for filling common process metadata fields in a struct. Each keyword names a struct field setter:

(fill-process-info event
  :pid-field       my-event-pid
  :uid-field       my-event-uid
  :timestamp-field my-event-timestamp
  :comm-field      my-event-comm-ptr
  :comm-size       16)

This expands to calls to get-current-pid-tgid, get-current-uid-gid, ktime-get-ns, and get-current-comm with the appropriate field setters.

BPF Helpers

BPF helper functions are kernel-provided routines callable from BPF programs. In Whistler, call them by name in function position. Arguments are passed in registers R1--R5 and the return value is in R0.

(get-current-pid-tgid)                      ; 0 args
(probe-read-user dst size src)              ; 3 args
(get-current-comm buf-ptr buf-size)         ; 2 args
(ringbuf-reserve map-name size flags)       ; 3 args (special: map arg)

The compiler validates argument counts at compile time.

Available helpers

Map helpers (1--3) and tail-call (12) are called through dedicated Whistler forms rather than by name. The table lists them for completeness.

Example

(defprog trace-fork (:type :tracepoint
                     :section "tracepoint/sched/sched_process_fork"
                     :license "GPL")
  (let ((tgid (get-current-pid-tgid))
        (pid  (cast u32 (>> tgid 32)))
        (ts   (ktime-get-ns)))
    (setf (getmap fork-times pid) ts))
  0)

Loops

dotimes

Iterate a fixed number of times. The bound must be a compile-time constant (an integer literal or a defconstant symbol).

(dotimes (var count)
  body...)

The loop variable is bound as a u32 starting at 0.

(defconstant +max-headers+ 8)

(dotimes (i +max-headers+)
  (when (= (load u8 ptr i) 0)
    (return i)))

do-user-ptrs

Iterate over a user-space array of pointers (e.g., ffi_type **). For each element within count (up to the compile-time constant max-count), reads the pointer via probe-read-user and binds it if non-null.

(do-user-ptrs (ptr-var base-ptr count max-count [:index idx])
  body...)

The :index keyword optionally names the loop index variable. If omitted, a gensym is used.

(defconstant +max-args+ 16)

(do-user-ptrs (atype-ptr (ffi-cif-arg-types cif)
                          (ffi-cif-nargs cif)
                          +max-args+
                          :index i)
  (probe-read-user ft (sizeof ffi-type) atype-ptr)
  (setf (stats-key-arg-types key i) (ffi-type-type-code ft)))

Expands to a dotimes with a bounds-guarded probe-read-user and a when-let null check on each pointer.

do-user-array

Iterate over a user-space array of typed elements (scalar or struct). Similar to do-user-ptrs but reads the elements themselves rather than pointers to them.

(do-user-array (var type base-ptr count max-count [:index idx])
  body...)

type is a scalar type (u8, u16, u32, u64) or a struct name. For scalars, var is bound to the loaded value. For structs, var is bound to a reusable stack buffer pointer that is overwritten each iteration.

(do-user-array (val u32 array-ptr nelems 64 :index i)
  (incf (getmap histogram val)))

Tail Calls

BPF tail calls transfer execution from one BPF program to another using a program array map. The call is a zero-cost jump -- no stack frame is pushed, and the callee inherits the caller's context.

Syntax

(tail-call prog-array-map index)

prog-array-map must be a map declared with :type :prog-array. The index is evaluated at runtime and used to look up a program file descriptor in the map.

If no program is loaded at the given index, or the index is out of range, execution falls through to the next instruction. This is not an error -- it is the standard BPF tail call contract.

Example: protocol dispatch

;; Jump table: protocol number -> program FD
(defmap jt :type :prog-array
  :key-size 4 :value-size 4 :max-entries 256)

(defmap proto-stats :type :array
  :key-size 4 :value-size 8 :max-entries 3)

(defconstant +stat-dispatched+ 0)

(defprog xdp-dispatch (:type :xdp :section "xdp" :license "GPL")
  (let ((data     (xdp-data))
        (data-end (xdp-data-end)))
    (when (> (+ data 34) data-end)
      (return XDP_PASS))
    (when (/= (eth-type data) +ethertype-ipv4+)
      (return XDP_PASS))
    (let ((proto (ipv4-protocol (+ data +eth-hdr-len+))))
      (declare (type u32 proto))
      (incf (getmap proto-stats +stat-dispatched+))
      ;; Tail call into protocol-specific handler.
      ;; Falls through to XDP_PASS if no handler is loaded.
      (tail-call jt proto)))
  XDP_PASS)

At load time, populate the jump table with program file descriptors:

bpftool map update name jt key 6 0 0 0 value pinned /sys/fs/bpf/tcp_handler
bpftool map update name jt key 17 0 0 0 value pinned /sys/fs/bpf/udp_handler

Notes

• The map must be :type :prog-array. Using another map type is a compile error.
• The BPF verifier limits tail call depth (typically 33).
• Tail calls and normal calls share the same stack, so deeply nested tail calls may hit the 512-byte stack limit.

Inline Assembly

The asm form emits a single raw BPF instruction. It is an escape hatch for cases where Whistler does not yet have a dedicated form for a particular BPF operation.

Syntax

(asm opcode dst-reg src-reg offset immediate)

All arguments are integer constants corresponding to BPF instruction fields:

Example

;; Emit a raw BPF_CALL instruction for helper #5 (ktime_get_ns)
(asm #x85 0 0 0 5)

Use this sparingly. Prefer Whistler's typed forms (load, store, helper calls by name) whenever possible -- they provide type safety, register allocation, and verifier-friendly patterns that raw assembly does not.

Macros

Whistler uses standard Common Lisp defmacro for user-defined macros. Full CL is available at compile time -- the compiler expands all macros before lowering to eBPF bytecode.

How expansion works

The compiler walks each form before compilation. If the head of a form is a known Whistler builtin (let, if, when, unless, and, or, dotimes, etc.) or a BPF helper name, it is not macroexpanded. Everything else goes through macroexpand-1 and the result is walked recursively.

This means:

• Whistler builtins (when, unless, and, or, cond, etc.) are compiler primitives, not CL macros. They cannot be redefined with defmacro.
• User macros expand normally into primitive forms.
• Protocol macros (from defheader, defstruct, deftracepoint) expand normally -- they are regular CL macros.
• setf expanders work: (setf (my-struct-field ptr) val) expands via defsetf.

Example: convenience macro

(defmacro with-map-value ((var map key) &body body)
  "Look up a map value and execute body with VAR bound to it.
   Skips body if key is not found."
  `(when-let ((,var (map-lookup ,map ,key)))
     ,@body))

(defprog my-prog (:type :xdp :section "xdp" :license "GPL")
  (with-map-value (p counters 0)
    (atomic-add p 0 1))
  XDP_PASS)

Example: code-generating macro

(defmacro define-port-checker (name port action)
  "Generate an XDP program that acts on a specific TCP port."
  `(defprog ,name (:type :xdp :section "xdp" :license "GPL")
     (with-tcp (data data-end tcp)
       (when (= (tcp-dst-port tcp) ,port)
         (return ,action)))
     XDP_PASS))

(define-port-checker drop-9999  9999 XDP_DROP)
(define-port-checker drop-8080  8080 XDP_DROP)

Both define-port-checker invocations expand at compile time into full defprog forms. The generated programs are compiled independently.

Compile-time computation

Since macros run in full CL, you can do arbitrary computation:

(defconstant +blocked-ports+ '(80 443 8080 9999))

(defmacro blocked-port-p (port)
  `(or ,@(mapcar (lambda (p) `(= ,port ,p)) +blocked-ports+)))

(defprog filter (:type :xdp :section "xdp" :license "GPL")
  (with-tcp (data data-end tcp)
    (when (blocked-port-p (tcp-dst-port tcp))
      (return XDP_DROP)))
  XDP_PASS)

The blocked-port-p macro generates (or (= port 80) (= port 443) (= port 8080) (= port 9999)) at compile time. No list data structure exists at runtime.

XDP

XDP (eXpress Data Path) programs run at the earliest possible point in the network receive path, before the kernel allocates an sk_buff. This makes them the fastest option for packet processing.

Section Name

Use "xdp" for a single program or "xdp/name" to distinguish multiple XDP programs in the same ELF:

(defprog my-filter (:type :xdp :section "xdp/my_filter")
  XDP_PASS)

Return Codes

Packet Context

The program receives an xdp_md pointer implicitly. Use the zero-argument macros xdp-data and xdp-data-end to get the packet boundaries:

(defprog check-len (:type :xdp)
  (let ((data (xdp-data))
        (data-end (xdp-data-end)))
    (if (< (- data-end data) 14)
        XDP_DROP
        XDP_PASS)))

Packet Parsing Macros

Whistler provides with-packet, with-tcp, and with-udp to safely parse protocol headers with automatic bounds checking.

with-packet binds data and data-end from the XDP context and checks a minimum packet length:

(with-packet (data data-end :min-len 14)
  ;; headers are bounds-checked; body only runs if valid
  ...)

with-tcp and with-udp build on with-packet to also parse transport layer headers:

(with-tcp (data data-end tcp)
  ;; Ethernet, IP, and TCP headers are bounds-checked
  (tcp-dst-port tcp)
  ...)

Example: Drop TCP Port 9999

(defprog drop-9999 (:type :xdp :section "xdp/drop_9999")
  (with-tcp (data data-end tcp)
    (when (= (tcp-dst-port tcp) 9999)
      (return XDP_DROP)))
  XDP_PASS)

Attachment

From userspace, attach with attach-xdp:

(attach-xdp prog "eth0" :mode "xdpgeneric")

Kprobe / Uprobe

Kprobes attach to kernel function entry (or return) points. Uprobes do the same for userspace functions. Both use the :kprobe program type.

Kprobes

Set the section name to "kprobe/function_name" to target a kernel function:

(defprog trace-exec (:type :kprobe
                     :section "kprobe/__x64_sys_execve")
  ;; runs each time execve is called
  0)

Uprobes

For userspace functions, use "uprobe/path":

(defprog trace-malloc (:type :kprobe
                       :section "uprobe//lib64/libc.so.6")
  0)

Return Probes

To trace when a function returns rather than when it is entered, use a kretprobe section:

(defprog trace-exec-ret (:type :kprobe
                         :section "kretprobe/__x64_sys_execve")
  0)

Context: pt_regs

The context is accessed implicitly. Use the zero-argument macros pt-regs-parm1 through pt-regs-parm6 for function arguments, and pt-regs-ret for the return value on return probes:

(defprog trace-exec (:type :kprobe
                     :section "kprobe/__x64_sys_execve")
  (let ((filename-ptr (pt-regs-parm1)))
    ;; filename-ptr holds the first argument to execve
    0))

Example: Trace execve Calls

Record the PID of every process that calls execve:

(defstruct exec-event
  (pid u32)
  (ts u64))

(defmap events :type :ringbuf
  :max-entries (* 256 1024))

(defprog trace-exec (:type :kprobe
                     :section "kprobe/__x64_sys_execve")
  (with-ringbuf (e events (sizeof exec-event))
    (setf (exec-event-pid e) (cast u32 (get-current-pid-tgid))
          (exec-event-ts e) (ktime-get-ns)))
  0)

Tracepoints

Tracepoint programs attach to stable kernel tracepoints. Unlike kprobes, tracepoints are part of the kernel ABI and less likely to change between versions.

Section Name

The section follows the format "tracepoint/category/event":

(defprog trace-fork (:type :tracepoint
                     :section "tracepoint/sched/sched_process_fork")
  0)

deftracepoint

Each tracepoint has a format file under tracefs that describes its fields. deftracepoint reads this format at compile time and generates typed accessor functions automatically:

(deftracepoint sched/sched-process-fork parent-pid child-pid)

This generates zero-argument accessor macros prefixed with tp-. For sched_process_fork, the above generates (tp-parent-pid) and (tp-child-pid). Each expands to a ctx-load at the correct offset, read from the kernel's format file at /sys/kernel/tracing/events/sched/sched_process_fork/format.

Example: Track Process Forks

Record parent and child PIDs for every fork:

(deftracepoint sched/sched-process-fork parent-pid child-pid)

(defstruct fork-event
  (parent-pid u32)
  (child-pid u32))

(defmap events :type :ringbuf
  :max-entries (* 256 1024))

(defprog trace-fork (:type :tracepoint
                     :section "tracepoint/sched/sched_process_fork")
  (with-ringbuf (e events (sizeof fork-event))
    (setf (fork-event-parent-pid e) (tp-parent-pid)
          (fork-event-child-pid e) (tp-child-pid)))
  0)

The tp-parent-pid and tp-child-pid accessors are zero-argument macros generated by deftracepoint. They read from the correct offsets in the tracepoint context structure, so there is no need to manually define field offsets.

Traffic Control (TC)

TC programs attach to the Linux traffic control layer via the clsact qdisc. They can filter packets on both ingress and egress, making them useful for policies that XDP cannot cover (XDP only sees inbound packets).

Section Name

Use "tc" or "tc/name":

(defprog my-tc-filter (:type :xdp :section "tc/my_filter")
  TC_ACT_OK)

Return Codes

Packet Parsing

TC programs operate on __sk_buff instead of xdp_md, so packet data offsets differ from XDP. Whistler provides with-tc-packet, with-tc-tcp, and with-tc-udp macros that mirror the XDP API but handle the __sk_buff layout:

(with-tc-tcp (data data-end tcp)
  ;; Ethernet, IP, and TCP headers are bounds-checked
  (tcp-dst-port tcp)
  ...)

Attachment

TC programs are attached to an interface's ingress or egress path through a clsact qdisc. The typical steps from userspace:

;; Add a clsact qdisc (idempotent)
(tc-add-clsact "eth0")

;; Attach to egress
(tc-attach prog "eth0" :direction :egress)

Example: Block Outbound Traffic to Port 4444

(defprog block-4444 (:type :xdp :section "tc/block_4444")
  (with-tc-tcp (data data-end tcp)
    (when (= (tcp-dst-port tcp) 4444)
      (return TC_ACT_SHOT)))
  TC_ACT_OK)

Cgroup

Cgroup BPF programs enforce per-cgroup network and socket policies. The kernel automatically sets expected_attach_type based on the section name, so the loader does not require extra configuration.

Program Subtypes

cgroup_skb

Per-cgroup packet filtering at the SKB level. Return 1 to allow, 0 to drop.

(defprog count-egress (:type :cgroup-skb
                     :section "cgroup_skb/egress")
  ;; allow all, but could inspect and drop
  1)

cgroup_sock

Socket lifecycle hooks. Return 1 to allow, 0 to deny.

(defprog audit-sock (:type :cgroup-sock
                     :section "cgroup/sock_create")
  ;; allow all socket creation
  1)

cgroup_sock_addr

Connection and message authorization. Return 1 to allow, 0 to block.

(defprog filter-connect (:type :cgroup-sock-addr
                         :section "cgroup/connect4")
  ;; allow all IPv4 connections
  1)

BPF Helpers

The following helpers are available in cgroup programs:

Attachment

Standalone

Use attach-cgroup with the cgroup filesystem path and the appropriate attach type constant:

(attach-cgroup prog "/sys/fs/cgroup"
               :attach-type +bpf-cgroup-inet-egress+)

Attach type constants:

with-bpf-session

Inside a with-bpf-session, bpf:attach auto-detects the attach type from the program's section name:

(with-bpf-session (session "count-egress.o")
  (bpf:attach session "count-egress"
              :cgroup-path "/sys/fs/cgroup"))

Example: Count Egress Packets

A complete program that counts outbound packets for the root cgroup.

BPF program

(defmap pkt-count :type :array
  :key-size 4
  :value-size 8
  :max-entries 1)

(defprog count-egress (:type :cgroup-skb
                     :section "cgroup_skb/egress")
  (let ((key (u32 0))
        (val (map-lookup pkt-count key)))
    (when val
      (atomic-add val 1)))
  1)

Standalone loader

(let* ((obj (bpf:open-object "count-egress.o"))
       (prog (bpf:find-program obj "count-egress"))
       (loaded (bpf:load-program prog)))
  (attach-cgroup loaded "/sys/fs/cgroup"
                 :attach-type +bpf-cgroup-inet-egress+)
  ;; read the counter
  (let ((map (bpf:find-map obj "pkt-count")))
    (format t "packets: ~a~%" (bpf:map-lookup map 0))))

with-bpf-session loader

(with-bpf-session (session "count-egress.o")
  (bpf:attach session "count-egress"
              :cgroup-path "/sys/fs/cgroup")
  ;; session auto-detaches on scope exit
  (let ((map (bpf:find-map session "pkt-count")))
    (loop
      (sleep 1)
      (format t "packets: ~a~%" (bpf:map-lookup map 0)))))

deftracepoint

deftracepoint reads a kernel tracepoint format file at macroexpand time and generates zero-cost accessor macros for each field.

Syntax

(deftracepoint category/event-name [field1 field2 ...])

category/event-name is a symbol like sched/sched-switch. Hyphens are converted to underscores for the filesystem lookup.

field1, field2, ... optionally restrict which fields to import. If omitted, all non-common_ fields are imported.

How it works

At macroexpand time, Whistler reads the tracefs format file at:

/sys/kernel/tracing/events/{category}/{event}/format

It parses each field declaration to extract the name, byte offset, size, signedness, and array dimensions. For each selected field, it generates a macro:

(tp-FIELD)  ;; expands to (ctx-load TYPE OFFSET)

The tp- prefix is fixed. Types are inferred from the field size:

Array fields additionally generate a -ptr accessor:

(tp-FIELD-ptr)  ;; expands to (+ (ctx-load u64 0) OFFSET)

This gives you a pointer into the tracepoint context buffer, suitable for probe-read-kernel or probe-read-str.

Examples

Import specific fields from sched_process_fork:

(deftracepoint sched/sched-process-fork parent-pid child-pid)

;; Now available:
;; (tp-parent-pid) -> (ctx-load u32 24)   ; exact offset from format file
;; (tp-child-pid)  -> (ctx-load u32 28)

Import all fields from sched_switch:

(deftracepoint sched/sched-switch)

;; Generates tp-prev-comm, tp-prev-comm-ptr, tp-prev-pid,
;; tp-prev-prio, tp-prev-state, tp-next-comm, tp-next-comm-ptr,
;; tp-next-pid, tp-next-prio

Use in a tracepoint program:

(defprog trace-fork (:type :tracepoint
                     :section "tracepoint/sched/sched_process_fork"
                     :license "GPL")
  (setf (getmap ppid-map (tp-child-pid)) (tp-parent-pid))
  0)

Permissions

The format file must be readable by the compiling user. If running without root:

sudo chmod a+r /sys/kernel/tracing/events/sched/sched_process_fork/format

If the file is not found, Whistler also checks the debugfs fallback path at /sys/kernel/debug/tracing/events/.

import-kernel-struct

import-kernel-struct reads the vmlinux BTF blob at macroexpand time and generates typed accessor macros for kernel struct fields.

Syntax

(import-kernel-struct struct-name [field1 field2 ...])

struct-name is a symbol like task-struct. Hyphens are converted to underscores for the BTF lookup.

field1, field2, ... optionally restrict which fields to import. If omitted, all fields are imported (including those from anonymous nested structs/unions, which are flattened).

How it works

At macroexpand time, Whistler:

1. Reads and parses /sys/kernel/btf/vmlinux (cached across expansions).
2. Finds the struct by name in the BTF type table.
3. Resolves each field's type through typedefs, const, volatile, etc.
4. Generates accessor macros using kernel-load (which compiles to probe-read-kernel + stack buffer + load).

For a struct like task_struct, this generates:

;; Scalar/pointer fields:
(task-struct-pid ptr)   ;; -> (kernel-load u32 ptr OFFSET)
(task-struct-tgid ptr)  ;; -> (kernel-load u32 ptr OFFSET)
(task-struct-flags ptr) ;; -> (kernel-load u32 ptr OFFSET)

;; Embedded struct fields:
(task-struct-mm ptr)    ;; -> (+ ptr OFFSET)  (returns typed pointer)

;; Size constant:
+task-struct-size+      ;; total size in bytes

Pointer fields resolve to u64. Embedded structs return a pointer offset (no probe-read -- you access their sub-fields with further kernel-loads). A (as-STRUCT ptr) cast macro is also generated for type-safe pointer tagging.

Pointer chasing

Kernel-load accessors compose naturally for pointer chasing:

(import-kernel-struct task-struct pid tgid mm)
(import-kernel-struct mm-struct exe-file)
(import-kernel-struct file f-path)

;; Chase: current task -> mm -> exe_file
(let* ((task (get-current-task))
       (mm   (task-struct-mm task))
       (exe  (mm-struct-exe-file mm)))
  ;; exe is now a kernel pointer to struct file
  ...)

Each accessor checks the typed-ptr tag at macroexpand time. If you pass a mm-struct pointer to a task-struct accessor, you get a compile-time error. Use (as-task-struct ptr) to cast if intentional.

CO-RE compatibility

The offsets come from the build host's BTF. For CO-RE relocatable programs, use core-load / core-ctx-load instead. import-kernel-struct is best suited for programs that will run on the same kernel they were compiled on.

Example

(import-kernel-struct task-struct pid tgid comm)

(defstruct exec-event
  (pid  u32)
  (tgid u32)
  (comm (array u8 16)))

(defmap events :type :ringbuf :max-entries 4096)

(defprog trace-exec (:type :kprobe
                     :section "kprobe/__x64_sys_execve"
                     :license "GPL")
  (let ((task (get-current-task)))
    (with-ringbuf (ev events (sizeof exec-event))
      (setf (exec-event-pid ev)  (task-struct-pid task)
            (exec-event-tgid ev) (task-struct-tgid task))
      (probe-read-kernel (exec-event-comm-ptr ev) 16
                         (+ task (task-struct-comm task)))))
  0)

Key points:

• kernel-load handles the probe-read-kernel dance automatically.
• Pointer fields return u64 values you can pass to further accessors.
• +STRUCT-SIZE+ is useful for probe-read-kernel buffer sizing.

Protocol Library

Whistler includes a compile-time protocol header library in protocols.lisp. All macros expand to (load TYPE ptr OFFSET) at compile time -- zero runtime cost.

defheader

Define custom protocol headers:

(defheader my-proto
  (field-a :offset 0 :type u32)
  (field-b :offset 4 :type u16 :net-order t))

Each field generates an accessor macro (my-proto-FIELD ptr). When :net-order t, the accessor wraps the load in ntohs / ntohl as appropriate for the field size.

Built-in headers

Ethernet (14 bytes)

IPv4 (20 bytes)

IPv6 (40 bytes)

Accessors: ipv6-ver-tc-flow, ipv6-payload-len, ipv6-nexthdr, ipv6-hop-limit, ipv6-src-addr-hi/lo, ipv6-dst-addr-hi/lo.

TCP (20 bytes)

UDP (8 bytes)

Accessors: udp-src-port, udp-dst-port, udp-length, udp-checksum.

ICMP (8 bytes)

Accessors: icmp-type, icmp-code, icmp-checksum, icmp-rest.

XDP context access

CO-RE-aware context loads for XDP programs:

(xdp-data)      ;; -> (core-ctx-load u32 0 xdp-md data)
(xdp-data-end)  ;; -> (core-ctx-load u32 4 xdp-md data-end)

Statement-oriented parsing (with-*)

These macros bind packet pointers, perform bounds checks, and early-return XDP_PASS on failure. They use flat guard structure (no nesting of success paths), which is optimal for the BPF verifier.

(with-packet (data data-end :min-len 34) ...)
(with-eth (data data-end) ...)
(with-ipv4 (data data-end ip) ...)
(with-tcp (data data-end tcp) ...)
(with-udp (data data-end udp) ...)

Example:

(defprog my-xdp (:type :xdp :section "xdp" :license "GPL")
  (with-tcp (data data-end tcp)
    ;; tcp is bound to the TCP header pointer
    ;; data, data-end are bound from XDP context
    (when (= (tcp-dst-port tcp) 80)
      (return XDP_DROP)))
  XDP_PASS)

Expression-oriented parsing (parse-*)

Return a pointer on success or 0 on failure. Use with when-let for pipeline-style composition:

(parse-eth data data-end)   ;; returns data or 0
(parse-ipv4 data data-end)  ;; returns IP header ptr or 0
(parse-tcp data data-end)   ;; returns TCP header ptr or 0
(parse-udp data data-end)   ;; returns UDP header ptr or 0

Example:

(let ((data (xdp-data))
      (data-end (xdp-data-end)))
  (when-let ((tcp (parse-tcp data data-end)))
    (incf (getmap stats (tcp-dst-port tcp)))))

TC variants

Traffic Control programs use __sk_buff context (data at offset 76, data_end at offset 80) and return TC_ACT_OK on early exit:

(tc-data)       ;; -> (ctx-load u32 76)
(tc-data-end)   ;; -> (ctx-load u32 80)

(with-tc-packet (data data-end :min-len N) ...)
(with-tc-eth (data data-end) ...)
(with-tc-ipv4 (data data-end ip) ...)
(with-tc-tcp (data data-end tcp) ...)
(with-tc-udp (data data-end udp) ...)

Constants

Loading Programs

The Whistler userspace loader is a pure Common Lisp BPF loader -- no libbpf dependency. It handles ELF parsing, map creation, relocation patching, and program loading via the bpf(2) syscall.

Setup

(asdf:load-system "whistler/loader")

with-bpf-object (recommended)

The simplest way to load a compiled BPF object:

(whistler/loader:with-bpf-object (obj "my-prog.bpf.o")
  ;; obj is a loaded bpf-object with maps created and programs loaded
  ;; Attach, read maps, etc.
  ...)
;; All resources (maps, programs, attachments) auto-closed here

with-bpf-object opens the ELF, creates all maps, patches map FD relocations, loads all programs into the kernel, and closes everything on exit (normal or error) via unwind-protect.

Manual lifecycle

For finer control:

(let ((obj (whistler/loader:open-bpf-object "my-prog.bpf.o")))
  ;; Parses ELF, extracts map definitions. Nothing loaded yet.

  (whistler/loader:load-bpf-object obj)
  ;; Creates maps, patches relocations, loads programs.

  ;; ... use obj ...

  (whistler/loader:close-bpf-object obj))
  ;; Detaches programs, closes all FDs.

Object accessors

After loading, look up maps and programs by name:

(whistler/loader:bpf-object-map obj "my_map")   ;; -> map-info or nil
(whistler/loader:bpf-object-prog obj "my_prog")  ;; -> prog-info or nil

Names use underscores (matching the ELF symbol table).

What happens during load

1. ELF parsing -- reads section headers, symbol table, map definitions from .maps, program bytecode from named sections, relocation entries.
2. Map creation -- calls BPF_MAP_CREATE for each map defined in the .maps section.
3. Relocation patching -- for each R_BPF_64_64 relocation, replaces the placeholder in the instruction stream with the real map FD.
4. Program loading -- calls BPF_PROG_LOAD for each program section. The program type is inferred from the section name (e.g., xdp, kprobe/..., tracepoint/..., cgroup_skb/...).

Attaching Programs

After loading, programs must be attached to a kernel hook point. All attachment functions return an attachment struct that can be passed to (detach att) for cleanup.

Kprobe

(attach-kprobe prog-fd "function_name")
(attach-kprobe prog-fd "function_name" :retprobe t)

Attaches to a kernel function entry (or return) point via the kprobe PMU.

Uprobe

(attach-uprobe prog-fd "/path/to/binary" "symbol_name")
(attach-uprobe prog-fd "/path/to/binary" "symbol_name" :retprobe t)

Resolves the symbol to a file offset via ELF parsing, then attaches via the uprobe PMU.

Tracepoint

(attach-tracepoint prog-fd "tracepoint/sched/sched_process_fork")
(attach-tracepoint prog-fd "sched/sched_process_fork")

Resolves the tracepoint ID from tracefs and opens a perf event. The tracepoint/ prefix is optional. Hyphens are converted to underscores for the filesystem lookup.

XDP

(attach-xdp prog-fd "eth0")
(attach-xdp prog-fd "eth0" :mode "xdpdrv")

Attaches an XDP program to a network interface. Mode options:

TC (Traffic Control)

(attach-tc prog-fd "eth0")
(attach-tc prog-fd "eth0" :direction "egress")

Attaches a TC classifier program. Sets up the clsact qdisc and pins the program to bpffs. Direction is "ingress" (default) or "egress".

Cgroup

(attach-cgroup prog-fd "/sys/fs/cgroup" +bpf-cgroup-inet-egress+)
(attach-cgroup prog-fd "/sys/fs/cgroup" +bpf-cgroup-inet-egress+ :flags 2)

Attaches a BPF program to a cgroup. The attach type must be one of the constants below. Optional :flags can include BPF_F_ALLOW_MULTI (2) or BPF_F_REPLACE (4).

Cgroup attach type constants

Convenience wrappers

For with-bpf-object users, these look up the program by name and track the attachment on the object (auto-detached on close):

(attach-obj-kprobe obj "prog_name" "function_name" :retprobe nil)
(attach-obj-uprobe obj "prog_name" "/path/to/bin" "symbol" :retprobe nil)
(attach-obj-cgroup obj "prog_name" "/sys/fs/cgroup" +bpf-cgroup-inet-egress+)

Detaching

(detach attachment)

Closes perf event FDs and runs any cleanup (e.g., removing TC filters, detaching from cgroups, removing XDP programs).

Map Operations

The loader provides userspace access to BPF maps via the bpf(2) syscall. The low-level API works with raw byte arrays, with integer helpers for the common scalar case.

Core operations

(map-lookup map-info key-bytes)
(map-lookup-int map-info key)

Returns the value as a byte array, or nil if the key is not found. For percpu maps, returns a vector of per-CPU byte arrays. map-lookup-int encodes the key and decodes the value as little-endian integers.

(map-update map-info key-bytes value-bytes)
(map-update map-info key-bytes value-bytes :flags +bpf-noexist+)
(map-update-int map-info key value)
(map-update-struct map-info key-bytes record 'my-struct)
(map-update-struct-int map-info key record 'my-struct)

Insert or update a key/value pair. For percpu maps, value-bytes can be a single byte array (replicated to all CPUs) or a vector of per-CPU arrays. map-update-int does the integer encoding for fixed-size scalar maps.

(map-delete map-info key-bytes)
(map-delete-int map-info key)
(map-delete-struct map-info record 'my-struct)

Delete a key from the map.

(map-get-next-key map-info key-bytes)
(map-get-next-key-int map-info &optional key)
(map-get-next-key-struct map-info 'my-struct &optional key-record)

Returns the next key after key-bytes, or nil when iteration is complete. Pass nil as the key to get the first key. The typed variants decode integer and struct keys for you.

Encoding helpers

(encode-int-key 42 4)       ;; -> #(42 0 0 0)  (4-byte LE)
(decode-int-value #(10 0 0 0 0 0 0 0))  ;; -> 10

encode-int-key encodes an integer as a little-endian byte array of the given size. decode-int-value decodes a little-endian byte array back to an integer.

Struct-valued maps

When a map value matches a defstruct layout, use the generated codecs through the typed helpers:

(defstruct stats-entry
  (packets u64)
  (drops   u64))

(let ((rec (make-stats-entry-record :packets 10 :drops 2)))
  (map-update-struct-int stats-map 0 rec 'stats-entry))

(let ((rec (map-lookup-struct-int stats-map 0 'stats-entry)))
  (when rec
    (format t "packets=~d drops=~d~%"
            (stats-entry-record-packets rec)
            (stats-entry-record-drops rec))))

The struct symbol determines which encode-* / decode-* functions are used.

The same approach works for struct keys:

(defstruct flow-key
  (src u32)
  (dst u32))

(let ((key (make-flow-key-record :src #x0a000001 :dst #x0a000002)))
  (map-delete-struct flows key 'flow-key))

Iteration example

Walk all entries in a hash map:

(let ((key nil))
  (loop
    (let ((next (map-get-next-key my-map key)))
      (unless next (return))
      (let ((val (map-lookup my-map next)))
        (when val
          (format t "~d -> ~d~%"
                  (decode-int-value next)
                  (decode-int-value val))))
      (setf key next))))

Typed key iteration is simpler when the key is scalar or structured:

(let ((key nil))
  (loop
    (setf key (map-get-next-key-int stats-map key))
    (unless key (return))
    (format t "next key: ~d~%" key)))

(let ((key nil))
  (loop
    (setf key (map-get-next-key-struct flows 'flow-key key))
    (unless key (return))
    (format t "~x -> ~x~%"
            (flow-key-record-src key)
            (flow-key-record-dst key))))

Percpu maps

For percpu-hash and percpu-array maps, map-lookup returns a vector of byte arrays (one per possible CPU). Each slot is 8-byte aligned as required by the kernel:

(let ((values (map-lookup percpu-map (encode-int-key 0 4))))
  (when values
    (dotimes (cpu (length values))
      (format t "CPU ~d: ~d~%" cpu (decode-int-value (aref values cpu))))))

Ring Buffer Consumer

The loader includes a pure-CL ring buffer consumer for BPF_MAP_TYPE_RINGBUF maps. It uses mmap for zero-copy access and epoll for efficient waiting.

Usage

(let ((consumer (open-ring-consumer map-info
                  (lambda (sap len)
                    ;; sap is a system-area-pointer to the event data
                    ;; len is the event length in bytes
                    (let ((buf (make-array len :element-type '(unsigned-byte 8))))
                      (dotimes (i len)
                        (setf (aref buf i) (sb-sys:sap-ref-8 sap i)))
                      (process-event buf))))))
  (unwind-protect
       (loop
         (ring-poll consumer :timeout-ms 1000))
    (close-ring-consumer consumer)))

API

open-decoding-ring-consumer

(open-decoding-ring-consumer map-info decoder callback) -> ring-consumer

Creates a ring buffer consumer that copies each event into an octet vector, decodes it with decoder, and passes the decoded object to callback. Use this when your ring buffer holds defstruct-defined records.

with-decoding-ring-consumer

(with-decoding-ring-consumer (consumer map-info decoder callback)
  body...)

Convenience macro that opens a decoding ring consumer, binds it to consumer, and guarantees cleanup with unwind-protect.

open-ring-consumer

(open-ring-consumer map-info callback) -> ring-consumer

Creates a ring buffer consumer. callback is called with (sap len) for each event -- a system-area-pointer and the event byte length. The callback runs synchronously during ring-poll.

Internally, this mmaps the consumer page (read-write) and the producer + data pages (read-only), then sets up an epoll instance on the map FD.

ring-poll

(ring-poll consumer :timeout-ms 100) -> event-count

Waits for ring buffer events via epoll, then consumes all available events. Returns the number of events processed. A timeout of 0 makes it non-blocking.

close-ring-consumer

(close-ring-consumer consumer)

Unmaps memory and closes the epoll FD. Always call this on cleanup.

Reading structured events

When your BPF program writes a defstruct-defined struct to the ring buffer, use the higher-level decoding helper:

;; Given: (defstruct conn-event (src-addr u32) (dst-addr u32) (port u16))
;; Whistler generates: decode-conn-event, conn-event-record-src-addr, etc.

(open-decoding-ring-consumer
 map-info
 #'decode-conn-event
 (lambda (ev)
   (format t "~a:~d~%" (conn-event-record-src-addr ev)
                        (conn-event-record-port ev))))

Inline BPF Sessions

with-bpf-session compiles BPF code at macroexpand time and loads it at runtime, all in one Lisp file. No separate compilation step.

How it works

graph TD
    A["<b>Macroexpand time</b><br/>bpf:map and bpf:prog forms extracted,<br/>compiled to BPF bytecode,<br/>embedded as literal byte arrays"] --> B["<b>Runtime</b><br/>Maps created, relocations patched,<br/>programs loaded into kernel,<br/>CL body executes"]
    B --> C["<b>Exit</b><br/>All maps, programs, and<br/>attachments closed automatically"]

    style A fill:#4a9eff,color:#fff
    style B fill:#f39c12,color:#fff
    style C fill:#2ecc71,color:#fff

Syntax

(with-bpf-session ()
  ;; BPF-side declarations (compiled at macroexpand time):
  (bpf:map NAME :type TYPE :key-size N :value-size N :max-entries N)
  (bpf:prog NAME (OPTIONS...) BODY...)

  ;; Userspace code (runs at runtime):
  (bpf:attach PROG-NAME TARGET ...)
  (bpf:map-ref MAP-NAME KEY)
  ;; ... any CL code ...
  )

bpf:map

Declares a BPF map. Same keyword arguments as defmap.

bpf:prog

Declares a BPF program. The options plist takes :type, :section, and :license. The body is Whistler BPF code.

bpf:attach

Attaches a loaded program. The attachment type is auto-detected from the program's section name:

• kprobe/... -- (bpf:attach prog "function_name")
• uprobe/... -- (bpf:attach prog "/path/to/binary" "symbol")
• tracepoint/... -- (bpf:attach prog "tracepoint/cat/event")
• cgroup_skb/... -- (bpf:attach prog "/sys/fs/cgroup")

bpf:map-ref

Reads a map value by integer key, returning the decoded integer value or nil:

(bpf:map-ref my-map 0)  ;; -> integer or nil

Package setup

The easiest setup is to work in whistler-loader-user, which already imports the compiler and loader entry points with the right shadowing in place:

(asdf:load-system "whistler/loader")
(in-package #:whistler-loader-user)

If you want your own application package, use whistler-loader-user as the reference layout instead of rebuilding the package story from scratch.

Kprobe example

(asdf:load-system "whistler/loader")
(in-package #:whistler-loader-user)

(whistler:defstruct call-event
  (pid u32)
  (ts  u64))

(with-bpf-session ()
  (bpf:map events :type :ringbuf :max-entries 4096)

  (bpf:prog trace-exec (:type :kprobe
                         :section "kprobe/__x64_sys_execve"
                         :license "GPL")
    (with-ringbuf (ev events (sizeof call-event))
      (setf (call-event-pid ev) (cast u32 (ash (get-current-pid-tgid) -32))
            (call-event-ts ev)  (ktime-get-ns)))
    0)

  (bpf:attach trace-exec "__x64_sys_execve")

  (let ((consumer (open-decoding-ring-consumer
                   (bpf-session-map 'events)
                   #'decode-call-event
                   (lambda (ev)
                     (format t "exec pid=~d ts=~d~%"
                             (call-event-record-pid ev)
                             (call-event-record-ts ev))))))
    (unwind-protect
         (loop (ring-poll consumer :timeout-ms 1000))
      (close-ring-consumer consumer))))

Cgroup example

(with-bpf-session ()
  (bpf:map pkt-count :type :array :key-size 4 :value-size 8 :max-entries 1)

  (bpf:prog count-egress
      (:type :cgroup-skb :section "cgroup_skb/egress" :license "GPL")
    (incf (getmap pkt-count 0))
    1)

  (bpf:attach count-egress "/sys/fs/cgroup")

  (loop repeat 10
        do (sleep 1)
           (format t "packets: ~d~%" (or (bpf:map-ref pkt-count 0) 0))))

The attach type (+bpf-cgroup-inet-egress+) is inferred automatically from the section name "cgroup_skb/egress".

Compilation Model

Whistler compiles Lisp s-expressions to eBPF bytecode through a 9-phase pipeline. Full Common Lisp is available at compile time -- user-defined macros, defconstant, defstruct, and import-kernel-struct all run during the macroexpansion phase.

Pipeline

graph LR
    A[Source] --> B[Load]
    B --> C[Macroexpand]
    C --> D[Constant Fold]
    D --> E[Lower to SSA IR]
    E --> F[SSA Optimize]
    F --> G[Register Alloc]
    G --> H[BPF Emit]
    H --> I[Peephole]
    I --> J[ELF Output]

    style A fill:#4a9eff,color:#fff
    style J fill:#2ecc71,color:#fff

1. Load

Read the source file. defmap, defprog, defstruct, defconstant, and deftracepoint forms are evaluated, populating the compiler's map and program tables.

2. Macroexpand

Recursively expand all macros in program bodies. Whistler built-in forms (if, let, setf, load, store, etc.) and BPF helpers are recognized and left intact. Everything else is expanded via macroexpand-1. This is what makes Whistler a real Lisp -- user macros compose freely with built-in forms.

3. Constant fold

Walk the expanded s-expression tree, replacing defconstant symbols with their integer values and folding arithmetic on constant arguments (+, -, *, /, <<, >>, &, |).

4. Lower to SSA IR

Translate surface-language forms into SSA (Static Single Assignment) intermediate representation with virtual registers and basic blocks. Each variable binding creates a fresh virtual register. Control flow (if, when, cond, dotimes) creates basic blocks with branch/jump instructions. Phi nodes are inserted at join points.

5. SSA optimize

Multiple optimization passes, run in sequence:

Canonicalization (to fixed point):

• Copy propagation
• Constant propagation
• Dead code elimination
• Dead destination elimination
• Eliminate trivial phis
• Simplify CFG
• Eliminate unreachable blocks

Domain-specific folds:

• Byte-swap comparison fusion (fold ntohs/ntohl into comparisons)
• Constant offset folding
• Tracepoint return elision

Loop and memory:

• Loop-invariant code motion
• Common subexpression elimination
• Forward stores to loads

SCCP (Sparse Conditional Constant Propagation):

• Propagate constants through phi nodes and fold unreachable branches

Cleanup:

• Dead code elimination
• Dead destination elimination
• Dead store elimination

Cross-block fusions:

• Lookup-delete fusion (merge map lookup + delete into single path)
• Hoist loads before helper calls
• Phi branch threading
• Bitmask check fusion
• Redundant branch cleanup

Final:

• Re-canonicalize after fusions
• Narrow ALU types (use 32-bit ops where safe)
• Split live ranges (improve register allocation)

6. Register allocation

Linear-scan register allocation with two pools:

R0 is reserved for helper return values and program exit code. R10 is the read-only frame pointer. R6 is reserved for the context pointer when needed.

When registers are exhausted, values spill to the 512-byte stack frame. Spill decisions consider value classification (packet pointers are expensive to spill; constants can be rematerialized).

7. BPF emission

Map allocated physical registers to BPF instructions. Handle stack layout, map FD placeholder loading (for later relocation), and CO-RE relocation tracking.

8. Peephole optimization

Post-emission cleanup on the BPF instruction list:

• Redundant mov elimination (mov rX, rX)
• Branch inversion (jCC +1; ja +N becomes j!CC +N)
• Jump-to-next elimination (ja +0)
• Jump threading (chains of unconditional jumps)
• Dead code after exit/unconditional jump
• Return value folding (mov rX, IMM; mov r0, rX; exit becomes mov r0, IMM; exit)
• Stack address folding
• Tail merge (identical exit sequences)

9. ELF emit

Write the final BPF instructions and metadata into an ELF64 relocatable object file. See ELF Output for section details.

BPF register table

eBPF constraints

ELF Output

Whistler produces standard ELF64 little-endian relocatable object files (ET_REL) with e_machine = EM_BPF (247). These are compatible with libbpf, bpftool, and the Whistler loader.

Section layout

A typical Whistler-generated .bpf.o contains these sections:

Program sections

Each defprog produces a section named after its :section option (e.g., xdp, kprobe/__x64_sys_execve, tracepoint/sched/sched_process_fork). The section contains raw BPF instructions (8 bytes each), marked SHF_ALLOC | SHF_EXECINSTR.

For multi-program ELF files (e.g., tail call dispatch), each program gets its own section and a STT_FUNC symbol.

Maps section

The .maps section contains 32-byte entries:

Each map has a corresponding STT_OBJECT global symbol in .symtab.

Relocations

Map references in BPF instructions use R_BPF_64_64 relocations. Each relocation entry is 16 bytes (SHT_REL, not RELA):

r_offset (8 bytes) -- byte offset of the ld_imm64 instruction
r_info   (8 bytes) -- ELF64_R_INFO(symbol_index, R_BPF_64_64)

The loader resolves these by patching the ld_imm64 instruction's src_reg to BPF_PSEUDO_MAP_FD and setting the immediate to the map FD.

Symbol table

The symbol table contains:

1. Null symbol (index 0)
2. Section symbols (STT_SECTION, STB_LOCAL) -- one per program section
3. Map symbols (STT_OBJECT, STB_GLOBAL) -- one per map, pointing into .maps
4. Function symbols (STT_FUNC, STB_GLOBAL) -- one per program, named after the defprog name (underscored)

Shared Header Generation

When compiling BPF programs, you often need matching struct definitions on both sides: the BPF program (Whistler) and the userspace consumer (C, Go, Rust, Python, or Common Lisp). Whistler generates these automatically.

CLI usage

whistler compile input.lisp -o output.bpf.o --gen c
whistler compile input.lisp -o output.bpf.o --gen go
whistler compile input.lisp -o output.bpf.o --gen rust
whistler compile input.lisp -o output.bpf.o --gen python
whistler compile input.lisp -o output.bpf.o --gen lisp
whistler compile input.lisp -o output.bpf.o --gen all

The --gen flag accepts one or more target languages. all generates every supported format.

Output files

Given output base name foo, Whistler produces:

What gets generated

Struct definitions

Every defstruct in the source file produces a matching struct in each target language. Array fields use the language's native array syntax:

;; Whistler source
(defstruct conn-event
  (src-addr u32)
  (dst-addr u32)
  (port     u16)
  (comm     (array u8 16)))

// C output
struct conn_event {
    uint32_t src_addr;
    uint32_t dst_addr;
    uint16_t port;
    uint8_t comm[16];
};

// Go output
type ConnEvent struct {
	SrcAddr uint32
	DstAddr uint32
	Port    uint16
	Comm    [16]uint8
}

#![allow(unused)]
fn main() {
// Rust output
#[repr(C)]
#[derive(Clone, Copy, Debug, Default)]
pub struct ConnEvent {
    pub src_addr: u32,
    pub dst_addr: u32,
    pub port: u16,
    pub comm: [u8; 16],
}
unsafe impl aya::Pod for ConnEvent {}
}

# Python output
class ConnEvent(ctypes.LittleEndianStructure):
    _fields_ = [
        ("src_addr", ctypes.c_uint32),
        ("dst_addr", ctypes.c_uint32),
        ("port", ctypes.c_uint16),
        ("comm", ctypes.c_uint8 * 16)
    ]

Constants

defconstant values defined in the source file are included in the generated headers:

(defconstant +event-type-tcp+ 1)

#define EVENT_TYPE_TCP 1

CL codec

The Common Lisp output includes defstruct with typed slots plus NAME-from-bytes and NAME-to-bytes functions for BPF map interop. These decode/encode structs from/to raw byte arrays.

Layout guarantee

All generated structs are guaranteed to match the BPF-side memory layout. Whistler computes field offsets at compile time and generates code that uses the same byte positions, so there is no alignment mismatch between the kernel and userspace sides.

CLI Reference

Whistler ships as a standalone binary (a saved SBCL image).

Commands

Version

whistler --version

Print the Whistler version string.

Help

whistler --help

Show usage information and available commands.

Compile

whistler compile INPUT [-o OUTPUT] [--gen LANG...]

Compile a Whistler source file to a BPF ELF object.

Examples:

# Compile to BPF object
whistler compile my-prog.lisp -o my-prog.bpf.o

# Compile and generate C + Go headers
whistler compile my-prog.lisp -o my-prog.bpf.o --gen c go

# Compile and generate all language bindings
whistler compile my-prog.lisp --gen all

Disassemble

whistler disasm INPUT

Load a Whistler source file, compile its first program, and print the BPF instructions in human-readable form.

whistler disasm my-prog.lisp

Doctor

whistler doctor

Run a local environment check for Whistler development. The report includes:

• kernel version
• whether sbcl, ip, and tc are available
• whether tracefs and /sys/kernel/btf/vmlinux are readable
• whether sbcl or ./whistler appear to have useful Linux capabilities set

Tracepoint with Ring Buffer

Capture new TCP connection events (SYN packets) via XDP and send them to userspace through a ring buffer.

BPF program

(in-package #:whistler)

;;; Event struct -- shared between BPF and userspace.
;;; defstruct generates both BPF accessor macros and CL-side codec
;;; (decode-conn-event -> conn-event-record struct).
(defstruct conn-event
  (src-addr  u32)
  (dst-addr  u32)
  (dst-port  u16)
  (proto     u8)
  (pad       u8))    ; align to 12 bytes

;;; Maps
(defmap events :type :ringbuf :max-entries 4096)

(defmap rb-stats :type :array
  :key-size 4 :value-size 8 :max-entries 2)

(defconstant +stat-events-sent+    0)
(defconstant +stat-events-dropped+ 1)

;;; Program
(defprog event-logger (:type :xdp :section "xdp" :license "GPL")
  (with-packet (data data-end :min-len 38)    ; Eth + IPv4 + TCP ports
    (when (= (eth-type data) +ethertype-ipv4+)
      (let* ((ip    (+ data +eth-hdr-len+))
             (proto (ipv4-protocol ip)))
        ;; TCP SYN packets only
        (when (and (= proto +ip-proto-tcp+)
                   (> (+ data 54) data-end))
          (return XDP_PASS))
        (when (= proto +ip-proto-tcp+)
          (let* ((tcp   (+ ip +ipv4-hdr-len+))
                 (flags (tcp-flags tcp)))
            ;; Only log new connections (SYN set, ACK not set)
            (when (and (logand flags +tcp-syn+)
                       (not (logand flags +tcp-ack+)))
              (with-ringbuf (event events (sizeof conn-event))
                (setf (conn-event-src-addr event) (ipv4-src-addr ip)
                      (conn-event-dst-addr event) (ipv4-dst-addr ip)
                      (conn-event-dst-port event) (tcp-dst-port tcp)
                      (conn-event-proto event) proto)
                (incf (getmap rb-stats +stat-events-sent+)))))))))
  XDP_PASS)

(compile-to-elf "ringbuf-events.bpf.o")

Key points

• Flat guards: with-packet does a single bounds check and early-returns XDP_PASS on failure. The nested when forms are flat guard checks, not deeply nested success paths. This structure is optimal for the BPF verifier, which tracks packet bounds along each code path.

• sizeof: (sizeof conn-event) resolves at compile time to the struct's total byte size (12 in this case). It works anywhere an integer constant is expected.

• with-ringbuf: Reserves space in the ring buffer, executes the body, and auto-submits on normal exit. If the ring buffer is full, the reservation returns 0 and the body is skipped entirely (guarded by an internal when).

• Dual struct: defstruct generates both BPF accessor macros for the kernel side (conn-event-src-addr, etc.) and a CL record type (conn-event-record) plus decode-conn-event for the userspace side. With --gen c, you also get a matching C struct for non-Lisp consumers.

Kernel Struct Traversal

Read kernel data structures using import-kernel-struct and send execution events to userspace.

BPF program

(in-package #:whistler)

;;; Import kernel struct fields from vmlinux BTF
(import-kernel-struct task-struct pid tgid comm)

;;; Userspace event struct
(defstruct exec-event
  (pid  u32)
  (tgid u32)
  (comm (array u8 16)))

;;; Maps
(defmap events :type :ringbuf :max-entries 4096)

;;; Kprobe program
(defprog trace-exec (:type :kprobe
                     :section "kprobe/__x64_sys_execve"
                     :license "GPL")
  (let ((task (get-current-task)))
    (with-ringbuf (ev events (sizeof exec-event))
      (setf (exec-event-pid ev)  (task-struct-pid task)
            (exec-event-tgid ev) (task-struct-tgid task))
      (probe-read-kernel (exec-event-comm-ptr ev) 16
                         (+ task (task-struct-comm task)))))
  0)

(compile-to-elf "exec-events.bpf.o")

Key points

• CO-RE caveat: import-kernel-struct reads BTF from the build host at compile time. The generated offsets are correct for that specific kernel version. For portable programs that must run across kernels, use core-load / core-ctx-load with BTF relocations instead.

• kernel-load: Each accessor like (task-struct-pid task) expands to (kernel-load u32 task OFFSET), which compiles to:

  (let ((buf (struct-alloc 4)))
    (probe-read-kernel buf 4 (+ task OFFSET))
    (load u32 buf 0))
  

  The BPF verifier requires probe_read_kernel for kernel pointers -- direct dereference is not allowed.

• Pointer chasing: For pointer fields (e.g., task_struct->mm), the accessor returns a u64 kernel pointer. Chain accessors naturally:

  (import-kernel-struct task-struct mm)
  (import-kernel-struct mm-struct exe-file)
  
  (let* ((task (get-current-task))
         (mm   (task-struct-mm task))
         (exe  (mm-struct-exe-file mm)))
    ...)
  

• comm field: comm is a char[16] array in task_struct. The (task-struct-comm task) accessor returns an address offset (embedded struct/array path), so use probe-read-kernel to copy it into your event struct.

XDP Tail Call Dispatch

Use a prog-array map to dispatch XDP processing to per-protocol handler programs via tail calls.

BPF programs

(in-package #:whistler)

;;; Jump table: protocol number -> program FD
(defmap jt :type :prog-array
  :key-size 4 :value-size 4 :max-entries 256)

;;; Per-protocol counters
(defmap proto-stats :type :array
  :key-size 4 :value-size 8 :max-entries 3)

(defconstant +stat-dispatched+ 0)
(defconstant +stat-tcp+ 1)
(defconstant +stat-udp+ 2)

;;; Dispatcher -- entry point
(defprog xdp-dispatch (:type :xdp :section "xdp" :license "GPL")
  (let ((data     (xdp-data))
        (data-end (xdp-data-end)))
    (when (> (+ data 34) data-end)
      (return XDP_PASS))
    (when (/= (eth-type data) +ethertype-ipv4+)
      (return XDP_PASS))
    (let ((proto (ipv4-protocol (+ data +eth-hdr-len+))))
      (declare (type u32 proto))
      (incf (getmap proto-stats +stat-dispatched+))
      (tail-call jt proto)))
  XDP_PASS)

;;; TCP handler (separate program, same ELF)
(defprog tcp-handler (:type :xdp :section "xdp/tcp" :license "GPL")
  (incf (getmap proto-stats +stat-tcp+))
  XDP_PASS)

;;; UDP handler (separate program, same ELF)
(defprog udp-handler (:type :xdp :section "xdp/udp" :license "GPL")
  (incf (getmap proto-stats +stat-udp+))
  XDP_PASS)

(compile-to-elf "tail-call-dispatch.bpf.o")

Loading and wiring

After compilation, load all programs and populate the jump table:

# Load all programs from the multi-program ELF
bpftool prog loadall tail-call-dispatch.bpf.o /sys/fs/bpf/tcd

# Wire protocol handlers into the jump table
bpftool map update name jt \
  key 6 0 0 0 value pinned /sys/fs/bpf/tcd/tcp_handler
bpftool map update name jt \
  key 17 0 0 0 value pinned /sys/fs/bpf/tcd/udp_handler

# Attach the dispatcher to a network interface
ip link set dev eth0 xdp pinned /sys/fs/bpf/tcd/xdp_dispatch

Key points

• Multi-program ELF: Each defprog with a distinct :section name produces a separate program section in the ELF. Use bpftool prog loadall to load them all and pin each to bpffs.

• Tail-call semantics: (tail-call jt proto) compiles to the BPF_TAIL_CALL helper. If the key exists in the prog-array, execution transfers to that program with the same context and no return. If the key is missing (no handler loaded), execution falls through to the next instruction -- here, XDP_PASS.

• Shared maps: All programs in the same ELF share map definitions. The proto-stats array is accessible from both the dispatcher and the handlers.

• bpftool population: prog-array maps cannot be populated from BPF code. Insert program FDs from userspace using bpftool or the loader's map-update.

Inline Session

A complete inline BPF session -- compilation, loading, attachment, and event consumption in a single Lisp file, with no separate compilation step.

Full example

(asdf:load-system "whistler/loader")
(in-package #:whistler-loader-user)

;;; Struct definition -- generates both BPF and CL sides
(whistler:defstruct call-event
  (pid u32)
  (ts  u64))

;;; Inline BPF session
(defun run ()
  (with-bpf-session ()
    ;; BPF side (compiled at macroexpand time)
    (bpf:map events :type :ringbuf :max-entries 4096)

    (bpf:prog trace-exec (:type :kprobe
                           :section "kprobe/__x64_sys_execve"
                           :license "GPL")
      (with-ringbuf (ev events (sizeof call-event))
        (setf (call-event-pid ev) (cast u32 (ash (get-current-pid-tgid) -32))
              (call-event-ts ev)  (ktime-get-ns)))
      0)

    ;; Userspace side (runs at runtime)
    (bpf:attach trace-exec "__x64_sys_execve")

    (with-decoding-ring-consumer (consumer
                                  (bpf-session-map 'events)
                                  #'decode-call-event
                                  (lambda (ev)
                                    (format t "exec pid=~d ts=~d~%"
                                            (call-event-record-pid ev)
                                            (call-event-record-ts ev))))
      (handler-case
          (loop (ring-poll consumer :timeout-ms 1000))
        (sb-sys:interactive-interrupt ()
          (format t "~&Detaching.~%"))))))

(run)

Run:

sudo sbcl --load my-tracer.lisp

Key points

• Lifecycle: with-bpf-session manages the full BPF lifecycle. Maps, programs, and attachments are created on entry and cleaned up on exit. with-decoding-ring-consumer handles ring consumer cleanup even on Ctrl-C.

• bpf: prefix: bpf:map and bpf:prog are compiled at macroexpand time. bpf:attach and bpf:map-ref expand to runtime loader calls. Everything else is plain CL code that runs at load time.

• Package setup: whistler-loader-user is the intended default package for interactive work. It already imports the compiler and loader symbols with the right shadowing in place.

• Map access: *bpf-session* is a special variable bound during the session. Use bpf-session-map for named lookup:

  (bpf-session-map 'events)
  

  For simple integer reads, (bpf:map-ref map-name key) is more convenient. For struct-valued maps, use map-lookup-struct-int and map-update-struct-int with the generated record codecs.

Cgroup Packet Counter

Count egress packets for all processes in a cgroup using cgroup_skb/egress. Two approaches: standalone (compile to ELF) and inline (with-bpf-session).

Approach 1: Inline session

The simplest approach -- compile and load in one form:

(require :asdf)
(asdf:load-system "whistler/loader")
(use-package :whistler/loader)

(with-bpf-session ()
  (bpf:map pkt-count :type :array :key-size 4 :value-size 8 :max-entries 1)

  (bpf:prog count-egress
      (:type :cgroup-skb :section "cgroup_skb/egress" :license "GPL")
    (incf (getmap pkt-count 0))
    1)  ;; 1 = allow (SK_PASS for cgroup_skb)

  (bpf:attach count-egress "/sys/fs/cgroup")
  (format t "Counting egress packets on /sys/fs/cgroup...~%")
  (loop repeat 10
        do (sleep 1)
           (format t "  packets: ~d~%" (or (bpf:map-ref pkt-count 0) 0))))

The bpf:attach macro detects the cgroup_skb/egress section name and automatically calls attach-cgroup with +bpf-cgroup-inet-egress+.

Approach 2: Standalone ELF

Compile to an ELF file first, then load and attach separately.

BPF source

(in-package #:whistler)

(defmap pkt-count :type :array
  :key-size 4 :value-size 8 :max-entries 1)

(defprog count-egress
    (:type :cgroup-skb :section "cgroup_skb/egress" :license "GPL")
  (incf (getmap pkt-count 0))
  1)

(compile-to-elf "/tmp/cgroup-count.bpf.o")

Userspace loader

(asdf:load-system "whistler/loader")

(whistler/loader:with-bpf-object (obj "/tmp/cgroup-count.bpf.o")
  (let ((map (whistler/loader:bpf-object-map obj "pkt_count")))
    (whistler/loader:attach-obj-cgroup
     obj "count_egress" "/sys/fs/cgroup"
     whistler/loader:+bpf-cgroup-inet-egress+)

    (format t "Counting egress packets on /sys/fs/cgroup...~%")
    (loop repeat 10
          do (sleep 1)
             (let ((val (whistler/loader:map-lookup
                         map
                         (whistler/loader:encode-int-key 0 4))))
               (format t "  packets: ~d~%"
                       (if val (whistler/loader:decode-int-value val) 0))))))

Key points

• Return value: cgroup_skb programs return 1 to allow the packet (SK_PASS) or 0 to drop it. This counter always returns 1, so it observes without blocking traffic.

• Cgroup path: /sys/fs/cgroup is the root cgroup on cgroup2 systems, affecting all processes. Use a more specific path (e.g., /sys/fs/cgroup/user.slice/...) to monitor only certain processes.

• Auto-detection: The loader infers expected_attach_type automatically from the ELF section name (cgroup_skb/egress maps to +bpf-cgroup-inet-egress+). In the inline session, bpf:attach does this too -- no explicit constant needed.

• Cleanup: Both with-bpf-session and with-bpf-object detach the program and close file descriptors automatically on exit.

Running

Requires root (or CAP_BPF + CAP_NET_ADMIN):

sudo sbcl --load cgroup-counter.lisp

Expected output:

Counting egress packets on /sys/fs/cgroup...
  packets: 42
  packets: 97
  packets: 158
  ...

Cgroup Outbound Firewall

A process-level outbound firewall using three cooperating cgroup eBPF programs. Reimplements ebpf-cgroup-firewall (originally Go + C) in Whistler.

What it does

Traditional firewalls are IP-based and machine-wide. This firewall attaches to a cgroup, so it targets a single process or group of processes. It:

1. Intercepts outbound connections (cgroup/connect4) -- redirects DNS to a local proxy (for domain-level decisions) and HTTP/HTTPS to a transparent MITM proxy (for URL-level decisions).
2. Tracks socket-to-port correlation (sockops) -- maps source ports back to client socket cookies so egress can identify which connection a packet belongs to.
3. Enforces firewall rules (cgroup_skb/egress) -- checks every outbound packet against IP allowlists, emits events with PID, IP, port, and decision to a ring buffer for userspace logging.

Architecture

                    ┌──────────────────────────────────────┐
                    │          Userspace Loader             │
                    │  (populates maps, reads ring buffer)  │
                    └──────────┬───────────────────────────┘
                               │ ring buffer events
          ┌────────────────────┼────────────────────────┐
          │                    │                         │
   ┌──────┴───────┐    ┌──────┴───────┐    ┌────────────┴──────┐
   │  connect4     │    │  sockops     │    │  cgroup_skb/egress │
   │ (sock_addr)   │    │ (sock_ops)   │    │  (skb filter)     │
   │               │    │              │    │                    │
   │ Redirect DNS, │    │ Track src    │    │ Check allowlists,  │
   │ HTTP, HTTPS   │    │ port→cookie  │    │ emit events,       │
   │ to proxies    │    │ on connect   │    │ allow/deny packets  │
   └───────────────┘    └──────────────┘    └────────────────────┘
          │                    │                         │
          └────────────────────┼────────────────────────┘
                               │
                          Shared Maps
               (socket-pid, original-ip/port,
                src-port→cookie, allowlists)

BPF source

The complete source is in examples/cgroup-firewall.lisp. Key patterns demonstrated:

Multi-program coordination via shared maps

Seven maps connect the three programs. connect4 stores original destinations before redirecting; sockops stores port-to-cookie mappings; egress reads both to reconstruct the full picture:

;; socket cookie → original destination IP (before redirect)
(defmap sock-client-to-original-ip :type :hash
  :key-size 8 :value-size 4 :max-entries 262144
  :map-flags 1)  ; BPF_F_NO_PREALLOC

;; source port → client socket cookie (for proxy correlation)
(defmap src-port-to-sock-client :type :hash
  :key-size 2 :value-size 8 :max-entries 262144
  :map-flags 1)

Setf-able context access for connection redirection

ctx is a setf-able place for BPF context struct fields. The compiler resolves field names from the program type automatically — (ctx user-ip4) instead of (ctx u32 4). This is what makes transparent proxying work -- the application thinks it's connecting to the original destination, but the kernel sends the traffic to localhost:

;; Read the original destination
(let ((user-ip4 (ctx user-ip4)))
  ;; Redirect to localhost proxy
  (setf (ctx user-ip4) +localhost-nbo+)
  (setf (ctx user-port) (htons 8080)))

Built-in protocol headers

Cgroup/skb programs can't do direct packet access -- they must copy packet data onto the stack via skb-load-bytes, then read fields from the buffer. Whistler's built-in protocol headers (ipv4-*, tcp-*, udp-* from defheader) work on any pointer, including stack buffers. Port accessors auto-convert to host byte order, so comparisons use plain integers:

(let* ((pkt (make-pkt-buf))
       (rc (skb-load-bytes (ctx-ptr) 0 pkt +ipv4-hdr-len+)))
  (let ((protocol (ipv4-protocol pkt))
        (daddr    (ipv4-dst-addr pkt)))
    ;; Reuse buffer for transport header
    (skb-load-bytes (ctx-ptr) +ipv4-hdr-len+ pkt +udp-hdr-len+)
    (when (= (udp-dst-port pkt) 53)     ; host byte order, no htons needed
      ...)))

Events via ringbuf-output

Every firewall decision is reported to userspace. The event struct is built on the stack and copied to the ring buffer in a single helper call:

(defstruct event
  (pid         u32)
  (port        u16)
  (allowed     u8)
  (pad0        u8)
  (ip          u32)
  (original-ip u32)
  (event-type  u8)
  (pad1        u8)
  (dns-txid    u16)
  (pid-resolved u8)
  (redirected  u8)
  (pad2        u16))

(let ((evt (make-event)))
  (setf (event-pid evt) pid
        (event-port evt) (ntohs port)
        (event-allowed evt) 1
        (event-ip evt) (ntohl daddr)
        (event-event-type evt) +http-redirect+)
  (ringbuf-output events evt (sizeof event) 0))

This is more compact than with-ringbuf (which uses reserve+field-stores+submit) and matches the pattern clang generates from C's bpf_ringbuf_output().

Compiling

./whistler compile examples/cgroup-firewall.lisp -o firewall.bpf.o

This produces a single ELF with three program sections (cgroup/connect4, sockops, cgroup_skb/egress) and seven shared maps.

Key points

• Three program types: This example uses cgroup-sock-addr (to modify connection destinations), cgroup-sock (to observe socket events), and cgroup-skb (to filter packets). Each has a different context struct with different available fields.

• Setf-able context: Most cgroup programs only read context fields via (ctx field-name). cgroup/connect4 writes via (setf (ctx field-name) ...) to user-ip4 and user-port, redirecting connections -- this is what makes it a transparent proxy rather than just an observer.

• Runtime constants: The original Go implementation uses .rodata rewriting to set proxy ports, PID, and firewall mode at load time. The Whistler version uses placeholder values that should be set by the loader before attaching.

• IPv6: Not supported. IPv6 packets are blocked in egress and reported with event type +packet-ipv6+.

• Return values: cgroup/connect4 always returns 1 (allow the connection -- enforcement happens in egress). cgroup_skb/egress returns 1 to allow or 0 to drop the packet.

Comparison with the Go/C original