Guix system container daemon
When developing system containers it is useful to be able to test them on the underlying host. Particularly when using AI agents. This little tool allows starting system containers through a network call. It validates the originating IP and whether this is a true container.
Still, make sure this only runs on systems that have limited functionality. I use it on my reduced AI box where the agents run in containers.
A daemon that starts Guix system containers (guix system container ...
launchers from /gnu/store) on request. You can run it by hand, but
it is meant to run as root from shepherd (Guix System) or systemd
(Debian). It refuses to start otherwise: check Privileges
required.
Network protocol
Line based, one request per connection:
START <hash|/gnu/store/HASH-run-container> [<fingerprint>]\n
→ OK <pid>\n container started (detached)
→ ERR <reason>\n rejected
KILL <pid>\n
→ OK killed <pid>\n SIGKILL sent to a container started by this daemon
→ ERR <reason>\n unknown pid, not ours, etc.
LIST\n
→ OK\n + one "<pid> <name>" line per running container
LOG <hash|/gnu/store/HASH-run-container>\n
→ OK\n + raw log contents (boot output of the container)
FINGERPRINT\n
→ OK <fingerprint>\n the daemon's fingerprint
The daemon runs at most one container at a time: START is
refused with ERR already running container <pid> while a previously
started container is still alive — KILL it (or --list to find it)
first.
The client run-guix-container.rb speaks this protocol:
./run-guix-container.rb --daemon 127.0.0.1:8989 HASH
GUIX_CONTAINER_DAEMON=127.0.0.1:8989 ./run-guix-container.rb HASH
./run-guix-container.rb --daemon 127.0.0.1:8989 --list
./run-guix-container.rb --daemon 127.0.0.1:8989 --kill PID
./run-guix-container.rb --daemon 127.0.0.1:8989 --log HASH
Inspect a running container (as root):
nsenter -at $PID /run/current-system/profile/bin/bash --login
Daemon fingerprint
Start the daemon with --fingerprint WORD, a short word or version
like v0.1:
run-guix-system-container --listen 127.0.0.1 --port 8989 --fingerprint v0.1
If no fingerprint is given, the daemon generates a random hash at startup. The fingerprint makes sure everyone talks to the daemon (and thus the containers) they expect:
- clients can query it (
FINGERPRINTcommand) or pass the expected fingerprint withSTART <hash> <fingerprint>— a mismatch is refused withERR fingerprint mismatch - the client has
--fingerprint WORD, which verifies the daemon viaFINGERPRINTbefore doing anything else - every started container receives the fingerprint in its
GUIX_CONTAINER_FINGERPRINTenvironment variable; a container OS can show it at login (e.g. write it to/etc/issue), so an ssh session immediately reveals which daemon generation started it
Shutdown behaviour
On SIGINT or SIGTERM the daemon terminates the containers it started: SIGTERM to all of them, wait up to 10 seconds for them to exit, then SIGKILL any survivors, and exit. Containers started by a previous daemon instance are not tracked and not touched.
Access control
The daemon listens on one address only, given by --listen IP
(default 127.0.0.1, i.e. loopback — only local clients can reach
it). Pass an explicit address to expose it further, e.g.
--listen 0.0.0.0 for all IPv4 interfaces, or a specific address such
as --listen 192.168.1.10. IPv6 addresses work too (--listen ::1).
There is no per-client ACL: whoever can reach the listening address may start containers. Restrict access at the network level (bind to a dedicated interface, firewall the port, or use a private network between the host and its containers).
Every request is logged to stderr with a timestamp and client address.
Validation
Before a container script is exec'd, the daemon verifies:
- the name resolves to exactly
HASH-run-container(32-char nix-base32 hash, unique in the store) - the file is a direct child of
/gnu/store, executable, and immutable (no write bits — Guix store items are0555root-owned) - the file is structurally a generated Guix system container launcher:
- first line is a guile shebang
(
#!/gnu/store/...-guile-.../bin/guile ...) — this also rules out ELF binaries, which start with\x7fELF - it calls
call-with-container, the container entry point - it sets
GUIX_NEW_SYSTEM, as generated system containers do - it references
linux-container
A regular binary, an ELF file, or a script that merely mentions these strings but is not a generated launcher will not pass.
Build & install
cd bin
gcc -O2 -Wall -o run-guix-system-container run-guix-system-container.c
# Debian:
sudo install -o root -g root -m 0755 run-guix-system-container /usr/local/bin/
# Guix System: add it to a channel/package or keep it in a root-owned
# directory, e.g. /run/current-system/profile/bin/
# Run the unit tests:
cd ../test && ./run.sh
Privileges required
Container creation needs real root on the host: CAP_SETUID (to
write the container's 65536-entry uid map) and CAP_SYS_ADMIN (to
mount its file systems, /proc and /dev). Without them every
START succeeds in forking a launcher that dies within milliseconds
and leaves nothing behind: the client sees OK <pid> and then a
LIST that comes back empty.
So the daemon checks at startup and exits with status 1 when
geteuid() != 0 or when CapEff from /proc/self/status lacks
CAP_SETUID or CAP_SYS_ADMIN (root in a unit with a narrowed
CapabilityBoundingSet=), e.g.:
error: run-guix-system-container must run as root (uid 1000): container
creation needs CAP_SETUID and CAP_SYS_ADMIN, without them every
START fails
Container boot output goes to /var/log/guix-containers, or wherever
--log-dir DIR (or $GUIX_CONTAINER_LOG_DIR) points -- the flag wins over
the variable, and the default above applies when neither is given. If that
directory cannot be created or written (read-only /var, hardened unit, a
daemon that is not system-wide) the daemon only warns -- containers still
run -- but a boot failure then leaves no log to read. With
ProtectSystem=strict either point --log-dir at a writable path or set
LogsDirectory=guix-containers (equivalently
ReadWritePaths=/var/log/guix-containers).
Running it
systemd (Debian)
Basic unit, /etc/systemd/system/guix-container-daemon.service:
[Unit]
Description=Guix system container launcher daemon
After=network.target
[Service]
ExecStart=/usr/local/bin/run-guix-system-container --listen 127.0.0.1 --port 8989 --fingerprint v0.1
Restart=on-failure
[Install]
WantedBy=multi-user.target
A hardened variant — the daemon only needs to read the store, listen
on a socket, fork and exec, so most systemd sandboxing can be enabled
(note: ProtectSystem=strict would hide /gnu/store unless made
read-only-visible, which is the default anyway — keep it read-only):
[Unit]
Description=Guix system container launcher daemon
After=network.target
[Service]
ExecStart=/usr/local/bin/run-guix-system-container --listen 127.0.0.1 --port 8989 --fingerprint v0.1
Restart=on-failure
# Logging goes to stderr; let journald capture it.
StandardOutput=journal
StandardError=journal
# The daemon forks detached containers (setsid), which must survive a
# daemon restart — do not use KillMode=control-group here.
KillMode=process
# Sandbox: the daemon needs /gnu/store (read-only), /dev/null, the
# socket, and nothing else writable.
ProtectSystem=strict
ReadOnlyPaths=/gnu/store
# Container boot logs; without this /var/log is read-only and the
# daemon can only warn that boot output will be lost.
LogsDirectory=guix-containers
ProtectHome=true
PrivateTmp=true
NoNewPrivileges=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_INET AF_INET6
CapabilityBoundingSet=CAP_SETUID CAP_SETGID CAP_SYS_ADMIN
RestrictSUIDSGID=true
# Network filtering (cgroup BPF): only allow loopback traffic, deny
# everything else — a second layer on top of --listen 127.0.0.1.
IPAddressDeny=any
IPAddressAllow=localhost
# Optionally log traffic from/to the unit:
IPAccounting=yes
[Install]
WantedBy=multi-user.target
Enable and watch the log:
sudo systemctl daemon-reload
sudo systemctl enable --now guix-container-daemon
sudo journalctl -u guix-container-daemon -f # watch the log
systemd network filtering
Beyond --listen, systemd can restrict the unit's network at the
cgroup level with IPAddressAllow= / IPAddressDeny= (enforced with
BPF, applies to both directions, and to anything the unit's processes
spawn — including the started containers, since they are children of
the unit's cgroup):
# allow only loopback, deny everything else
IPAddressDeny=any
IPAddressAllow=localhost
Rules accept addresses, prefixes (192.168.1.0/24), and the special
values any and localhost. Order does not matter; the most
specific match wins. Combined with RestrictAddressFamilies= this
gives belt-and-braces isolation: even a bug that made the daemon bind
0.0.0.0 could not be reached from outside.
Two related options:
PrivateNetwork=yes— gives the unit its own network namespace; then nothing, not even the host's loopback, can reach it except via a veth/VRF setup. Usually too strong for this daemon.IPAccounting=yes— counts bytes in/out per unit, useful for auditing.
Note: containers started by the daemon inherit the unit's cgroup and
thus its IPAddress rules — if containers need real network access,
keep the rules on the daemon minimal or move started containers to a
scope with Delegate= semantics.
shepherd (Guix System)
Add a service to the OS configuration:
(simple-service 'guix-container-da shepherd-root-service-type
(list (shepherd-service
(provision '(guix-container-daemon))
(requirement '(user-processes))
(start #~(make-forkexec-constructor
(list #$(file-append my-tools "/bin/run-guix-system-container")
"--listen" "127.0.0.1"
"--port" "8989"
"--fingerprint" "v0.1")
#:log-file "/var/log/guix-container-daemon.log"))
(stop #~(make-kill-destructor)))))
Worked example: a service container other programs talk to
The pattern that matters in practice is a container whose services the host
then uses over loopback. --network shares the host's network namespace, so a
service listening on a port inside the container is reachable at
127.0.0.1:PORT on the host -- no port forwarding and no NAT. (The daemon's
own --port is a different thing: it is the socket you send START to.)
Build first, start second. guix system container ... builds and runs the
container in the foreground, which is not what you want when a daemon is going
to own it:
# 1. get the launcher's derivation, then build the launcher
guix system container -d --network --share=/srv/share config.scm # -> /gnu/store/HASH-run-container.drv
guix build /gnu/store/HASH-run-container.drv # -> /gnu/store/HASH-run-container
# 2. let the daemon start it (it validates the launcher, detaches it, and
# enforces one container at a time)
./run-guix-container.rb --daemon 127.0.0.1:8989 CONTAINER_HASH
# 3. it is now running; talk to its services on the host
curl http://127.0.0.1:PORT/...
--share=HOSTDIR mounts a host directory inside the container at the same
path. A --share whose host path does not exist at container start is
dropped silently -- Guix does not warn -- so create the directory first, and if
a container cannot see a shared path, check it exists before looking at the
container.
Two timings worth knowing when testing (measured on a Slurm cluster container,
2026-11-12): a container reaches its services in about 2 s from START
when the store paths are already built, and about 45 s on its first boot,
where activation still has to generate keys and lay out its spool and log
directories. If a test client gives up after a few seconds, it is measuring the
first case and losing against the second.
Boot output is the way to see what a container is doing: LOG HASH over the
protocol, --log HASH from the client, or /var/log/guix-containers/<pid>.log
on the host (which the daemon warns it cannot write without
LogsDirectory=/ReadWritePaths=, see the unit files below).
Containers and VMs
Container definitions live in services/, VM definitions in vms/.
All of them provide ssh
and are minimal (hello + ssh server). Containers share the host's
whole /gnu/store read-only; the VM image contains only the closure of
the system, so it exposes no host file systems at all.
Containers are built and started like this:
guix system container --network services/hello-container.scm
./run-guix-container.rb --daemon 127.0.0.1:8900 <HASH>
The VM is built as a disk image and run under qemu (user-mode networking forwards host port 2223 to guest ssh port 22):
guix system image -t qcow2 vms/hello-vm.scm
cp /gnu/store/...-image.qcow2 /tmp/vm.qcow2 && chmod u+w /tmp/vm.qcow2
qemu-system-x86_64 -m 1024 -drive file=/tmp/vm.qcow2,format=qcow2 \
-netdev user,id=n0,hostfwd=tcp:127.0.0.1:2223-:22 \
-device virtio-net-pci,netdev=n0 -nographic
Definition Description
hello-container.scm Bare-bones test container running 'hello' with an ssh server on port 2222. Root login with etc/hello-container-key (host key pinned from etc/ssh-container-key, so the identification never changes). Smoke test: ssh -p 2222 -i etc/hello-container-key root@127.0.0.1 then run /run/current-system/profile/bin/hello.
debug-container.scm Same OS as hello-container, plus the container fingerprint service (shows the starting daemon generation in /etc/issue) and a DBG2 monitor that dumps ps, herd status and syslog into the container log every 20s. Use this to debug boot problems; its host-name is "debug-container".
hello-vm.scm The same minimal OS as a SELF-CONTAINED qcow2 VM. Unlike the containers, the VM's /gnu/store holds only this system's closure (~260 items, not the whole host store) and nothing from the host is mounted. SSH on guest port 22, forwarded to host port 2223. Serial console on ttyS0 for debugging; root password "hello". Host SSH keys are generated at first boot and persist in the (writable) image.
The packages field is slim on purpose: just bash,
coreutils, findutils, grep, sed, gawk, tar, gzip,
shadow, procps, guile and hello (%base-packages would
add ~400 binaries: nano, man-db, sudo, inetutils with
its ftp and telnet, wget, pciutils, ...). ssh tools
come in via the openssh service, 'herd' via the
shepherd service, and the guix service is deleted.
BNW (Bayesian Network Webserver) is also deployable through this daemon:
bnw-container.scm modernized container: nginx (foreground
service), php-fpm, BNW with GN2 handshake
(XSS-fixed); web port 8881, ssh 2222
vms/bnw-vm.scm the same as a self-contained qcow2 VM
(2 GiB RAM suggested); web 8881, ssh 22
Why a privileged daemon? (and the rootless alternative)
guix system container needs real root on the host:
- mount/PID/network namespaces are created with real root (no uid mapping involved),
--networkneedsCAP_NET_ADMINto set up the interface,- a shepherd runs as root inside the container.
A sandboxed (rootless) agent cannot provide any of that, which is
precisely why this daemon exists: root listens on a socket and the
sandbox sends START/KILL/LIST/LOG. Note that fixing AppArmor on the
host (e.g. adding the k permission for the guix-daemon//guix-builder
profile) only unblocks the build daemon's builders — it does not
change this privilege boundary.
Rootless alternative for smoke tests. guix shell -C runs entirely
on unprivileged user namespaces and works from an unprivileged
account. Two conditions apply:
- unprivileged user namespaces must be allowed — check
sysctl kernel.apparmor_restrict_unprivileged_userns; on systems where it is1(Ubuntu 24.04 and newer) each program needs an AppArmor profile before it can create user namespaces. Where it is0or unset, nothing extra is needed; - the profile must live in a writable place — a bare
guix shell -Cfails when/var/guix/profilesis read-only. Use an explicit profile:
mkdir -p /tmp/rootless-test
guix package -p /tmp/rootless-test/profile -i coreutils # built via the host daemon
guix shell -C -p /tmp/rootless-test/profile -- /bin/sh -c 'id'
Inside such a container you are still yourself (your own uid, e.g.
uid=1000), there
is no shepherd, no sshd and no guix system services — but a stack of
foreground daemons (nginx + php-fpm + octave, ports > 1024, data via
--share=...) runs fine for fast zero-privilege smoke tests.
Use both as they fit: keep the daemon for deployment-grade testing of
real guix system environments (service management, ssh, host keys,
production parity), and use the rootless shell path for quick,
privilege-free checks straight from the sandbox.
Other options researched:
- an unprivileged guix-daemon (Guix ≥ 2025, "build daemon drops its privileges", see Build Environment Setup in the manual) removes the need for a root daemon for builds — irrelevant when a root daemon is already running, and it does not help with running system containers;
- rootless Podman also runs containers as an unprivileged user
(user namespaces + subordinate UID/GID service) — but gives
Docker-style isolation, not
guix systemsemantics, and needs subuid/subgid setup; - PRoot is the pre-userns fallback (ptrace-based, slow) — only relevant on kernels without user namespace support.
Files
bin/run-guix-system-container.c— the daemon.bin/run-guix-container.rb— unprivileged client; validates the hash locally before talking to the daemon.test/— unit tests for the container-launcher and daemon fingerprint logic (test/run.sh).
Licence
BSD 3-Clause -- see LICENSE.
