These path based syscalls are not in kTracedSyscalls, and the generated policy
ends in DEFAULT ALLOW, so they run unobserved and never reach the report.
Measured against the report of a run under --seccomp_unotify, using the glibc
wrappers rather than raw syscall numbers, on a file and on a symlink to it:
truncate() shrank the file from 21 to 3 bytes inside the jail, verified from
outside. The report attributed no truncate to that path.
setfattr set an attribute on a file inside the jail; the attribute was still
there after the run, and the report attributed nothing to that path.
glibc faccessat(AT_EACCESS) produced no entry at all, and no faccessat
appeared either, so glibc had issued faccessat2.
faccessat2 is the same case as the statx entry added in c20246a: modern glibc
reaches for it first and only falls back to the traced faccessat on older
kernels. A plain access(2) is still traced by its own entry; what goes unseen
is the check made through faccessat().
truncate changes a file's contents without opening it, so nothing in the report
marks that path as written.
The l* variants are listed separately, the same way stat and lstat are, and are
exercised through a symlink so that all eleven entries are covered.
The flags argument of faccessat2 is left undecoded, the same as every other
AT_* entry in the table.
buildKafelPolicy() lists the kafel_name of every table entry. kafel resolves
those names from a per-architecture table, and those tables are not equally
complete, so a name that resolves on one architecture can be unknown on
another. kafel_compile() then fails, preparePolicy() returns false, and nsjail
does not start at all. Reproduced on amd64 by replacing one kafel_name with a
name kafel does not know:
[E] preparePolicy():160 Could not compile the default unotify seccomp policy:
3:275: Undefined identifier `no_such_syscall'
[F] main():386 Couldn't prepare sandboxing policy
exit 255
No report file is written; the process never runs.
Two entries are in that position today. Checked by taking __NR_ from the cross
compiler for each target and looking each kafel_name up in that target's kafel
table, counting only the entries the #ifdef keeps:
amd64 38 kept, 0 skipped, none unknown to kafel
aarch64 23 kept, 15 skipped, none unknown to kafel
riscv64 22 kept, 16 skipped, none unknown to kafel
i386 37 kept, 1 skipped, openat2 unknown to kafel
arm 37 kept, 1 skipped, openat2 and statx unknown to kafel
openat2 has been in the table since ee2d05e and statx since c20246a. m68k and
the mips targets were not checked, no cross compiler at hand. debian/control
says Architecture: any, so i386 and armhf are built.
The syscall number is a compile time constant of the architecture being built
for, so emitting it removes the dependency on how complete kafel's name table
happens to be. kafel accepts SYSCALL[<nr>] (parser.y) and C style comments
(lexer.l), so the name stays in the generated policy as a comment and keeps it
readable.
Measured on amd64 with one workload and a fresh build per form: the name based
policy, the numeric one, and the numeric one with the comment produce identical
syscall distributions in the report. With the numeric form a kafel_name that
kafel does not know no longer prevents startup, and the syscall is still
reported under its display_name.
Cap retained guest-to-host TCP payload across flows so a stalled sink cannot grow the supervisor toward the per-flow limit times the maximum flow count.
Add statx to the seccomp user-notify traced-syscall table
(unotify/syscall_defs.h::kTracedSyscalls). That table is the single source
of truth -- the kafel policy string and the argument decoder are both
derived from it -- so this one entry is the whole change.
Modern glibc/coreutils (ls, stat) issue statx before falling back to
newfstatat/openat when probing a path. Because statx was not traced, a
stat of a path that is not present in the jail's mount namespace was never
observed at all: under a restrictive mount set, `ls /opt` and `stat /opt`
produced no access record, while openat-based reads (cat, head) were
recorded normally. Tracing statx closes that gap.
statx(dirfd, path, flags, mask, statxbuf) has arg0 = dirfd and arg1 =
path, the same shape as newfstatat, so it reuses the existing DIRFD/PATH
arg roles. Guarded by #ifdef __NR_statx (defined on x86_64 and arm64).
kafel already knows the statx syscall, so no policy-side change is needed.
Tested by running a command under --seccomp_unotify with a mount set that
excludes /opt. Before: `stat /opt` and `ls /opt` did not appear in the
report. After: /opt appears with exists_in_jail:false, exists_in_main:true
via a statx syscall, while a genuinely missing path still reports
exists_in_main:false and an allowlisted path still reports
exists_in_jail:true.
An expected EPERM after denying setgroups in a new user namespace can still leave inherited supplementary groups attached to the child. Query the remaining group count and abort before exec unless the credential state is actually clean.
Add regression coverage for both an inherited-group failure and a clean no-groups launch.
applyPolicy() installed the seccomp-unotify listener only when pipefd != -1.
In modes that pass pipefd == -1 (e.g. MODE_STANDALONE_EXECVE) the install was
silently skipped, and prepareAndCommit() then returned early because the classic
seccomp filter is empty -- so a sandboxee for which seccomp_unotify was
explicitly requested ran with NO seccomp policy at all, with no error emitted.
Refuse to continue instead of silently dropping the requested policy: if
seccomp_unotify is set but there is no supervisor to receive the notification fd
(pipefd == -1), log an error and fail.
Verified: `nsjail -Me --seccomp_unotify --seccomp_string 'DEFAULT ALLOW' -- ...`
now aborts with a clear error instead of launching the process unfiltered.
The guest is untrusted and fully controls the acknowledgement number of
every TCP segment it emits. tcp_process_data() accepted any forward ACK
(acked_bytes > 0) without checking that it stayed within the send window,
so the guest could acknowledge data that was never sent.
Each such ACK advances tx_acked_offset by up to 2^31-1. When tx_buffer is
empty the erase step below is a no-op, so tx_acked_offset accumulates and
can be driven past 2^32 with a few pure ACK segments. push_to_guest() then
computes:
int32_t available = tx_buffer.size() - tx_acked_offset;
const uint8_t* data = tx_buffer.data() + tx_acked_offset + in_flight;
available is an int32_t truncation of a size_t subtraction, so an oversized
tx_acked_offset wraps it back positive and defeats the in_flight >= available
guard. data then points at least 2 GB past the heap buffer and
tcp_send_packet() reads up to NSTUN_MTU bytes from it.
The nstun network loop runs in the nsjail parent (supervisor) process, and
SIGSEGV is not handled, so the out of bounds read lets an untrusted jailed
process crash its own supervisor. The minimum out of bounds distance is 2 GB,
so under ASLR the access reliably faults rather than returning data.
Add the RFC 793 receive check (SEG.ACK <= SND.NXT): reject ACKs whose
sequence is beyond seq_to_guest. This keeps tx_acked_offset within
tx_buffer.size(), which the framing in push_to_guest() relies on.
Adds --use_core_scheduling proto flag that calls prctl(PR_SCHED_CORE_CREATE) in the child process, giving each jail its own SMT scheduling group. Prevents sharing a physical core with other tenants and mitigates L1TF/MDS side-channel attacks in multi-tenant environments. Requires Linux >= 5.14; guarded with #ifdef PR_SCHED_CORE for compatibility with older kernel headers.
The protobuf API marks PrintToString as nodiscard, so ignoring its
return value broke the build under -Werror=unused-result. Check the
result and bail out with a warning if formatting fails.
setupArgv() unconditionally called set_path(argv[optind]) when positional
arguments were present after --, clobbering any path previously set via
the -x / --exec_file flag. The intended behavior, documented in the help
text ("File to exec (default: argv[0])"), is for --exec_file to win and
fall back to argv[0] only when not given.
Remove the unconditional set_path; the existing fallback a few lines
below already sets the path from argv[0] when --exec_file was not
provided.
ENOENT means the thread has moved on (killed or interrupted).
EINPROGRESS indicates either misuse by sending again after
a successful send (which we're not doing) but we also see it
when a process is exiting, perhaps related to thread shutdown.
Authored-by: rsc@google.com
Tested-by: rsc@google.com
Most of the system calls involved in the unotify poll loop were subtly wrong.
The biggest problem was the reuse of the previous message's req.id with
isTargetAlive to decide whether to exit the loop. The loop can be watching
many threads and many processes, any one of which might exit at any time
(especially if the last observed system call was exit!), so it does not make
sense to focus on a specific thread except within the context of a pending
syscall being evaluated.
SECCOMP_IOCTL_NOTIF_ID_VALID is for one purpose and one purpose only:
checking after an access by pid that the pid was not reused, invalidating
whatever was just read. The only time that purpose applies is between
parseSyscall and addStat, which is now the only time that the loop calls
isTargetAlive.
The loop was misusing isTargetAlive to decide when to exit the loop.
Now it correctly waits until a POLLHUP event.
The loop was misusing isTargetAlive to decide whether to call
SECCOMP_IOCTL_NOTIF_SEND. That's going to do the same check itself,
because otherwise there would be a race between the check and the send.
That redundant check is deleted.
The loop was also misusing isTargetAlive after a failed
SECCOMP_IOCTL_NOTIF_SEND to decide whether to exit the loop.
As before, one target being dead does not imply that all the traced targets
are dead, so that check is deleted. On failure, if the error is EINTR,
the send is tried again. If the error is ENOENT, that means the thread is
no longer blocked in the system call (either due to an interrupt or signal),
so we stop the send attempt but continue the processing loop.
Otherwise we print the error error, stop the send, but continue the processing loop.
The hangs seem to have been caused by a few different failure modes:
1. The loop could have been exiting prematurely, hanging the syscalls
waiting for judgement.
2. In one strace log I read carefully, the loop kept running after POLLHUP,
hanging in SECCOMP_IOCTL_NOTIF_RECV. I believe this may have been
fixed between 6.6 and 6.12.
3. The loop was not reacting well at all to send failing because the syscall
had been interrupted. This manifested as needing to run Go programs
with GODEBUG=asyncpreemptoff=1 to let them run at all.
With these changes, Go programs using signals work just fine.
Authored-by: rsc@google.com
Tested-by: rsc@google.com
initParent registers a defer{nl_cache_free(link_cache)} RAII guard that
runs on all exit paths, but the error returns inside the iface_own loop
and the cloneIface block also called nl_cache_free explicitly, causing
a double-free when moveToNs or cloneIface fails.
Remove the redundant explicit frees; the defer guard is sufficient.
When nsjail creates a new process in a new PID namespace (CLONE_NEWPID) using the direct kernel syscall clone/clone3 (introduced in d1f332b), glibc's internal PID/TID cache is not updated for the child process.
As a result, calling the glibc wrapper `sched_setaffinity(0, ...)` inside the child process causes glibc to inadvertently pass the cached parent's TID to the kernel instead of 0 (current thread). Since the parent's TID does not exist within the new PID namespace, the kernel returns ESRCH (No such process).
This commit fixes the issue by bypassing the glibc wrapper and invoking the `sched_setaffinity` syscall directly via `util::syscall`. This ensures that `0` is passed accurately to the kernel, referring to the current thread.
