Wrong seccomp BPF architecture when kernel and userland are running on different architectures

[PiotrSikora]

Describe the bug

Let me preface this by saying that I understand that this is an edge case, and maybe there is no desire to ever support such environment, but I find it quite interesting, so perhaps it's worth fixing.

I've been playing with apple/container lately, and one interesting thing about it is that regardless of the selected container architecture, the Linux VM is always running natively as aarch64, and if the userland is built for x86_64, then the individual processes are started under Rosetta emulation, thanks to the binfmt registration.

This unfortunately results in some issues when running nixos/nix containers for x86_64 architecture.

One such issue is that the sandbox builder is trying to load seccomp BPF for the architecture of the userland (as determined at the build-time), and not that of the running kernel.

See: src/libstore/unix/build/linux-derivation-builder.cc#L40

Steps To Reproduce

Start NixOS/nix container for x86_64 architecture using apple/container on macOS:

% container run -it --rm --arch x86_64 nixos/nix:2.33.2

Setup build environment (fails):

# nix develop nixpkgs#hello
error:
       … while setting up the build environment

       error: unable to load seccomp BPF program: Invalid argument
#

Setup build environment without seccomp BPF (works):

# nix --no-filter-syscalls develop nixpkgs#hello
#

Expected behavior

nix develop build environment starts with seccomp BPF without any issues.

Metadata

# nix-env --version
nix-env (Nix) 2.33.2

Additional context

The container contains userland built for x86_64, which is running as x86_64 under Rosetta emulation:

# uname -m
x86_64

Nix assumes that the whole system is x86_64:

# nix config show system
x86_64-linux

But the kernel is really running on aarch64:

# nix-shell --quiet -p sysctl --run 'sysctl -n kernel.arch'
aarch64

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[xokdvium]

I assume this would also surface with qemu-user?

[PiotrSikora]

Correct.

[xokdvium]

It might make sense to run some functional tests with qemu-user with a cross build of nix once this bug is fixed.