Container image with llama.cpp built for Vulkan on top of Mesa's Venus driver, for running AI models on the GPU inside a Lima krunkit VM on Apple Silicon.

This replaces quay.io/slopezpa/fedora-vgpu-llama, which pins a June 2025 llama.cpp and cannot load models using architectures added since (see lima-vm/lima#5349).

Build and test locally

Build inside a krunkit VM to get a native arm64 image. git is only needed inside the builder stage, so the VM itself does not need it.

limactl start --vm-type=krunkit --cpus 8 --memory 12 --name=llamabuild
limactl shell llamabuild

cd hack/krunkit-llama
nerdctl build --build-arg LLAMA_CPP_REF=master -t fedora-vgpu-llama:dev .

LLAMA_CPP_REF accepts any branch or tag. Use master while iterating and a release tag (for example b7000) for anything published.

Test with a model bind-mounted from the host:

nerdctl run --rm -ti --device /dev/dri -v ~/models:/models fedora-vgpu-llama:dev

# inside the container
vulkaninfo --summary
llama-cli -m /models/Qwen3-1.7B-Q8_0.gguf -b 512 -ngl 99 -p "Introduce yourself"

vulkaninfo must report Virtio-GPU Venus. If it reports llvmpipe instead, the patched Mesa from the COPR was not picked up and inference will run on the CPU.