Fails to boot any VM due to panic in hostagent

[s1gnate-sync]

Description

Bug
In 90% cases lima start NAME fails to boot VM due to panic in hostagent in attempt of creating iso file (via go-diskfs package). It doesn't matter if I clean cache or remove whole lima dir.

Env

• MacOS 15.1.1 on Apple Silicon
• lima installed via brew (limactl version 1.0.4, go-diskfs version 1.4.1)
• Using unmodified
• No special changes made to configuration/templates

Steps

• Create VM
• Start VM

Obviously I expect it to start, not to panic...

Log sample

~>limactl start rekcod --debug
DEBU[0000] interpreting argument "rekcod" as an instance name
INFO[0000] Creating an instance "rekcod" from template://default (Not from template://rekcod)
WARN[0000] This form is deprecated. Use `limactl create --name=rekcod template://default` instead
INFO[0000] debug mode detected, adding more guest agent candidates: /opt/_output/share/lima/lima-guestagent.Linux-aarch64
DEBU[0000] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/rekcod/vz-identifier")
INFO[0000] Using the existing instance "rekcod"
DEBU[0000] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/docker/vz-identifier")
DEBU[0000] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/podman/vz-identifier")
DEBU[0000] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/random/vz-identifier")
DEBU[0000] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/rekcod/vz-identifier")
DEBU[0000] Make sure "user-v2" network is stopped
DEBU[0000] Make sure usernet network is stopped
DEBU[0000] Make sure "shared" network is stopped
DEBU[0000] Make sure "bridged" network is running
DEBU[0000] Make sure "host" network is stopped
INFO[0000] Starting the instance "rekcod" with VM driver "vz"
DEBU[0000] [hostagent] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/user/.lima/rekcod/vz-identifier")
INFO[0000] [hostagent] debug mode detected, adding more guest agent candidates: /opt/_output/share/lima/lima-guestagent.Linux-aarch64
DEBU[0000] [hostagent] Creating iso file /Users/user/.lima/rekcod/cidata.iso
DEBU[0000] [hostagent] Using /var/folders/y7/xf8my40d7k931ln6fr03tgd80000gn/T/com.apple.shortcuts.mac-helper//diskfs_iso3036248355 as workspace
INFO[0000] [hostagent] panic: runtime error: invalid memory address or nil pointer dereference
INFO[0000] [hostagent] [signal SIGSEGV: segmentation violation code=0x2 addr=0xf8 pc=0x101148304]
INFO[0000] [hostagent] goroutine 1 [running]:
INFO[0000] [hostagent] github.com/diskfs/go-diskfs/filesystem/iso9660.walkTree.func1({0x14000173f80, 0x69}, {0x101a91768
INFO[0000] [hostagent] 0x14000163240}, {0x0?,
INFO[0000] [hostagent] 0x0?})
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/diskfs/go-diskfs@v1.4.1/filesystem/iso9660/finalize.go:948 +0x344
INFO[0000] [hostagent] path/filepath.walkDir({0x14000173f80, 0x69}, {0x101a91768, 0x14000163240}, 0x1400035cc90)
INFO[0000] [hostagent] 	/opt/homebrew/Cellar/go/1.23.5/libexec/src/path/filepath/path.go:310
INFO[0000] [hostagent] path/filepath.walkDir({0x14000192380, 0x65}, {0x101a917a0, 0x140003a7530}, 0x1400035cc90)
INFO[0000] [hostagent] 	/opt/homebrew/Cellar/go/1.23.5/libexec/src/path/filepath/path.go:332 +0x21c
INFO[0000] [hostagent] path/filepath.WalkDir(
INFO[0000] [hostagent] 0x65}, 0x1400035cc90)
INFO[0000] [hostagent] 	/opt/homebrew/Cellar/go/1.23.5/libexec/src/path/filepath/path.go:400 +0x78
INFO[0000] [hostagent] github.com/diskfs/go-diskfs/filesystem/iso9660.walkTree({0x14000192380, 0x65}
INFO[0000] [hostagent] )
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/diskfs/go-diskfs@v1.4.1/filesystem/iso9660/finalize.go:917 +0x98
INFO[0000] [hostagent] github.com/diskfs/go-diskfs/filesystem/iso9660.(*FileSystem).Finalize(0x140000203f0, {0xf0?
INFO[0000] [hostagent] 0x3?, 0x0
INFO[0000] [hostagent] ?, {0x10156b940?, 0x101a80c58?}})
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/diskfs/go-diskfs@v1.4.1/filesystem/iso9660/finalize.go:480 +0x240
INFO[0000] [hostagent] github.com/lima-vm/lima/pkg/iso9660util.Write({0x14000044ea0, 0x23}, {0x10156b940, 0x6
INFO[0000] [hostagent] , {
INFO[0000] [hostagent] )
INFO[0000] [hostagent] 	/private/tmp/lima-20250121-4853-uuipyh/lima-1.0.4/pkg/iso9660util/iso9660util.go:57 +0x290
INFO[0000] [hostagent] github.com/lima-vm/lima/pkg/cidata.GenerateISO9660({0x1400029a060, 0x18
INFO[0000] [hostagent] 0x16f433800?, 0x140000021c0?}, 0x140002a4008, 0x100a429ac?, 0x1400035d718?, {0x0, 0x0
INFO[0000] [hostagent] }, 0x10
INFO[0000] [hostagent] )
INFO[0000] [hostagent] 	/private/tmp/lima-20250121-4853-uuipyh/lima-1.0.4/pkg/cidata/cidata.go:432 +0x9dc
INFO[0000] [hostagent] github.com/lima-vm/lima/pkg/hostagent.New
INFO[0000] [hostagent] {0x16f433800, 0x6}, {0x101a80cb8, 0x14000292000}, 0x140005740e0, {0x0, 0x0, 0x0?
INFO[0000] [hostagent] )
INFO[0000] [hostagent] 	/private/tmp/lima-20250121-4853-uuipyh/lima-1.0.4/pkg/hostagent/hostagent.go:141 +0x298
INFO[0000] [hostagent] main.hostagentAction(
INFO[0000] [hostagent] 0x1, 0x101567de5?})
INFO[0000] [hostagent] 	/private/tmp/lima-20250121-4853-uuipyh/lima-1.0.4/cmd/limactl/hostagent.go:85 +0x4e0
INFO[0000] [hostagent] github.com/spf13/cobra.(*Command).execute(0x140005dcf08, {0x140005e0300, 0x6, 0x6})
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:985 +0x834
INFO[0000] [hostagent] github.com/spf13/cobra.(*Command).ExecuteC(0x14000542c08)
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x344
INFO[0000] [hostagent] github.com/spf13/cobra.(*Command).Execute(...)
INFO[0000] [hostagent] 	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041
INFO[0000] [hostagent] main.main(
INFO[0000] [hostagent] :28 +0x20
FATA[0000] host agent process has exited: exit status 2

[s1gnate-sync]

I actually have solution, but I just not sure if it makes sense to fix it here, but it's just silly one line...

The bug depends on the parent process in which under various conditions special variable DIRHELPER_USER_DIR_SUFFIX is set. And if it's set to specific value it breaks limactl.

# started terminal.app
~>go env GOGCCFLAGS
-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/y7/xf8my40d7k931ln6fr03tgd80000gn/T/go-build966448874=/tmp/go-build -gno-record-gcc-switches -fno-common

# started alacritty which I installed via brew
~>go env GOGCCFLAGS
-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/y7/xf8my40d7k931ln6fr03tgd80000gn/T/com.apple.shortcuts.mac-helper//go-build2936258725=/tmp/go-build -gno-record-gcc-switches -fno-common
~>

The problematic spot is "...lper//go-bui..." which I believe must be tolerated by everyone, who cares about doubled sep., but it does break go-diskfs..

If I patch limactl like so everything works like a breeze.

index 74ef0bbe..2f739c93 100644
--- a/pkg/iso9660util/iso9660util.go
+++ b/pkg/iso9660util/iso9660util.go
@@ -37,6 +37,7 @@ func Write(isoPath, label string, layout []Entry) error {
         // go-embed unfortunately needs unix path
         workdir = filepath.ToSlash(workdir)
     }
+    workdir = filepath.Clean(workdir)
     logrus.Debugf("Creating iso file %s", isoFile.Name())
     logrus.Debugf("Using %s as workspace", workdir)
     fs, err := iso9660.Create(isoFile, 0, 0, 0, workdir)

WDYT? We can spawn author of faulty package but it seems like we shouldn't trust system to much and it's better to have some sort of validation

[jandubois]

Please file a bug report (or even better a PR) upstream. If it looks like there is no updated release forthcoming, then we can add the call in Lima with a comment pointing to the upstream bug. But we should always try to get issues resolved upstream before we apply workarounds.

[afbjorklund]

There is a similar segfault on Windows, in the same library (in that case when using unexpected path separators):

• Remove unnecessary platform specific workaround in iso9660util #3149

[s1gnate-sync]

@afbjorklund @jandubois I fixed it in upstream diskfs/go-diskfs#279

But be ready lima would require additional changes to make things work as in 1.5.0 version signature for iso9660.Create function has changed and now expects backend.Storage instead of os.File. Lime currently is on 1.4~ something.

[s1gnate-sync]

Keeping everyone in the loop - here is a preparation for a fix - #3187

[s1gnate-sync]

FYI PR for fix diskfs/go-diskfs#280

[s1gnate-sync]

THe fix has been merge to master so this issue should disappear

[s1gnate-sync]

@AkihiroSuda what's the next steps after fixing upstream dependency? Do you need to release new version of lima which would use updated go-diskfs?

[AkihiroSuda]

@AkihiroSuda what's the next steps after fixing upstream dependency? Do you need to release new version of lima which would use updated go-diskfs?

Yes, after go-diskfs makes a release

[jandubois]

Yes, after go-diskfs makes a release

FYI go-diskfs has been updated