Begin using Zig to build on Linux #321
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mlugg
force-pushed
the
zig-build
branch
2 times, most recently
from
bf64014 to
31e04d9
Compare
This commit gets the Linux build of SAR working via `zig build` on Zig 0.14.0 (and current dev builds). This solves the problems we've been having with glibc versions, because Zig is able to provide far older glibc versions than many toolchains upon request. I am somewhat arbitrarily targeting glibc 2.28, which seems to work. In theory, Zig *could* be used to build SAR for Windows, although it's more finickey than on Linux because of ABI differences between the `gnu` and `msvc` ABIs. SAR relies on the MSVC C++ ABI on Windows in order for vtable and data layout to match the game modules, so we need to target `x86-windows-msvc`. This unfortunately makes cross-compilation essentially impossible, because Zig cannot distribute the MSVC library files. As a part of this commit, several libraries are now built on-the-fly rather than being linked as binary blobs in the repo. These libraries are: * libx265 (a dependency of ffmpeg, for sar_render) * SFML * discord-rpc The reason for this is that these three libraries contain C++ code, and using Zig's build system means we are using LLVM's libc++ instead of GCC's libstdc++. These c++ stdlib implementations have different ABIs, so any code linking to libc++ -- i.e. any C++ code -- needed recompiling. Given that I was going to have to rebuild these dependencies anyway, I figured it made sense to actually integrate their builds. These dependencies are not vendored. Instead, Zig's package manager is used to fetch tarballs on-the-fly. Mostly, these tarballs come directly from upstreams; SFML is an exception, because their 2.5.x branch needed changes to build with modern libc++, so the URL in `build.zig.zon` is a fork with minor tweaks. Long-term, it would make sense to migrate *all* of our dependencies to work like this, and on Windows too; vendoring binary blobs in the repo is pretty clearly a Bad Thing. However, that's out of scope for this PR.
This is being replaced with build.zig.
This change isn't *necessary* like the C++ dependencies were, but since we're already handling *some* dependencies through the Zig build system, we may as well embrace that a little more where it's easy to minimize repo size and maximise trust (by not linking unreproducible binaries into SAR builds).
principle of least privilege. still need to figure out codeql before merge or remove it and pray we don't add any vulnerabilities
This new release of setup-zig allows determining the version of Zig to fetch based off of the `minimum_zig_version` specified in `build.zig.zon`, which means we only need to specify it in one place instead of three.
Feel free to exclude this commit, but I kinda like it; it just says how long it took to do everything. It can be noisy if your build graph is more complicated, but ours is relatively simple, so I find it harmless.
This is unnecessary, because it is unused; it only risks becoming out-of-sync and causing confusion.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This will probably need a bit more work, but I'm putting it up to gather some thoughts. I'm obviously biased, but to me, this seems like a pretty good path forward.
This PR gets the Linux build of SAR working via
zig buildon Zig 0.14.0 (and current dev builds). This solves the problems we've been having with glibc versions, because Zig is able to provide far older glibc versions than many toolchains upon request. I am somewhat arbitrarily targeting glibc 2.28, which seems to work.In theory, Zig could be used to build SAR for Windows, although it's more finickey than on Linux because of ABI differences between the
gnuandmsvcABIs. SAR relies on the MSVC C++ ABI on Windows in order for vtable and data layout to match the game modules, so we need to targetx86-windows-msvc. This unfortunately makes cross-compilation essentially impossible, because Zig cannot distribute the MSVC library files.As a part of this commit, several libraries are now built on-the-fly rather than being linked as binary blobs in the repo. These libraries are:
The reason for this is that these three libraries contain C++ code, and using Zig's build system means we are using LLVM's libc++ instead of GCC's libstdc++. These c++ stdlib implementations have different ABIs, so any code linking to libc++ -- i.e. any C++ code -- needed recompiling. Given that I was going to have to rebuild these dependencies anyway, I figured it made sense to actually integrate their builds.
These dependencies are not vendored. Instead, Zig's package manager is used to fetch tarballs on-the-fly. Mostly, these tarballs come directly from upstreams; SFML is an exception, because their 2.5.x branch needed changes to build with modern libc++, so the URL in
build.zig.zonis a fork with minor tweaks.Long-term, it would make sense to migrate all of our dependencies to work like this, and on Windows too; vendoring binary blobs in the repo is pretty clearly a Bad Thing. However, that's out of scope for this PR.
The CI/CD workflows are updated to use
zig buildon Linux, using my setup-zig action to install the compiler.