Skip to content

feature: Add Bootloader=uki_prebuilt for distro-shipped UKIs - #4247

Open
src-up wants to merge 2 commits into
systemd:mainfrom
src-up:uki-prebuilt
Open

feature: Add Bootloader=uki_prebuilt for distro-shipped UKIs#4247
src-up wants to merge 2 commits into
systemd:mainfrom
src-up:uki-prebuilt

Conversation

@src-up

@src-up src-up commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Some distributions (like Fedora) ship their own prebuilt Unified Kernel Images (UKIs). In some use cases, it's helpful to use mkosi to build images that consume these distro-shipped UKIs rather than building a UKI from scratch.

One such use case is testing cvmutils, where the goal is to replace ad-hoc shell scripts that handle image building with mkosi. However, doing so currently requires extra scripts to bridge the gap.

This PR fills that gap by allowing mkosi to natively consume prebuilt UKIs, eliminating the need for any additional scripts.

Fixes #4174

@src-up

src-up commented Mar 27, 2026

Copy link
Copy Markdown
Contributor Author

Tested on Fedora 43 with kernel-uki-virt and ShimBootloader=none — boots successfully via UEFI → EFI/BOOT/BOOTX64.EFI (UKI).

Shim integration (ShimBootloader=signed) is not yet working and will be addressed in a follow-up.

@MoltenArmor

Copy link
Copy Markdown
Contributor

This is a bit confusing. Looking at install_uki(), doesn't mkosi already copy a prebuilt UKI from /usr/lib/modules/<kver>/ directly when Bootloader=uki-signed is set with UnifiedKernelImages=auto, or when UnifiedKernelImages=signed is set explicitly?

    if (
        context.config.bootloader.is_signed()
        and context.config.unified_kernel_images == UnifiedKernelImage.auto
    ) or context.config.unified_kernel_images == UnifiedKernelImage.signed:
        for p in (context.root / "usr/lib/modules" / kver).glob("*.efi"):
            log_step(f"Installing prebuilt UKI at {p} to {boot_binary}")
            copyfile2(p, boot_binary)
            break

@src-up

src-up commented Mar 30, 2026

Copy link
Copy Markdown
Contributor Author

Looking at install_uki(), doesn't mkosi already copy a prebuilt UKI from /usr/lib/modules// directly when Bootloader=uki-signed is set with UnifiedKernelImages=auto, or when UnifiedKernelImages=signed is set explicitly?

i think , currently with Bootloader=uki-signed , stub check fails...

build_image -> install_kernel -> want_uki = True -> stub check fires → DIES: systemd-stub not found

systemd-stub is used to build UKIs from scratch. so i think the uki-signed assumes systemd-stub would be present, which is not needed for prebuilt UKIs.

Even if we add systemd-stub to list of packages, we get
A bootable image was requested but no kernel was found

I believe this is because, install_uki is only called if gen_kernel_images returns something and assumption there is that both a raw vmlinuz AND a signed .efi alongside it in /usr/lib/modules// will exist. Since they dont exist for distros like fedora, gen_kernel_images returns empty and code never reaches install_uki..

  • install_uki() is only called from the gen_kernel_images() loop, which looks for vmlinuz* that is not a UKI
  • So the "find distro-signed UKI" code inside install_uki() is designed for distros that ship both a raw vmlinuz AND a signed .efi alongside it in /usr/lib/modules//
  • kernel-uki-virt (fedora prebuilt UKI) only ships the .efi — no separate vmlinuz — so gen_kernel_images() yields nothing and that path is never reached

uki-prebuilt does not make this assumption..

@behrmann behrmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Fedora ships a prebuilt UKI, then this should be covered by the tests as well and an image with a prebuilt UKI should be built.

Comment thread mkosi/__init__.py Outdated
Comment thread mkosi/__init__.py Outdated
Comment thread mkosi/__init__.py Outdated
Comment thread mkosi/resources/man/mkosi.1.md Outdated
@src-up
src-up force-pushed the uki-prebuilt branch 2 times, most recently from 89292fe to 2efe904 Compare April 9, 2026 20:40
@src-up

src-up commented Apr 9, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann Thanks for the review.

Tried two approaches for test coverage:

  • Injecting --package=kernel-uki-virt at test time — fails because --incremental=strict rejects the cache manifest mismatch.
  • Adding kernel-uki-virt to mkosi.conf.d/fedora/mkosi.conf — fails with No match for argument: kernel-uki-virt on rawhide

Open to suggestions on how to handle this.

Rest of the comments were addressed.

@src-up
src-up force-pushed the uki-prebuilt branch 3 times, most recently from ff85ba3 to 3a308d3 Compare April 15, 2026 06:32
@src-up
src-up requested a review from behrmann April 15, 2026 11:03
@src-up

src-up commented Apr 29, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann
moved kernel-uki-virt to the arch-specific x86_64.conf and arm64.conf configs (where the package exists), and excluded ppc64le from the test since it doesn't support SMBIOS anyway.

One other thing needed: the CI test framework injects params like systemd.unit= and systemd.firstboot=no via kernel_command_line at build time — but for a prebuilt UKI those can't be baked in. Fixed by passing them at runtime via SMBIOS type-11 (io.systemd.stub.kernel-cmdline-extra), which systemd-stub reads from the prebuilt UKI.

Please take a look when you get a chance.

@src-up

src-up commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

@daandemeyer any thoughts on this ?

@src-up

src-up commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann please review

@behrmann

behrmann commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

@behrmann please review

On vacation currently, will review sometime in the next few days.

@src-up
src-up force-pushed the uki-prebuilt branch 2 times, most recently from db2b453 to 76105a1 Compare June 8, 2026 18:36
@src-up

src-up commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

On vacation currently, will review sometime in the next few days.

@behrmann no worries. I have rebased from latest main

For the new uki-prebuilt path, the sequence is:

  • mkosi/bootloader.py: gen_kernel_images() switches to gen_prebuilt_uki_images() when uki-prebuilt is selected, and that path picks a UKI .efi from /usr/lib/modules//.

  • mkosi/init.py (install_kernel): mkosi copies that selected UKI to the EFI boot target (boot_binary, i.e. shim second stage or fallback EFI path) via copyfile2(...).

CI log matches this flow, with the raw line: Installing prebuilt distro UKI .../usr/lib/modules/.../vmlinuz-virt.efi to .../efi/EFI/BOOT/grubx64.EFI.

Relevant CI job: integration-test (fedora, fedora, ubuntu-24.04)

Comment thread mkosi/qemu.py
@src-up

src-up commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann can we get these merged ?

@behrmann

Copy link
Copy Markdown
Contributor

@src-up Oh, geez, I thought I had already. I was talking to Vitaly (?) about this two weeks ago. I'm terribly sorry.

I've just went through the logic again and notice I had misunderstood @MoltenArmor's initial comment about the codepath for prebuilt UKIs.

Having gone through the code again, your analysis why that code isn't used is correct (install_uki not being called if gen_kernel_images doesn't return anything). Having looked at this for the past hour, I don't think this ever worked, because the the logic to not pick up EFI binaries in gen_kernel_images predates (2023, in #2166) the logic of installing those files in install_uki (b68a3ff in 2024).

I think this should work if gen_kernel_images tried to look for UKIs in gen_kernel_images when uki-signed is used. The addition of gen_prebuilt_uki_images and in install_kernel should then not be necessary. The added test is still very valuable, since, as I think, if this never worked, this is due to it not being tested.

I would be grateful if you could have a look whether the above idea makes this work. Sorry for bungling up the communication on this so much.

@src-up

src-up commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@src-up Oh, geez, I thought I had already. I was talking to Vitaly (?) about this two weeks ago. I'm terribly sorry.

Yes, Vitaly mentioned he met and spoke to you @behrmann

I would be grateful if you could have a look whether the above idea makes this work. Sorry for bungling up the communication on this so much.

No worries :=) . Thanks for reviewing this again. Much appreciated.

I tried putting the changes together like you suggested and came up with this commit src-up@2d3c99b. Let me know if this is more in line with what you were thinking.

I tested the below commands on a fedora box and they pass

bin/mkosi -f box -- true
tools/integration-test-setup.sh fedora fedora
bin/mkosi box -- pytest -m integration --distribution fedora --capture=no --verbose     'tests/test_boot.py::test_bootloader[uki-signed]'

@behrmann

behrmann commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@src-up Looks good, but I have to think this through more fully, I'll come back to you about this tomorrow.

@behrmann

behrmann commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

@src-up Sorry, late again. I left you a few commits on the commit. Thank you!

@src-up
src-up force-pushed the uki-prebuilt branch 3 times, most recently from deb0860 to a4c4281 Compare July 10, 2026 16:52
@src-up

src-up commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann hey, so I have moved that commit src-up@2d3c99b as part of this PR. Let me know what u think of the changes now ...

  • Guard: replaced is_signed() with a want_prebuilt_uki() helper. Also added match=config_make_enum_matcher(Bootloader) to make Bootloader= usable in [Match] sections.
  • Glob: narrowed vmlinuz* → vmlinuz*.efi in the UKI-yielding branch.
  • kernel-uki-virt: Since multiple [Match] sections in the same file are ANDed, and we have packages that are needed for other Bootloader modes, moved to new x86_64-uki-signed.conf / arm64-uki-signed.conf with [Match] Bootloader=uki-signed.

@src-up
src-up force-pushed the uki-prebuilt branch 2 times, most recently from 7b77aa6 to 342fcf4 Compare July 10, 2026 18:06

@keszybz keszybz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, but the commit has not commit message. Please add a proper commit message.

Comment thread mkosi/qemu.py Outdated
@src-up

src-up commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@keszybz thanks for the review. I added the commit message and updated the comment. Let me know if this can be merged now.

@behrmann behrmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost done

Comment thread mkosi/qemu.py Outdated
Comment thread tests/test_boot.py Outdated
Comment thread tests/test_boot.py Outdated
@src-up
src-up force-pushed the uki-prebuilt branch 2 times, most recently from c491669 to b611da8 Compare July 19, 2026 23:41
@src-up

src-up commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann Thanks for mentioning about CI and your patience on this PR. I should have checked that myself first.

Post the ruff fix, the fedora CI tests were also failing due to cache manifest not matching the saved one.

With Packages=kernel-uki-virt, the package appears in the cache manifest. Since the CI cache was built without it, the manifests differ and --incremental=strict aborts the build. I changed it to VolatilePackages= so that kernel-uki-virt is installed on top of the cache without affecting the manifest.

The fedora/fedora CI test passes now, but one of them (opensuse) ais still failing due to openSUSE GPG key verification error - perhaps unrelated changes.

@behrmann

Copy link
Copy Markdown
Contributor

@behrmann Thanks for mentioning about CI and your patience on this PR. I should have checked that myself first.

No worries. Your work on this has been very diligent, thank you.

The fedora/fedora CI test passes now, but one of them (opensuse) ais still failing due to openSUSE GPG key verification error - perhaps unrelated changes.

That is unrelated. The key is not yet updated in a release version of distribution-gpg-keys

Post the ruff fix, the fedora CI tests were also failing due to cache manifest not matching the saved one.

With Packages=kernel-uki-virt, the package appears in the cache manifest. Since the CI cache was built without it, the manifests differ and --incremental=strict aborts the build. I changed it to VolatilePackages= so that kernel-uki-virt is installed on top of the cache without affecting the manifest.

The workaround seems like a hack. Maybe for this particular piece of the matrix --incremental=yes or --incremental=no can be set—we do have --incremental=no for the extension image tests. You can try that in CI or I'll try to test this locally once I find time. It's just my first idea, I might be wrong.

Some distributions (like Fedora) ship their own prebuilt Unified Kernel
Images (UKIs).  In some use cases, it's helpful to use mkosi to build
images that consume these distro-shipped UKIs rather than building a UKI
from scratch.

This commit allows mkosi to natively consume prebuilt UKIs,
eliminating the need for any additional scripts.

Fixes systemd#4174
@src-up

src-up commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Hey @behrmann With "incremental=no", it is showing 7 failed tests and i think "default-initrd.cache does not exist" in logs means the cache is getting removed.

Here is link to the raw log output of one of the 7 failing CI checks, specifically these lines

2026-07-20T19:50:51.6024838Z ‣ /home/runner/work/mkosi/mkosi/mkosi.cache/fedora~rawhide~x86-64~default-initrd.cache does not exist, not reusing cached images
2026-07-20T19:50:51.6026197Z ‣ Strict incremental mode is enabled and cache for image default-initrd is out-of-date
2026-07-20T19:50:51.6026874Z ‣ (Build once with '-i yes' to update the image cache)
2026-07-20T19:50:51.6246787Z Traceback (most recent call last):
2026-07-20T19:50:51.6253892Z   File "/home/runner/work/mkosi/mkosi/mkosi/run.py", line 104, in uncaught_exception_handler
2026-07-20T19:50:51.6254536Z     yield
2026-07-20T19:50:51.6254913Z   File "/usr/lib64/python3.15/contextlib.py", line 116, in inner
2026-07-20T19:50:51.6255582Z     return func(*args, **kwds)
2026-07-20T19:50:51.6256086Z   File "/home/runner/work/mkosi/mkosi/mkosi/__main__.py", line 44, in main
2026-07-20T19:50:51.6256675Z     run_verb(args, tools, images, resources=resources)
2026-07-20T19:50:51.6257204Z     ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-07-20T19:50:51.6257764Z   File "/home/runner/work/mkosi/mkosi/mkosi/__init__.py", line 5251, in run_verb
2026-07-20T19:50:51.6258302Z     die(
2026-07-20T19:50:51.6258540Z     ~~~^
2026-07-20T19:50:51.6259100Z         f"Strict incremental mode is enabled and cache for image {c.image} is out-of-date",
2026-07-20T19:50:51.6259767Z         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-07-20T19:50:51.6260305Z         hint="Build once with '-i yes' to update the image cache",
2026-07-20T19:50:51.6260790Z         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-07-20T19:50:51.6261164Z     )
2026-07-20T19:50:51.6261399Z     ^
2026-07-20T19:50:51.6261767Z   File "/home/runner/work/mkosi/mkosi/mkosi/log.py", line 35, in die
2026-07-20T19:50:51.6262608Z     sys.exit(1)
2026-07-20T19:50:51.6262877Z     ~~~~~~~~^^^
2026-07-20T19:50:51.6263138Z SystemExit: 1

I can try "incremental=yes" also but wanted to show you these first.

@src-up

src-up commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann I tried with both incremental=yes and incremental=no.

Where things stand

  1. incremental=strict CI (default used by all tests) - test_bootloader[uki-signed] fails. Cache is built without kernel-uki-virt, but when test_bootloader[uki-signed] runs with Packages=kernel-uki-virt , the manifest mismatches and build aborted.
  2. incremental=no test_bootloader[uki-signed] passes, but both default-initrd and main caches are unconditionally deleted (no manifest check at all). The 10 subsequent tests fail with "does not exist". Deletion of cache happens because the way code is written, if force flag is on (which is for all tests), cache is deleted if cache is ignored which is always true for incremental=no.
  3. incremental=yes test_bootloader[uki-signed] passes, but rebuilds and saves both caches with kernel-uki-virt. The 10 subsequent tests fail with "Cache manifest mismatch" — the saved cache now has kernel-uki-virt but those tests build without Bootloader=uki-signed.
  4. VolatilePackages= works but i do see ur point, its a hack.

Honestly, not sure how to proceed here.

@behrmann

Copy link
Copy Markdown
Contributor

Thanks for trying this out. I have a couple of deadlines I need to handle, so I won't be looking at this seriously before the week after next (the first of August that is).

@src-up

src-up commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

@behrmann Hey , Let me know if this week is a good time to figure out what path to take forward. Thanks !

@behrmann behrmann mentioned this pull request Aug 5, 2026
@behrmann

behrmann commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

I just opened #4410 to look into this.

@behrmann

behrmann commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

@src-up after playing around with this for a bit, I think the most concise solution is to set a different cache key for this particular test. You can find this as the last commit on #4410. The changes before that are unnecessary, I only factored out that particular instance from the test initially, because I was looking at the ordering of the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Support distro UKI + shim direct boot via BOOTX64.CSV

4 participants