LBI: Add /etc/bootc/bound-images.d

[cgwalters]

The problem of "agents" is long-running and difficult. With bootc we have made it trivial to embed them in the base OS image. However, while that creates a simple story and generally works, it lifecycle binds the OS to the agent, which is not always desired.

Today Fedora CoreOS basically reimplements minimal cloud support in a generic way in ignition/afterburn. cloud-init handles many things, but not all. There's vmware-guest-agent etc.

We're having some discussions in Fedora-derivative land about trying to make "generic" bootc systems, and I think one possible approach here that would feel nice is to add support for /etc/bootc/bound-images.d paralleling our existing /usr/lib/bootc/bound-images.d.

The idea here is basically that we could support shipping e.g. cloud-init or vmware-guest-agent as a privileged container, and in a bootc install to-existing-root scenario the installer could dynamically detect the platform at install time and add relevant agents into /etc/bootc/bound-images.d. So by default they'd be required for OS upgrades, and the image data would continue to live in the bootc c/storage instance. The only "mutable" state would be that their presence is requested via /etc and not /usr - decoupling them from the fully generic base OS.

OR perhaps arguably...we add dynamism to the current LBI in a systemd-style way ConditionVirtualization=vmware or ConditionKernelCommandLine=ignition.platform.id=vmware or so.

[jlebon]

OR perhaps arguably...we add dynamism to the current LBI in a systemd-style way ConditionVirtualization=vmware or ConditionKernelCommandLine=ignition.platform.id=vmware or so.

Yeah, I think specifically for the agent case, that's probably a better fit so that we still control the quadlet definitions from in-band. (And obviously it's a generically useful enhancement too.)

[jlebon]

One downside of this approach I think is that it would hit bootstrapping issues wrt [RHEL-82921] [RFE] Support cloud registry credential provider in container tools.