LVM disk mounts in config.toml not working as expected

[bdunne]

Given a Containerfile similar to:

FROM quay.io/centos-bootc/centos-bootc:stream10
RUN dnf -y install postgresql-server

And config.toml including:

[[customizations.disk.partitions]]
type = "lvm"
name = "vg_system"
minsize = "20 GiB"

[[customizations.disk.partitions.logical_volumes]]
name = "lv_var_lib"
mountpoint = "/var/lib"
fs_type = "xfs"
minsize = "10 GiB"

If I podman run -it --rm --entrypoint bash my_image, I can see:

bash-5.2# ls -al /var/lib/pgsql/
total 4
drwx------. 1 postgres postgres  48 May 28 00:48 .
drwxr-xr-x. 1 root     root     552 May 28 00:52 ..
-rw-r--r--. 1 postgres postgres  85 Mar 16 00:00 .bash_profile
drwx------. 1 postgres postgres   0 Mar 16 00:00 backups
drwx------. 1 postgres postgres   0 Mar 16 00:00 data

However when I boot the qcow2 disk image that was produced using the same Containerfile and config.toml, I see:

bash-5.2# ls -al /var/lib/pgsql/
total 8
drwx------.  2 postgres postgres   27 May 28 01:41 .
drwxr-xr-x. 34 root     root     4096 May 28 01:41 ..
-rw-r--r--.  1 postgres postgres   85 May 28 01:41 .bash_profile

If I unmount /var/lib then I can see the same directory structure that is in the container image (including data & backups.)

1. Is this expected behavior? I was expecting all filesystems to be mounted before laying down the image similar to a traditional install using kickstart, however, it appears that the image is installed on the root filesystem and the mounts are on top of that.

2. Is there a known workaround or fix for this?

[supakeen]

This is not expected behavior; it's a bug, everything should have been mounted before we installed onto the system.

I'd need to look for a workaround if it exists and the bug needs to be fixed.

[bdunne]

This doesn't appear to be LVM-specific or BIB-specific, I was able to reproduce the same problem with kickstart installations using regular partitions and LVM.

[supakeen]

@bdunne In that case it might be bootc specific since both (bootc-)image-builder and anaconda call bootc install to-filesystem /some/path under which we set up all the mount points. Perhaps worth it to file an issue there and link it back here?