custom python3-labgrid overrides userconfig.yaml

[tretter]

The /etc/labgrid/userconfig.yaml is installed with a bbappend to the python3-labgrid package.

Since I'm using a patched version of the python3-labgrid, my userconfig.yaml is replaced with the default one whenever I'm deploying my custom python3-labgrid package.

This is surprising, because I would have expected that the userconfig.yaml is preserved as it is preserved on the RAUC updates.

Maybe the default userconfig.yaml should be moved to a separate package to make it independent of the python3-labgrid package?

(As a workaround, I could modify the default userconfig.yaml in my BSP to deploy my configuration, but I think that deviates a lot from the usual behavior of the userconfig.yaml.)

[hnez]

Hi,

I am working on a new stable release and am trying to decide whether this should be fixed before the release.

Could you provide me with some more information to help me better understand the issue?

The /etc/labgrid/userconfig.yaml is installed with a bbappend to the python3-labgrid package.

We do indeed install a couple of labgrid config files that are custom to meta-lxatac, namely these:

• /etc/labgrid/configuration.yaml
• /etc/labgrid/environment
• /etc/labgrid/labgrid.conf
• /etc/labgrid/userconfig.yaml

The configuration.yaml is in fact extra custom because it only works in conjunction with our custom configuration.yaml, due to an import:

meta-lxatac/meta-lxatac-software/recipes-devtools/python/python3-labgrid/configuration.yaml

Lines 16 to 19 in f364507

	## Include user configuration
	## ==========================
	{% include 'userconfig.yaml' %}

This would make deploying the userconfig.yaml as a separate package a bit tricky, since we would either have to use a .bbappend anyways to deploy our custom configuration.yaml or would conflict with the unmodified python3-labgrid package when trying to deploy a configuration.yaml as well.

Since I'm using a patched version of the python3-labgrid, my userconfig.yaml is replaced with the default one whenever I'm deploying my custom python3-labgrid package.

Could you specify how you are patching and deploying your python3-labgrid version? Are you building customized images? In that case it should be possible to stack another python3-labgrid_%.bbappend on top of the one provided by meta-lxatac and apply your patches that way.

This is surprising, because I would have expected that the userconfig.yaml is preserved as it is preserved on the RAUC updates.

Could it be that the userconfig.yaml is in fact preserved but a stock configuration.yaml is installed, which does not include it?

[tretter]

This would make deploying the userconfig.yaml as a separate package a bit tricky, since we would either have to use a .bbappend anyways to deploy our custom configuration.yaml or would conflict with the unmodified python3-labgrid package when trying to deploy a configuration.yaml as well.

Maybe the userconfig.yaml could be moved to a separate package and the python3-labgrid depend on the package that contains the configuration files?

Since I'm using a patched version of the python3-labgrid, my userconfig.yaml is replaced with the default one whenever I'm deploying my custom python3-labgrid package.

Could you specify how you are patching and deploying your python3-labgrid version?

I'm using devtool with a local git tree. My patches are patches that I submitted to upstream or backported from upstream. Thus, I'm using the setup to test my own patches and I keep them around until a new labgrid version lands in the LXATAC and I can get rid of my local patches.

Are you building customized images? In that case it should be possible to stack another python3-labgrid_%.bbappend on top of the one provided by meta-lxatac and apply your patches that way.

No, I'm trying to avoid a custom image and having to maintain my patches. It's a temporary workaround until my patches are upstream.

I could patch the userconfig.yaml in my local BSP checkout to persist my local changes. That would work for exactly for my single LXATAC, which has my exact configuration and as long I'm deploying my own version of labgrid. Once I returned to the upstream version, I will forget about the changes to my userconfig.yaml. If I ever come into the situation that I need to build labgrid with local patches, I will have forgotten that I need to manually preserve the userconfig.yaml and lose my changes to the config, again.

This is surprising, because I would have expected that the userconfig.yaml is preserved as it is preserved on the RAUC updates.

Could it be that the userconfig.yaml is in fact preserved but a stock configuration.yaml is installed, which does not include it?

No, the configuration.yaml is fine, but the userconfig.yaml is replaced with the default.

[hnez]

I still don't understand this completely, sorry.

I'm using devtool with a local git tree. My patches are patches that I submitted to upstream or backported from upstream. Thus, I'm using the setup to test my own patches and I keep them around until a new labgrid version lands in the LXATAC and I can get rid of my local patches.

And then you generate a RAUC bundle and install that on your TAC?

---

No, the configuration.yaml is fine, but the userconfig.yaml is replaced with the default.

The generated rootfs always contains the default userconfig.yaml (it contains it twice to be exact. Once as /etc/labgrid/userconfig.yaml and a copy at /etc/labgrid/userconfig.yaml.default).
The default /etc/labgrid/userconfig.yaml is then replaced with the previous one by a RAUC hook:

meta-lxatac/meta-lxatac-software/recipes-core/bundles/files/hook.sh

Lines 74 to 79 in f364507

	migrate /etc/hostname
	migrate /etc/machine-id
	migrate /etc/labgrid/environment
	migrate /etc/labgrid/userconfig.yaml
	migrate /etc/github-act-runner/sessions.json
	migrate /etc/github-act-runner/settings.json

This should also happen with your modified python3-labgrid, since it is completely independent from it.

[tretter]

I'm using devtool with a local git tree. My patches are patches that I submitted to upstream or backported from upstream. Thus, I'm using the setup to test my own patches and I keep them around until a new labgrid version lands in the LXATAC and I can get rid of my local patches.

And then you generate a RAUC bundle and install that on your TAC?

No, I don't generate a RAUC bundle, and I don't use RAUC for deployment. Thus, the RAUC hook is useless in this case.

My workflow is devtool build python3-labgrid && devtool deploy-target python3-labgrid. This keeps the base system intact, and the build and deploy only affects the python3-labgrid package.

[hnez]

No, I don't generate a RAUC bundle, and I don't use RAUC for deployment. Thus, the RAUC hook is useless in this case.

Now it all makes sense, thanks for the clarification!

I just tend to use RAUC bundles when developing for the TAC (I rarely need to do iterative debugging on the TAC these days. I mostly just update some piece of software, throw it against the GitHub CI by opening a PR and install the generated RAUC bundle and it usually just works).

My workflow is devtool build python3-labgrid && devtool deploy-target python3-labgrid. This keeps the base system intact, and the build and deploy only affects the python3-labgrid package.

I would be okay with splitting off the userconfig.yaml to make this workflow work for you, but would'nt give it a high priority, since I don't think it is likely that someone else will be affected by this and because it would be possible to work around it by placing ssh lxatac-12345 cp /etc/labgrid/userconfig.yaml /tmp and ssh lxatac-12345 cp /tmp/userconfig.yaml /etc/labgrid around your devtool deploy-target calls.

[tretter]

I [...] would'nt give it a high priority, since I don't think it is likely that someone else will be affected by this and because it would be possible to work around it [...]

Low priority or even closing this issue as won't fix is fine by me, since there are various workarounds.