Skip to content

Introduce TOML-based configuration to replace data.py#491

Draft
glehmann wants to merge 4 commits intogln/passlib-integration-kuyqfrom
gln/config-migration-to-toml-mxrr
Draft

Introduce TOML-based configuration to replace data.py#491
glehmann wants to merge 4 commits intogln/passlib-integration-kuyqfrom
gln/config-migration-to-toml-mxrr

Conversation

@glehmann
Copy link
Copy Markdown
Member

@glehmann glehmann commented Apr 21, 2026

The previous data.py approach had several shortcomings:

  • No base configuration: users had to start from scratch or copy data.py-dist
  • No environment management: working with multiple infra required to manually
    copy the data.py files
  • No clear default and customization separation: the data.py, once copied
    from data.py-dist and customized, was mixing the defaults and the
    customizations

The new TOML config system addresses this by:

  • Providing a base config.toml maintained by the project with sensible defaults,
    which can be updated without user action
  • Supporting per-environment override files (config..toml) so users only
    specify what differs from the defaults, making their customizations explicit

Options have been grouped by sections for clarity.

Configuration sections and their contents:

  • [host]: default XAPI credentials (user, password)
  • [hosts]: per-host credential overrides
  • [network]: management network name
  • [pxe]: PXE server settings (arp_server)
  • [vm]: VM settings (default_sr, images, equivalents, def_url)
  • [install]: installer settings (answerfiles, iso_remaster, isos)
  • [guest_tools]: guest tools packages and installer ISOs (win, other, installed, download_url)
  • [ssh]: SSH client settings (pubkey, output_max_lines, ignore_banner)
  • [storage]: storage backend configs (nfs, nfs4, nfs_iso, cifs_iso, cephfs, moosefs, lvmoiscsi)
  • Root keys: objects_name_prefix, dns_server

Also includes config-schema.json, lib/config_loader.py (Pydantic models),
and scripts/migrate_data_py.py to help users migrate from data.py.

This PR is part of a complex set of PRs containing 6 PRs:

  1. master
  2. Use passlib in place of legacycrypt #396master
  3. "Introduce TOML-based configuration to replace data.py" (this PR) → Use passlib in place of legacycrypt #396
  4. refactor: make BaseCmdResult and subclasses generic to eliminate runtime type narrowing #492master
  5. misc type related enhancements #460refactor: make BaseCmdResult and subclasses generic to eliminate runtime type narrowing #492
  6. chore: add ty type checker and reorder make commands for faster feedback #476misc type related enhancements #460
  7. Add Docker-based compatibility test kit for external validation #431refactor: make BaseCmdResult and subclasses generic to eliminate runtime type narrowing #492, Use passlib in place of legacycrypt #396

The previous data.py approach had several shortcomings:
- No base configuration: users had to start from scratch or copy data.py-dist
- No environment management: working with multiple infra required to manually
  copy the data.py files
- No clear default and customization separation: the data.py, once copied
  from data.py-dist and customized, was mixing the defaults and the
  customizations

The new TOML config system addresses this by:
- Providing a base config.toml maintained by the project with sensible defaults,
  which can be updated without user action
- Supporting per-environment override files (config.<name>.toml) so users only
  specify what differs from the defaults, making their customizations explicit

Options have been grouped by sections for clarity.

Configuration sections and their contents:
- [host]: default XAPI credentials (user, password)
- [hosts]: per-host credential overrides
- [network]: management network name
- [pxe]: PXE server settings (arp_server)
- [vm]: VM settings (default_sr, images, equivalents, def_url)
- [install]: installer settings (answerfiles, iso_remaster, isos)
- [guest_tools]: guest tools packages and installer ISOs (win, other, installed, download_url)
- [ssh]: SSH client settings (pubkey, output_max_lines, ignore_banner)
- [storage]: storage backend configs (nfs, nfs4, nfs_iso, cifs_iso, cephfs, moosefs, lvmoiscsi)
- Root keys: objects_name_prefix, dns_server

Also includes config-schema.json, lib/config_loader.py (Pydantic models),
and scripts/migrate_data_py.py to help users migrate from data.py.

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
To make easier editing within our editors.
This format is compatible with multiple toml lsp, including taplo.

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
Configuration is now done in config.toml and config.*.toml

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
Comment thread config-schema.json
"type": "object",
"properties": {
"path": { "type": "string" },
"net-url": { "type": "string" },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you should choose a convention "snake_case" or "kebab-case", what do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants