✨ feat(toml): declare factor group defaults and per-run overrides (#4050)

Author: gaborbernat

docs/changelog/4045.feature.rst

@@ -0,0 +1,3 @@
+A labeled factor group can now declare a ``default`` for ``{factor:label}`` to fall back on when no factor of that group
+is active in the environment name. Setting ``TOX_FACTOR_<label>`` resolves that label to a given value for a single run
+- by :user:`gaborbernat`.

docs/explanation.rst

@@ -345,6 +345,12 @@ have labeled one shape and left the other out. Nesting covers both with one rule
 and the value describes its factors. Adding a shape later means describing another value, not teaching every earlier
 shape one more key.
 
+A labeled factor value is part of the environment name, which is what makes ``tox run -e test-3.14-django50`` mean one
+thing. That leaves no way to name a version the matrix does not list on the command line, since ``>=4.2,<4.3`` cannot
+serve as a factor. ``TOX_FACTOR_<label>`` sidesteps that by leaving the name alone. The run goes through the environment
+the matrix generated, and only ``{factor:label}`` resolves elsewhere. The cost is a name that no longer tells the whole
+story, so the override suits a one-off check rather than a recorded configuration.
+
 For the configuration reference, see :ref:`env-base-templates`. For practical recipes, see :ref:`howto_env_base_matrix`.
 
 Overrides and substitution

docs/how-to/usage.rst

@@ -1183,6 +1183,12 @@ lists and range dicts take a label:
     ]
     description = "Test {factor:django_version} on {factor:py_version}"
 
+To try a version the matrix does not list, set ``TOX_FACTOR_<label>`` for that run:
+
+.. code-block:: console
+
+    $ env TOX_FACTOR_django_version=django61 tox run -e django-py314-django50
+
 See :ref:`env-base-templates` for the full reference.
 
 ***************************

docs/reference/config.rst

@@ -209,6 +209,8 @@ This generates 4 environments: ``django-py312-django42``, ``django-py312-django5
 - A labeled dict: ``{ ecosystem = ["oci", "python"] }`` (same as a list, but registers the label for substitution)
 - A labeled range dict: ``{ py_version = { prefix = "3.", start = 12, stop = 14 } }`` (a range that also registers a
   label)
+- A labeled values dict: ``{ django_version = { values = ["django42", "django50"], default = "django50" } }`` (a list
+  with a declared default)
 - Mixed in the same ``factors`` list for Cartesian products
 
 The template name itself does not appear as a runnable environment -- only the generated names do.
@@ -277,6 +279,42 @@ For ``task-oci-pw``, the description resolves to ``Run oci on pw``. Labeled dict
 **Defaults** -- ``{factor:label:fallback}`` uses ``fallback`` when the label is unknown, following the same convention
 as ``{env:VAR:default}``.
 
+A factor group can declare its own default instead, so every use site stays short:
+
+.. code-block:: toml
+
+    [env_base.test]
+    factors = [
+        { py_version = { prefix = "3.", start = 12, stop = 14 } },
+        { django_version = { values = ["django42", "django50"], default = "django50" } },
+    ]
+    description = "Test {factor:django_version} on Python {factor:py_version}"
+
+A range dict takes ``default`` in the same table as ``prefix``.
+
+The declared default must be one of the group's own factors. It applies when no factor of the group is active in the
+environment name, which happens in an ``[env.NAME]`` section outside the matrix. A ``{factor:label:fallback}`` written
+at the use site wins over it.
+
+.. versionadded:: 4.61
+
+    The ``values`` and ``default`` keys.
+
+**Overriding a label for one run** -- set ``TOX_FACTOR_<label>`` to make ``{factor:<label>}`` resolve to that value,
+whatever the environment name says:
+
+.. code-block:: console
+
+    $ env TOX_FACTOR_django_version=django61 tox run -e test-3.14-django50
+
+This installs ``Django61`` while still running the ``test-3.14-django50`` environment, which is useful for a one-off
+check against a version the matrix does not list. The variable only applies to labels the configuration declares, and it
+does not change which environments exist or what they are called.
+
+.. versionadded:: 4.61
+
+    The ``TOX_FACTOR_<label>`` override.
+
 **Comparison with conditionals** -- ``{factor:label}`` replaces nested ``replace = "if"`` chains when the substituted
 value equals the factor name itself. For values that don't match factor names (e.g., mapping ``pw`` to
 ``production-west-cluster``), use ``replace = "if"`` with ``factor.NAME`` conditions.

docs/tutorial/getting-started.rst

@@ -222,6 +222,10 @@ whichever value the environment picked:
     commands = [["pytest"]]
     description = "run the tests under Python {factor:py_version}"
 
+A group can also name the value to assume when an environment carries none of its factors, which saves repeating a
+fallback at every mention. To check one run against a version the matrix does not list, set ``TOX_FACTOR_py_version``
+for that run.
+
 See :ref:`env-base-templates` for details.
 
 ***************************

src/tox/config/loader/replacer.py

@@ -28,6 +28,8 @@
 REPLACE_END: Final[str] = "}"
 BACKSLASH_ESCAPE_CHARS: Final[tuple[str, ...]] = (ARG_DELIMITER, REPLACE_START, REPLACE_END, "[", "]")
 MAX_REPLACE_DEPTH: Final[int] = 100
+# lets a run resolve a labeled factor to a value the configuration does not list, without renaming environments
+_FACTOR_ENV_PREFIX: Final[str] = "TOX_FACTOR_"
 
 
 class MatchRecursionError(ValueError):
@@ -323,14 +325,16 @@ def replace_factor(conf: Config, args: list[str], conf_args: ConfigLoadArgs) ->
         raise MatchError(msg)
     label = args[0]
     default = ARG_DELIMITER.join(args[1:]) if len(args) > 1 else ""
-    labels = conf.factor_labels
-    if label not in labels or conf_args.env_name is None:
+    group = conf.factor_labels.get(label)
+    if group is None or conf_args.env_name is None:
         return default
+    if override := os.environ.get(f"{_FACTOR_ENV_PREFIX}{label}"):
+        return override
     env_factors = set(conf_args.env_name.split("-"))
-    for value in labels[label]:
+    for value in group.values:
         if value in env_factors:
             return value
-    return default
+    return default or group.default or ""
 
 
 __all__ = [

src/tox/config/loader/toml/_product.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from dataclasses import dataclass
 from itertools import product
 from typing import TYPE_CHECKING
 
@@ -11,6 +12,14 @@
     from tox.config.loader.toml._api import TomlTypes
 
 
+@dataclass(frozen=True)
+class FactorGroup:
+    """``default`` stands in wherever an environment name carries none of ``values``."""
+
+    values: list[str]
+    default: str | None = None
+
+
 def expand_product(value: dict[str, TomlTypes]) -> list[str]:
     """Expand a product dict into a flat list of environment names.
 
@@ -60,6 +69,21 @@ def expand_factor_group(group: TomlTypes) -> list[str]:
     raise TypeError(msg)
 
 
+def extract_default(group: TomlTypes, values: list[str]) -> str | None:
+    if not isinstance(group, dict):
+        return None
+    table = group if "prefix" in group else next(iter(group.values()), None)
+    if not isinstance(table, dict) or (default := table.get("default")) is None:
+        return None
+    if not isinstance(default, str):
+        msg = f"factor group 'default' must be a string, got {type(default).__name__}"
+        raise TypeError(msg)
+    if default not in values:
+        msg = f"factor group 'default' {default!r} is not one of its factors: {', '.join(values)}"
+        raise TypeError(msg)
+    return default
+
+
 def extract_label(group: TomlTypes) -> str | None:
     if isinstance(group, dict) and "prefix" not in group and len(group) == 1:
         return str(next(iter(group)))
@@ -71,12 +95,18 @@ def _expand_labeled(label: str, values: TomlTypes) -> list[str]:
         msg = f"'{label}' is reserved and cannot be used as a factor label"
         raise TypeError(msg)
     if isinstance(values, dict):
-        if "prefix" not in values:
-            msg = f"labeled factor group '{label}' maps to a dict without a 'prefix' key, so it is not a range"
-            raise TypeError(msg)
-        return _expand_range(values)
+        if "prefix" in values:
+            return _expand_range(values)
+        if (listed := values.get("values")) is not None:
+            if not isinstance(listed, list):
+                msg = f"labeled factor group '{label}' 'values' must be a list, got {type(listed).__name__}"
+                raise TypeError(msg)
+            return [str(v) for v in listed]
+        msg = f"labeled factor group '{label}' maps to a dict with neither a 'prefix' nor a 'values' key"
+        raise TypeError(msg)
     if not isinstance(values, list):
-        msg = f"labeled factor group '{label}' must map to a list or a range dict, got {type(values).__name__}"
+        msg = f"labeled factor group '{label}' must map to a list, a range dict, or a values dict, "
+        msg += f"got {type(values).__name__}"
         raise TypeError(msg)
     return [str(v) for v in values]
 
@@ -101,7 +131,9 @@ def _expand_range(range_dict: dict[str, TomlTypes]) -> list[str]:
 
 __all__ = [
     "_RESERVED_LABELS",
+    "FactorGroup",
     "expand_factor_group",
     "expand_product",
+    "extract_default",
     "extract_label",
 ]

src/tox/config/main.py

@@ -13,6 +13,7 @@
     from collections.abc import Iterable, Iterator, Sequence
 
     from tox.config.loader.api import Loader, OverrideMap
+    from tox.config.loader.toml._product import FactorGroup
 
     from .cli.parser import Parsed
     from .loader.memory import MemoryLoader
@@ -139,7 +140,7 @@ def overrides(self) -> OverrideMap:
         return self._overrides
 
     @property
-    def factor_labels(self) -> dict[str, list[str]]:
+    def factor_labels(self) -> dict[str, FactorGroup]:
         return getattr(self._src, "_factor_labels", {})
 
     @property

src/tox/config/source/toml_pyproject.py

@@ -8,7 +8,7 @@
 
 from tox.config.loader.section import Section
 from tox.config.loader.toml import TomlLoader
-from tox.config.loader.toml._product import expand_factor_group, extract_label
+from tox.config.loader.toml._product import FactorGroup, expand_factor_group, extract_default, extract_label
 from tox.config.types import MissingRequiredConfigKeyError
 from tox.report import HandledError
 
@@ -183,9 +183,9 @@ def _table_at(content: dict[str, TomlTypes], keys: tuple[str, ...]) -> dict[str,
     return content
 
 
-def _build_env_base_map(env_base_content: dict[str, TomlTypes]) -> tuple[dict[str, str], dict[str, list[str]]]:
+def _build_env_base_map(env_base_content: dict[str, TomlTypes]) -> tuple[dict[str, str], dict[str, FactorGroup]]:
     result: dict[str, str] = {}
-    all_labels: dict[str, list[str]] = {}
+    all_labels: dict[str, FactorGroup] = {}
     for base_name, config in env_base_content.items():
         if not isinstance(config, dict):
             msg = f"env_base.{base_name} must be a table"
@@ -202,9 +202,10 @@ def _build_env_base_map(env_base_content: dict[str, TomlTypes]) -> tuple[dict[st
             for idx, g in enumerate(factors_raw):
                 values = expand_factor_group(g)
                 expanded.append(values)
-                all_labels[str(idx)] = values
+                group = FactorGroup(values=values, default=extract_default(g, values))
+                all_labels[str(idx)] = group
                 if (label := extract_label(g)) is not None:
-                    all_labels[label] = values
+                    all_labels[label] = group
             names = ["-".join(combo) for combo in product(*expanded)]
         else:
             names = [str(f) for f in factors_raw]
@@ -213,20 +214,21 @@ def _build_env_base_map(env_base_content: dict[str, TomlTypes]) -> tuple[dict[st
     return result, all_labels
 
 
-def _extract_env_list_labels(env_list_raw: TomlTypes) -> dict[str, list[str]]:
+def _extract_env_list_labels(env_list_raw: TomlTypes) -> dict[str, FactorGroup]:
     if not isinstance(env_list_raw, list):
         return {}
-    labels: dict[str, list[str]] = {}
+    labels: dict[str, FactorGroup] = {}
     for item in env_list_raw:
         if isinstance(item, dict) and "product" in item:
             raw_groups = item["product"]
             if not isinstance(raw_groups, list):
                 continue
             for idx, g in enumerate(raw_groups):
                 values = expand_factor_group(g)
-                labels[str(idx)] = values
+                group = FactorGroup(values=values, default=extract_default(g, values))
+                labels[str(idx)] = group
                 if (label := extract_label(g)) is not None:
-                    labels[label] = values
+                    labels[label] = group
     return labels
 
 

src/tox/session/cmd/schema.py

@@ -204,10 +204,21 @@ def gen_schema(state: State) -> int:
                     "prefix": {"type": "string"},
                     "start": {"type": "integer"},
                     "stop": {"type": "integer"},
+                    "default": {"type": "string"},
                 },
                 "additionalProperties": False,
                 "description": "range factor group: expands to prefix+N for N in [start, stop]",
             },
+            "factor_values_dict": {
+                "type": "object",
+                "required": ["values"],
+                "properties": {
+                    "values": {"type": "array", "items": {"type": "string"}},
+                    "default": {"type": "string"},
+                },
+                "additionalProperties": False,
+                "description": "factor group with an explicit list and a default used when none of it is active",
+            },
             "factor_labeled_dict": {
                 "type": "object",
                 "minProperties": 1,
@@ -217,6 +228,7 @@ def gen_schema(state: State) -> int:
                     "oneOf": [
                         {"type": "array", "items": {"type": "string"}},
                         {"$ref": "#/definitions/factor_range_dict"},
+                        {"$ref": "#/definitions/factor_values_dict"},
                     ]
                 },
                 "description": "labeled factor group for {factor:label} substitution",