Skip to content

Commit 4a8f153

Browse files
committed
Support Kedro 1.x while preserving 0.19 compatibility
- widen the Kedro dependency from `>0.18.4, <=0.19.9` to `>0.18.4, <2.0` so projects can upgrade to Kedro 1.x to pick up security fixes - keep backward compatibility for Kedro 0.19 via an AbstractConfigLoader shim - remove dead Kedro 0.16 compatibility code and the unused semver dependency - replace the Pydantic v1-only deep_update import in tests with a stdlib helper - update docs and changelog for the revised Kedro compatibility story - make MLflow-related tests self-contained so the default test suite passes without optional mlflow/kedro-mlflow packages installed
1 parent f6d6974 commit 4a8f153

12 files changed

Lines changed: 1244 additions & 377 deletions

CHANGELOG.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,24 @@
22

33
## [Unreleased]
44

5+
### Changed
6+
7+
- Widened Kedro dependency to `>0.18.4, <2.0` to support both Kedro 0.19.x and 1.x
8+
- Updated minimum Python version to 3.9 and upper bound to <3.13
9+
- Added compatibility shim for `AbstractConfigLoader` removal in Kedro 1.x
10+
- Updated documentation: dynamic configuration examples now use `OmegaConfigLoader` with `oc.env` resolver instead of deprecated `TemplatedConfigLoader`
11+
12+
### Removed
13+
14+
- Removed `ContextHelper16` class (Kedro 0.16 compatibility — dead code since minimum is 0.18.4)
15+
- Removed `semver` dependency (only used for obsolete version check)
16+
17+
### Fixed
18+
19+
- Replaced Pydantic v1-only `pydantic.utils.deep_update` import in tests with stdlib implementation
20+
- Fixed internal module path in `test_kfpclient.py` for `KedroContext` mock patch
21+
- Fixed mlflow-related tests to not require `mlflow`/`kedro_mlflow` packages installed (using `sys.modules` stubbing)
22+
523
## [0.10.0] - 2026-04-27
624

725
- Add support for `kedro 0.19.9`

docs/conf.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
author = "GetInData"
2929

3030
myst_substitutions = {
31-
"tested_kedro": "0.17.7",
31+
"tested_kedro": "1.0.0",
3232
"release": release,
3333
}
3434

docs/source/02_installation/02_configuration.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -176,12 +176,14 @@ run_config:
176176
177177
## Dynamic configuration support
178178
179-
`kedro-kubeflow` contains hook that enables [TemplatedConfigLoader](https://kedro.readthedocs.io/en/stable/kedro.config.TemplatedConfigLoader.html).
180-
It allows passing environment variables to configuration files. It reads all environment variables following `KEDRO_CONFIG_<NAME>` pattern, which you
181-
can later inject in configuration file using `${name}` syntax.
179+
`kedro-kubeflow` supports dynamic configuration via Kedro's `OmegaConfigLoader` with the `oc.env` resolver.
180+
It allows passing environment variables to configuration files. It reads all environment variables following `KEDRO_CONFIG_<NAME>` pattern, which you
181+
can later inject in configuration file using `${oc.env:KEDRO_CONFIG_NAME}` syntax.
182182

183-
There are two special variables `KEDRO_CONFIG_COMMIT_ID`, `KEDRO_CONFIG_BRANCH_NAME` with support specifying default when variable is not set,
184-
e.g. `${commit_id|dirty}`
183+
There are two special variables `KEDRO_CONFIG_COMMIT_ID`, `KEDRO_CONFIG_BRANCH_NAME` with support specifying default when variable is not set,
184+
e.g. `${oc.env:KEDRO_CONFIG_COMMIT_ID, dirty}`
185+
186+
> **Note:** The legacy `EnvTemplatedConfigLoader` is deprecated and will be removed in a future release. Please migrate to `OmegaConfigLoader` with `oc.env` resolver.
185187

186188
## Extra volumes
187189
You can mount additional volumes (such as `emptyDir`) to specific nodes by using `extra_volumes` config node.

kedro_kubeflow/context_helper.py

Lines changed: 9 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,21 @@
22
from functools import cached_property, lru_cache
33
from typing import Any, Dict
44

5-
from kedro import __version__ as kedro_version
65
from kedro.config import (
7-
AbstractConfigLoader,
86
MissingConfigException,
97
OmegaConfigLoader,
108
)
9+
10+
try:
11+
from kedro.config import AbstractConfigLoader
12+
except ImportError:
13+
# AbstractConfigLoader was removed in Kedro 1.0;
14+
# OmegaConfigLoader is the only config loader from that version on.
15+
AbstractConfigLoader = OmegaConfigLoader
16+
1117
from kedro.framework.session import KedroSession
1218
from omegaconf import DictConfig, OmegaConf
1319
from omegaconf.resolvers import oc
14-
from semver import VersionInfo
1520

1621
from .config import PluginConfig
1722

@@ -146,20 +151,4 @@ def kfp_client(self):
146151

147152
@staticmethod
148153
def init(metadata, env):
149-
version = VersionInfo.parse(kedro_version)
150-
if version.match(">=0.17.0"):
151-
return ContextHelper(metadata, env)
152-
else:
153-
return ContextHelper16(metadata, env)
154-
155-
156-
class ContextHelper16(ContextHelper):
157-
"""KedroKubeflowConfig vairant for compatibility with Kedro 1.6"""
158-
159-
@property
160-
def project_name(self):
161-
return self.context.project_path.name
162-
163-
@property
164-
def context(self):
165-
return self.session.load_context()
154+
return ContextHelper(metadata, env)

poetry.lock

Lines changed: 1159 additions & 295 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,16 +34,15 @@ line-length = 121
3434
line-length = 121
3535

3636
[tool.isort]
37-
known_third_party = ["click", "google", "kedro", "kfp", "kubernetes", "tabulate", "pydantic", "semver", "setuptools"]
37+
known_third_party = ["click", "google", "kedro", "kfp", "kubernetes", "tabulate", "pydantic", "setuptools"]
3838

3939
[tool.poetry.dependencies]
40-
python = ">=3.8,<3.12"
41-
kedro = ">0.18.4, <=0.19.9"
40+
python = ">=3.9,<3.13"
41+
kedro = ">0.18.4, <2.0"
4242
# kedro-viz = { version = "<8", optional = true} # because of pydantic conflict
4343
click = ">=8.0.4"
4444
kfp = ">=1.8.12,<2.0" # blocking pydantic upgrade
4545
tabulate = ">=0.8.7"
46-
semver = "~=2.10"
4746
fsspec = ">=2021.4"
4847
pyyaml = ">=6.0,<7.0"
4948
google-auth = { version = "<3", optional = true}

tests/common.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,14 @@
1-
from pydantic.utils import deep_update
1+
import copy
2+
3+
4+
def deep_update(mapping, updating_mapping):
5+
updated_mapping = copy.deepcopy(mapping)
6+
for k, v in updating_mapping.items():
7+
if k in updated_mapping and isinstance(updated_mapping[k], dict) and isinstance(v, dict):
8+
updated_mapping[k] = deep_update(updated_mapping[k], v)
9+
else:
10+
updated_mapping[k] = v
11+
return updated_mapping
212

313

414
class MinimalConfigMixin:

tests/test_cli.py

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -270,19 +270,17 @@ def test_init_with_github_actions(self, cwd):
270270
assert "kedro kubeflow upload-pipeline" in content
271271
assert "kedro kubeflow schedule" in content
272272

273-
@patch("mlflow.start_run")
274-
@patch("mlflow.set_tag")
275-
@patch("mlflow.get_experiment_by_name")
276-
def test_mlflow_start(self, get_experiment_by_name_mock, set_tag_mock, start_run_mock):
273+
def test_mlflow_start(self):
277274
context_helper = MagicMock(ContextHelper)
278275
config = dict(context_helper=context_helper)
279276
runner = CliRunner()
280-
get_experiment_by_name_mock.return_value = type("obj", (object,), {"experiment_id": 47})
281-
start_run_mock.return_value = namedtuple("InfoObject", "info")(
277+
mlflow_mock = MagicMock()
278+
mlflow_mock.get_experiment_by_name.return_value = type("obj", (object,), {"experiment_id": 47})
279+
mlflow_mock.start_run.return_value = namedtuple("InfoObject", "info")(
282280
namedtuple("RunIdObject", "run_id")("MLFLOW_RUN_ID")
283281
)
284282

285-
with TemporaryDirectory() as temp_dir:
283+
with patch.dict("sys.modules", {"mlflow": mlflow_mock}), TemporaryDirectory() as temp_dir:
286284
run_id_file_path = f"{temp_dir}/run_id"
287285
result = runner.invoke(
288286
mlflow_start,
@@ -295,7 +293,7 @@ def test_mlflow_start(self, get_experiment_by_name_mock, set_tag_mock, start_run
295293
with open(run_id_file_path) as f:
296294
assert f.read() == "MLFLOW_RUN_ID"
297295

298-
set_tag_mock.assert_called_with("kubeflow_run_id", "KUBEFLOW_RUN_ID")
296+
mlflow_mock.set_tag.assert_called_with("kubeflow_run_id", "KUBEFLOW_RUN_ID")
299297

300298
@patch("kubernetes.client")
301299
@patch("kubernetes.config")

tests/test_context_helper.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111
from kedro_kubeflow.config import PluginConfig
1212
from kedro_kubeflow.context_helper import (
1313
ContextHelper,
14-
ContextHelper16,
1514
EnvTemplatedConfigLoader,
1615
)
1716

@@ -20,11 +19,6 @@
2019

2120

2221
class TestContextHelper(unittest.TestCase, MinimalConfigMixin):
23-
def test_init_different_kedro_versions(self):
24-
25-
with patch("kedro_kubeflow.context_helper.kedro_version", "0.16.0"):
26-
ch = ContextHelper.init(None, None)
27-
assert isinstance(ch, ContextHelper16)
2822

2923
def test_project_name(self):
3024
metadata = Mock()

tests/test_hooks.py

Lines changed: 25 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
import os
22
import unittest
3-
from unittest.mock import patch
4-
5-
import mlflow
3+
from unittest.mock import MagicMock, patch
64

75
from kedro_kubeflow.auth import AuthHandler
86
from kedro_kubeflow.hooks import MlflowIapAuthHook, MlflowTagsHook # NOQA
@@ -19,43 +17,39 @@ def test_should_inject_token_when_env_is_set(self, obtain_id_token):
1917
obtain_id_token.assert_called_with()
2018

2119

22-
@patch.object(mlflow, "set_tag")
2320
class TestMlflowTagsHook(unittest.TestCase):
24-
def test_should_set_mlflow_tags(self, mlflow_set_tag):
25-
with environment({"KUBEFLOW_RUN_ID": "KFP_123"}):
21+
def test_should_set_mlflow_tags(self):
22+
mlflow_mock = MagicMock()
23+
with patch.dict("sys.modules", {"mlflow": mlflow_mock, "kedro_mlflow": MagicMock()}), environment(
24+
{"KUBEFLOW_RUN_ID": "KFP_123"}
25+
):
2626
MlflowTagsHook().before_node_run()
2727

28-
mlflow_set_tag.assert_called_with("kubeflow_run_id", "KFP_123")
28+
mlflow_mock.set_tag.assert_called_with("kubeflow_run_id", "KFP_123")
2929

30-
def test_should_not_set_mlflow_tags_when_kubeflow_run_id_env_is_not_set(self, mlflow_set_tag):
31-
with environment({}, delete_keys=["KUBEFLOW_RUN_ID"]):
30+
def test_should_not_set_mlflow_tags_when_kubeflow_run_id_env_is_not_set(self):
31+
mlflow_mock = MagicMock()
32+
with patch.dict("sys.modules", {"mlflow": mlflow_mock, "kedro_mlflow": MagicMock()}), environment(
33+
{}, delete_keys=["KUBEFLOW_RUN_ID"]
34+
):
3235
MlflowTagsHook().before_node_run()
3336

34-
mlflow_set_tag.assert_not_called()
37+
mlflow_mock.set_tag.assert_not_called()
3538

36-
def test_should_not_set_mlflow_tags_when_kubeflow_run_id_env_is_empty(self, mlflow_set_tag):
37-
with environment({"KUBEFLOW_RUN_ID": ""}):
39+
def test_should_not_set_mlflow_tags_when_kubeflow_run_id_env_is_empty(self):
40+
mlflow_mock = MagicMock()
41+
with patch.dict("sys.modules", {"mlflow": mlflow_mock, "kedro_mlflow": MagicMock()}), environment(
42+
{"KUBEFLOW_RUN_ID": ""}
43+
):
3844
MlflowTagsHook().before_node_run()
3945

40-
mlflow_set_tag.assert_not_called()
41-
42-
def test_should_not_set_mlflow_tags_when_mlflow_is_not_enabled(self, mlflow_set_tag):
43-
# given
44-
real_import = __builtins__["__import__"]
45-
46-
def mlflow_import_disabled(name, *args, **kw):
47-
if name == "mlflow":
48-
raise ImportError
49-
return real_import(name, *args, **kw)
46+
mlflow_mock.set_tag.assert_not_called()
5047

51-
__builtins__["__import__"] = mlflow_import_disabled
52-
53-
# when
48+
@patch("kedro_kubeflow.hooks.is_mlflow_enabled", return_value=False)
49+
def test_should_not_set_mlflow_tags_when_mlflow_is_not_enabled(self, _):
50+
mlflow_mock = MagicMock()
5451
with environment({"KUBEFLOW_RUN_ID": "KFP_123"}):
55-
MlflowTagsHook().before_node_run()
56-
57-
# then
58-
mlflow_set_tag.assert_not_called()
52+
with patch.dict("sys.modules", {"mlflow": mlflow_mock}):
53+
MlflowTagsHook().before_node_run()
5954

60-
# cleanup
61-
__builtins__["__import__"] = real_import
55+
mlflow_mock.set_tag.assert_not_called()

0 commit comments

Comments
 (0)