Add a CI check to ensure packages entries in pyproject.toml are up to date #29

VaniHaripriya · 2025-12-17T15:01:35Z

Description of your changes:

Added a new CI check to validate the package entries in pyproject.toml.

Sample run for an example of the CI working.

Checklist:

Pre-Submission Checklist

All tests and CI checks pass
Pre-commit hooks pass without errors
You have signed off your commits
The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

Additional Checklist Items for New or Updated Components/Pipelines:

metadata.yaml includes fresh lastVerified timestamp
All required files are present and complete
OWNERS file lists appropriate maintainers
README provides clear documentation with usage examples
Component follows snake_case naming convention
No security vulnerabilities in dependencies

hbelmiro · 2025-12-29T20:36:50Z

scripts/validate_package_entries/tests/test_validate_package_entries.py

+        (tmp_path / "__init__.py").write_text("")
+
+        components_dir = tmp_path / "components"
+        components_dir.mkdir()
+        (components_dir / "__init__.py").write_text("")
+
+        training_dir = components_dir / "training"
+        training_dir.mkdir()
+        (training_dir / "__init__.py").write_text("")


This code is duplicated. It appears 3 times in this file.

Removed the duplicate code and created common directory structure for the tests.

hbelmiro · 2025-12-29T20:38:11Z

scripts/validate_package_entries/validate_package_entries.py

+    if (repo_root / "__init__.py").exists():
+        packages.add("kfp_components")
+
+    def _discover_recursive(directory: Path, base_package: str) -> None:


base_package is unused.

Updated the function to use base_package.

hbelmiro · 2025-12-29T20:42:06Z

scripts/validate_package_entries/validate_package_entries.py

+    except FileNotFoundError:
+        raise RuntimeError(f"pyproject.toml not found at {pyproject_path}")
+    except tomllib.TOMLDecodeError as e:
+        raise RuntimeError(f"Failed to parse pyproject.toml: {e}")


The tomllib.TOMLDecodeError is caught and re-raised as a RuntimeError, which loses the original traceback context.

Suggested change

raise RuntimeError(f"Failed to parse pyproject.toml: {e}")

raise RuntimeError(f"Failed to parse pyproject.toml: {e}") from e

hbelmiro · 2025-12-29T20:53:22Z

scripts/validate_package_entries/tests/test_validate_package_entries.py

+        assert not is_valid
+        assert len(errors) == 1
+        assert "Extra packages" in errors[0]
+        assert "kfp_components.components.nonexistent" in errors[0]


Don't we need to verify kfp_components.components too?

Good catch! added an assert statement for kfp_components.components.

hbelmiro · 2025-12-29T21:01:51Z

scripts/validate_package_entries/validate_package_entries.py

+import sys
+import tomllib
+from pathlib import Path
+from typing import Set


The Set import from typing isn't needed in Python 3.9+. You can use builtin set[str] directly (like you already do with list[str] on lines 90 and 102). Consider updating lines 24, 30, and 66 to use set[str] and removing this import for consistency.

Removed Set import and updated with set[str].

google-oss-prow · 2026-01-05T19:00:24Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mprahl for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hbelmiro · 2026-01-06T13:30:24Z

scripts/validate_package_entries/tests/test_validate_package_entries.py

+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from validate_package_entries import (
+    discover_packages,
+    read_pyproject_packages,
+    validate_package_entries,
+)


We're no longer using sys.path.insert(0, str(Path(__file__).parent.parent)). We're creating a __init.py__ file and using relative imports instead.
PTAL at https://github.com/kubeflow/pipelines-components/blob/main/scripts/README.md#import-conventions for more info.

Updated to use relative imports.

hbelmiro · 2026-01-06T13:34:26Z

scripts/validate_package_entries/validate_package_entries.py

+def get_repo_root() -> Path:
+    """Get the repository root directory."""
+    return Path(__file__).parent.parent.parent.resolve()


I've sent a PR that includes this method as a lib for reuse.
#57
If that PR is merged before this one, please use that instead of duplicating it here.

Updated to reuse get_repo_root() from common utility functions.

We need to hold this PR until #57 gets merged.

docs/CONTRIBUTING.md

hbelmiro · 2026-01-09T19:20:36Z

scripts/validate_package_entries/validate_package_entries.py

+    if (repo_root / "__init__.py").exists():
+        packages.add("kfp_components")
+
+    def _discover_recursive(directory: Path, base_package: str) -> None:


Optional:
Is this defined inside another function by purpose?
Personally, I think it's ugly. I understand though that it allows us to use packages without passing it as a parameter.
I'm fine with leaving it as it is, unless it was not intentional.

Updated _discover_recursive as a separate function that takes packages as a parameter.

Signed-off-by: VaniHaripriya <[email protected]>

hbelmiro

/lgtm

google-oss-prow bot requested review from HumairAK and mprahl December 17, 2025 15:01

google-oss-prow bot added the size/XL label Dec 17, 2025

VaniHaripriya marked this pull request as draft December 17, 2025 15:02

google-oss-prow bot added the do-not-merge/work-in-progress label Dec 17, 2025

VaniHaripriya force-pushed the package-validation-ci branch from f753cce to 73677bf Compare December 17, 2025 20:08

google-oss-prow bot added size/L and removed size/XL labels Dec 17, 2025

VaniHaripriya force-pushed the package-validation-ci branch 2 times, most recently from 0d33245 to eddc10f Compare December 17, 2025 20:17

VaniHaripriya marked this pull request as ready for review December 17, 2025 20:17

google-oss-prow bot removed the do-not-merge/work-in-progress label Dec 17, 2025

VaniHaripriya force-pushed the package-validation-ci branch 3 times, most recently from 660ff58 to ddc342c Compare December 18, 2025 22:53

hbelmiro reviewed Dec 29, 2025

View reviewed changes

VaniHaripriya force-pushed the package-validation-ci branch 3 times, most recently from 3f582eb to 1b515f5 Compare January 5, 2026 19:00

hbelmiro reviewed Jan 6, 2026

View reviewed changes

VaniHaripriya force-pushed the package-validation-ci branch 5 times, most recently from 7809865 to 1fdbbea Compare January 7, 2026 22:12

VaniHaripriya requested a review from hbelmiro January 7, 2026 22:13

hbelmiro reviewed Jan 9, 2026

View reviewed changes

test:Add CI check to ensure package entries are updated

79cc441

Signed-off-by: VaniHaripriya <[email protected]>

VaniHaripriya force-pushed the package-validation-ci branch from 0da92b7 to 79cc441 Compare January 9, 2026 20:36

VaniHaripriya requested a review from hbelmiro January 9, 2026 20:37

hbelmiro reviewed Jan 9, 2026

View reviewed changes

google-oss-prow bot assigned hbelmiro Jan 9, 2026

google-oss-prow bot added the lgtm label Jan 9, 2026

mprahl merged commit f75900f into kubeflow:main Jan 12, 2026
15 of 16 checks passed

	raise RuntimeError(f"Failed to parse pyproject.toml: {e}")
	raise RuntimeError(f"Failed to parse pyproject.toml: {e}") from e

Add a CI check to ensure packages entries in pyproject.toml are up to date #29

Add a CI check to ensure packages entries in pyproject.toml are up to date #29

Uh oh!

Conversation

VaniHaripriya commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Submission Checklist

Additional Checklist Items for New or Updated Components/Pipelines:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

google-oss-prow bot commented Jan 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hbelmiro Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hbelmiro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

VaniHaripriya commented Dec 17, 2025 •

edited

Loading

hbelmiro Jan 9, 2026 •

edited

Loading