feat: Add UV package manager support for Python projects #4616
Conversation
- Implement UvPyprojectTomlHandler to parse pyproject.toml with [tool.uv] sections - Implement UvLockHandler to parse uv.lock lockfiles - Add BaseUvPythonLayout for UV project assembly logic - Add is_uv_pyproject_toml() helper for UV project detection - Add parse_dependency_requirement() helper for dependency parsing - Support standard dependencies, dev-dependencies, and optional groups - Handle UV lockfile format with [[package]] entries and markers - Add comprehensive test fixtures and test cases Signed-off-by: Shekhar Suman [email protected] Signed-off-by: Shekhar <[email protected]>
levi42x
force-pushed
the
feature/parse-uv_package
branch
from
64d01f0 to
900452d
Compare
AyanSinhaMahapatra
left a comment
AyanSinhaMahapatra
left a comment
There was a problem hiding this comment.
Thanks++ @levi42x for the PR, much needed update! ❤️
This is a great start and a step in the right direction, but needs some more updates and love. See my comments for your consideration.
| yield from yield_dependencies_from_package_resource(resource) | ||
| return | ||
|
|
||
| if codebase.has_single_resource: |
There was a problem hiding this comment.
This should be at the beginning of the function
| package = pypi.UvLockHandler.parse(test_file) | ||
| expected_loc = self.get_test_loc('pypi/uv/attrs-uv.lock-expected.json') | ||
| self.check_packages_data(package, expected_loc, regen=REGEN_TEST_FIXTURES) | ||
|
|
There was a problem hiding this comment.
Can you also add an end-to-end example with a --package scan (see the test file for examples, add a test similarly) in a uv package manifest layout and show that the package assembly works as expected. You've added an assemble function, but this is not tested otherwise.
| @@ -0,0 +1,23 @@ | |||
| [project] | |||
| name = "attrs-example" | |||
There was a problem hiding this comment.
Please use real world examples, and include the link
There was a problem hiding this comment.
If you include a file that is large, please shorten it to only keep the relevant parts which you need to test the parser for, as we want to keep filesize and repo size as minimal as possible
There was a problem hiding this comment.
also find a file which has all/almost all the values which could be present, for example this fake example doesn't have licenses, copyrights and other usable package data fields.
|
|
||
| extra_data = {} | ||
| extra_data['python_version'] = requires_python | ||
| extra_data['lock_version'] = version |
There was a problem hiding this comment.
| extra_data['lock_version'] = version | |
| extra_data['lock_version'] = lock_version |
this would be confusing with the proper package version, let's be as descriptive as possible in variable names to improve readability
There was a problem hiding this comment.
this is also likely wrong as you're using the same variable for version
| extra_data['lock_version'] = version | ||
|
|
||
| package_data = dict( | ||
| datasource_id=cls.datasource_id, |
There was a problem hiding this comment.
there are a lot more package data fields which can be parsed and added, please modify to include those. See
scancode-toolkit/src/packagedcode/models.py
Line 548 in c05a13d
| type=cls.default_package_type, | ||
| primary_language='Python', | ||
| extra_data=extra_data, | ||
| dependencies=dependencies, |
There was a problem hiding this comment.
we also need to populate URLs with packageURL fields, see
scancode-toolkit/src/packagedcode/pypi.py
Line 794 in c05a13d
|
|
||
| def is_uv_pyproject_toml(location): | ||
| with open(location, 'r') as file: | ||
| data = file.read() |
There was a problem hiding this comment.
| data = file.read() | |
| if "tool.uv" in file.read(): | |
| return True | |
| return False |
| yield models.PackageData.from_data(package_data, package_only) | ||
|
|
||
|
|
||
| def parse_dependency_requirement(requirement, scope='dependencies', is_runtime=True): |
There was a problem hiding this comment.
Could this be replaced by using get_requires_dependencies? The processing looks similar.
Or maybe abstract out similar code into functions that both these functions use? As we atleast have some code duplication here.
2 participants
Fixes #4501
Description
This PR implements support for parsing Python UV package manager manifests in scancode-toolkit.
Changes
UvPyprojectTomlHandler: Parsespyproject.tomlfiles with[tool.uv]sectionsUvLockHandler: Parsesuv.locklockfiles with full dependency resolutionBaseUvPythonLayout: Manages UV project assembly and resource coordinationis_uv_pyproject_toml()for UV project detectionparse_dependency_requirement()for PEP 508 dependency parsingTasks
Run tests locally to check for errors.
Signed-off-by: Shekhar Suman [email protected]