[Bug]: py_pex_binary incompatible with newer rules_python versions

[kwlzn]

What happened?

when attempting to version bump rules_python to the latest, we observed failures during pex builds coming from the aspect_rules_py==1.6.0 py_pex_binary rule:

ERROR: /workspaces/.../tools/manufacturing_test/eng_tools/io_gui/BUILD:94:14: Building PEX binary //tools/manufacturing_test/eng_tools/io_gui:io_gui_pex failed: (Exit 1): pex failed: error executing PyPex command (from target //tools/manufacturing_test/eng_tools/io_gui:io_gui_pex) bazel-out/k8-opt-exec-ST-ff2b6a5febf8/bin/external/aspect_rules_py~/py/tools/pex/pex '--sys-path=rules_python~~pip~pypi_310_flatbuffers/site-packages' ... (remaining 88 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Traceback (most recent call last):
  File "/home/vscode/.cache/bazel/_bazel_vscode/d9fd8dd9485b3d1d1994c3846baefa35/sandbox/processwrapper-sandbox/61/execroot/_main/bazel-out/k8-opt-exec-ST-ff2b6a5febf8/bin/external/aspect_rules_py~/py/tools/pex/pex.runfiles/aspect_rules_py~/py/tools/pex/main.py", line 162, in <module>
    dist = Distribution.load(dep)
  File "/home/vscode/.cache/bazel/_bazel_vscode/d9fd8dd9485b3d1d1994c3846baefa35/sandbox/processwrapper-sandbox/61/execroot/_main/bazel-out/k8-opt-exec-ST-ff2b6a5febf8/bin/external/aspect_rules_py~/py/tools/pex/pex.runfiles/.pex.venv/lib/python3.10/site-packages/pex/dist_metadata.py", line 885, in load
    return cls(location=location, metadata=DistMetadata.load(location))
  File "/home/vscode/.cache/bazel/_bazel_vscode/d9fd8dd9485b3d1d1994c3846baefa35/sandbox/processwrapper-sandbox/61/execroot/_main/bazel-out/k8-opt-exec-ST-ff2b6a5febf8/bin/external/aspect_rules_py~/py/tools/pex/pex.runfiles/.pex.venv/lib/python3.10/site-packages/pex/dist_metadata.py", line 807, in load
    raise MetadataError(
pex.dist_metadata.MetadataError: Failed to determine project name and version for distribution at bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages.

where the referenced dir contained only python sources and no e.g. dist-info:

$ find bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing/tests
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing/tests/data
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing/tests/data/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing/tests/data/pass
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/typing/tests/data/pass/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples/cffi
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples/cffi/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples/numba
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/random/_examples/numba/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples/cython
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples/cython/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples/limited_api
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy/core/tests/examples/limited_api/__init__.py
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy.libs
bazel-out/k8-opt-release/bin/external/rules_python~~pip~pypi_310_numpy/site-packages/numpy.libs/__init__.py

Version

Development (host) and target OS/architectures: Linux x86_64

Output of bazel --version: 7.6.0

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file: bazel_dep(name = "aspect_rules_py", version = "1.6.0")

Language(s) and/or frameworks involved: python

How to reproduce

combine these rulesets in `MODULE.bazel` and build a test pex with thirdparty deps:


bazel_dep(name = "aspect_rules_py", version = "1.6.0")
bazel_dep(name = "rules_python", version = "1.5.1")

Any other information?

No response

[kwlzn]

seems like rules_python==1.4.1 might be the last compatible release

[Cartman75]

I can use aspect_rules_py 1.6.1 and rules_python 1.5.0 before the above error starts happening.

But I get this error if I upgrade to rules_python 1.5.1 as well.