Skip to content

install: Support "pkg @ https://…/pkg.whl" syntax #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+19 −14
Open

install: Support "pkg @ https://…/pkg.whl" syntax #254

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
900feb7
install: Support "pkg @ https://…/pkg.whl" syntax
chrysn Sep 17, 2025
6a739e5
transaction: Split overly complex function
chrysn Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 19 additions & 14 deletions micropip/transaction.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,21 +80,15 @@ async def add_requirement(self, req: str | Requirement) -> None:
try:
as_req = constrain_requirement(Requirement(req), self.constrained_reqs)
except InvalidRequirement:
as_req = None

if as_req:
if as_req.name and len(as_req.specifier):
return await self.add_requirement_inner(as_req)
if as_req.url:
req = as_req.url

if urlparse(req).path.endswith(".whl"):
# custom download location
wheel = WheelInfo.from_url(req)
check_compatible(wheel.filename)
return await self.add_wheel(wheel, extras=set(), specifier="")
Comment on lines -91 to -95
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to check the parsed url after converting the str to Requirement here. We don't support non .whl suffixes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this patch, attempting to load a non-wheel (trying with a tarball) resulted in a 2-deep parsing error. With this PR, this doesn't change for the await micropip.install("https://.../...tar.gz") case, but attempting to install package @ https://…/package….tar.gz gives a clearer "Invalid wheel filename (extension must be '.whl'): 'aiocoap-0.4.14.tar.gz'" error. (Previously, using the @ syntax also gave the old error).

I don't think we need further checking: Running with a non-wheel gives an error either way, the error message gives a way forward, and the less we explicitly check and just let the errors be clear, the easier any later additions can be. (I don't think we'll need support for .tar.gz ever because there are platform-independent wheels, but then again, I've used Python long enough to remember when it laid eggs, so maybe something else comes along).

Old error 
Traceback (most recent call last):
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/_parser.py", line 62, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/_parser.py", line 80, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
                             ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/_parser.py", line 124, in _parse_requirement_details
    marker = _parse_requirement_marker(
        tokenizer,
    ...<5 lines>...
        ),
    )
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/_parser.py", line 145, in _parse_requirement_marker
    tokenizer.raise_syntax_error(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        f"Expected end or semicolon (after {after})",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        span_start=span_start,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/_tokenizer.py", line 167, in raise_syntax_error
    raise ParserSyntaxError(
    ...<3 lines>...
    )
micropip._vendored.packaging.src.packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after name and no valid version specifier)
    https://files.pythonhosted.org/packages/28/76/de52f7fa51ddbb5255f7b80d8965903bfa718420ac5d21b0852f5d81c1b1/aiocoap-0.4.14.tar.gz
         ^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/lib/python3.13/site-packages/micropip/package_manager.py", line 198, in install
    await transaction.gather_requirements(requirements)
  File "/lib/python3.13/site-packages/micropip/transaction.py", line 75, in gather_requirements
    await asyncio.gather(*requirement_promises)
  File "/lib/python3.13/site-packages/micropip/transaction.py", line 98, in add_requirement
    return await self.add_requirement_inner(Requirement(req))
                                            ~~~~~~~~~~~^^^^^
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/requirements.py", line 38, in __init__
    raise InvalidRequirement(str(e)) from e
micropip._vendored.packaging.src.packaging.requirements.InvalidRequirement: Expected end or semicolon (after name and no valid version specifier)
    https://files.pythonhosted.org/packages/28/76/de52f7fa51ddbb5255f7b80d8965903bfa718420ac5d21b0852f5d81c1b1/aiocoap-0.4.14.tar.gz
         ^
New error when installing with @ syntax 
>>> await micropip.install("aiocoap @ https://files.pythonhosted.org/packages/28/76/de52f7fa51ddbb5255f7b80d8965903bfa718420ac5d21b0852f5
d81c1b1/aiocoap-0.4.14.tar.gz")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/lib/python3.13/site-packages/micropip/package_manager.py", line 205, in install
    await transaction.gather_requirements(requirements)
  File "/lib/python3.13/site-packages/micropip/transaction.py", line 74, in gather_requirements
    await asyncio.gather(*requirement_promises)
  File "/lib/python3.13/site-packages/micropip/transaction.py", line 91, in add_requirement
    return await self.add_requirement_inner(as_req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.13/site-packages/micropip/transaction.py", line 199, in add_requirement_inner
    wheel = WheelInfo.from_url(req.url)
  File "/lib/python3.13/site-packages/micropip/wheelinfo.py", line 92, in from_url
    name, version, build, tags = parse_wheel_filename(file_name)
                                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/lib/python3.13/site-packages/micropip/_utils.py", line 86, in parse_wheel_filename
    return parse_wheel_filename_orig(filename)
  File "/lib/python3.13/site-packages/micropip/_vendored/packaging/src/packaging/utils.py", line 98, in parse_wheel_filename
    raise InvalidWheelFilename(
        f"Invalid wheel filename (extension must be '.whl'): {filename!r}"
    )
micropip._vendored.packaging.src.packaging.utils.InvalidWheelFilename: Invalid wheel filename (extension must be '.whl'): 'aiocoap-0.4.14
.tar.gz'
>>>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh, okay. Thanks for the traceback. That makes sense.

if urlparse(req).path.endswith(".whl"):
# custom download location
wheel = WheelInfo.from_url(req)
check_compatible(wheel.filename)
return await self.add_wheel(wheel, extras=set(), specifier="")
else:
raise

return await self.add_requirement_inner(Requirement(req))
return await self.add_requirement_inner(as_req)

def check_version_satisfied(
self, req: Requirement, *, allow_reinstall: bool = False
Expand Down Expand Up @@ -201,6 +195,17 @@ def eval_marker(e: dict[str, str]) -> bool:
logger.info("Requirement already satisfied: %s (%s)", req, ver)
return

if req.url:
wheel = WheelInfo.from_url(req.url)
check_compatible(wheel.filename)
return await self.add_wheel(wheel, extras=req.extras, specifier="")
else:
await self._add_requirement_by_name(req)

async def _add_requirement_by_name(
self,
req: Requirement,
) -> None:
try:
if self.search_pyodide_lock_first:
if await self._add_requirement_from_pyodide_lock(req):
Expand Down
Loading