Skip to content

Implement --uploaded-prior-to to filter packages by upload-time #13520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e1e72e7
Implement --upload-before
uranusjr May 20, 2024
c8b2481
Fix merge errors and rename to "exclude-newer-than"
notatallshaw Aug 6, 2025
a53d08a
Make common parse_iso_time
notatallshaw Aug 6, 2025
6db6c94
Add documentation on how to specify explicit timezone
notatallshaw Aug 6, 2025
c0ec2ec
Add exclude-newer tests
notatallshaw Aug 6, 2025
2bf1d3a
Pass exclude-newer-than to isolated build install
notatallshaw Aug 6, 2025
d5adbda
NEWS ENTRY
notatallshaw Aug 6, 2025
c374b2c
Fix linting
notatallshaw Aug 6, 2025
bc162b2
Merge branch 'main' into exclude-newer-than
notatallshaw Aug 8, 2025
72f363d
Add helpful error message on incorrect datetime format
notatallshaw Aug 9, 2025
181a7ca
Merge branch 'main' into exclude-newer-than
notatallshaw Aug 9, 2025
007caf6
Update tests/functional/test_exclude_newer.py
notatallshaw Aug 15, 2025
b53c5e8
Merge branch 'main' into exclude-newer-than
notatallshaw Aug 15, 2025
0f1bc46
Add `--no-deps` to request installs to not download unneeded packages
notatallshaw Aug 19, 2025
ad90024
Remove excessive functional tests
notatallshaw Aug 19, 2025
6ff91a4
Clean up test_finder tests
notatallshaw Aug 19, 2025
fbe923d
Update `test_handle_exclude_newer_than_naive_dates` comparison
notatallshaw Aug 19, 2025
703cdc4
Improve parameter formatting of `test_handle_exclude_newer_than_with_…
notatallshaw Aug 19, 2025
6cf2bec
Get exclude_newer_than from option
notatallshaw Aug 19, 2025
4713c6d
Add exclude-newer-than to the lock command
notatallshaw Aug 19, 2025
841ae12
Remove change in list, links, and wheel
notatallshaw Aug 19, 2025
61ec9b0
Update docs and news items to make clear index needs to provide `uplo…
notatallshaw Aug 19, 2025
e1f274a
Change name to uploaded prior to
notatallshaw Aug 21, 2025
e592e95
Add `--uploaded-prior-to` to the user guide
notatallshaw Aug 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/html/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,61 @@ e.g. http://example.com/constraints.txt, so that your organization can store and
serve them in a centralized place.


.. _`Filtering by Upload Time`:


Filtering by Upload Time
=========================

The ``--uploaded-prior-to`` option allows you to filter packages by their upload time
to an index, only considering packages that were uploaded before a specified datetime.
This can be useful for creating reproducible builds by ensuring you only install
packages that were available at a known point in time.

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install --uploaded-prior-to=2025-03-16T00:00:00Z SomePackage
.. tab:: Windows

.. code-block:: shell
py -m pip install --uploaded-prior-to=2025-03-16T00:00:00Z SomePackage
The option accepts ISO 8601 datetime strings in several formats:

* ``2025-03-16`` - Date in local timezone
* ``2025-03-16 12:30:00`` - Datetime in local timezone
* ``2025-03-16T12:30:00Z`` - Datetime in UTC
* ``2025-03-16T12:30:00+05:00`` - Datetime in UTC offset

For consistency across machines, use either UTC format (with 'Z' suffix) or UTC offset
format (with timezone offset like '+05:00'). Local timezone formats may produce different
results on different machines.

.. note::

This option only works with package indexes that provide upload-time metadata
(such as PyPI). When upload-time information is not available, packages are not
filtered and installation continues normally.

You can combine this option with other filtering mechanisms like constraints files:

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install -c constraints.txt --uploaded-prior-to=2025-03-16 SomePackage
.. tab:: Windows

.. code-block:: shell
py -m pip install -c constraints.txt --uploaded-prior-to=2025-03-16 SomePackage
.. _`Dependency Groups`:


Expand Down
2 changes: 2 additions & 0 deletions news/13520.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add ``--uploaded-prior-to`` option to only consider packages uploaded prior to
a given datetime when the ``upload-time`` field is available from an index.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ max-complexity = 33 # default is 10
[tool.ruff.lint.pylint]
max-args = 15 # default is 5
max-branches = 28 # default is 12
max-returns = 13 # default is 6
max-returns = 14 # default is 6
max-statements = 134 # default is 50

######################################################################################
Expand Down
2 changes: 2 additions & 0 deletions src/pip/_internal/build_env.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ def install(
args.append("--pre")
if finder.prefer_binary:
args.append("--prefer-binary")
if finder.uploaded_prior_to:
args.extend(["--uploaded-prior-to", finder.uploaded_prior_to.isoformat()])
args.append("--")
args.extend(requirements)
with open_spinner(f"Installing {kind}") as spinner:
Expand Down
49 changes: 49 additions & 0 deletions src/pip/_internal/cli/cmdoptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
from pip._internal.models.format_control import FormatControl
from pip._internal.models.index import PyPI
from pip._internal.models.target_python import TargetPython
from pip._internal.utils.datetime import parse_iso_datetime
from pip._internal.utils.hashes import STRONG_HASHES
from pip._internal.utils.misc import strtobool

Expand Down Expand Up @@ -796,6 +797,54 @@ def _handle_dependency_group(
help="Ignore the Requires-Python information.",
)


def _handle_uploaded_prior_to(
option: Option, opt: str, value: str, parser: OptionParser
) -> None:
"""
This is an optparse.Option callback for the --uploaded-prior-to option.
Parses an ISO 8601 datetime string. If no timezone is specified in the string,
local timezone is used.
Note: This option only works with indexes that provide upload-time metadata
as specified in the simple repository API:
https://packaging.python.org/en/latest/specifications/simple-repository-api/
"""
if value is None:
return None

try:
uploaded_prior_to = parse_iso_datetime(value)
# Use local timezone if no offset is given in the ISO string.
if uploaded_prior_to.tzinfo is None:
uploaded_prior_to = uploaded_prior_to.astimezone()
parser.values.uploaded_prior_to = uploaded_prior_to
except ValueError as exc:
msg = (
f"invalid --uploaded-prior-to value: {value!r}: {exc}. "
f"Expected an ISO 8601 datetime string, "
f"e.g '2023-01-01' or '2023-01-01T00:00:00Z'"
)
raise_option_error(parser, option=option, msg=msg)


uploaded_prior_to: Callable[..., Option] = partial(
Option,
"--uploaded-prior-to",
dest="uploaded_prior_to",
metavar="datetime",
action="callback",
callback=_handle_uploaded_prior_to,
type="str",
help=(
"Only consider packages uploaded prior to the given date time. "
"Accepts ISO 8601 strings (e.g., '2023-01-01T00:00:00Z'). "
"Uses local timezone if none specified. Only effective when "
"installing from indexes that provide upload-time metadata."
),
)

no_build_isolation: Callable[..., Option] = partial(
Option,
"--no-build-isolation",
Expand Down
1 change: 1 addition & 0 deletions src/pip/_internal/cli/req_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -348,4 +348,5 @@ def _build_package_finder(
link_collector=link_collector,
selection_prefs=selection_prefs,
target_python=target_python,
uploaded_prior_to=options.uploaded_prior_to,
)
1 change: 1 addition & 0 deletions src/pip/_internal/commands/download.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ def add_options(self) -> None:
self.cmd_opts.add_option(cmdoptions.no_use_pep517())
self.cmd_opts.add_option(cmdoptions.check_build_deps())
self.cmd_opts.add_option(cmdoptions.ignore_requires_python())
self.cmd_opts.add_option(cmdoptions.uploaded_prior_to())

self.cmd_opts.add_option(
"-d",
Expand Down
2 changes: 2 additions & 0 deletions src/pip/_internal/commands/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ def add_options(self) -> None:
cmdoptions.add_target_python_options(self.cmd_opts)

self.cmd_opts.add_option(cmdoptions.ignore_requires_python())
self.cmd_opts.add_option(cmdoptions.uploaded_prior_to())
self.cmd_opts.add_option(cmdoptions.pre())
self.cmd_opts.add_option(cmdoptions.json())
self.cmd_opts.add_option(cmdoptions.no_binary())
Expand Down Expand Up @@ -103,6 +104,7 @@ def _build_package_finder(
link_collector=link_collector,
selection_prefs=selection_prefs,
target_python=target_python,
uploaded_prior_to=options.uploaded_prior_to,
)

def get_available_package_versions(self, options: Values, args: list[Any]) -> None:
Expand Down
1 change: 1 addition & 0 deletions src/pip/_internal/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ def add_options(self) -> None:
),
)

self.cmd_opts.add_option(cmdoptions.uploaded_prior_to())
self.cmd_opts.add_option(cmdoptions.ignore_requires_python())
self.cmd_opts.add_option(cmdoptions.no_build_isolation())
self.cmd_opts.add_option(cmdoptions.use_pep517())
Expand Down
1 change: 1 addition & 0 deletions src/pip/_internal/commands/lock.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ def add_options(self) -> None:
self.cmd_opts.add_option(cmdoptions.src())

self.cmd_opts.add_option(cmdoptions.ignore_requires_python())
self.cmd_opts.add_option(cmdoptions.uploaded_prior_to())
self.cmd_opts.add_option(cmdoptions.no_build_isolation())
self.cmd_opts.add_option(cmdoptions.use_pep517())
self.cmd_opts.add_option(cmdoptions.no_use_pep517())
Expand Down
1 change: 1 addition & 0 deletions src/pip/_internal/commands/wheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ def add_options(self) -> None:
self.cmd_opts.add_option(cmdoptions.requirements())
self.cmd_opts.add_option(cmdoptions.src())
self.cmd_opts.add_option(cmdoptions.ignore_requires_python())
self.cmd_opts.add_option(cmdoptions.uploaded_prior_to())
self.cmd_opts.add_option(cmdoptions.no_deps())
self.cmd_opts.add_option(cmdoptions.progress_bar())

Expand Down
25 changes: 25 additions & 0 deletions src/pip/_internal/index/package_finder.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from __future__ import annotations

import datetime
import enum
import functools
import itertools
Expand Down Expand Up @@ -111,6 +112,7 @@ class LinkType(enum.Enum):
format_invalid = enum.auto()
platform_mismatch = enum.auto()
requires_python_mismatch = enum.auto()
upload_too_late = enum.auto()


class LinkEvaluator:
Expand All @@ -132,6 +134,7 @@ def __init__(
target_python: TargetPython,
allow_yanked: bool,
ignore_requires_python: bool | None = None,
uploaded_prior_to: datetime.datetime | None = None,
) -> None:
"""
:param project_name: The user supplied package name.
Expand All @@ -149,6 +152,8 @@ def __init__(
:param ignore_requires_python: Whether to ignore incompatible
PEP 503 "data-requires-python" values in HTML links. Defaults
to False.
:param uploaded_prior_to: If set, only allow links uploaded prior to
the given datetime.
"""
if ignore_requires_python is None:
ignore_requires_python = False
Expand All @@ -158,6 +163,7 @@ def __init__(
self._ignore_requires_python = ignore_requires_python
self._formats = formats
self._target_python = target_python
self._uploaded_prior_to = uploaded_prior_to

self.project_name = project_name

Expand All @@ -176,6 +182,14 @@ def evaluate_link(self, link: Link) -> tuple[LinkType, str]:
reason = link.yanked_reason or "<none given>"
return (LinkType.yanked, f"yanked for reason: {reason}")

if link.upload_time is not None and self._uploaded_prior_to is not None:
if link.upload_time >= self._uploaded_prior_to:
reason = (
f"Upload time {link.upload_time} not "
f"prior to {self._uploaded_prior_to}"
)
return (LinkType.upload_too_late, reason)

if link.egg_fragment:
egg_info = link.egg_fragment
ext = link.ext
Expand Down Expand Up @@ -593,6 +607,7 @@ def __init__(
format_control: FormatControl | None = None,
candidate_prefs: CandidatePreferences | None = None,
ignore_requires_python: bool | None = None,
uploaded_prior_to: datetime.datetime | None = None,
) -> None:
"""
This constructor is primarily meant to be used by the create() class
Expand All @@ -614,6 +629,7 @@ def __init__(
self._ignore_requires_python = ignore_requires_python
self._link_collector = link_collector
self._target_python = target_python
self._uploaded_prior_to = uploaded_prior_to

self.format_control = format_control

Expand All @@ -637,6 +653,7 @@ def create(
link_collector: LinkCollector,
selection_prefs: SelectionPreferences,
target_python: TargetPython | None = None,
uploaded_prior_to: datetime.datetime | None = None,
) -> PackageFinder:
"""Create a PackageFinder.
Expand All @@ -645,6 +662,8 @@ def create(
:param target_python: The target Python interpreter to use when
checking compatibility. If None (the default), a TargetPython
object will be constructed from the running Python.
:param uploaded_prior_to: If set, only find links uploaded prior
to the given datetime.
"""
if target_python is None:
target_python = TargetPython()
Expand All @@ -661,6 +680,7 @@ def create(
allow_yanked=selection_prefs.allow_yanked,
format_control=selection_prefs.format_control,
ignore_requires_python=selection_prefs.ignore_requires_python,
uploaded_prior_to=uploaded_prior_to,
)

@property
Expand Down Expand Up @@ -720,6 +740,10 @@ def prefer_binary(self) -> bool:
def set_prefer_binary(self) -> None:
self._candidate_prefs.prefer_binary = True

@property
def uploaded_prior_to(self) -> datetime.datetime | None:
return self._uploaded_prior_to

def requires_python_skipped_reasons(self) -> list[str]:
reasons = {
detail
Expand All @@ -739,6 +763,7 @@ def make_link_evaluator(self, project_name: str) -> LinkEvaluator:
target_python=self._target_python,
allow_yanked=self._allow_yanked,
ignore_requires_python=self._ignore_requires_python,
uploaded_prior_to=self._uploaded_prior_to,
)

def _sort_links(self, links: Iterable[Link]) -> list[Link]:
Expand Down
15 changes: 15 additions & 0 deletions src/pip/_internal/models/link.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
from __future__ import annotations

import datetime
import functools
import itertools
import logging
import os
import posixpath
import re
import urllib.parse
import urllib.request
from collections.abc import Mapping
from dataclasses import dataclass
from typing import (
Expand All @@ -15,6 +17,7 @@
NamedTuple,
)

from pip._internal.utils.datetime import parse_iso_datetime
from pip._internal.utils.deprecation import deprecated
from pip._internal.utils.filetypes import WHEEL_EXTENSION
from pip._internal.utils.hashes import Hashes
Expand Down Expand Up @@ -207,6 +210,7 @@ class Link:
"requires_python",
"yanked_reason",
"metadata_file_data",
"upload_time",
"cache_link_parsing",
"egg_fragment",
]
Expand All @@ -218,6 +222,7 @@ def __init__(
requires_python: str | None = None,
yanked_reason: str | None = None,
metadata_file_data: MetadataFile | None = None,
upload_time: datetime.datetime | None = None,
cache_link_parsing: bool = True,
hashes: Mapping[str, str] | None = None,
) -> None:
Expand All @@ -239,6 +244,8 @@ def __init__(
no such metadata is provided. This argument, if not None, indicates
that a separate metadata file exists, and also optionally supplies
hashes for that file.
:param upload_time: upload time of the file, or None if the information
is not available from the server.
:param cache_link_parsing: A flag that is used elsewhere to determine
whether resources retrieved from this link should be cached. PyPI
URLs should generally have this set to False, for example.
Expand Down Expand Up @@ -272,6 +279,7 @@ def __init__(
self.requires_python = requires_python if requires_python else None
self.yanked_reason = yanked_reason
self.metadata_file_data = metadata_file_data
self.upload_time = upload_time

self.cache_link_parsing = cache_link_parsing
self.egg_fragment = self._egg_fragment()
Expand Down Expand Up @@ -300,6 +308,12 @@ def from_json(
if metadata_info is None:
metadata_info = file_data.get("dist-info-metadata")

upload_time: datetime.datetime | None
if upload_time_data := file_data.get("upload-time"):
upload_time = parse_iso_datetime(upload_time_data)
else:
upload_time = None
Comment on lines +311 to +315
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that the JSON version of the Simple API is used in the vast majority of installs, but we should probably support this feature with the HTML Simple API if possible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature doesn't exist in the HTML version of the API, it only exists as addition JSON fields in the spec: https://peps.python.org/pep-0700/#specification

For the HTML version of the API, there is no change from version 1.0. For the JSON version of the API, the following changes are made:

IMO this was a short sighted choice by the spec authors, as it has prevented certain simple index libraries from supporting this feature. But we can't change the past.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, thanks for the information. /me wonders if it'd be useful to propose a v1.1 of the HTML spec to bring it to feature parity with the JSON API, but I don't have the time for that, haha.


# The metadata info value may be a boolean, or a dict of hashes.
if isinstance(metadata_info, dict):
# The file exists, and hashes have been supplied
Expand All @@ -325,6 +339,7 @@ def from_json(
yanked_reason=yanked_reason,
hashes=hashes,
metadata_file_data=metadata_file_data,
upload_time=upload_time,
)

@classmethod
Expand Down
Loading
Loading