Skip to content

🐛 fix(pypi): drop a hosted project once it serves nothing - #2192

Open
gaborbernat wants to merge 2 commits into
mainfrom
fix/purge-hosted-project-rows-1326
Open

🐛 fix(pypi): drop a hosted project once it serves nothing#2192
gaborbernat wants to merge 2 commits into
mainfrom
fix/purge-hosted-project-rows-1326

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

Trashing a hosted project's last file left its display row standing, so the root Simple index kept advertising a project whose detail page had started answering 404. That breaks the PEP 691 contract rather than showing stale text, and the stronger claim is what gives the fix something to answer to. I reproduced it through the served surface on e64cc8308 before writing anything: upload a wheel, DELETE /root/pypi/peryxpkg/, and detail answers 404 while the root index answers 200 with {"meta":{"api-version":"1.4"},"projects":[{"name":"peryxpkg"}]}. That test is the first commit here and fails without the rest.

Validating it corrected two things in the issue, and I wrote both back into #1326 so the next reader skips the trace. delete_upload is test-only, and so is mutate_uploads, leaving UploadMutation::Delete and the remove_upload_row behind it unreachable outside tests. The served DELETE soft-deletes: remove_files_with_webhook hands mutate_uploads_and_overrides a replacement record with trashed set, and removes no row. So the defect reproduces on the trash path, not on a purge.

A hosted project now keeps its untrashed and total upload counts in one row, and the transaction that moves the upload row writes it. Both numbers share a row so a single read and write keep them consistent, and both move with the upload rather than after it, because a count kept in a second transaction would trade the inconsistency this closes for a window where the listing and the rows disagree. list_projects reads them in one scan per index instead of a lookup per project.

The row exists only where uploads happened, so a cached project carries none and stays listed on the strength of its page. That distinction is safe because an index is hosted or cached and never both, so a given {index}/{normalized} cannot hold uploads and a cached page at once. A row peryx cannot decode counts as serving, so a listing loses no project to a decode failure. Trashing the last file keeps the project row, which is what lets a restore return the spelling the project was published under; removing the last record takes both rows, so a later upload writes its own spelling.

Two of the issue's four criteria have no production trigger, and this PR does not claim otherwise. Nothing removes an upload record: the served DELETE trashes, and retention plans and exports without applying, its own ExportHeader doc referring to "a later apply" that does not exist. That reads as a boundary nobody has built rather than a second defect, so this drives the purge criteria through the store API, where they pin the invariant for whenever an apply phase arrives. HTTP tests cover the two reachable criteria end to end.

Closes #1326

Deleting every file of a hosted project leaves its display row behind, so the
root Simple index keeps advertising a project whose detail page answers 404.
This fails on e64cc83 and is the demonstration the fix has to satisfy.
Trashing a project's last file left its display row standing, so the root Simple
index went on advertising a project whose detail page had started answering 404.
The project row cannot answer on its own whether anything is still served, and
list_projects returned every project holding one.

A hosted project now carries its untrashed and total upload counts in a single
row, written in the transaction that moved the upload row. Both numbers share a
row so one read and one write keep them consistent, and both move with the
upload rather than after it: a count maintained separately would trade this
inconsistency for a window where the listing and the rows disagree.

The row exists only where uploads happened, so a cached project carries none and
stays listed on the strength of its page. Trashing the last file keeps the
project row for a restore to return the published spelling; removing the last
record takes both rows, so a later upload writes its own spelling.

Nothing in production removes an upload record: the served DELETE trashes it and
retention only plans. The two purge criteria are driven through the store API so
the invariant holds when an apply phase arrives.
@gaborbernat gaborbernat added the bug Something isn't working label Sep 4, 2026
@codspeed-hq

codspeed-hq Bot commented Sep 4, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks1


Comparing fix/purge-hosted-project-rows-1326 (397eaf4) with main (d75b16a)

Open in CodSpeed

Footnotes

  1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove hosted project rows after final purge

1 participant