🐛 fix(pypi): drop a hosted project once it serves nothing - #2192
Open
gaborbernat wants to merge 2 commits into
Open
🐛 fix(pypi): drop a hosted project once it serves nothing#2192gaborbernat wants to merge 2 commits into
gaborbernat wants to merge 2 commits into
Conversation
Deleting every file of a hosted project leaves its display row behind, so the root Simple index keeps advertising a project whose detail page answers 404. This fails on e64cc83 and is the demonstration the fix has to satisfy.
Trashing a project's last file left its display row standing, so the root Simple index went on advertising a project whose detail page had started answering 404. The project row cannot answer on its own whether anything is still served, and list_projects returned every project holding one. A hosted project now carries its untrashed and total upload counts in a single row, written in the transaction that moved the upload row. Both numbers share a row so one read and one write keep them consistent, and both move with the upload rather than after it: a count maintained separately would trade this inconsistency for a window where the listing and the rows disagree. The row exists only where uploads happened, so a cached project carries none and stays listed on the strength of its page. Trashing the last file keeps the project row for a restore to return the published spelling; removing the last record takes both rows, so a later upload writes its own spelling. Nothing in production removes an upload record: the served DELETE trashes it and retention only plans. The two purge criteria are driven through the store API so the invariant holds when an apply phase arrives.
Merging this PR will not alter performance
Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trashing a hosted project's last file left its display row standing, so the root Simple index kept advertising a project whose detail page had started answering
404. That breaks the PEP 691 contract rather than showing stale text, and the stronger claim is what gives the fix something to answer to. I reproduced it through the served surface one64cc8308before writing anything: upload a wheel,DELETE /root/pypi/peryxpkg/, and detail answers404while the root index answers200with{"meta":{"api-version":"1.4"},"projects":[{"name":"peryxpkg"}]}. That test is the first commit here and fails without the rest.Validating it corrected two things in the issue, and I wrote both back into #1326 so the next reader skips the trace.
delete_uploadis test-only, and so ismutate_uploads, leavingUploadMutation::Deleteand theremove_upload_rowbehind it unreachable outside tests. The servedDELETEsoft-deletes:remove_files_with_webhookhandsmutate_uploads_and_overridesa replacement record withtrashedset, and removes no row. So the defect reproduces on the trash path, not on a purge.A hosted project now keeps its untrashed and total upload counts in one row, and the transaction that moves the upload row writes it. Both numbers share a row so a single read and write keep them consistent, and both move with the upload rather than after it, because a count kept in a second transaction would trade the inconsistency this closes for a window where the listing and the rows disagree.
list_projectsreads them in one scan per index instead of a lookup per project.The row exists only where uploads happened, so a cached project carries none and stays listed on the strength of its page. That distinction is safe because an index is hosted or cached and never both, so a given
{index}/{normalized}cannot hold uploads and a cached page at once. A row peryx cannot decode counts as serving, so a listing loses no project to a decode failure. Trashing the last file keeps the project row, which is what lets a restore return the spelling the project was published under; removing the last record takes both rows, so a later upload writes its own spelling.Two of the issue's four criteria have no production trigger, and this PR does not claim otherwise. Nothing removes an upload record: the served
DELETEtrashes, and retention plans and exports without applying, its ownExportHeaderdoc referring to "a later apply" that does not exist. That reads as a boundary nobody has built rather than a second defect, so this drives the purge criteria through the store API, where they pin the invariant for whenever an apply phase arrives. HTTP tests cover the two reachable criteria end to end.Closes #1326