feat: add migrate state store and CLI parsers - #653
Draft
SgtPooki wants to merge 6 commits into
Draft
Conversation
First slice of the migrate command split (#652): the SQLite-backed resume state store (migrate.db schema and lifecycle transitions), trustless-gateway CAR URL construction, size/int/CID-list flag parsers, and duration formatting. No CLI wiring yet; the command lands at the end of the stack.
FilOzzy
added
team/filecoin-pin
A garbage-collected secondary copy was hidden from secondary scheduling forever: the NOT EXISTS in subPiecesMissingSecondary matched rows of any status. It now excludes only live and terminal states, matching subPiecesNeedingUpload. deleteSubPieceForRebuild blocked a rebuild only for a committed primary, so a committed secondary's on-chain breadcrumb could be deleted and the piece re-added later. Any committed row now blocks, and the member reset to pending happens inside the same transaction so a crash cannot strand done rows with no member file. Also: recordUploadParked no longer demotes a committed row, a unique index makes double-packing a member impossible, parseSize accepts decimal spellings as binary aliases, and a stale docstring goes. Adds a MigrationDB unit test file covering these transitions.
documentation/migrate.md and the migrate glossary entries move from the final slice (#657) to the first so the pipeline design, resume guarantees, and exit contract get agreement before the code that implements them is reviewed. The doc describes the completed stack; flags it mentions land in later slices.
The stderr log() claimed stdout stays reserved for machine-readable output, but that is a convention no other filecoin-pin command has: human-facing output goes through src/utils/cli-logger.ts (clack in TTY, console.log otherwise) and only errors hit stderr. Later slices switch their call sites to cli-logger; a --json mode is a possible follow-up if scripted consumption is ever needed.
parseSize and parsePositiveInt are generic CLI flag parsers, not migrate concepts; they move next to formatFileSize (parseSize's inverse) in src/utils/cli-helpers.ts so the next command with a size or count flag reuses them instead of writing a third parser. parseCidList stays in migrate's util: it parses migrate's input file format. Folding payments/setup.ts's TiB-per-month regex into parseSize is a possible follow-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Part 1 of 5 splitting #652 into reviewable slices. Adds the migrate command's SQLite resume state store (
migrate.dbschema and lifecycle transitions: pending, packed, parked, add_unconfirmed, committed), trustless-gateway CAR URL construction, the size/int flag parsers (shared insrc/utils/cli-helpers.tsnext toformatFileSize), and the CID-list parser. Usesnode:sqlite(DatabaseSync, WAL), unflagged since Node 23.4.Also carries documentation/migrate.md and the migrate glossary entries: the design doc rides in the first slice so the pipeline, resume guarantees, and exit contract get agreement before the code that implements them. Start the review there.
No CLI wiring yet.. the command activates in part 5, so this is dead code until the stack lands. Code review focus: the schema and the state transitions, they are the resume model everything else leans on.
How to verify
Notes / risks
Stack: #652 is the umbrella with the full pipeline description. Merge order is 1 through 5; each PR's base is the previous branch.