feat: support incremental scan between 2 snapshots #1470
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CTTY
reviewed
Jul 16, 2025
crates/iceberg/src/scan/context.rs
Outdated
| } | ||
| } | ||
| } | ||
|
|
||
| struct Ancestors { |
There was a problem hiding this comment.
This can be handy for validation in SnapshotProducer as well, maybe we should move this to a new module util/snapshot.rs
There was a problem hiding this comment.
btw, what kind of validation are you talking about? It seems currently there's no such place where Ancestors can be used
There was a problem hiding this comment.
Snapshot validation is not implemented yet, I have a stale PR for that. My PR also comes with the logic of scanning ancestors, but I like your version much better and would love to reuse what you have here when we restart the snapshot validation work
Signed-off-by: xxchan <[email protected]>
Signed-off-by: xxchan <[email protected]>
Signed-off-by: xxchan <[email protected]>
xxchan
force-pushed
the
xxchan/naval-heron
branch
from
a3f5068 to
d87a9d2
Compare
This refactoring addresses the PR comment suggesting that the Ancestors struct could be reused by SnapshotProducer and should be moved to a dedicated utilities module for better code organization. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
xxchan
force-pushed
the
xxchan/naval-heron
branch
from
d87a9d2 to
d5fb028
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
close #1469
What changes are included in this PR?
Added
to_snapshot_idandfrom_snapshot_idto the table scan API.Are these changes tested?
unit test