Skip to content

Feature request: support to update snapshot summary #1329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dentiny opened this issue May 15, 2025 · 1 comment · May be fixed by #1336
Open

Feature request: support to update snapshot summary #1329

dentiny opened this issue May 15, 2025 · 1 comment · May be fixed by #1336
Labels
enhancement New feature or request

Comments

@dentiny
Copy link
Contributor

dentiny commented May 15, 2025 •
edited
Loading

Is your feature request related to a problem or challenge?

Context: I would like to set some self-defined metadata (pretty small, less than 100 bytes) for a particular snapshot in the current snapshot.

Search around the code, I think the best place to store might be snapshot summary.

iceberg-rust/crates/iceberg/src/spec/snapshot.rs

Line 74 in f9061cc

/// Other summary data.

Interface-wise I imagine something similar to pyiceberg:

  • We expose set_properties and get_properties interface for Transaction

Describe the solution you'd like

There're a lot of bad workarounds (i.e. store the metadata into puffin files, hack the TableMetadata, etc), but I would expect the summary update operation to be exposed at transaction layer.

Willingness to contribute

I can contribute to this feature independently
@dentiny dentiny added the enhancement New feature or request label May 15, 2025
@dentiny
Copy link
Contributor Author

dentiny commented May 15, 2025 •
edited
Loading

It's worth noting, existing interfaces only allow to modify table properties, but it's completely different from snapshot properties, and not expected to embed arbitrary metadata:

iceberg-rust/crates/iceberg/src/spec/table_metadata.rs

Lines 135 to 138 in f9061cc

///A string to string map of table properties. This is used to control settings that
/// affect reading and writing and is not intended to be used for arbitrary metadata.
/// For example, commit.retry.num-retries is used to control the number of commit retries.
pub(crate) properties: HashMap<String, String>,

@dentiny dentiny linked a pull request May 15, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Introduce snapshot summary properties dentiny/iceberg-rust
1 participant
@dentiny