AVRO-4031: [Rust] builder header #3096
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g
pushed a commit
that referenced
this pull request
Aug 16, 2024
* AVRO-4031: [Rust] allow setting has_header in builder * AVRO-4031: [Rust] add test for appending with multiple writers * AVRO-4031: [Rust] Prefix the new IT test name with `avro_4031_` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> --------- Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> Co-authored-by: Martin Tzvetanov Grigorov <[email protected]> (cherry picked from commit 14361c0)
|
Thank you, @MarcoLugo ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
AVRO-4031
What is the purpose of the change
This PR allows for more flexibility when creating new avro writers with the goal of appending to an existing file. Current functions are convenient wrappers but do not allow fully configuring the parameters for the new writer when it's meant for appending to an existing file.
Some more context pertaining to my particular use case: parameters such as block size become quite important when using compression so being able to append with a new writer using the same, larger-than-default block size the first writer used can make a significant difference for compression.
Verifying this change
Added a new test.
Documentation
Yes, now it is possible to create Writer for appending with any other parameter available when creating the initial writer.
The newly exposed field (i.e.
has_header) is documented automatically thanks to theTypedBuildermacro.