fix(PartitionedOutput): Add int128 converter in PrestoIterativePartitioningSerializer - #2459
Open
xin-zhang2 wants to merge 4 commits into
Open
Conversation
`PrestoIterativePartitioningSerializer` can now serialize struct columns, including structs nested inside structs, with nulls at any level. Serializing a struct column does not write into the vector handed to `append()`. Given `ROW(ROW(ROW(INTEGER)))` where the outer struct is null at row 0 and the inner struct is null at row 2, combining the rows the outer struct discards back into the caller's own null bitmap would leave the caller's inner struct reporting two nulls instead of one, so the mask that tells a child which rows survive is built separately: only when a level really drops rows, and in a buffer of its own when both the level and an ancestor drop rows. A struct whose null bitmap is allocated but holds no null does not count as having nulls. Such a bitmap drops no row, so treating it as nullable would reject `ROW(ROW(VARCHAR))` with "Variable-width columns nested under a ROW with nulls are not supported" and push fixed-width leaves off the bulk copy path onto one stream write per value. A struct column can arrive CONSTANT-encoded, which produces a `PartitionedConstantVector`; that is reported as an unsupported encoding rather than relying on a cast that returns null. The ROW block footer reuses the same null-section writer as simple columns instead of repeating the Velox-to-Presto bitmap conversion. Also adds nested-ROW coverage to the benchmarks: the `PartitionedVector` type generator was off by one nesting level, so its one-level case built a plain BIGINT column, and `ExchangeBenchmark` reused a single null window at every level, so no level ever discarded a row its child would otherwise have written. An unused `PartitionedVector` setter and a test that asserted nothing are removed.
…itioningSerializer
xin-zhang2
force-pushed
the
PartitionedOutput-int128
branch
from
September 1, 2026 13:43
1dd0397 to
5d5005d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.