-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP Add partitioning push down #23432
Open
dain
wants to merge
10
commits into
trinodb:master
Choose a base branch
from
dain:apply-partitioning
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
+1,251
−734
Commits on Sep 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 23cf63b - Browse repository at this point
Copy the full SHA 23cf63bView commit details -
Configuration menu - View commit details
-
Copy full SHA for a6958ef - Browse repository at this point
Copy the full SHA a6958efView commit details -
Optimize ActualProperties compatible checks
Check boolean nullsAndAnyReplicated field before more complex fields
Configuration menu - View commit details
-
Copy full SHA for ba97284 - Browse repository at this point
Copy the full SHA ba97284View commit details -
Configuration menu - View commit details
-
Copy full SHA for b9b5805 - Browse repository at this point
Copy the full SHA b9b5805View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0766ed6 - Browse repository at this point
Copy the full SHA 0766ed6View commit details
Commits on Sep 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ae974d9 - Browse repository at this point
Copy the full SHA ae974d9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2321964 - Browse repository at this point
Copy the full SHA 2321964View commit details -
Add partitioning push down and update Hive partitioning
Add partitioning push down to table scan which connector can use to activate optional partitioning, or choose between multiple partitioning strategies. This replaces the existing Metadata makeCompatiblePartitioning and getCommonPartitioningHandle methods used exclusively by Hive
Configuration menu - View commit details
-
Copy full SHA for 5cf85cd - Browse repository at this point
Copy the full SHA 5cf85cdView commit details -
Add partitioning push down to Iceberg
Add support for pushing plan partitioning into Iceberg when Iceberg tables use hash bucked partitioning. This enables co-located joins which can be significantly more efficient. Additionally, since Iceberg supports multiple independent partitioning functions, a table can effectively have multiple distributions, which makes the optimization more effective. This feature can be controlled with the iceberg.bucket-execution-mode configuration property and the bucket_execution session property. Mode can be set to NEVER, AUTO, or ALWAYS. AUTO is the default and enables bucked execution when the bucket count is equal to or greater than the current node count.
Configuration menu - View commit details
-
Copy full SHA for 56f4735 - Browse repository at this point
Copy the full SHA 56f4735View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ebd691 - Browse repository at this point
Copy the full SHA 1ebd691View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.