-
Notifications
You must be signed in to change notification settings - Fork 952
Add fallback to HStack
lowering in cudf-polars
#19163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @rjzamora, looks great.
Do you think this could affect any other operations? Do you think we should do some assert in _lower_ir_pwise
to validate that everything actually is piecewise (probably not, but maybe...)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm curious if the test is failing only with pytest-xdist
turned on?
df = ( | ||
pl.DataFrame({"dt": dates, "a": [3, 7, 5, 9, 2, 1]}) | ||
.with_columns(pl.col("dt").str.strptime(pl.Datetime("ns"))) | ||
.lazy() | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
df = ( | |
pl.DataFrame({"dt": dates, "a": [3, 7, 5, 9, 2, 1]}) | |
.with_columns(pl.col("dt").str.strptime(pl.Datetime("ns"))) | |
.lazy() | |
) | |
df = ( | |
pl.LazyFrame({"dt": dates, "a": [3, 7, 5, 9, 2, 1]}) | |
.with_columns(pl.col("dt").str.strptime(pl.Datetime("ns"))) | |
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like engine='gpu'
raises when I use this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we don't support this strptime
deducing the format I think. (cc @brandon-b-miller who implemented this stuff).
Ahh - Looks like we have a serialization/hashing error when the new test runs in distributed mode :/ |
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tiny suggestions, but I think this looks good.
/merge |
Description
Closes #19150
(Although, we can probably do a bit more work to avoid fallback for some non-pointwise expressions)
We were not validating that the items in
HStack.columns
were all "pointwise" before lowering to partitionwise logic. This adds the possibility for single-partition fallback.Checklist