Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-36818][PYTHON] Fix filtering a Series by a boolean Series #34061

Closed
wants to merge 6 commits into from
Closed

[SPARK-36818][PYTHON] Fix filtering a Series by a boolean Series #34061

wants to merge 6 commits into from

Conversation

xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Sep 21, 2021 •
edited
Loading

What changes were proposed in this pull request?

Fix filtering a Series (without a name) by a boolean Series.

Why are the changes needed?

A bugfix. The issue is raised as databricks/koalas#2199.

Does this PR introduce any user-facing change?

Yes.

From

>>> psser = ps.Series([0, 1, 2, 3, 4])
>>> ps.set_option('compute.ops_on_diff_frames', True)
>>> psser.loc[ps.Series([True, True, True, False, False])]
Traceback (most recent call last):
...
KeyError: 'none key'

To

>>> psser = ps.Series([0, 1, 2, 3, 4])
>>> ps.set_option('compute.ops_on_diff_frames', True)
>>> psser.loc[ps.Series([True, True, True, False, False])]
0    0
1    1
2    2
dtype: int64

How was this patch tested?

Unit test.

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Test build #143482 has finished for PR 34061 at commit da176bc.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47993/

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47993/

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Test build #143486 has finished for PR 34061 at commit 36672c5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47997/

@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47997/

@xinrong-meng xinrong-meng marked this pull request as ready for review September 21, 2021 23:10
@SparkQA
Copy link

SparkQA commented Sep 21, 2021

Test build #143489 has finished for PR 34061 at commit e4ebcee.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

ueshin
ueshin reviewed Sep 22, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48000/

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Test build #143490 has finished for PR 34061 at commit d91bd01.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48001/

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48000/

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48001/

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Test build #143510 has finished for PR 34061 at commit 70b3cdc.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48021/

ueshin
ueshin approved these changes Sep 22, 2021
View reviewed changes
Copy link
Member

@ueshin ueshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@ueshin
Copy link
Member

ueshin commented Sep 22, 2021

Thanks! merging to master/3.2

ueshin pushed a commit that referenced this pull request Sep 22, 2021
@xinrong-meng @ueshin
[SPARK-36818][PYTHON] Fix filtering a Series by a boolean Series
423cff4 
### What changes were proposed in this pull request?
Fix filtering a Series (without a name) by a boolean Series.

### Why are the changes needed?
A bugfix. The issue is raised as databricks/koalas#2199.

### Does this PR introduce _any_ user-facing change?
Yes.

#### From
```py
>>> psser = ps.Series([0, 1, 2, 3, 4])
>>> ps.set_option('compute.ops_on_diff_frames', True)
>>> psser.loc[ps.Series([True, True, True, False, False])]
Traceback (most recent call last):
...
KeyError: 'none key'

```

#### To
```py
>>> psser = ps.Series([0, 1, 2, 3, 4])
>>> ps.set_option('compute.ops_on_diff_frames', True)
>>> psser.loc[ps.Series([True, True, True, False, False])]
0    0
1    1
2    2
dtype: int64
```

### How was this patch tested?
Unit test.

Closes #34061 from xinrong-databricks/filter_series.

Authored-by: Xinrong Meng <[email protected]>
Signed-off-by: Takuya UESHIN <[email protected]>
(cherry picked from commit 6a5ee02)
Signed-off-by: Takuya UESHIN <[email protected]>
@ueshin ueshin closed this in 6a5ee02 Sep 22, 2021
@SparkQA
Copy link

SparkQA commented Sep 22, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48021/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ueshin ueshin ueshin approved these changes

Assignees
No one assigned
Labels
CORE PYTHON
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xinrong-meng @SparkQA @ueshin