Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement] Support Spark 4 #1805

Open
3 tasks done
EnricoMi opened this issue Jun 17, 2024 · 0 comments · Fixed by #1806 · May be fixed by #1814
Open
3 tasks done

[Improvement] Support Spark 4 #1805

EnricoMi opened this issue Jun 17, 2024 · 0 comments · Fixed by #1806 · May be fixed by #1814

Comments

@EnricoMi
Copy link
Contributor

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

Support Spark 4.

How should we improve?

This requires the Spark client to compile for Scala 2.13 and work with Spark 4. The Spark 3 client is compatible to Spark 4, but users might hesitate to use the Spark 3 client with Spark 4, so Uniffle should provide a Spark 4 client, which is basically based on Spark 3.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@EnricoMi EnricoMi mentioned this issue Jun 17, 2024
Merged
EnricoMi added a commit that referenced this issue Jun 19, 2024
@EnricoMi
[#1805][part-1] build: Support Scala 2.13 (#1806)
ea1070c 
### What changes were proposed in this pull request?

Support compiling Spark client against Scala 2.13 dependencies.

### Why are the changes needed?

Starting with Spark 4, Scala 2.13 is required.

Contributes to: #1805

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

CI
@EnricoMi EnricoMi reopened this Jun 19, 2024
@EnricoMi EnricoMi linked a pull request Jun 19, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@EnricoMi