Skip to content

Commit 8b14629

Browse files
committed
[SPARK-45670][CORE] SparkSubmit does not support --total-executor-cores when deploying on K8s
Remove Kubernetes from the support list of `--total-executor-cores` in SparkSubmit `--total-executor-cores` does not take effect in Spark on K8s, [the comments from original PR](apache#19717 (comment)) also proves that The output of `spark-submit --help` changed ```patch ... - Spark standalone, Mesos and Kubernetes only: + Spark standalone and Mesos only: --total-executor-cores NUM Total cores for all executors. ... ``` Pass GA and review. No Closes apache#43536 from pan3793/tec. Authored-by: Cheng Pan <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 75aed56 commit 8b14629

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -679,7 +679,7 @@ private[spark] class SparkSubmit extends Logging {
679679
confKey = EXECUTOR_CORES.key),
680680
OptionAssigner(args.executorMemory, STANDALONE | MESOS | YARN | KUBERNETES, ALL_DEPLOY_MODES,
681681
confKey = EXECUTOR_MEMORY.key),
682-
OptionAssigner(args.totalExecutorCores, STANDALONE | MESOS | KUBERNETES, ALL_DEPLOY_MODES,
682+
OptionAssigner(args.totalExecutorCores, STANDALONE | MESOS, ALL_DEPLOY_MODES,
683683
confKey = CORES_MAX.key),
684684
OptionAssigner(args.files, LOCAL | STANDALONE | MESOS | KUBERNETES, ALL_DEPLOY_MODES,
685685
confKey = FILES.key),

core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -567,7 +567,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
567567
| --kill SUBMISSION_ID If given, kills the driver specified.
568568
| --status SUBMISSION_ID If given, requests the status of the driver specified.
569569
|
570-
| Spark standalone, Mesos and Kubernetes only:
570+
| Spark standalone and Mesos only:
571571
| --total-executor-cores NUM Total cores for all executors.
572572
|
573573
| Spark standalone, YARN and Kubernetes only:

0 commit comments

Comments
 (0)