[SPARK-44542][CORE] Eagerly load SparkExitCode class in exception han…

…dler ### What changes were proposed in this pull request? 1. eagerly load SparkExitCode class in the the SparkUncaughtExceptionHandler ### Why are the changes needed? In some extreme case, it's possible for SparkUncaughtExceptionHandler's exit/halt process function calls throw an exception if the SparkExitCode is not loaded earlier, See corresponding jira: [SPARK-44542](https://issues.apache.org/jira/browse/SPARK-44542) for more details. By eagerly load SparkExitCode class, we can make sure at least the halt/exit would work properly. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No logic change, hence no new UTs. Closes #42195 from advancedxy/SPARK-44542. Authored-by: Xianjin <[email protected]> Signed-off-by: Sean Owen <[email protected]>
apache · Jul 31, 2023 · 32498b3 · 32498b3
1 parent af6e177
commit 32498b3
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/core/src/main/scala/org/apache/spark/util/SparkUncaughtExceptionHandler.scala b/core/src/main/scala/org/apache/spark/util/SparkUncaughtExceptionHandler.scala
@@ -28,6 +28,12 @@ import org.apache.spark.internal.Logging
 private[spark] class SparkUncaughtExceptionHandler(val exitOnUncaughtException: Boolean = true)
   extends Thread.UncaughtExceptionHandler with Logging {
 
+  locally {
+    // eagerly load SparkExitCode class, so the System.exit and runtime.halt have a chance to be
+    // executed when the disk containing Spark jars is corrupted. See SPARK-44542 for more details.
+    val _ = SparkExitCode.OOM
+  }
+
   override def uncaughtException(thread: Thread, exception: Throwable): Unit = {
     try {
       // Make it explicit that uncaught exceptions are thrown when container is shutting down.