[query] Daemonize Py4JQueryDriver http server (hail-is#15050)

PR hail-is#14698 refactored the startup of the py4j http server so that calling
`pyHttpServer` created and started the sever. In the process, it deleted
the daemon thread that the server was started from originally. This
caused the `httpServer.start` method to spawn a non-daemon thread as it
was being called from the main thread py4j was using to do its work.
Simply calling `setExecutor(null)` is not sufficent for this purpose.

Manually stopping the server, like with `hl.stop()` would solve the
issue since the JVM would exit and spark would note that the job was
complete after the python process exited as well.

We once again create a thread that itself starts the http server, and
then start that thread in `pyHttpServer`.

This should solve the hanging in dataproc that bisection showed was
introduced in hail-is#14698.

## Security Assessment
- This change cannot impact the Hail Batch instance as deployed by Broad
Institute in GCP

Author: chrisvittal

hail/hail/src/is/hail/backend/driver/Py4JQueryDriver.scala

@@ -403,23 +403,40 @@ final class Py4JQueryDriver(backend: Backend) extends Closeable {
 
       // 0 => let the OS pick an available port
       private[this] val httpServer = HttpServer.create(new InetSocketAddress(0), 10)
+      httpServer.createContext("/", runRpc(_: HttpExchange))
 
       @nowarn def port: Int = httpServer.getAddress.getPort
       override def close(): Unit = httpServer.stop(10)
 
       // This HTTP server *must not* start non-daemon threads because such threads keep the JVM
       // alive. A living JVM indicates to Spark that the job is incomplete. This does not manifest
-      // when you run jobs in a local pyspark (because you'll Ctrl-C out of Python regardless of
-      // the JVM's state) nor does it manifest in a Notebook (again, you'll kill the Notebook kernel
+      // when you run jobs in a local pyspark (because you'll Ctrl-C out of Python regardless of the
+      // JVM's state) nor does it manifest in a Notebook (again, you'll kill the Notebook kernel
       // explicitly regardless of the JVM). It *does* manifest when submitting jobs with
       //
       //     gcloud dataproc submit ...
       //
       // or
       //
       //     spark-submit
-      httpServer.createContext("/", runRpc(_: HttpExchange))
+      //
+      // setExecutor(null) ensures the server creates no new threads:
+      //
+      // > If this method is not called (before start()) or if it is called with a null Executor,
+      // > then a default implementation is used, which uses the thread which was created by
+      // > the start() method.
+      //
+      /* Source:
+       * https://docs.oracle.com/en/java/javase/11/docs/api/jdk.httpserver/com/sun/net/httpserver/HttpServer.html#setExecutor(java.util.concurrent.Executor) */
       httpServer.setExecutor(null) // ensures the server creates no new threads
-      httpServer.start()
+
+      // Note that simply calling httpServer.start() from a non-daemon thread will spawn a
+      // non-daemon thread itself.
+      private[this] val thread = new Thread(new Runnable() {
+        def run(): Unit = httpServer.start()
+      })
+
+      thread.setDaemon(true)
+      thread.start()
     }
 }