[SPARK-45600][SQL][PYTHON][FOLLOW-UP] Make Python data source registr…

…ation session level ### What changes were proposed in this pull request? This PR is a followup of #44305. It already works properly with the session-level. ### Why are the changes needed? To remove unnecessary TODO JIRA. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing Ci in this PR should verify them ### Was this patch authored or co-authored using generative AI tooling? No. Closes #44487 from HyukjinKwon/SPARK-45600. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
apache · Dec 26, 2023 · 439ec6b · 439ec6b
1 parent 3ee2db8
commit 439ec6b
Showing 1 changed file with 0 additions and 2 deletions.
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
@@ -650,7 +650,6 @@ object DataSource extends Logging {
                 // Found the data source using fully qualified path
                 dataSource
               case Failure(error) =>
-                // TODO(SPARK-45600): should be session-based.
                 val isUserDefinedDataSource = SparkSession.getActiveSession.exists(
                   _.sessionState.dataSourceManager.dataSourceExists(provider))
                 if (provider1.startsWith("org.apache.spark.sql.hive.orc")) {
@@ -679,7 +678,6 @@ object DataSource extends Logging {
           }
         case head :: Nil =>
           // there is exactly one registered alias
-          // TODO(SPARK-45600): should be session-based.
           val isUserDefinedDataSource = SparkSession.getActiveSession.exists(
             _.sessionState.dataSourceManager.dataSourceExists(provider))
           // The source can be successfully loaded as either a V1 or a V2 data source.