Commit a8f817d
[SPARK-54745][PYTHON] Fix PySpark import error caused by missing UnixStreamServer on Windows
### What changes were proposed in this pull request?
This PR fixes an error caused by `socketserver.UnixStreamServer` not being available on Windows. We define a fallback `AccumulatorUnixServer` to raise an exception on construction and inform the user to disable `spark.python.unix.domain.socket.enabled`.
### Why are the changes needed?
`import pyspark` fails with the following message on Windows since PySpark 4.1.0:
```
sedona\spark\__init__.py:19: in <module>
import pyspark
.venv\Lib\site-packages\pyspark\__init__.py:71: in <module>
from pyspark.accumulators import Accumulator, AccumulatorParam
.venv\Lib\site-packages\pyspark\accumulators.py:324: in <module>
class AccumulatorUnixServer(socketserver.UnixStreamServer):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E AttributeError: module 'socketserver' has no attribute 'UnixStreamServer'
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually test this on Windows 11
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53546 from Kontinuation/fix-uds-windows-compat.
Authored-by: Kristin Cowalcijk <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>1 parent 532de64 commit a8f817d
1 file changed
+30
-15
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
345 | 345 | | |
346 | 346 | | |
347 | 347 | | |
348 | | - | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
363 | 378 | | |
364 | 379 | | |
365 | 380 | | |
| |||
0 commit comments