support gevent-using apps when workers>2 after #2183

[markjm]

Summary

#2796 shows the issue for gevent usage including a demo for the issue I am trying to solve.

This change updated multi-processing to use socket instead of pipe because it is patchable! Feel free to try out the demo script i included in the discussion and see that it fails without this change and passes with it. FWIW we have been running with this patch since Jan with no issue

Thanks for the project, let me know if this patch is worthwhile to you!

Checklist

• I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• [n/a] I've updated the documentation accordingly.

I am missing a few lines of coverage - can come back to work on those if this is a change you would be interested in pursuing!

[codspeed-hq]

Merging this PR will not alter performance

✅ 24 untouched benchmarks

---

_{Comparing markjm:markjm/multi-processing (3e4b2de) with main (e8a31bc)}

[hrv-dys]

Interesting approach. Replacing multiprocessing.Pipe with socket.socketpair makes the IPC mechanism compatible with gevent's monkey-patching, since gevent patches socket but not multiprocessing.Connection.

Some thoughts:

1. The ping/pong protocol change is straightforward — sendall(b"ping") / recv(4) instead of conn.send(b"ping") / conn.recv(). The explicit 4-byte recv matches the 4-byte "ping"/"pong" messages, which is correct.

2. Error handling in ping() — catching (TimeoutError, OSError) is right. The finally block resetting to blocking mode with its own try/except OSError is defensive — if the socket was closed between the recv and the reset, it won't crash. Solid.

3. The __del__ method on Process — this is a safety net for socket cleanup. It's fine as a fallback, but __del__ in CPython has known issues with reference cycles and GC ordering. The explicit cleanup in join() and start() (closing the unused end) is the primary cleanup mechanism, which is good.

4. always_pong() now catches OSError — needed because socket.recv raises OSError when the socket is closed, unlike Connection.recv which raises EOFError. The test test_process_ping_after_socket_closed validates this.

5. The with_running_supervisor context manager in tests is a nice cleanup — ensures the supervisor thread gets joined even if the test fails. The should_exit.set() in the finally block is a good fallback. Reduces the chance of zombie threads leaking between tests.

6. One concern: socket.socketpair() creates AF_UNIX sockets on Unix. On Windows, Python's socket.socketpair() creates a TCP loopback pair, which should still work but is slightly different in terms of buffering behavior. Since the messages are tiny (4 bytes), this shouldn't matter in practice, but it's a behavioral difference from multiprocessing.Pipe on Windows.

All 14 multiprocess tests pass (1 skipped — SIGBREAK on macOS, pre-existing).

[Kludex]

Interesting. I'll see if I have time to check it soon.

[markjm]

thanks! let me know if you need anything from me!

[cubic-dev-ai]

No issues found across 2 files

_{Re-trigger cubic}

[Kludex]

This doesn't seem enough - can you try with gevent monkeypatching to see if it is? @markjm