Exception ignored in PyodideHandle.__del__ playwright._impl._errors.Error: Event loop is closed! Is Playwright already stopped?

[tomjakubowski]

Using a PyodideHandle in a fixture with the playwright runner, I am seeing a warning about an ignored exception when cleaning up the test.

Test case:

# tomtests/test_handle_bug.py
import pytest
from pytest_pyodide import run_in_pyodide


@pytest.fixture
@run_in_pyodide
def macguffin(selenium):
    from pytest_pyodide.decorator import PyodideHandle

    s = {"a": 1}
    return PyodideHandle(s)


@run_in_pyodide
def test_macguffin(selenium, macguffin):
    assert macguffin["a"] == 1

# uv sync --all-extras
# uv run pytest --runner playwright --rt chrome tomtests/

Also pushed here: tomjakubowski@23cd219

Error message:

Exception ignored in: <function PyodideHandle.__del__ at 0x101ffa3e0>
Traceback (most recent call last):
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/decorator.py", line 88, in __del__
    self.selenium.run_js(
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/runner.py", line 262, in run_js
    return self.run_js_inner(code, check_code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/runner.py", line 415, in run_js_inner
    retval = self.driver.evaluate(wrapper % (code, check_code))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 8530, in evaluate
    self._sync(
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 104, in _sync
    raise Error("Event loop is closed! Is Playwright already stopped?")
playwright._impl._errors.Error: Event loop is closed! Is Playwright already stopped?

Note that this doesn't cause the test to fail, and the exit code is 0.

I see a similar error message when running with --runner selenium:

Exception ignored in: <function PyodideHandle.__del__ at 0x1032263e0>
Traceback (most recent call last):
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/decorator.py", line 88, in __del__
    self.selenium.run_js(
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/runner.py", line 262, in run_js
    return self.run_js_inner(code, check_code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/pytest_pyodide/runner.py", line 362, in run_js_inner
    retval = self.driver.execute_async_script(wrapper % (code, check_code))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 550, in execute_async_script
    return self.execute(command, {"script": script, "args": converted_args})["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 427, in execute
    response = self.command_executor.execute(driver_command, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 404, in execute
    return self._request(command_info[0], url, body=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 428, in _request
    response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/_request_methods.py", line 143, in request
    return self.request_encode_body(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/_request_methods.py", line 278, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/poolmanager.py", line 443, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/Users/tom/github.com/pyodide/pytest-pyodide/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=60847): Max retries exceeded with url: /session/d6624be6a6c2269e04186b871e101b60/execute/async (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1034bb7a0>: Failed to establish a new connection: [Errno 61] Connection refused'))

I think that the handle's finalizer is being called after the event loop is closed, and the call to eval the reference count decrement is what's throwing.

[tomjakubowski]

I'll take a crack at fixing this. It seems pretty straightforward: add a method to SeleniumType/_BrowserBaseRunner called like is_closed() which returns if the page is closed or not, and then check that in ProxyHandle.__del__ before calling run_js.

WIP commit here, it fixes the problem for Playwright but not for Selenium. I'm unsure to go about a regression test for this; maybe a PyodideHandle with a mocked out selenium driver returning false from is_closed() and assert that run_js() is not called when the PyodideHandle is deleted?