Example test that would generate "Windows fatal exception: access violation"

[trajano]

This is a test I wrote to try and get the "Windows fatal exception: access violation" alluded to by #11593 and #7634

This test creates a Thread that would run an asyncio server. In my real test I was planning to run a server on the side started up as a separate thread along side TestContainer docker containers. However, if there's an error I can't seem to see it anywhere and the multiple trace reports that are practically useless in my situation are all I have left with.

import asyncio
from threading import Thread

import pytest
from testcontainers.kafka import KafkaContainer

from confluent_kafka import Consumer
@pytest.fixture(scope="module")
def kafka_container():
    with KafkaContainer() as container:
        yield container

@pytest.fixture
def failing_fixture(kafka_container):
    async def kafka_listener():
        conf = {
            'bootstrap.servers': 'kafka_container.get_bootstrap_server()',
            'group.id': 'my-consumer-group',
            'auto.offset.reset': 'earliest',
        }
        consumer = Consumer(conf)
        consumer.subscribe(['my-topic'])
        try:
            while True:
                await asyncio.to_thread(consumer.poll, 1.0)
        finally:
            consumer.close()

    async def asyncio_server(time:int):
        await asyncio.sleep(time)
        raise RuntimeError(f"foo {kafka_container.get_bootstrap_server()}")

    async def gather():
        await asyncio.gather(kafka_listener(), asyncio_server(5),asyncio_server(15),asyncio_server(10),asyncio_server(50) )
    def the_server():
        asyncio.run(gather())

    thread = Thread(the_server())
    thread.start()
    print("STARTED")
    yield
    print("JOINING")
    thread.join()
    print("JOINED")

@pytest.mark.usefixtures("failing_fixture")
def test_threading():
    pass

In this case I made a simple Kafka listener that would just poll indefinitely and a asyncio server that raises a runtime error after a few seconds. All of which I gathered together.

When it triggers the exception, it spits out those thread traces like

Windows fatal exception: access violation

Thread 0x00026c28 (most recent call first):
  File "C:\Users\trajano\scoop\apps\python\3.13.2\Lib\threading.py", line 1092 in join
  File "C:\Users\trajano\scoop\apps\python\3.13.2\Lib\concurrent\futures\thread.py", line 239 in shutdown
  File "C:\Users\trajano\scoop\apps\python\3.13.2\Lib\asyncio\base_events.py", line 627 in _do_shutdown
  File "C:\Users\trajano\scoop\apps\python\3.13.2\Lib\threading.py", line 992 in run
  File "C:\Users\trajano\scoop\apps\python\3.13.2\Lib\threading.py", line 1041 in _bootstrap_inner

[trajano]

I kind of wished pytest-dev/pytest-rerunfailures#112 was actually supported but it does not seem to work on module level fixtures.

[The-Compiler]

"Windows fatal exception: access violation" basically is the equivalent of "an exception occurred somewhere". This particular case will most likely have nothing to do with the others you linked.

You might want to try passing -s to pytest to see if there is any output. If that doesn't help, your best bet is to find out how to debug things on a C level.

[trajano]

Not necessarily need to be on the C level, but maybe some way of detecting if there are any remaining threads or unawaited asyncio tasks or unresolved concurrent.future.Futures by the end of the test. I think pytest just presumes they're closed and perhaps that is what triggered the exception.

[RonnyPfannschmidt]

As far as I'm aware the Kafka objects are not threadsafe

The mix of asyncio and threading is concerning