[BUG]: aiohttp client memory leak with ddtrace >= 4.9

[akursar]

Tracer Version(s)

4.9

Python Version(s)

3.14

Pip Version(s)

uv

Bug Report

When aiohttp is patched, making sequential, large requests shows a clear and linear increase in max rss memory. This does not happen when aiohttp is not traced. This also does not happen when ddtrace <4.9.

See reproducible code attached, which fails when ddtrace >=4.9, and succeeds when ddtrace <4.9. You can change the constraint in the top and uv run ./test_demo.py.

As the ten-MB GET requests are performed in a loop, the max rss jumps up 10 MB until the assertion on max fails:

i=0, mem=138
i=1, mem=148
i=2, mem=159
i=3, mem=169
i=4, mem=179
i=5, mem=189
i=6, mem=199
...
i=33, mem=471
i=34, mem=481
i=35, mem=491
i=36, mem=501
i=37, mem=511
i=38, mem=521
FAILED test_demo.py::test_mem_usage - AssertionError: Using more than 512 MB at loop iteration 38

I think this is due to #17576 . If I remove the

if resp is not None:
    ctx.event.set_response(resp)

from patch.py, the provided test succeeds (wherein the max rss is very flat throughout the loop), so I suspect something introduced by this change is hanging on to the resp and preventing it from being gc.

Reproduction Code

test_demo.py

# /// script
# requires-python = ">=3.14"
# dependencies = [
#     "aiohttp",
#     "ddtrace>=4.9",
#     "fastapi",
#     "pytest",
#     "pytest-asyncio",
#     "uvicorn",
# ]
# ///

from ddtrace import patch

patch(aiohttp=True)

import resource
import socket
import threading
from collections.abc import Iterator
from time import sleep
import sys

import aiohttp
from fastapi import FastAPI, Response
import pytest
import uvicorn


def max_rss_mb() -> int:
    m = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    if sys.platform == "darwin":  # macOS
        return int(m / (1024 * 1024))
    # Linux
    return int(m / 1024)


@pytest.fixture
def server_url() -> Iterator[str]:
    """Run a FastAPI app on a random port."""

    app = FastAPI()
    content = bytes(10 * 1024 * 1024)

    @app.get("/")
    async def get() -> Response:
        return Response(content=content)

    server = uvicorn.Server(config=uvicorn.Config(app=app, log_config=None))
    (sock := socket.socket()).bind(("127.0.0.1", 0))
    (thread := threading.Thread(target=server.run, kwargs={"sockets": [sock]})).start()

    try:
        while not server.started:
            sleep(0.01)
        address, port = sock.getsockname()
        yield f"http://{address}:{port}/"
    finally:
        server.should_exit = True
        thread.join()


@pytest.mark.asyncio
async def test_mem_usage(server_url: str) -> None:

    # 50 iters of downloading 10 MB should not exceed 500 MB of mem (unless there's a bug)
    iters = 50
    max_mem_mb = 512

    async with aiohttp.ClientSession() as s:
        for i in range(iters):
            async with s.get(server_url) as r:
                content = await r.read()
                assert len(content) > 1024 * 1024
            mem = max_rss_mb()
            print(f"{i=}, {mem=}")
            assert mem < max_mem_mb, f"Using more than {max_mem_mb} MB at loop iteration {i}"


if __name__ == "__main__":
    pytest.main()

Error Logs

No response

Libraries in Use

aiohttp

Operating System

No response