Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ResourceWarning unclosed socket #758

Merged
merged 1 commit into from
Jan 23, 2024

Conversation

admirito
Copy link

@admirito admirito commented Aug 7, 2023

This PR fixes issue #710 by properly closing the underlying socket. It first uses pool._put_conn to keep the connection in the pool, and later removes and closes it when the context manager exits.

I was unsure about the exact purpose of the ConnectonRemove class, so I made minimal changes to minimize the risk of breaking the code and there may be better solutions for fixing this issue.

For example, the urllib3.connectionpool.HTTPConnectionPool will utilize a weakref to terminate pool connections. By appending our connection to it, it will also take care of closing our connection. So another solution could be to modify the __exit__ in patch.ConnectionRemover method and add our connection to the pool:

class ConnectionRemover:
    ...

    def __exit__(self, *args):
        for pool, connections in self._connection_pool_to_connections.items():
            for connection in connections:
                if isinstance(connection, self._connection_class):
                    pool._put_conn(connection)

This PR fixes issue kevin1024#710 by properly closing the underlying socket. It
first uses `pool._put_conn` to keep the connection in the pool, and
later removes and closes it when the context manager exits.

I was unsure about the exact purpose of the `ConnectonRemove` class,
so I made minimal changes to minimize the risk of breaking the code
and there may be better solutions for fixing this issue.

For example, the `urllib3.connectionpool.HTTPConnectionPool` will
utilize a weakref to terminate pool connections. By appending our
connection to it, it will also take care of closing our connection. So
another solution could be to modify the `__exit__` in
`patch.ConnectionRemover` method and add our connection to the pool:

```py
class ConnectionRemover:
    ...

    def __exit__(self, *args):
        for pool, connections in self._connection_pool_to_connections.items():
            for connection in connections:
                if isinstance(connection, self._connection_class):
                    pool._put_conn(connection)
```
@graingert
Copy link
Collaborator

It looks like the tests are failing here, but the logs have expired @admirito can you rebase this on the default branch?

@graingert
Copy link
Collaborator

Can you also test that this is fixed by enabling filterwarnings=error in the pytest config

@graingert
Copy link
Collaborator

I've added some more commits to get the tests passing with filterwarnings=error #801

@graingert
Copy link
Collaborator

this seems to cause a problem in tests/integration/test_urllib3.py::test_post[https]

@graingert graingert merged commit 17c78bf into kevin1024:master Jan 23, 2024
1 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants