creating/binding a socket in a fixture causes it to not close #53

altendky · 2021-12-24T03:28:02Z

Describe the bug
While creating and binding a socket in a test and then closing it allows the same port to be reopened later in the test, if the socket was originally created and bound in a fixture then it fails claiming OSError: [Errno 98] Address already in use.

To Reproduce
Complete input and output below, but the tl;dr is...

================================================= test session starts ==================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/altendky/repos/chia-blockchain
plugins: monitor-1.6.3
collected 2 items                                                                                                      

x.py F.                                                                                                          [100%]

======================================================= FAILURES =======================================================
________________________________________________ test_a_socket[fixture] ________________________________________________

just_a_socket = ('fixture', <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>)

    def test_a_socket(just_a_socket):
        where, sock = just_a_socket
        if where == "test":
            print("creating and binding in test")
            sock = make_a_bound_socket()
    
        sock.close()
    
        sock2 = socket.socket()
>       sock2.bind(address)
E       OSError: [Errno 98] Address already in use

x.py:37: OSError
------------------------------------------------ Captured stdout setup -------------------------------------------------
creating and binding in fixture
=============================================== short test summary info ================================================
FAILED x.py::test_a_socket[fixture] - OSError: [Errno 98] Address already in use
============================================= 1 failed, 1 passed in 0.09s ==============================================

import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)

def make_a_bound_socket():
    sock = socket.socket()
    sock.bind(address)
    return sock


@pytest.fixture(params=["fixture", "test"])
def just_a_socket(request):
    where = request.param

    if where == "fixture":
        print("creating and binding in fixture")
        sock = make_a_bound_socket()
    else:
        sock = None

    yield where, sock


def test_a_socket(just_a_socket):
    where, sock = just_a_socket
    if where == "test":
        print("creating and binding in test")
        sock = make_a_bound_socket()

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)

https://gist.github.com/altendky/656bd59340288be28e8e3eb9a8d6f1b8

Copy/paste this into a terminal in an empty directory on a system with Python 3.9 available

cat > x.py << EOF
import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)

def make_a_bound_socket():
    sock = socket.socket()
    sock.bind(address)
    return sock


@pytest.fixture(params=["fixture", "test"])
def just_a_socket(request):
    where = request.param

    if where == "fixture":
        print("creating and binding in fixture")
        sock = make_a_bound_socket()
    else:
        sock = None

    yield where, sock


def test_a_socket(just_a_socket):
    where, sock = just_a_socket
    if where == "test":
        print("creating and binding in test")
        sock = make_a_bound_socket()

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)


def main():
    subprocess.run(["pytest", "--capture", "no", __file__], check=True)


# yuck
if __name__ == "__main__":
    main()
EOF
cat x.py
python3.9 -m venv venv
venv/bin/python --version --version
venv/bin/python -m pip install --upgrade pip setuptools wheel
venv/bin/pip install attrs==21.2.0 iniconfig==1.1.1 packaging==21.3 pluggy==1.0.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 toml==0.10.2
venv/bin/pip freeze
venv/bin/pytest x.py
venv/bin/pip install attrs==21.2.0 certifi==2021.10.8 charset-normalizer==2.0.9 idna==3.3 iniconfig==1.1.1 memory-profiler==0.60.0 packaging==21.3 pluggy==1.0.0 psutil==5.8.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 pytest-monitor==1.6.3 requests==2.26.0 toml==0.10.2 urllib3==1.26.7
venv/bin/pip freeze
venv/bin/pytest x.py
uname -a
lsb_release -a

To get this session

$ cat > x.py << EOF
> import socket
> import subprocess
> 
> import pytest
> 
> 
> address = ("127.0.0.1", 33125)
> 
> def make_a_bound_socket():
>     sock = socket.socket()
>     sock.bind(address)
>     return sock
> 
> 
> @pytest.fixture(params=["fixture", "test"])
> def just_a_socket(request):
>     where = request.param
> 
>     if where == "fixture":
>         print("creating and binding in fixture")
>         sock = make_a_bound_socket()
>     else:
>         sock = None
> 
>     yield where, sock
> 
> 
> def test_a_socket(just_a_socket):
>     where, sock = just_a_socket
>     if where == "test":
>         print("creating and binding in test")
>         sock = make_a_bound_socket()
> 
>     sock.close()
> 
>     sock2 = socket.socket()
>     sock2.bind(address)
> 
> 
> def main():
>     subprocess.run(["pytest", "--capture", "no", __file__], check=True)
> 
> 
> # yuck
> if __name__ == "__main__":
>     main()
> EOF

$ cat x.py
import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)

def make_a_bound_socket():
    sock = socket.socket()
    sock.bind(address)
    return sock


@pytest.fixture(params=["fixture", "test"])
def just_a_socket(request):
    where = request.param

    if where == "fixture":
        print("creating and binding in fixture")
        sock = make_a_bound_socket()
    else:
        sock = None

    yield where, sock


def test_a_socket(just_a_socket):
    where, sock = just_a_socket
    if where == "test":
        print("creating and binding in test")
        sock = make_a_bound_socket()

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)


def main():
    subprocess.run(["pytest", "--capture", "no", __file__], check=True)


# yuck
if __name__ == "__main__":
    main()

$ python3.9 -m venv venv

$ venv/bin/python --version --version
Python 3.9.5 (default, Jun  3 2021, 15:18:23) 
[GCC 9.3.0]

$ venv/bin/python -m pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.1.1)
Collecting pip
  Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (56.0.0)
Collecting setuptools
  Using cached setuptools-60.1.0-py3-none-any.whl (952 kB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 56.0.0
    Uninstalling setuptools-56.0.0:
      Successfully uninstalled setuptools-56.0.0
  Attempting uninstall: pip
    Found existing installation: pip 21.1.1
    Uninstalling pip-21.1.1:
      Successfully uninstalled pip-21.1.1
Successfully installed pip-21.3.1 setuptools-60.1.0 wheel-0.37.1

$ venv/bin/pip install attrs==21.2.0 iniconfig==1.1.1 packaging==21.3 pluggy==1.0.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 toml==0.10.2
Collecting attrs==21.2.0
  Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting iniconfig==1.1.1
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging==21.3
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pluggy==1.0.0
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting py==1.11.0
  Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting pyparsing==3.0.6
  Using cached pyparsing-3.0.6-py3-none-any.whl (97 kB)
Collecting pytest==6.2.5
  Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Installing collected packages: pyparsing, toml, py, pluggy, packaging, iniconfig, attrs, pytest
Successfully installed attrs-21.2.0 iniconfig-1.1.1 packaging-21.3 pluggy-1.0.0 py-1.11.0 pyparsing-3.0.6 pytest-6.2.5 toml-0.10.2

$ venv/bin/pip freeze
attrs==21.2.0
iniconfig==1.1.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
toml==0.10.2

$ venv/bin/pytest x.py
================================================= test session starts ==================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/altendky/repos/chia-blockchain
collected 2 items                                                                                                      

x.py ..                                                                                                          [100%]

================================================== 2 passed in 0.01s ===================================================

$ venv/bin/pip install attrs==21.2.0 certifi==2021.10.8 charset-normalizer==2.0.9 idna==3.3 iniconfig==1.1.1 memory-profiler==0.60.0 packaging==21.3 pluggy==1.0.0 psutil==5.8.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 pytest-monitor==1.6.3 requests==2.26.0 toml==0.10.2 urllib3==1.26.7
Requirement already satisfied: attrs==21.2.0 in ./venv/lib/python3.9/site-packages (21.2.0)
Collecting certifi==2021.10.8
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer==2.0.9
  Using cached charset_normalizer-2.0.9-py3-none-any.whl (39 kB)
Collecting idna==3.3
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Requirement already satisfied: iniconfig==1.1.1 in ./venv/lib/python3.9/site-packages (1.1.1)
Collecting memory-profiler==0.60.0
  Using cached memory_profiler-0.60.0-py3-none-any.whl
Requirement already satisfied: packaging==21.3 in ./venv/lib/python3.9/site-packages (21.3)
Requirement already satisfied: pluggy==1.0.0 in ./venv/lib/python3.9/site-packages (1.0.0)
Collecting psutil==5.8.0
  Using cached psutil-5.8.0-cp39-cp39-manylinux2010_x86_64.whl (293 kB)
Requirement already satisfied: py==1.11.0 in ./venv/lib/python3.9/site-packages (1.11.0)
Requirement already satisfied: pyparsing==3.0.6 in ./venv/lib/python3.9/site-packages (3.0.6)
Requirement already satisfied: pytest==6.2.5 in ./venv/lib/python3.9/site-packages (6.2.5)
Collecting pytest-monitor==1.6.3
  Using cached pytest_monitor-1.6.3-py3-none-any.whl (14 kB)
Collecting requests==2.26.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Requirement already satisfied: toml==0.10.2 in ./venv/lib/python3.9/site-packages (0.10.2)
Collecting urllib3==1.26.7
  Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Requirement already satisfied: wheel in ./venv/lib/python3.9/site-packages (from pytest-monitor==1.6.3) (0.37.1)
Installing collected packages: urllib3, psutil, idna, charset-normalizer, certifi, requests, memory-profiler, pytest-monitor
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.9 idna-3.3 memory-profiler-0.60.0 psutil-5.8.0 pytest-monitor-1.6.3 requests-2.26.0 urllib3-1.26.7

$ venv/bin/pip freeze
attrs==21.2.0
certifi==2021.10.8
charset-normalizer==2.0.9
idna==3.3
iniconfig==1.1.1
memory-profiler==0.60.0
packaging==21.3
pluggy==1.0.0
psutil==5.8.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
pytest-monitor==1.6.3
requests==2.26.0
toml==0.10.2
urllib3==1.26.7

$ venv/bin/pytest x.py
================================================= test session starts ==================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/altendky/repos/chia-blockchain
plugins: monitor-1.6.3
collected 2 items                                                                                                      

x.py F.                                                                                                          [100%]

======================================================= FAILURES =======================================================
________________________________________________ test_a_socket[fixture] ________________________________________________

just_a_socket = ('fixture', <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>)

    def test_a_socket(just_a_socket):
        where, sock = just_a_socket
        if where == "test":
            print("creating and binding in test")
            sock = make_a_bound_socket()
    
        sock.close()
    
        sock2 = socket.socket()
>       sock2.bind(address)
E       OSError: [Errno 98] Address already in use

x.py:37: OSError
------------------------------------------------ Captured stdout setup -------------------------------------------------
creating and binding in fixture
=============================================== short test summary info ================================================
FAILED x.py::test_a_socket[fixture] - OSError: [Errno 98] Address already in use
============================================= 1 failed, 1 passed in 0.09s ==============================================

$ uname -a
Linux p1 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

Expected behavior
I expect sockets created and bound in a fixture to be able to be closed.

Desktop (please complete the following information):

OS: Linux (Ubuntu 20.04.3 LTS)
Python version: Python 3.9.5 (default, Jun 3 2021, 15:18:23) [GCC 9.3.0]
Pytest version: 6.2.5
pytest-monitor version: 1.6.3

Additional context
I probably won't get to it tonight, but I will try to see what I can learn about pytest-monitor towards the end of fixing this. I tested back to pytest-monitor v1.0.0 and it seems to be present throughout.

The text was updated successfully, but these errors were encountered:

js-dieu · 2021-12-24T08:54:28Z

Hello @altendky

Thanks for the time spent on this. However, I am unsure to get it right.
You already bind your socket in your fixture and try to rebind it in your test. In your fixture, the where will be evaluated to 'fixture' and is then returned to the stack in your test. Thus, you bind twice the same object.

Additionally, you probably want to close your socket after the yield statement in your fixture.
Please correct me if I misunderstood your problem.

altendky · 2021-12-24T14:31:36Z

Before rebinding to the port, in either case, the first socket is closed. Looking at the pytest output in the full session it shows that the second parametrization where the socket is created in the test there is no error (be sure to expand the collapsed sections in the OP and below here). Closing the socket in the fixture during teardown would normally make sense, but it isn't relevant to this example since it would occur after the error. Note that the failing test with the socket from the fixture is run first so it can't be caused by an issue not closing the socket from the passing case which is second.

I probably shouldn't have parametrized the tests. Here's a more explicit non-parametrized recreation.

import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)


@pytest.fixture(scope="function")
def just_a_socket():
    sock = socket.socket()
    sock.bind(address)
    yield sock
    sock.close()


def test_a_socket_from_a_fixture(just_a_socket):
    sock = just_a_socket

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)


def test_a_socket_from_the_test():
    sock = socket.socket()
    sock.bind(address)

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock.close()


def main():
    subprocess.run(["pytest", "--capture", "no", "--verbose", __file__], check=True)


# yuck
if __name__ == "__main__":
    main()

commands

cat > x.py << EOF
import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)


@pytest.fixture(scope="function")
def just_a_socket():
    sock = socket.socket()
    sock.bind(address)
    yield sock
    sock.close()


def test_a_socket_from_a_fixture(just_a_socket):
    sock = just_a_socket

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)


def test_a_socket_from_the_test():
    sock = socket.socket()
    sock.bind(address)

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock.close()


def main():
    subprocess.run(["pytest", "--capture", "no", "--verbose", __file__], check=True)


# yuck
if __name__ == "__main__":
    main()
EOF
cat x.py
python3.9 -m venv venv
venv/bin/python --version --version
venv/bin/python -m pip install --upgrade pip setuptools wheel
venv/bin/pip install attrs==21.2.0 iniconfig==1.1.1 packaging==21.3 pluggy==1.0.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 toml==0.10.2
venv/bin/pip freeze
venv/bin/pytest x.py
venv/bin/pip install attrs==21.2.0 certifi==2021.10.8 charset-normalizer==2.0.9 idna==3.3 iniconfig==1.1.1 memory-profiler==0.60.0 packaging==21.3 pluggy==1.0.0 psutil==5.8.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 pytest-monitor==1.6.3 requests==2.26.0 toml==0.10.2 urllib3==1.26.7
venv/bin/pip freeze
venv/bin/pytest x.py
uname -a
lsb_release -a

session

$ cat > x.py << EOF
> import socket
> import subprocess
> 
> import pytest
> 
> 
> address = ("127.0.0.1", 33125)
> 
> 
> @pytest.fixture(scope="function")
> def just_a_socket():
>     sock = socket.socket()
>     sock.bind(address)
>     yield sock
>     sock.close()
> 
> 
> def test_a_socket_from_a_fixture(just_a_socket):
>     sock = just_a_socket
> 
>     sock.close()
> 
>     sock2 = socket.socket()
>     sock2.bind(address)
> 
> 
> def test_a_socket_from_the_test():
>     sock = socket.socket()
>     sock.bind(address)
> 
>     sock.close()
> 
>     sock2 = socket.socket()
>     sock2.bind(address)
>     sock.close()
> 
> 
> def main():
>     subprocess.run(["pytest", "--capture", "no", "--verbose", __file__], check=True)
> 
> 
> # yuck
> if __name__ == "__main__":
>     main()
> EOF

$ cat x.py
import socket
import subprocess

import pytest


address = ("127.0.0.1", 33125)


@pytest.fixture(scope="function")
def just_a_socket():
    sock = socket.socket()
    sock.bind(address)
    yield sock
    sock.close()


def test_a_socket_from_a_fixture(just_a_socket):
    sock = just_a_socket

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)


def test_a_socket_from_the_test():
    sock = socket.socket()
    sock.bind(address)

    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock.close()


def main():
    subprocess.run(["pytest", "--capture", "no", "--verbose", __file__], check=True)


# yuck
if __name__ == "__main__":
    main()

$ python3.9 -m venv venv

$ venv/bin/python --version --version
Python 3.9.5 (default, Jun  3 2021, 15:18:23) 
[GCC 9.3.0]

$ venv/bin/python -m pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.1.1)
Collecting pip
  Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (56.0.0)
Collecting setuptools
  Using cached setuptools-60.1.0-py3-none-any.whl (952 kB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 56.0.0
    Uninstalling setuptools-56.0.0:
      Successfully uninstalled setuptools-56.0.0
  Attempting uninstall: pip
    Found existing installation: pip 21.1.1
    Uninstalling pip-21.1.1:
      Successfully uninstalled pip-21.1.1
Successfully installed pip-21.3.1 setuptools-60.1.0 wheel-0.37.1

$ venv/bin/pip install attrs==21.2.0 iniconfig==1.1.1 packaging==21.3 pluggy==1.0.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 toml==0.10.2
Collecting attrs==21.2.0
  Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting iniconfig==1.1.1
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging==21.3
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pluggy==1.0.0
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting py==1.11.0
  Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting pyparsing==3.0.6
  Using cached pyparsing-3.0.6-py3-none-any.whl (97 kB)
Collecting pytest==6.2.5
  Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Installing collected packages: pyparsing, toml, py, pluggy, packaging, iniconfig, attrs, pytest
Successfully installed attrs-21.2.0 iniconfig-1.1.1 packaging-21.3 pluggy-1.0.0 py-1.11.0 pyparsing-3.0.6 pytest-6.2.5 toml-0.10.2

$ venv/bin/pip freeze
attrs==21.2.0
iniconfig==1.1.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
toml==0.10.2

$ venv/bin/pytest x.py
================================================= test session starts ==================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/altendky/repos/chia-blockchain
collected 2 items                                                                                                      

x.py ..                                                                                                          [100%]

================================================== 2 passed in 0.01s ===================================================

$ venv/bin/pip install attrs==21.2.0 certifi==2021.10.8 charset-normalizer==2.0.9 idna==3.3 iniconfig==1.1.1 memory-profiler==0.60.0 packaging==21.3 pluggy==1.0.0 psutil==5.8.0 py==1.11.0 pyparsing==3.0.6 pytest==6.2.5 pytest-monitor==1.6.3 requests==2.26.0 toml==0.10.2 urllib3==1.26.7
Requirement already satisfied: attrs==21.2.0 in ./venv/lib/python3.9/site-packages (21.2.0)
Collecting certifi==2021.10.8
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer==2.0.9
  Using cached charset_normalizer-2.0.9-py3-none-any.whl (39 kB)
Collecting idna==3.3
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Requirement already satisfied: iniconfig==1.1.1 in ./venv/lib/python3.9/site-packages (1.1.1)
Collecting memory-profiler==0.60.0
  Using cached memory_profiler-0.60.0-py3-none-any.whl
Requirement already satisfied: packaging==21.3 in ./venv/lib/python3.9/site-packages (21.3)
Requirement already satisfied: pluggy==1.0.0 in ./venv/lib/python3.9/site-packages (1.0.0)
Collecting psutil==5.8.0
  Using cached psutil-5.8.0-cp39-cp39-manylinux2010_x86_64.whl (293 kB)
Requirement already satisfied: py==1.11.0 in ./venv/lib/python3.9/site-packages (1.11.0)
Requirement already satisfied: pyparsing==3.0.6 in ./venv/lib/python3.9/site-packages (3.0.6)
Requirement already satisfied: pytest==6.2.5 in ./venv/lib/python3.9/site-packages (6.2.5)
Collecting pytest-monitor==1.6.3
  Using cached pytest_monitor-1.6.3-py3-none-any.whl (14 kB)
Collecting requests==2.26.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Requirement already satisfied: toml==0.10.2 in ./venv/lib/python3.9/site-packages (0.10.2)
Collecting urllib3==1.26.7
  Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Requirement already satisfied: wheel in ./venv/lib/python3.9/site-packages (from pytest-monitor==1.6.3) (0.37.1)
Installing collected packages: urllib3, psutil, idna, charset-normalizer, certifi, requests, memory-profiler, pytest-monitor
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.9 idna-3.3 memory-profiler-0.60.0 psutil-5.8.0 pytest-monitor-1.6.3 requests-2.26.0 urllib3-1.26.7

$ venv/bin/pip freeze
attrs==21.2.0
certifi==2021.10.8
charset-normalizer==2.0.9
idna==3.3
iniconfig==1.1.1
memory-profiler==0.60.0
packaging==21.3
pluggy==1.0.0
psutil==5.8.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
pytest-monitor==1.6.3
requests==2.26.0
toml==0.10.2
urllib3==1.26.7

$ venv/bin/pytest x.py
================================================= test session starts ==================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/altendky/repos/chia-blockchain
plugins: monitor-1.6.3
collected 2 items                                                                                                      

x.py F.                                                                                                          [100%]

======================================================= FAILURES =======================================================
_____________________________________________ test_a_socket_from_a_fixture _____________________________________________

just_a_socket = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>

    def test_a_socket_from_a_fixture(just_a_socket):
        sock = just_a_socket
    
        sock.close()
    
        sock2 = socket.socket()
>       sock2.bind(address)
E       OSError: [Errno 98] Address already in use

x.py:24: OSError
=============================================== short test summary info ================================================
FAILED x.py::test_a_socket_from_a_fixture - OSError: [Errno 98] Address already in use
============================================= 1 failed, 1 passed in 0.06s ==============================================

$ uname -a
Linux p1 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

altendky · 2021-12-24T14:42:15Z

Forcing it to run wrapped_function() instead of prof() seems to avoid the problem. So, perhaps I need to bump this over there. I'll see if I can make a recreation just using memory_profiler.memory_usage().

pytest-monitor/pytest_monitor/pytest_monitor.py

Lines 157 to 162 in a73b055

    
           if not PYTEST_MONITORING_ENABLED: 
        
               wrapped_function() 
        
           else: 
        
               if not pyfuncitem.session.config.option.mtr_disable_gc: 
        
                   gc.collect() 
        
               prof()

altendky · 2021-12-24T15:27:49Z

It does appear to be down in memory_profiler (or below?). pythonprofilers/memory_profiler#342 I would personally leave this open until it is resolved so it is easy to find if anyone else runs into this. But, at the moment, it doesn't look like there's any action needed on your part in terms of fixing this.

js-dieu · 2021-12-24T17:19:52Z

Ok got your point.
My intuition drives me to memory_profiler at first glance since the profiling launches a monitoring process apart. I'll check this out next week...

altendky · 2021-12-24T17:22:28Z

Already filed a bug there since I could recreate with just it. Link above. I'll follow up there if I manage any more debugging.

altendky mentioned this issue Dec 24, 2021

Sockets opened outside profiling do not close properly inside profiling pythonprofilers/memory_profiler#342

Open

altendky mentioned this issue Jan 1, 2022

run tests in CI via coverage Chia-Network/chia-blockchain#9704

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

creating/binding a socket in a fixture causes it to not close #53

creating/binding a socket in a fixture causes it to not close #53

altendky commented Dec 24, 2021

js-dieu commented Dec 24, 2021

altendky commented Dec 24, 2021

altendky commented Dec 24, 2021

altendky commented Dec 24, 2021

js-dieu commented Dec 24, 2021

altendky commented Dec 24, 2021

creating/binding a socket in a fixture causes it to not close #53

creating/binding a socket in a fixture causes it to not close #53

Comments

altendky commented Dec 24, 2021

js-dieu commented Dec 24, 2021

altendky commented Dec 24, 2021

altendky commented Dec 24, 2021

altendky commented Dec 24, 2021

js-dieu commented Dec 24, 2021

altendky commented Dec 24, 2021