Can't debug my pytest unittests from VS Code

[pauloneves]

Environment data

• debugpy version: 1.2.1
• OS and version: Windows 10 with latest updates
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.7.9 official installer
• Using VS Code or Visual Studio: VS Code 1.53.1 64 bits extracted from zip

Actual behavior

VS Code does not stops at my breakpoints, nor from running from test tabs , nor trying to stepping into a file, nor running python -m debugpy and trying to attach to it.

Expected behavior

I want to be able to click in the highlighted icon below of one of my pytest unit tests:

and to have the debugger stop at this breakpoint:

and be able to step through it.

Steps to reproduce:

I'm working inside a venv and its python exe is set as my VS Code python interpreter. Here is the venv installed packages.

Package                Version
---------------------- -------------------
appdirs                1.4.4
arrow                  0.17.0
attrs                  20.3.0
binaryornot            0.4.4
black                  19.10b0
brotlipy               0.7.0
certifi                2020.12.5
cffi                   1.14.3
chardet                3.0.4
click                  7.1.2
conda                  4.9.2
conda-package-handling 1.7.2
cookiecutter           1.7.2
cryptography           3.2.1
idna                   2.10
Jinja2                 2.11.2
jinja2-time            0.2.0
MarkupSafe             1.1.1
menuinst               1.4.16
mypy-extensions        0.4.3
pathspec               0.7.0
pip                    20.2.4
poyo                   0.5.0
pycosat                0.6.3
pycparser              2.20
pyOpenSSL              19.1.0
PySocks                1.7.1
python-dateutil        2.8.1
python-slugify         4.0.1
pywin32                227
regex                  2020.11.13
requests               2.24.0
rope                   0.18.0
ruamel-yaml            0.15.87
setuptools             50.3.1.post20201107
six                    1.15.0
text-unidecode         1.3
toml                   0.10.1
tqdm                   4.51.0
typed-ast              1.4.2
typing-extensions      3.7.4.3
urllib3                1.25.11
wheel                  0.36.2
win-inet-pton          1.1.0
wincertstore           0.2

I installed my module as editable running from inside the venv, in the root folder: pip install -e obabr

root is my workspace root folder. obabr is the name of the package I'm developing. My workspace is structured this way

root/obabr/
 | 
 ------------> src/mypackage/*py
|
------------->tests/*_test.py 

My setpup.py is in root/obabr/. This directory also has the file pytest.ini which content is:

[pytest]
norecursedirs =
    migrations

python_files =
    *_test.py
    tests.py
addopts =
    -ra
    --strict-markers
    --doctest-modules
    --doctest-glob=\*.rst
    --tb=short
testpaths =
    tests

I can successfully run my tests from the VS Code test tab:

When I click in the debug icon of my breakpointed test, the VS Code status bar becomes orange, then becomes:

Here is the output of the debug console:

============================= test session starts =============================
platform win32 -- Python 3.7.9, pytest-6.2.0, py-1.10.0, pluggy-0.13.1
rootdir: c:\dev\ingest_oba, configfile: obabr\pytest.ini
plugins: hypothesis-6.0.3, forked-1.3.0, profiling-1.7.0, timeout-1.4.2, xdist-2.2.0
collected 1 item

obabr\tests\download_test.py .                                           [100%]

- generated xml file: C:\Users\ppppp\AppData\Local\Temp\tmp-15396J3k1TtjEH4i3.xml -
============================== 1 passed in 0.13s ==============================

And nothing happens. :-(

Trying to attach to debugpy module

If from my activated workspace in a _ bash shell_ I run the following command:

python -m debugpy --wait-for-client --listen 0.0.0.0:5678 ../../Scripts/pytest.exe download_test.py

and from the debug tab I try to connect using this configuration in my launch.json:

    "configurations": [
        {
            "name": "Python: Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "localhost",
                "port": 5678
            }
        }

Then the test is executed, but my bookmarked command print(123) at top level of the test module does not stop. There are also other bookmarks inside my tests.

Trying a simple python file

It also fails the breakpoing if I put a bookmark in a simple .py file and try to debug with the following launch.json configuration:

        {
            "name": "Python: Arquivo Atual",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            //"console": "integratedTerminal"
            "console": "internalConsole",
            "cwd": "${workspaceRoot}/obabr",
        }

I've tried some variations in this launch configuration, but nothing works.

I feel I'm really missing the full power of VS Code as a dev env.

[fabioz]

If you create a simple test without all your structure, does it work?

i.e.:
Create a test_something.py with:

def test_it():
    print('here') # <- add breakpoint here

Also, can you add logging information to try to diagnose what's happening?

i.e.:

• Open VS Code
• Select the command Extensions: Open Extensions Folder
• Locate the Python extension directory, typically of the form ms-python.python-2020..***
• In that directory ensure you do not have any debug*.log files, if you do, please delete them
• Go back into VS Code and modify your launch.json to add the setting "logToFile": true, see below:

{
    "name": "Debug Tests",
    "type": "python",
    "request": "test",
    "console": "integratedTerminal",
    "logToFile": true
}

• Start debugging
• When done, go back into the extension directory and upload the debug*.log files into this GitHub issue.

[pauloneves]

Thanks for your attention, Fabio.

The simple test didn't work. I've already tried it, but did it again. The test appeared in my left panel (after reloading the panel), When I clicked in the bug icon, the status bar became orange, then blue and in the debug console I see that the test executed successfully.

Here is the complete output:

 /usr/bin/env 'DEBUGPY_LOG_DIR=c:\Users\ppppp\.vscode\extensions\ms-python.python-2021.2.582707922' c:\\dev\\ingest_oba\\Scripts\\python.exe c:\\Users\\ppppp\\.vscode\\extensions\\ms-python.python-2021.2.582707922\\pythonFiles\\lib\\python\\debugpy\\launcher 60179 -- c:\\Users\\ppppp\\.vscode\\extensions\\ms-python.python-2021.2.582707922\\pythonFiles\\pyvsc-run-isolated.py c:\\Users\\ppppp\\.vscode\\extensions\\ms-python.python-2021.2.582707922\\pythonFiles\\testlauncher.py c:\\dev\\ingest_oba pytest --override-ini junit_family=xunit1 --rootdir c:\\dev\\ingest_oba --junit-xml=C:\\Users\\ppppp\\AppData\\Local\\Temp\\tmp-55086i0NuaM2i55S.xml ./obabr/tests/simple_test.py::test_it 
============================= test session starts =============================
platform win32 -- Python 3.7.9, pytest-6.2.0, py-1.10.0, pluggy-0.13.1
rootdir: c:\dev\ingest_oba, configfile: obabr\pytest.ini
plugins: anyio-2.1.0, hypothesis-6.0.3, forked-1.3.0, profiling-1.7.0, timeout-1.4.2, xdist-2.2.0
collected 1 item

obabr\tests\simple_test.py .                                             [100%]

- generated xml file: C:\Users\ppppp\AppData\Local\Temp\tmp-55086i0NuaM2i55S.xml

Then I created this Launch configuration:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Arquivo Atual",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal"
        },
        {
            "name": "Debug Tests",
            "type": "python",
            "request": "test",
            "console": "integratedTerminal",
            "logToFile": true
        }
    ]
}

logToFile is underlined as an inexistent option, but it looks like it worked. I'm attaching all files

debugger.vscode_f6c644e6-0cde-4858-824b-5c1f2bbbe288.log
debugpy.adapter-14224.log
debugpy.launcher-992.log
debugpy.pydevd.18768.log
debugpy.server-18768.log

[frsann]

I'm having similar issue on MacOS 11.2.3, VSCode 1.54.3, Python Extension v2021.2.636928669.

In a clean folder the dummy test above works, as in the breakpoint stops the process. However, in one of our product-repos, it doesn't work. I can't share the logs (sensitive info) but if you tell me what to look for I'm more than happy to check.

[fabioz]

@pauloneves Sorry about the delay in getting back to you, it seems I missed your response...

This seems to happen because you're trying to debug library code.

i.e.: The logs show a message saying that the breakpoint is disabled because it's added in library code:

{
"type": "response", 
"request_seq": 9, 
"success": true, 
"command": "setBreakpoints",
 "body": {"breakpoints": [
    {"verified": false, "message": "Breakpoint in file excluded by filters.\nNote: may be excluded because of \"justMyCode\" option (default == true).Try setting \"justMyCode\": false in the debug configuration (e.g., launch.json).\n", "source": {"name": "cli.py", "path": "c:\\dev\\ingest_oba\\obabr\\src\\obabr\\cli.py"}, "line": 24}, 
    {"verified": false, "message": "Breakpoint in file excluded by filters.\nNote: may be excluded because of \"justMyCode\" option (default == true).Try setting \"justMyCode\": false in the debug configuration (e.g., launch.json).\n", "source": {"name": "cli.py", "path": "c:\\dev\\ingest_oba\\obabr\\src\\obabr\\cli.py"}, "line": 46}]}, 
"seq": 20, "pydevd_cmd_id": 502}

This happens because: site.getsitepackages() is returning c:\dev\ingest_oba.

So, the solution here is setting "justMyCode": "true" in your "request": "test" launch configuration.

@frsann please try to set the justMyCode in your "request": "test" launch config too... if it doesn't fix the issue, please create a separate issue with the same logs requested in: #544 (comment)

[pauloneves]

It works! But the solution was to set `"justMyCode": False

My code was a package and the tests were importing the package "from mypackage import mymodule` so they were considering the package as an external lib.

Maybe VSCode sould warn us when a bookmark is made inside a lib and justMyCode is True.

Here is my launch.json configuration:

        {
            "name": "Debug Tests",
            "type": "python",
            "request": "test",
            "console": "integratedTerminal",
            "python": "${command:python.interpreterPath}",
            "justMyCode": false
        }

[fabioz]

As a note, the notification and visibility on that should be done by VSCode itself (debugpy does what it should given the protocol: specify that the breakpoint is disabled with a message giving the details).

[aminechm]

what worked for me was clearing all .pyc and pycache recursively using : https://gist.github.com/hbsdev/a17deea814bc10197285

in both my local workspace and equivalent Docker container