Step into library import statement gives "Could not load source '<string>': Source unavailable."

[luabud]

Set a breakpoint to a import numpy statement and step into it. It'll raise: Could not load source '<string>': Source unavailable.

VS Code Insiders 1.33.0
Extension version 2019.3.6215

launch.json file:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        
        {
            "name": "Python-test",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "stopOnEntry": true,
            "cwd": "${workspaceFolder}",
        },
    ]
}

[DonJayamanne]

You are debugging non-user code, hence you need to turn that feature on.
Add the following into launch.json

"debugStdLib": true

[karthiknadig]

It should not send back '' though. If filename is '', we should return sourceReference. If there is no source reference, we should not return those frames.

[DonJayamanne]

It should not send back '' though. If filename is '', we should return sourceReference. If there is no source reference, we should not return those frames.

Aah yes, agreed, I thought step in would just automatically perform a step over.
Isn't that the expected behavior. I thought thats what it did in the past.

[karthiknadig]

You are correct. step-in effectively becomes step-over, when debugStdLib=false. But it seems to also send <string> as frame. I need to investigate that.

[karthiknadig]

This is what we get. I think message should describe why users are seeing this and what users can do about it. Message should be something along the lines of:

• If you intended to debug a site-package or standard library please set debugStdLib: true in launch.json for VS Code. For Visual Studio 2017/2019 set the debug option at Tools > Options > Python > Debugging.
• If you stepped into a @dataclass operator or exec, we may not be able to find the source for it. If you feel this is a bug please file a issue https://github.com/Microsoft/ptvsd/issues with repro steps.

[int19h]

<string> is not treated as stdlib, so when an stdlib module does an eval or exec, and you step in, you end up in something like this. You can see the relevant bits here: microsoft/ptvsd@d41e3a5 (note that the behavior wrt <string> is the same in both old and new code), and some discussion about this here: microsoft/ptvsd#1232 (comment).

We can't just exclude <string>, because anything that's run with python -c has that as the name, and then of course user code can also do eval or exec. If I remember correctly, in ptvsd 3, when it saw a frame like that, it walked one step back to see if the previous frame is user code or stdlib.

[karthiknadig]

Ahh. I see. So the only reason debugStdLib: true feels like it works because user has not yet reached the point where the exec or eval is called.

[int19h]

Is the behavior actually different depending on debugStdLib setting here? I would expect it to be the same, since this one frame is not considered stdlib either way. The only difference should be that if you have stdlib on, then a step-in should step into the first line of the numpy module itself - which would be the expected behavior. If you keep stepping till you hit the eval, then a step-in on that would still give the same message about "source unavailable".

That part is correct, I think - it is a frame, it doesn't have a source that we can possibly fetch, but you can still e.g. inspect variables, and even step (blindly). It's not an internal debugger frame, and could very well be running user code, so hiding it would be confusing.

So I think the real bug here is that a <string> frame caused by eval/exec in stdlib code is not itself treated as stdlib, and the rest is by design. We can improve the error message though - if I remember correctly, the part before the colon is from VSC, and the part after is whatever error message we return from our source handler.

[int19h]

(Side note: in the future, we might want to try decompiling such frames with something like 'uncompyle6', and providing the synthesized source via source reference.)

[juliussimonelli]

When I try to add "debugStdLib": true to launch.json I get an error saying Property debugStdLib is not allowed. But based on this issue it looks like should be working. I am using VSCode - Insiders Version 1.34.20-insider (1.34.20-insider).

[int19h]

We have changed it to "justMyCode" recently (which has to be set to false in this case), to be consistent with other VSC debuggers. You should get a prompt to fix it automatically in launch.json when you re-open your folder or workspace.

[juliussimonelli]

That worked. Thanks! By the way, when I click on "Quick Fix" it says No code actions available.

[parlad]

I was having the same issue

raw_input("enter input") fixed this.

[bored002]

i'm trying the same thing but when debugging my python code with "justMyCode": false and launching the debug i'm getting a KeyError exception on sys.path_importer_cache[path_item] in pkgutil.py
using :
python 3.6.8 64 bit
Vscode version : 1.35.1

[iamryanchia]

In the latest vscode 1.36.1, adding "justMyCode": false to launch.json will sovle this problem.