Debugging Pybind11 extension with Visual Studio Code (MacOS)

Question

I've been using pybind11 recently, and now that I'm getting the hang of it, I'm thrilled with it. It's an awesome piece of work. The final piece of the tool puzzle for doing pybind11 is the debug part. I've got command line debugging with lldb working using the following guide:

Debugging pybind11 with lldb

I've spent some time trying to get debug working with Visual Studio Code, with limited success. The first issue is that in order to set up an attach configuration, you need to specify the python executable (not the process id). I don't know how this is supposed to work if you have more than one active python process, which often happens.

Putting that aside, I set up a launch configuration to point to the ipython executable, which is the most convenient thing to use. When I try to start debugging, I get this:

Can anyone explain this?

If I change the executable to plain Python, I get this in the Debug Console:

Could not initialize Python interpreter - only native expressions will be available.
Launching: /Users/andy/anaconda3/envs/SciPy37/bin/python

But if go to the Terminal window, I can successfully enter Python expressions, and trigger breakpoints set in the code. Hooray! But there is one more problem. When one of my extensions needs to load another dylib, it can't find it.

>>> import block_test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so, 2): Symbol not found: _vDSP_vsmul
  Referenced from: /Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so

This makes some sense because_vDSP_vsmul is part of an Apple DSP accelerate library. But what I don't get is that I don't have this problem when I use the command line debug technique noted at the beginning of this post. Clearly this is somehow related to how dylibs are found, but why would this be different from the command line situation?

Any help on these issues would be great. Getting this debug stuff working in Visual Studio Code is the missing piece to absolutely amazing interoperability between Python and C++

Answer 1

I figured out how to get attach to process debugging to work with Visual Studio Code with iPython and on Mac OS Catalina. The following code in launch.json will work:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "type": "lldb",
      "request": "launch",
      "name": "Debug",
      "program": "path_to/bin/python",
      "args": [],
      "cwd": "${workspaceFolder}/build"
    },
    { 
      "name": "Debug Attach",
      "type": "lldb",
      "request": "attach",
      "program": "path_to/bin/python",
      "processId": "${command:pickProcess}",
      "MIMode": "gdb"
  },
  ]
}

First, before trying to start debugging, in a Terminal window, start IPython so you have a process to attach to.

I was initially running into problems with VSC complaining about not having a "program" entry in the json file, which seems irrelevant when attaching to process. It may in fact be completely irrelevant, but you have to add it to get things to work. I have not tried putting in a bogus entry for the program argument to see whether the value matter at all.

One other detail - sometime the breakpoints in VSC don't appear to be enabled, you get an unfilled circle instead of a red dot when you create them. But it seems you will hit the breakpoints regardless. However, based on my previous experience, it is best to import the package in iPython under test after attaching to the process, for best results.