How do I debug a function enqueued with rq?

Question

I am currently attempting to debug a function that's enqueued inside a rq queue in VS Code. However rq forks the process to produce its workers, which I think is why it is impossible to intercept the breakpoint.

I use the debugpy as a debugging library and I am able to break into the non-queued code and I know the function is called because it produces the proper output.

I have tried to set the worker class to simple-worker and set the max number of workers to 1 but that didn't work.

I have also try to explicitly call debugpy to listen to incoming connections inside the enqueued function. More concretely:

import rq

def worker_function():
    pass # Breakpoints placed here don't break
    debugpy.listen(('0.0.0.0', 5678)) # Adding these lines doesn't help
    debugpy.wait_for_client()
    debugpy.breakpoint() # Doesn't break

def parent_function(rq_queue: rq.Queue):
    rq_queue.enqueue(worker_function) # Breakpoints placed here work

And I launch the script with the following command:

# Assume that the rq worker and redis instance are up and running
# This is the command called by the docker-compose file
python3 -m debugpy --listen 0.0.0.0:5678 script.py

And on the VS Code side my launch configuration looks like this:

{
    "configurations": [    
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "0.0.0.0",
                "port": 5678
            },
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "/app"
                }
            ],
            "justMyCode": true
        }
    ]
}

Answer 1

I found a way to start a remote debug session by using remote-dbg.

First append these values to the container running rq worker

environment:
  PYTHONBREAKPOINT: remote_pdb.set_trace
  REMOTE_PDB_HOST: 0.0.0.0
  REMOTE_PDB_PORT: 4444
ports:
  - "4444:4444"

Add this to break into the code:

RemotePdb(host="0.0.0.0", port=4444).set_trace()

Finally access the debugger using telnet:

telnet localhost 4444