Know where the thread is blocked

Question

I have some python threads that end up eternally blocked waiting for something and I would like python to tell me which statement blocked them.

My case is the following: I am using some libraries that are causing my threads to block. I don't know the code of the libraries, so a first step in debugging is for python to tell me which statements are the guilty ones.

Is there a way for python to "tell me which threads are blocked now and which statement they are in"?

Answer 1

See this short example program that makes use of faulthandler.dump_traceback with all_threads=True (the default)

It creates 6 threads. 3 will run for a long time, getting "blocked" in the f1 function. The other three will run f2, which finishes after some short time and exits.

import faulthandler
import threading
import time


def f1():
    time.sleep(99999)

def f2():
    i = 1
    for j in range(10000):
        i += 1


for i in range(3):
    threading.Thread(target=f1).start()

for i in range(3):
    threading.Thread(target=f2).start()


time.sleep(1)

faulthandler.dump_traceback()

Upon running this program, you will notice it gets stuck (due to the open threads), but not before giving an output like so:

Thread 0x000040f8 (most recent call first):
  File "stack.py", line 7 in f1
  File "C:\Program Files\Python37\lib\threading.py", line 865 in run
  File "C:\Program Files\Python37\lib\threading.py", line 917 in _bootstrap_inner
  File "C:\Program Files\Python37\lib\threading.py", line 885 in _bootstrap

Thread 0x000034d8 (most recent call first):
  File "stack.py", line 7 in f1
  File "C:\Program Files\Python37\lib\threading.py", line 865 in run
  File "C:\Program Files\Python37\lib\threading.py", line 917 in _bootstrap_inner
  File "C:\Program Files\Python37\lib\threading.py", line 885 in _bootstrap

Thread 0x00003a60 (most recent call first):
  File "stack.py", line 7 in f1
  File "C:\Program Files\Python37\lib\threading.py", line 865 in run
  File "C:\Program Files\Python37\lib\threading.py", line 917 in _bootstrap_inner
  File "C:\Program Files\Python37\lib\threading.py", line 885 in _bootstrap

Current thread 0x00004bb8 (most recent call first):
  File "stack.py", line 24 in <module>

As we can see, the main thread is at line 24, (the location of the faulthandler.dump_traceback call which triggered this message), and 3 other threads are currently in f1, at line 7, time.sleep(99999) to be precise.

We don't necessarily know that they are "blocked", beyond the fact that we waited a reasonable amount of time before checking, 1 second in this case. If necessary, you could perhaps check repeatedly with a loop to monitor which threads remain in the same function after a while.