Events for stopping Threads vs ThreadPoolExecutor

Question

I'm trying to understand all the differences between using a ThreadPoolExecutor and just using threads in python. I tried to use a threadpoolexecutor, and it seemed to act as a blocking function (the context didnt seem to move on and allow the main thread to continue). When I use normal Threads, things work as intended.

I'm trying to understand all the differences between using a ThreadPoolExecutor and just using threads in python. I've found a simple example that shows a hole in my understanding:

import time
import threading
from concurrent.futures import ThreadPoolExecutor
event = threading.Event()

tot=0

def action():
    global tot
    while not event.is_set():
        time.sleep(1)
        tot+=1
    return


"""
Use a threadpool
"""
with ThreadPoolExecutor(max_workers = 4) as executor:
    executor.submit(action)
    executor.submit(action)
    executor.submit(action)
    executor.submit(action)
    
time.sleep(5.5)
event.set()
    
print(f"tot = {tot}")

My understanding is that the threadpool should start the four threads. Then after 5.5 seconds, they should be halted by the event being set. However, when I run this, it hangs, and never seems to leave the threadpool context.

If I replace the lower half with the following:

"""
dont use a threadpool
"""
submissions = [
        threading.Thread(target=action).start() for i in range(4)
    ]
time.sleep(5.5)

event.set()

print(f"tot = {tot}")

Everything works as I think it should, and it spits out 20

Clearly I'm deeply misunderstanding something important about ThreadPoolExecutors. Could somebody explain what I'm missing?

Answer 1

Part of the action of with ThreadPoolExecutor is that it waits for all threads to finish before it completes. It does a join on all of the threads. But none of your threads can finish until the event is set. You have deadlock.

Change your code to:

with ThreadPoolExecutor(max_workers = 4) as executor:
    executor.submit(action)
    executor.submit(action)
    executor.submit(action)
    executor.submit(action)
    
    time.sleep(5.5)
    event.set()

and everything works fine.