Python thread.join(timeout) not timing out

Question

I am using threading python module. I want to execute a function which runs an expression entered by a user. I want to wait for it to finish execution or until a timeout period is reached. The following code should timeout after 5 second, but it never times out.

def longFunc():
    # this expression could be entered by the user
    return 45 ** 10 ** 1000
 
thread = threading.Thread(target=longFunc, args=(), daemon=True)
thread.start()
thread.join(5.0)
print("end") # never reaches this point :(

Why is this and how can I fix this behaviour? Should I try to use multiprocessing instead?

Answer 1

I suspect in this case you're hitting an issue where the join can't execute while the global interpreter lock is held by the very long-running calculation, which I believe would happen as a single atomic operation. If you change longFunc to something that happens over multiple instructions, such as a busy loop, e.g.

def longFunc():
    while True:
        pass

then it works as expected. Is the single expensive calculation realistic for your situation or did the example just happen to hit a very bad case?

Using the multiprocessing module does appear to resolve this issue:

from multiprocessing import Process

def longFunc():
    # this expression could be entered by the user
    return 45 ** 10 ** 1000

if __name__ == "__main__":
    thread = Process(target=longFunc, args=(), daemon=True)
    thread.start()
    thread.join(5.0)
    print("end")

This prints "end" as expected.