Why does "event.set()" not break a "while not event.is_set()" loop in a sub-thread? (see code)

Question

I'm using events to communicate between my main code and a sub-thread. The thread contains a while loop that should get exited once stop_event.set() is called in the main code:

import time
from threading import *


def stream_data(stop_event, new_image_event):

    while not stop_event.is_set():
        new_image_event.wait()
        new_image_event.clear()

        # Do some stuff
        print('Stuff!')

    print('Successfully exited while loop!')
    return 


if __name__ == '__main__':

    # Initialising events
    stop_event = Event()
    new_image_event = Event()

    # Starting the thread
    thread = Thread(target=stream_data, args=(stop_event, new_image_event, ))
    thread.start()

    # Do stuff before setting the stop event, while generating a 'new' new_image_event every 1s
    for i in range(5):
        new_image_event.set()
        time.sleep(1)

    # generating the stop event
    stop_event.set()
    print('stop_event has been set')

    # joining back the thread
    thread.join()

Output:

Stuff!
Stuff!
Stuff!
Stuff!
Stuff!
stop_event has been set

So the new_image_event does it's job, and at the end the stop_event is successfully set. But for some reason this does not break the while loop in the thread, and 'Successfully exited while loop!' is never printed. Why? And how can I solve this (preferably without resorting to classes and/or self)?

Answer 1

You are trying to satisfy waiting for new_image_event while remaining responsive to stop_event at the same time. As Paul correctly points out in his diagnosis of your issue, you cannot respond to the stop_event while you are in a wait for another event. His solution, however, is wrong.

I think your best solution is to leverage the timeout mechanism of Event.wait(). Per the documentation:

This method returns True if and only if the internal flag has been set to true, either before the wait call or after the wait starts, so it will always return True except if a timeout is given and the operation times out.

In other words, change your thread loop to the following:

def stream_data(stop_event, new_image_event):
    while not stop_event.is_set():
        # Adjust the following timeout as appropriate. It controls
        # the amount of "dead time" before this thread responds
        # to `stop_event` if `new_image_event` was not raised
        # during the timeout.
        if new_image_event.wait(0.1):
            new_image_event.clear()
            # Do some stuff
            print('Stuff!')
        time.sleep(0.001)  # Breathe, dawg

    print('Successfully exited while loop!')
    return

This produces the result you expect:

Stuff!
Stuff!
Stuff!
Stuff!
Stuff!
stop_event has been set
Successfully exited while loop!

p.s. The 1 ms sleep at the bottom of the loop is not strictly necessary, but I find it often useful so that the thread has a chance to "breathe" instead of incessantly chasing its own tail :)