Python multithreading, Queues for messaging and avoiding processor hogging

Question

I have a process sends messages between threads using Queues.

# receiver.py
class Receiver(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.daemon = True
        self.inbox = Queue.Queue()

    def run(self):
        while True:
            if not self.inbox.empty():
                msg = self.inbox.get()
                # do other stuff


# main.py
def main():
    R1 = Receiver()
    R2 = Receiver()
    R1.start()
    R2.start()

    # spin up child threads that can also stuff messages into Receiver() inboxes

    while True:
        msg = "You're hogging processor time"
        R1.inbox.put(msg)
        R2.inbox.put(msg)
        # do a whole bunch more fancy stuff

if __name__ == '__main__':
    main()

When I look at the processor time % alloted to this process, it's usually pinned at > 90%.

Is there a better paradigm besides a while-True-check-inbox? I've tried sleeps, but the threads need to respond immediately.

Answer 1

Queue.get will wait (block) until there's something in the queue. During this wait, the thread will sleep, allowing other threads (and processes) to run.

So just remove your check for self.inbox.empty():

def run(self):
    while True:
        msg = self.inbox.get()
        # do other stuff