Search code examples
pythonmultithreadingqueueproducer-consumer

How to lock queue for one thread?


I have a queue that has several producer and consumer threads adding and removing items. Every once in a while, another thread needs to lock the queue, remove and process every item in the queue, and then release the queue back to the rest of the threads. How can I keep blocking other threads between successive calls of queue.get().


Solution

  • Thanks to martineau, who has a couple of good approaches to the solution in the comments. Here is an example class that locks the queue while processing all items in the queue:

    import queue
    from threading import Thread
    
    
    class PriorityConsumer(Thread):
    
        def __init__(self, q):
            super(PriorityConsumer, self).__init__()
            self.q = q
        
        def process(self, x):
            pass
    
        def run(self):
            with self.q.mutex:
                while self.q.queue:
                    x = self.q.queue.popleft()
                    self.process(x)
    
    
    q = queue.Queue()
    p_consumer = PriorityConsumer(q)
    p_consumer.start()