I am working on a larger application that requires eventlet, and now requires rabbitMQ as well. It appears that eventlet is causing the pika consumer thread to block execution of additional workers. I know Pika is not considered thread-safe, so I have everything, including the connection, within it's own thread. I would assume the blocking connection should only block the consumer thread. How can I get pika and eventlet to work together? In the below example, the workers thread never prints anything out, but commenting out eventlet.monkey_patch()
allows both threads to execute.
import threading
import pika
import eventlet
eventlet.monkey_patch()
def callback(ch, method, properties, body):
print body
ch.basic_ack(delivery_tag=method.delivery_tag)
def consumer():
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='test', durable=True,
exclusive=False, auto_delete=False)
channel.basic_consume(callback, queue='test')
channel.start_consuming()
def start_consumer_thread():
# initialize a listener thread
consumer_thread = threading.Thread(target=consumer)
consumer_thread.start()
def worker():
start_consumer_thread()
for x in range(1,10000):
print x
x = threading.Thread(target=worker())
x.start()
Pika and eventlet.monkey_patch
are not compatible. You will have to use eventlet
without patching system calls, if that is possible.