I'm developing a server which has to process a big amount of small data requests.
The server is developed with ZeroMQ, using the PULL/PUSH pattern.
Basically each request process consists in storing its data (message) in Redis.
I would like to have better performance by processing each request with gevent.
The following code doesn't work (messages are not stored in Redis).
from gevent import monkey
monkey.patch_all()
from redis import Redis, StrictRedis
from redis import connection
import zmq
import gevent
context = zmq.Context()
socket = context.socket(zmq.PULL)
socket.bind("tcp://*:5000")
connection.socket = gevent.socket
redis = Redis()
def enqueue(message):
redis.lpush('work_queue', message)
while True:
message = socket.recv()
#print message
gevent.spawn(enqueue, message)
If I remove gevent stuff, then the code works (messages are stored correctly).
P.S. I have just started to play with zeromq and gevent.
Update: I wanted to implement a simple task queue. In ended up using Celery. The server (which use zmq) asynchronously starts celery tasks (and it works great).
To use zeromq with gevent, you have to import de zmq.green module.
import zmq.green as zmq