Search code examples
pythonredispipeline

using pipeline for bulk processing in redis (python example)

I'm trying to clear my concept of pipelining implemented in redis, using a python client. Take the following example:

my_server = redis.Redis(connection_pool=POOL)
for obj_id in list_of_obj_ids:
    hash_name="n:"+str(obj_id)
    sorted_set = "s:"+str(obj_id)
    if my_server.exists(hash_name):
        my_server.hset(hash_name,'val',0)
    if my_server.zcard(sorted_set):
        my_server.zadd(sorted_set, hash_name, time.time())

I.e. I'm updating multiple hashes via iterating through a for loop. How can I accomplish this kind of bulk update via pipelining? From what I've read, the following is what comes to my mind:

my_server = redis.Redis(connection_pool=POOL)
p = my_server.pipeline()
for obj_id in list_of_obj_ids:
    hash_name="n:"+str(obj_id)
    sorted_set="s:"+str(obj_id)
    if p.exists(hash_name):
        p.hset(hash_name,'val',0)
    if p.zcard(sorted_set):
        p.zadd(sorted_set, hash_name, time.time())
p.execute()

Is this correct?

Solution

  • Read about what pipelining is/does and then you'll understand why this won't work - until you execute the pipeline, none of the commands in it will be sent to the server. That makes your conditional statements miss the purpose you had in mind.