Search code examples
pythonredismaster-slave

Implement a dynamic membership protocol using redis to keep the client list of workers updated


I am doing a master-worker architecture assignment with XMLRPC in Python and now I am supposed to implement a dynamic membership protocol so the workers list on the client gets updated everytime the master modifies it (worker added or deleted) so the client does not need to request it manually or before running a command.

The teacher mentioned it can be done through events or group communication (I will implement a manager node to ping workers which also needs the same dynamic protocol) so I thought about using sockets which is an event based architecture but the teacher told me I am better off using indirect communication with Redis or RabbitMQ.

The thing is I do not really know how to implement a Redis message listener on Python since I just find blocking examples with while True followed by redis.get_message() most of the time (using redis-py). Could you help me with that?

Thanks a lot in advance.


Solution

  • Thanks to Mephalich on Redis' Discord server I discovered an interesting way to achieve that which consists of using run_in_thread.

    import redis
    def hnd(msg):
       print(msg)
      # some real work  should be done in this handler over the msg arrived through the pubsub channel
    
    
    r = redis.Redis(...)
    p = r.pubsub()
    p.psubscribe(**{'cmdchannels*': hnd})
    thread = p.run_in_thread(sleep_time=0.001)