Search code examples
rediscluster-computingfailoverredis-sentinel

How to manage a situation where designated redis master on a cluster is down on startup


As I understand it if redis is to run on a cluster of machines, there will be several slaves and one master. And slaves can be made to connect to the master upon startup with a command. And then sentinels on each machine on the cluster could connect to the master to find out all slaves and then switch one of them to master if current master goes down for some reason.

Now, my question is - How can we fail over when the designated master is down when we start the redis servers on the cluster ? the SLAVEOF command will fail and there will not be a master and therefore nothing for the sentinels to get the config data from. How do we solve this problem ? Thanks !


Solution

  • So Redis Cluster which is still in development is meant to solve this exact problem.
    http://redis.io/topics/cluster-spec

    Zookeeper, HAProxy and a number other tools could be added to your setup to achieve this automated failover. On its own though Redis with sentinels only self manages a 1 piece of the puzzle.

    Your application now needs to update its config to point to the new master. So you need to add failover logic to your Redis connection logic. I don't know any easy solutions to point you to as it depends on what your application is written in.

    I asked a similar question before more specifically with Java Redis/Jedis no single point of failure and automated failover

    and got some good information back.