My team is switching from a single instance redis to a cluster of three in order to be more resilient. We sadly faced a crash when migrating and that raised questions.
Here is our configuration :
When it crashed, log indicates that it could not write to disk "No space left on device".
We had the same configuration on a testing environment, we filled it and it did not failed. But it failed on production environment. Is there an explanation for this ?
When looking at some documentation, it says that the persistence should be 3 times the ram memory, can you explain why ?
If you have any other advice on that matter, I would gladly read them.
Thank you
If you want to persist Redis data set, the disk size should be at least 2 times the size of the data set (might not be the same size of the memory size, since the memory size is not the same as the RDB/AOF file size).
Because when Redis does a new dump, it creates a new RDB file, and writes data set to it. Finally it renames the new file to the original RDB file, when it finishes dumping. So when Redis does a dump, it needs at least 2 times the size of data set, i.e. 2 RDB files.
Also, if your Redis instance has a replica, when the replica does a full sync, it needs to create a new RDB file, which also needs disk space, and send the whole file to replica. However, high version of Redis can use diskless replication, which does not need to dump a file to disk.