We are using KubeDB in our cluster to manage our DB's.
So Redis is deployed via a KubeDB Redis object and KubeDB attaches a PVC to the Redis pod.
Unfortunately KubeDB doesn't support any restoring or backing up of Redis dumps (yet).
For the backup our solution is to have a CronJob running which copies the dump.rdb from the Redis pod into the job pod and then uploads it to S3.
For the restoring of the dump I wanted to do the same, just the other way around. Have a temporary pod which downloads the S3 backup and then copies it over to the Redis pod into the dump.rdb location.
The redis.conf looks like this:
....
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /data
....
The copying works. The dump.rdb is in the correct location with the correct permissions. I verified this by starting a second redis-server in the Redis pod using the same redis.conf. The dump.rdb is being loaded into the server without a problem.
However, since I don't want to manually start a second redis-server, I restarted the Redis pod (by kubectl delete pods) for the pod to pickup the copied dump.rdb.
Everytime I delete the pod, the dump.rdb is deleted and a new dump.rdb is being created with a much smaller size (93 bytes).
I don't believe it is a PVC issue since I have created a few files to test whether they are deleted as well. They are not. Only the dump.rdb.
Why does this happen? I am expecting Redis to just restore the DB from the dump.rdb and not create a new one.
EDIT: Yeah, size of dump.rdb is around 47 GB. Redis version is 4.0.11.
Sooo, a few hours later, my teammate remembered that Redis executes a save to dump on shutdown.
Instead of deleting the pod using kubectl delete pod I now changed the code to run a SHUTDOWN NOSAVE using the redis-cli.
kubectl exec <redis-pod> -- /bin/bash -c 'redis-cli -a $(cat /usr/local/etc/redis/redis.conf | grep "requirepass " | sed -e "s/requirepass //g") SHUTDOWN NOSAVE'