Search code examples
spring-bootredistransactionshigh-availabilityspring-data-redis

How to serve the old data while updating it in Redis with high availability?


I have a spring boot application that periodically updates hash data in redis and exposes this data via a REST API. Hash contains about 800-900k records.

Every iteration I'm deleting the key and inserting them again in a redis transaction with multi/exec.

The key point here is that the availability, inserting data takes about 25-30 seconds and I need to serve the old data during that 25-30 seconds.

Currently we are using the commands below to update the data;

MULTI
DEL "key"
HSET "key" "x" "value"
HSET "key" "y" "value"
..
..
EXEC 

As you know, all commands after multi are queued in redis and once the EXEC command is executed, all queued commands are executed.

Differently, I thought that the below algorithm would be better for availability;

MULTI
HSET "key_temp" "x" "value"
HSET "key_temp" "y" "value"
..
..
DELETE "key"
RENAMENX "key_temp" "key"
EXEC

In solution 1, during exec command execution, I am deleting the key first, and in that time (25 seconds), API is not able to serve the old data.

In solution 2, the old data will be served while inserting new one, and the data only will not be available in DELETE and RENAMENX command execution which probably takes 1-2 ms.

I wanted to know your feedback and if there is better solution for this problem? Thank you.


Solution

  • Option2 looks good. Few things you can improve:

    1. no need to include the HSET in the transaction
    HSET "key_temp" "x" "value"
    HSET "key_temp" "y" "value"
    ..
    ..
    DELETE "key"
    MULTI
    DEL "key"
    RENAMENX "key_temp" "key"
    EXEC
    
    1. Update multi fields in the same HSET
    HSET "key_temp" "x" "value" "y" "value"