Search code examples
javaspring-bootredisjedisspring-data-redis

Spring-Data-Redis with Jedis putIfAbsent for distributed lock - incorrect behavior

I have some issues in a usage of Spring Data Redis to create a distributed lock. For this it is used the putIfAbsent method from the CacheManager.

From a high level perspective the operation looks something like this:

if (manager.putIfAbsent(parameters) == null) {
   executeOperation();
}

From the implementation of the putIfAbsent it seems the setNX operation from the underlying Jedis driver, is used.

The code of the Spring implementation looks something like:

if (!connection.setNX(keyBytes, value)) {
   return connection.get(keyBytes);
}
maintainKnownKeys(element, connection);
processKeyExpiration(element, connection);

The connection's setNX is just a delegation to the actual client operation. On the implementation of that method there is something like:

JedisConverters.toBoolean(jedis.setnx(key, value));

So I encountered two separate issues:

  1. My executeOperation() was simultanously executed by two separate processes. (Just a few occurences of this issue).

  2. I reached a situation were the key remained and was not expired. This means the code processKeyExpiration(element, connection) was not executed. This means the setNx executed as the key was not added and returned before that statement, but in fact the key was added.

Most of the time everything works fine. The key serializer is StringRedisSerializer.

I am using: spring-data-redis 1.8.23.RELEASE jedis 2.9.3

Can there be some environmental issues that are not treated correctly by Jedis, or something like this ? Has anyone reached something like this ? Is any fix that can be tried, a library update ?

Solution

  • So I did a bit of more analysis on this. And it seems that due to the fact of how putIfAbsent is implemented is prone to race conditions if used in multiple processes / threads.

    This fact is because the series of commands for putIfAbsent implementation, setNX, expire, get, is not transactional and due to proper conditions (long lasting operations, not proper eviction logic) it is prone to incorrect behavior.

    A similar explanation, of how to conduct distributed locking based on Redis setNX operation, can be found here Redis setNX command.

    The implementation from putIfAbsent is prone to incorrect behavior when used for locking, in a similar fashion that is described into the above documentation.

    So to sum up, probably it isn't the best idea to use putIfAbsent with Jedis driver for distributed locking, at least on that version of Spring, since from 2.x.x I know the implementation is changed a bit.