RedisClusterClient, one connection or one connection per thread

Question

I'm trying to decide whether or not to use multiple stateful connections across more than one thread or only reuse one. I'm using java.

Even though it says for StatefulRedisClusterConnectionImpl that the connection is thread-safe to a Redis Cluster and that multiple threads may share one, I am getting better performance results when using one connection per thread.

This is my code testing code:

final int THREADS = 5;
List<RedisAdvancedClusterCommands<String, String>> cmdLIst = new LinkedList<>();
for (int j = 0; j < THREADS; j++) {
    cmdLIst.add(redisClient.connect().sync());
}
RedisAdvancedClusterCommands<String, String> cmd = redisClient.connect().sync();
HashMap<Integer, List<String>> keysPerNode = new HashMap<>();
for (int i = 0; i < THREADS; i++) {

    List<String> keys = generateKeys();
    keysPerNode.put(i, keys);
    Thread insertThread = new InsertClass(cmd, keys, startLatch, endLatch);
    //Thread insertThread = new InsertClass(cmdLIst.get(i), keys, startLatch, endLatch);
    insertThread.setName("Insert thread(" + i + ")");

    insertThread.start();
}

Thread.sleep(5000);

for (int j = 0; j < THREADS; j++) {
    startLatch.countDown();
}

long start = System.currentTimeMillis();
endLatch.await();
printElapsed((System.currentTimeMillis() - start) / 1000);
cmdLIst.forEach(RedisClusterCommands::close);
cmd.close();

InsertClass extends Thread.
I was shifting between using cmd where I reuse connection for all threads and using cmdLIst where every connection uses it's own connection. My setup is 4 masters on 4 remote (code is not running on the same machine as redis cluster) machines (+ slaves).
Every thread performs 80k set operations for random keys and than 80k get operations for these same keys. Also I tried to do that same thing, but with 1 key (80k set's for that one key + 80k get's for that same key, same key for all threads). Result is always the same, when using multiple connections (1 connection per thread), I get around 20 second difference (95 sec vs 115 sec).

Is there some kind of drawback when using one connection per thread? Or some undesired consequences that I am not aware of, because it seems to me that this will be preferred way of using this API.

Answer 1

Lettuce connections are thread-safe. A single connection can be shared across multiple threads. The only exceptions where you should not share a connection is when you're using transactions (not applicable for Redis Cluster) or blocking Redis commands (such as BLPOP, BRPOP).

A Redis cluster connection creates up to

permanent TCP connections where n is the number of Redis Cluster nodes.

If you have a small cluster and a reasonable number of threads per application instance, you can apply the connection per thread pattern. If your cluster /application scale reaches a size at which you get an unreasonable number of connections, then you might be forced to use a connection per app instance.

In general, using a single connection involves fewer resources. The default pipelining operation mode sends commands from invoking threads as soon as the command is invoked – the connection does not wait until a previous command is finished.