AWS Redis : JedisConnectionException: java.net.SocketException: Too many open files

Question

I have configured redis using AWS Elasticache and connecting it through tomcat installed on AWS EC2.

Following is my code:

private JedisPool jedisPool = null;

@PostConstruct
private void initialiseJedisPools() {
    try {
        String redisHost = RESOURCES.getProperty("redis.master.host");
        if(Objects.nonNull(redisHost)) {
            Integer redisPort = Integer.valueOf(RESOURCES.getProperty("redis.master.port"));
            jedisPool = new JedisPool(redisHost, redisPort);
        }
    } catch (NumberFormatException exception) {
        logger.error("Exception occurred while initialising jedis pool.", exception);
    }
}

public void addKey(String key, String value, Integer secondsToExpire) {
    if (Objects.nonNull(jedisPool)) {
        try (Jedis jedis = jedisPool.getResource()) {
            jedis.set(key, value);
            if (Objects.nonNull(secondsToExpire)) {
                jedis.expire(key, secondsToExpire.intValue());
            }
        } catch (JedisException jedisException) {
            logger.error("Exception thrown while adding key in cache.", jedisException);
        }
    }
}

Often a while I get the following error and I have to restart tomcat to make it work.

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
        at redis.clients.util.Pool.getResource(Pool.java:53)
        at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
        .
        .
        .
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Too many open files
        at redis.clients.jedis.Connection.connect(Connection.java:207)
        at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
        at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
        at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
        at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
        at redis.clients.util.Pool.getResource(Pool.java:49)
        ... 10 more
Caused by: java.net.SocketException: Too many open files
        at java.net.Socket.createImpl(Socket.java:460)
        at java.net.Socket.getImpl(Socket.java:520)
        at java.net.Socket.setReuseAddress(Socket.java:1449)
        at redis.clients.jedis.Connection.connect(Connection.java:174)
        ... 17 more

I have tried increasing the ulimit for open files, configuring a JedisPoolConfig for maxTotal, maxIdle, minIdle, etc, but no success.

Please suggest.

Answer 1

We have also faced similar issue in past for jedis connection with error "java.net.SocketException: Too many open files"

This happens due to jedis connection not getting closed.

call jedisPool.returnResourse(jedis) or jedis.close()