Search code examples
springspring-bootspring-cloud-configspring-data-redislettuce

Resource leak detected when using spring-data-redis on cloud foundry


We develop a spring-boot service, which offers a rest api (spring-webflux) and sends data via RabbitMQ (spring-rabbit). The service is deployed on cloud foundry and we use spring-boot in version 2.1.4. We added spring-boot-starter-data-redis to use redis to cache some data and we got the following error:

[io.netty.util.ResourceLeakDetector] [] LEAK: HashedWheelTimer.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
    io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:284)
    io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:217)
    io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:196)
    io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:178)
    io.netty.util.HashedWheelTimer.<init>(HashedWheelTimer.java:162)
    io.lettuce.core.resource.DefaultClientResources.<init>(DefaultClientResources.java:169)
    io.lettuce.core.resource.DefaultClientResources$Builder.build(DefaultClientResources.java:532)
    io.lettuce.core.resource.DefaultClientResources.create(DefaultClientResources.java:233)
    io.lettuce.core.AbstractRedisClient.<init>(AbstractRedisClient.java:98)
    io.lettuce.core.RedisClient.<init>(RedisClient.java:87)
    io.lettuce.core.RedisClient.create(RedisClient.java:124)
    org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.lambda$createClient$7(LettuceConnectionFactory.java:971)
    java.base/java.util.Optional.orElseGet(Unknown Source)
    org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createClient(LettuceConnectionFactory.java:971)
    org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.afterPropertiesSet(LettuceConnectionFactory.java:273)
    org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:88)
    org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator.create(RedisConnectionFactoryCreator.java:31)
    org.springframework.cloud.Cloud.getServiceConnector(Cloud.java:288)
    org.springframework.cloud.Cloud.getSingletonServiceConnector(Cloud.java:202)
    org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:260)
    org.springframework.cloud.config.java.CloudServiceConnectionFactory.redisConnectionFactory(CloudServiceConnectionFactory.java:242)
    ...

This error only happens when we run the service on cloud foundry, if we run it locally, we don't get any error.

We don't do any configuration of the connection factory or the stringRedisTemplate on our side and only use stringRedisTemplate, which is configured by the spring-autoconfiguration. We use following configuration for redis on cloud foundry:

@Configuration
@Profile( "cloud" )
public class CloudSpecificConfig extends AbstractCloudConfig {

   @Bean
   public RedisConnectionFactory redisConnectionFactory() {
      return connectionFactory().redisConnectionFactory();
   }
}

And this is how we use the template

@Component
@RequiredArgsConstructor
public final class RequestUtil {

   private final StringRedisTemplate myRedisTemplate;

   public String cacheId(String id, String value) {
      myRedisTemplate.opsForValue().set( id, value );

   }
}

These are our spring dependencies:

<properties>
  <spring-boot-version>2.1.4.RELEASE</spring-boot-version>
</properties>

  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-webflux</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-configuration-processor</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-actuator</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.cloud</groupId>
     <artifactId>spring-cloud-starter-sleuth</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-cloud-connectors</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-amqp</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>
  <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-data-redis</artifactId>
     <version>${spring-boot-version}</version>
  </dependency>

We are quite confused on our side, since we didn't do any specific configuration on our side. It looks for us like there is something wrong with the spring configuration on the cloud. Are we doing something wrong? Do we need to configure something differently? Is this a bug?


Solution

  • In the end the issue disappeared on our side, because we changed the redis client from lettuce to jedis.

    We had the problem with lettuce that we would lose the connection to our redis service on our cloud infrastructure. But since there was an update to the redis service at same time as we changed the client, we don't really know if it was related to lettuce.

    Maybe there also just something wrong in the auto-configuration in conjunction with the redis service on our cloud instructure, which is based on cloudfoundry