I am trying to batch insert in redis list. Surprisingly right after inserting 11000 elements in redis list, i am getting this error while inserting next batch.
2022-10-07 01:30:44 [main] DEBUG com.dd - Inserted 10500 of batch 21 in 0.030744911
2022-10-07 01:30:45 [main] DEBUG com.dd - Current batch size: 500
2022-10-07 01:30:45 [main] DEBUG com.dd - Inserted 11000 of batch 22 in 0.05900122
2022-10-07 01:30:46 [main] DEBUG com.dd - Current batch size: 500
2022-10-07 01:30:46 [main] DEBUG com.dd - Error occurred while importing to redis
org.springframework.data.redis.RedisSystemException: Unknown redis exception; nested exception is java.lang.NullPointerException
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:53)
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:43)
at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:777)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.convertJedisAccessException(JedisClusterListCommands.java:348)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:53)
at org.springframework.data.redis.connection.DefaultedRedisConnection.rPush(DefaultedRedisConnection.java:559)
at org.springframework.data.redis.connection.DefaultStringRedisConnection.rPush(DefaultStringRedisConnection.java:895)
at org.springframework.data.redis.core.DefaultListOperations.lambda$rightPushAll$9(DefaultListOperations.java:242)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188)
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96)
at org.springframework.data.redis.core.DefaultListOperations.rightPushAll(DefaultListOperations.java:242)
at com.dd.redis.RedisClusterRepository.addToList(RedisClusterRepository.java:32)
at com.dd.processBatch(CardAppImport.java:120)
at com.dd.importV(CardAppImport.java:91)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at com.dd.main(MigrationApplication.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRunMojo.java:558)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at redis.clients.jedis.Protocol.sendCommand(Protocol.java:100)
at redis.clients.jedis.Protocol.sendCommand(Protocol.java:85)
at redis.clients.jedis.Connection.sendCommand(Connection.java:115)
at redis.clients.jedis.BinaryClient.rpush(BinaryClient.java:341)
at redis.clients.jedis.BinaryJedis.rpush(BinaryJedis.java:1142)
at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:583)
at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:580)
at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:106)
at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:49)
at redis.clients.jedis.BinaryJedisCluster.rpush(BinaryJedisCluster.java:585)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:51)
... 40 common frames omitted
2022-10-07 01:30:46 [main] DEBUG com.dd.mport - Error during import
org.springframework.data.redis.RedisSystemException: Unknown redis exception; nested exception is java.lang.NullPointerException
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:53)
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:43)
at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:777)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.convertJedisAccessException(JedisClusterListCommands.java:348)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:53)
at org.springframework.data.redis.connection.DefaultedRedisConnection.rPush(DefaultedRedisConnection.java:559)
at org.springframework.data.redis.connection.DefaultStringRedisConnection.rPush(DefaultStringRedisConnection.java:895)
at org.springframework.data.redis.core.DefaultListOperations.lambda$rightPushAll$9(DefaultListOperations.java:242)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188)
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96)
at org.springframework.data.redis.core.DefaultListOperations.rightPushAll(DefaultListOperations.java:242)
at com.dd.RedisClusterRepository.addToList(RedisClusterRepository.java:32)
at com.dd.processBatch(CardAppImport.java:120)
at com.dd.importp(CardAppImport.java:91)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at com.dd.main(MigrationApplication.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRunMojo.java:558)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at redis.clients.jedis.Protocol.sendCommand(Protocol.java:100)
at redis.clients.jedis.Protocol.sendCommand(Protocol.java:85)
at redis.clients.jedis.Connection.sendCommand(Connection.java:115)
at redis.clients.jedis.BinaryClient.rpush(BinaryClient.java:341)
at redis.clients.jedis.BinaryJedis.rpush(BinaryJedis.java:1142)
at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:583)
at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:580)
at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:106)
at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:49)
at redis.clients.jedis.BinaryJedisCluster.rpush(BinaryJedisCluster.java:585)
at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:51)
... 40 common frames omitted
Here is my redis configuration
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory cf) {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(cf);
redisTemplate.setKeySerializer(new StringRedisSerializer());
redisTemplate.setHashKeySerializer(new StringRedisSerializer());
redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
And here is method that is adding elements to redis list
public Long addToList(String key, List<String> list) {
return redisTemplate.opsForList().rightPushAll(key, list.stream().toArray(String[]::new));
}
After getting the error my application stops. After that if i restart my application it starts to insert in redis list again, inserts 11000 elements and stops again. So now i have 22000 elements in redis list. Why is this happening?
As the exception message is not very much specific, i am struggling to find the problem.
If it is consistently failing after 11000 elements, I suggest you to check values that are being inserted in the batch which is failing. May be some value in that batch is null or not of String type.