Redis列表批量插入11000条后触发RedisSystemException问题排查
问题
尝试向Redis列表中批量插入数据,插入11000条元素后,下一批数据插入时触发RedisSystemException,嵌套异常为java.lang.NullPointerException。报错后应用停止,重启后会再次插入数据,但插入11000条后仍会失败,目前列表已有22000条元素。
报错日志
2022-10-07 01:30:44 [main] DEBUG com.dd - Inserted 10500 of batch 21 in 0.030744911 2022-10-07 01:30:45 [main] DEBUG com.dd - Current batch size: 500 2022-10-07 01:30:45 [main] DEBUG com.dd - Inserted 11000 of batch 22 in 0.05900122 2022-10-07 01:30:46 [main] DEBUG com.dd - Current batch size: 500 2022-10-07 01:30:46 [main] DEBUG com.dd - Error occurred while importing to redis org.springframework.data.redis.RedisSystemException: Unknown redis exception; nested exception is java.lang.NullPointerException at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:53) at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:43) at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:777) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.convertJedisAccessException(JedisClusterListCommands.java:348) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:53) at org.springframework.data.redis.connection.DefaultedRedisConnection.rPush(DefaultedRedisConnection.java:559) at org.springframework.data.redis.connection.DefaultStringRedisConnection.rPush(DefaultStringRedisConnection.java:895) at org.springframework.data.redis.core.DefaultListOperations.lambda$rightPushAll$9(DefaultListOperations.java:242) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188) at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96) at org.springframework.data.redis.core.DefaultListOperations.rightPushAll(DefaultListOperations.java:242) at com.dd.redis.RedisClusterRepository.addToList(RedisClusterRepository.java:32) at com.dd.processBatch(CardAppImport.java:120) at com.dd.importV(CardAppImport.java:91) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) at com.dd.main(MigrationApplication.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRunMojo.java:558) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException: null at redis.clients.jedis.Protocol.sendCommand(Protocol.java:100) at redis.clients.jedis.Protocol.sendCommand(Protocol.java:85) at redis.clients.jedis.Connection.sendCommand(Connection.java:115) at redis.clients.jedis.BinaryClient.rpush(BinaryClient.java:341) at redis.clients.jedis.BinaryJedis.rpush(BinaryJedis.java:1142) at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:583) at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:580) at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:106) at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:49) at redis.clients.jedis.BinaryJedisCluster.rpush(BinaryJedisCluster.java:585) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:51) ... 40 common frames omitted 2022-10-07 01:30:46 [main] DEBUG com.dd.mport - Error during import org.springframework.data.redis.RedisSystemException: Unknown redis exception; nested exception is java.lang.NullPointerException at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:53) at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:43) at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:777) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.convertJedisAccessException(JedisClusterListCommands.java:348) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:53) at org.springframework.data.redis.connection.DefaultedRedisConnection.rPush(DefaultedRedisConnection.java:559) at org.springframework.data.redis.connection.DefaultStringRedisConnection.rPush(DefaultStringRedisConnection.java:895) at org.springframework.data.redis.core.DefaultListOperations.lambda$rightPushAll$9(DefaultListOperations.java:242) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188) at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96) at org.springframework.data.redis.core.DefaultListOperations.rightPushAll(DefaultListOperations.java:242) at com.dd.RedisClusterRepository.addToList(RedisClusterRepository.java:32) at com.dd.processBatch(CardAppImport.java:120) at com.dd.importp(CardAppImport.java:91) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) at com.dd.main(MigrationApplication.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRunMojo.java:558) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException: null at redis.clients.jedis.Protocol.sendCommand(Protocol.java:100) at redis.clients.jedis.Protocol.sendCommand(Protocol.java:85) at redis.clients.jedis.Connection.sendCommand(Connection.java:115) at redis.clients.jedis.BinaryClient.rpush(BinaryClient.java:341) at redis.clients.jedis.BinaryJedis.rpush(BinaryJedis.java:1142) at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:583) at redis.clients.jedis.BinaryJedisCluster$48.execute(BinaryJedisCluster.java:580) at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:106) at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:49) at redis.clients.jedis.BinaryJedisCluster.rpush(BinaryJedisCluster.java:585) at org.springframework.data.redis.connection.jedis.JedisClusterListCommands.rPush(JedisClusterListCommands.java:51) ... 40 common frames omitted
Redis配置代码
@Bean public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory cf) { RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>(); redisTemplate.setConnectionFactory(cf); redisTemplate.setKeySerializer(new StringRedisSerializer()); redisTemplate.setHashKeySerializer(new StringRedisSerializer()); redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer()); redisTemplate.afterPropertiesSet(); return redisTemplate; }
插入列表方法代码
public Long addToList(String key, List<String> list) { return redisTemplate.opsForList().rightPushAll(key, list.stream().toArray(String[]::new)); }
问题排查与解决方案
核心原因分析
从栈追踪看,NPE发生在redis.clients.jedis.Protocol.sendCommand(Protocol.java:100),说明发送Redis命令时存在空对象,结合重复出现的11000条插入节点,可能的触发点:
- Jedis版本bug:旧版Jedis(如2.x)在集群模式下,重试机制或连接管理存在缺陷,当处理大列表或连接池资源耗尽时,会获取到空连接对象。
- 批量数据含null元素:如果待插入的
List<String>中存在null值,转换为数组后传递给rightPushAll会导致命令参数为空,触发NPE。 - 连接池配置不足:连接池最大连接数过小,批量插入时连接耗尽,后续获取连接返回null。
- Redis集群异常:插入到11000条时,目标key对应的槽位所在节点出现连接中断、槽位迁移等情况,导致连接失效。
解决方案
- 校验并过滤空数据:修改插入方法,过滤list中的null元素:
public Long addToList(String key, List<String> list) { List<String> nonNullList = list.stream().filter(Objects::nonNull).collect(Collectors.toList()); if (nonNullList.isEmpty()) { return 0L; } return redisTemplate.opsForList().rightPushAll(key, nonNullList.toArray(new String[0])); } - 升级Jedis版本:将Jedis依赖升级到3.x或4.x稳定版,修复旧版本集群连接的NPE问题。
- 优化连接池配置:调整Redis连接池参数,避免连接耗尽,示例配置:
spring.redis.jedis.pool.max-active=200 spring.redis.jedis.pool.max-idle=50 spring.redis.jedis.pool.min-idle=10 spring.redis.jedis.pool.max-wait=3000ms - 添加异常重试机制:在插入逻辑中捕获
RedisSystemException,进行有限次数的重试,同时确保重试时重新获取连接:public Long addToListWithRetry(String key, List<String> list, int retryTimes) { int count = 0; while (count < retryTimes) { try { return addToList(key, list); } catch (RedisSystemException e) { count++; if (count == retryTimes) { throw e; } try { Thread.sleep(1000); } catch (InterruptedException ignored) {} } } return 0L; } - 检查Redis集群状态:确认集群所有节点在线、槽位分配稳定,无节点故障或正在进行的槽位迁移操作。
内容的提问来源于stack exchange,提问作者mahfuj asif
相关产品推荐
相关产品推荐

