Spring Boot项目Redis周期性Connection Reset问题排查与解决咨询

问题现象 #

项目启动后Redis可正常读写，但闲置约6分钟后会抛出Connection Reset异常；异常抛出后立即重试可恢复正常，但再次闲置约6分钟后问题复现。

异常栈信息 #

2024-02-22 11:49:05,174 ERROR [communityScheduler_Worker-2] o.q.c.JobRunShell [JobRunShell.java:211] Job communityJobGroup.postScoreRefreshJob threw an unhandled Exception: 
org.springframework.data.redis.RedisSystemException: Redis exception
    at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:72)
    at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41)
    at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:40)
    at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:38)
    at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:256)
    at org.springframework.data.redis.connection.lettuce.LettuceConnection.await(LettuceConnection.java:969)
    at org.springframework.data.redis.connection.lettuce.LettuceConnection.lambda$doInvoke$4(LettuceConnection.java:826)
    at org.springframework.data.redis.connection.lettuce.LettuceInvoker$Synchronizer.invoke(LettuceInvoker.java:665)
    at org.springframework.data.redis.connection.lettuce.LettuceInvoker.just(LettuceInvoker.java:94)
    at org.springframework.data.redis.connection.lettuce.LettuceSetCommands.sCard(LettuceSetCommands.java:63)
    at org.springframework.data.redis.connection.DefaultedRedisConnection.sCard(DefaultedRedisConnection.java:824)
    at org.springframework.data.redis.connection.DefaultStringRedisConnection.sCard(DefaultStringRedisConnection.java:755)
    at org.springframework.data.redis.core.DefaultSetOperations.lambda$size$17(DefaultSetOperations.java:264)
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:406)
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:373)
    at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:97)
    at org.springframework.data.redis.core.DefaultSetOperations.size(DefaultSetOperations.java:264)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.data.redis.core.BoundOperationsProxyFactory$BoundOperationsMethodInterceptor.doInvoke(BoundOperationsProxyFactory.java:177)
    at org.springframework.data.redis.core.BoundOperationsProxyFactory$BoundOperationsMethodInterceptor.invoke(BoundOperationsProxyFactory.java:148)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
    at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:244)
    at jdk.proxy4/jdk.proxy4.$Proxy121.size(Unknown Source)
    at com.nowcoder.community.quartz.PostScoreRefreshJob.execute(PostScoreRefreshJob.java:63)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: io.lettuce.core.RedisException: java.net.SocketException: Connection reset
    at io.lettuce.core.internal.Exceptions.bubble(Exceptions.java:83)
    at io.lettuce.core.internal.Futures.awaitOrCancel(Futures.java:250)
    at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:74)
    at org.springframework.data.redis.connection.lettuce.LettuceConnection.await(LettuceConnection.java:967)
    ... 25 common frames omitted
Caused by: java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
    at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426)
    at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:254)
    at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
    at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:842)

环境与配置 #

基础环境 #

• Redis版本：7.0.12（Docker容器部署）
• Spring Boot版本：3.1.5
• JDK版本：17
• Redis客户端：spring-boot-starter-data-redis（默认使用Lettuce）

Redis配置（application.properties） #

spring.data.redis.host=${community.spring.data.redis.host}
spring.data.redis.port=${community.spring.data.redis.port}
spring.data.redis.password=${community.spring.data.redis.password}
spring.data.redis.database=${community.spring.data.redis.database}

spring.data.redis.lettuce.pool.max-active=15
spring.data.redis.lettuce.pool.max-idle=15
spring.data.redis.lettuce.pool.min-idle=15
spring.data.redis.lettuce.pool.max-wait=10000

Docker启动Redis脚本 #

docker run --restart=always -p 6379:6379 \
-v ./data:/data --name redis \
--network community --network-alias redis -d \
redis:7.0.12  --requirepass 

临时解决方案 #

创建定时任务每分钟执行一次Redis操作，强制保持连接活跃，但该方案会额外消耗网络资源，并非最优解。

问题原因 #

1. Redis主动断开闲置连接：Redis的timeout参数若设置为360秒（6分钟），会主动断开超过该时间无交互的连接，而Lettuce连接池未及时更新连接状态，再次使用时触发Connection Reset。
2. 中间网络设备断开连接：服务器防火墙、云服务商安全组或负载均衡可能存在闲置连接超时策略，自动断开长时间无交互的TCP连接。
3. Lettuce连接池未配置连接验证：当前仅设置了连接池大小参数，未开启连接有效性检查，无法识别已被断开的无效连接。

解决办法 #

1. 配置Lettuce连接池的连接有效性验证 #

在application.properties中添加以下配置，自动剔除无效连接：

# 获取连接时验证有效性，无效则丢弃并新建
spring.data.redis.lettuce.pool.test-on-borrow=true
# 空闲时定期验证连接有效性
spring.data.redis.lettuce.pool.test-while-idle=true
# 空闲连接验证间隔（毫秒）
spring.data.redis.lettuce.pool.time-between-eviction-runs=60000
# 空闲连接最小存活时间（毫秒，可选）
spring.data.redis.lettuce.pool.min-evictable-idle-time-millis=300000

2. 调整Redis服务器的闲置超时时间 #

若不需要Redis主动断开闲置连接，可将timeout设为0（禁用该功能）；若需保留断开策略，可设置更长时间（如3600秒）。修改Docker启动脚本：

docker run --restart=always -p 6379:6379 \
-v ./data:/data --name redis \
--network community --network-alias redis -d \
redis:7.0.12  --requirepass your_password --timeout 0

3. 开启Lettuce的TCP保活机制 #

通过TCP层发送心跳包，避免中间网络设备断开连接：

spring.data.redis.lettuce.tcp-keepalive=true
# TCP保活空闲时间（秒，可选，默认由系统决定）
spring.data.redis.lettuce.tcp-keep-idle=300

4. 配置Redis操作超时时间 #

避免因长时间阻塞导致连接异常：

spring.data.redis.timeout=30000

内容的提问来源于stack exchange，提问作者Xiang Zhou