Spring Boot+Hikari池+JBoss部署下DB2连接超时警告排查

问题描述 #

基于Spring Boot的应用采用HikariCP管理DB2数据库连接，部署在JBoss上。日志显示连接池状态充足：

23:09:07,246 INFO  [stdout] (HikariPool-3 housekeeper) 2024-02-08 23:09:07.246 | DEBUG | com.zaxxer.hikari.pool.HikariPool:421   | HikariPool-3 - Before cleanup stats (total=30, active=0, idle=30, waiting=0)
23:09:07,246 INFO  [stdout] (HikariPool-3 housekeeper) 2024-02-08 23:09:07.246 | DEBUG | com.zaxxer.hikari.pool.HikariPool:421   | HikariPool-3 - After cleanup  stats (total=30, active=0, idle=30, waiting=0)
23:09:07,246 INFO  [stdout] (HikariPool-3 housekeeper) 2024-02-08 23:09:07.246 | DEBUG | com.zaxxer.hikari.pool.HikariPool:518   | HikariPool-3 - Fill pool skipped, pool is at sufficient level.

但无新建连接需求的情况下，20秒后仍出现连接超时警告：

23:09:20,158 WARN  [org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory] (HikariPool-1 connection adder) IJ030027: Destroying connection that is not valid, due to the following exception: com.ibm.db2.jcc.t4.b@28052227: com.ibm.db2.jcc.am.DisconnectNonTransientConnectionException: [jcc][t4][2030][11211][4.31.10] A communication error occurred during operations on the connection's underlying socket, socket input stream, 
or socket output stream.  Error location: Reply.fill() - socketInputStream.read (-1).  Message: Read timed out. ERRORCODE=-4499, SQLSTATE=08001
    at com.ibm.db2.jcc.am.b7.a(b7.java:338)
    at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:464)
    at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
    at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1341)
    at sun.security.ssl.SSLSocketImpl.access$300(SSLSocketImpl.java:73)
    at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:957)
    at com.ibm.db2.jcc.t4.y.b(y.java:230)
    ... 31 more

当前HikariCP配置：

connection-timeout=180000
minimum-idle=30
maximum-pool-size=700
idle-timeout=120000
max-lifetime=300000
auto-commit=true
leak-detection-threshold=180000

Hikari Version - 4.0.3

原因分析 #

1. 双重连接池冲突：JBoss自带的JCA连接池与HikariCP形成嵌套结构，Hikari从JBoss数据源获取连接时，JBoss会对连接做有效性校验，若此时DB2端或中间网络已断开连接，就会触发超时错误。
2. 连接存活时间不匹配：Hikari的max-lifetime设为5分钟，但DB2服务器或防火墙的连接超时时间可能更短，导致池内空闲连接被提前断开，Hikari未及时清理时，JBoss校验连接就会报错。
3. 校验逻辑不同步：Hikari的housekeeper刚完成连接清理，JBoss后续的连接分配校验仍会检测到已失效的连接。

解决办法 #

1. 禁用JBoss连接池，直接使用HikariCP #

移除JBoss中该数据源的JCA配置，在Spring Boot中直接配置HikariCP连接DB2：

spring.datasource.url=jdbc:db2://your-db-host:port/dbname
spring.datasource.username=xxx
spring.datasource.password=xxx
spring.datasource.driver-class-name=com.ibm.db2.jcc.DB2Driver
spring.datasource.hikari.connection-timeout=180000
spring.datasource.hikari.minimum-idle=30
spring.datasource.hikari.maximum-pool-size=700
spring.datasource.hikari.idle-timeout=120000
spring.datasource.hikari.max-lifetime=300000
spring.datasource.hikari.auto-commit=true
spring.datasource.hikari.leak-detection-threshold=180000

2. 匹配连接存活时间设置 #

• 查询DB2服务器的IDLE_TIMEOUT参数，确保Hikari的max-lifetime比该值小至少30秒
• 确认防火墙连接超时时间，同样让max-lifetime短于该值
• 添加连接校验超时配置：

  spring.datasource.hikari.validation-timeout=5000
  

3. 配置HikariCP自身的连接校验 #

让Hikari负责连接有效性校验，提前清理失效连接：

spring.datasource.hikari.connection-test-query=SELECT 1 FROM SYSIBM.SYSDUMMY1
spring.datasource.hikari.validation-interval=30000

确保validation-interval小于idle-timeout，定期校验空闲连接。

4. 同步JBoss与Hikari配置（保留双重池时） #

• 设置JBoss连接池的idle-timeout小于Hikari的max-lifetime
• 让JBoss的validation-query与Hikari保持一致，避免重复校验
• 关闭JBoss自动连接清理，交由Hikari管理

内容的提问来源于stack exchange，提问作者Coder