IBM MQ连接断开(MQRC_CONNECTION_BROKEN)问题排查求助
问题描述
- 无消息发布到主题时,触发IBM MQ调用失败错误,
compcode=2(MQCC_FAILED),reason=2009(MQRC_CONNECTION_BROKEN);有消息发布时,JMS监听器自动恢复正常。 - 偶发异常:连续数月无错误,突然抛出1-2条错误后自行恢复。
- 已尝试更换IBM MQ及依赖版本、从javax迁移到jakarta,问题仍存在。
依赖版本信息
<java.version>17</java.version> <spring.boot.version>3.1.3</spring.boot.version> <springframework.version>6.0.11</springframework.version> <jakarta.annotation.version>2.1.1</jakarta.annotation.version> <jakarta.xml.version>4.0.0</jakarta.xml.version> <jakarta.xml.soap.version>3.0.0</jakarta.xml.soap.version> <jakarta.jws.version>3.0.0</jakarta.jws.version> <jakarta.jms.version>3.1.0</jakarta.jms.version> <jakarta.servlet.version>6.0.0</jakarta.servlet.version> <jakarta.mail.version>1.1.0</jakarta.mail.version> <ibm.mq.jakarta.version>9.3.3.0</ibm.mq.jakarta.version> <tomcat.annotations.version>10.1.12</tomcat.annotations.version> <tomcat.version>11.0.0-M4</tomcat.version>
完整错误栈
Message : com.ibm.msg.client.jakarta.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred. An error has occurred with the IBM MQ JMS connection. Use the linked exception to determine the cause of this error. Class : class com.ibm.msg.client.jakarta.jms.DetailedJMSException Stack : com.ibm.msg.client.jakarta.wmq.common.internal.Reason.reasonToException(Reason.java:595) : com.ibm.msg.client.jakarta.wmq.common.internal.Reason.createException(Reason.java:215) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.disconnect(WMQSession.java:829) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.close(WMQSession.java:783) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:632) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:355) : com.ibm.mq.jakarta.jms.MQSession.close(MQSession.java:280) : org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:109) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1289) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1135) : java.lang.Thread.run(null:-1) Caused by [1] --> Message : com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN'). Class : class com.ibm.mq.MQException Stack : com.ibm.msg.client.jakarta.wmq.common.internal.Reason.createException(Reason.java:203) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.disconnect(WMQSession.java:829) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.close(WMQSession.java:783) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:632) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:355) : com.ibm.mq.jakarta.jms.MQSession.close(MQSession.java:280) : org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:109) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1289) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1135) : java.lang.Thread.run(null:-1) Caused by [2] --> Message : com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9206: Error sending data to host '{HOST} {HOST}'. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2009],3={HOST} {HOST}),4=TCP,5=RemoteTCPConnection.send(byte [ ],int,int,int,int)] Class : class com.ibm.mq.jmqi.JmqiException Stack : com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.send(RemoteTCPConnection.java:1830) : com.ibm.mq.jmqi.remote.impl.RemoteConnection.wrapSend(RemoteConnection.java:3027) : com.ibm.mq.jmqi.remote.impl.RemoteConnection.sendTSH(RemoteConnection.java:2793) : com.ibm.mq.jmqi.remote.impl.RemoteSession.sendTSH(RemoteSession.java:793) : com.ibm.mq.jmqi.remote.impl.RemoteSession.sendTSH(RemoteSession.java:709) : com.ibm.mq.jmqi.remote.api.RemoteFAP.MQDISC(RemoteFAP.java:1678) : com.ibm.mq.jmqi.remote.api.RemoteFAP.MQDISC(RemoteFAP.java:1613) : com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.MQDISC(InterceptedJmqiImpl.java:410) : com.ibm.mq.ese.jmqi.ESEJMQI.MQDISC(ESEJMQI.java:396) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.disconnect(WMQSession.java:817) : com.ibm.msg.client.jakarta.wmq.internal.WMQSession.close(WMQSession.java:783) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:632) : com.ibm.msg.client.jakarta.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:355) : com.ibm.mq.jakarta.jms.MQSession.close(MQSession.java:280) : org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:109) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1289) : org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1135) : java.lang.Thread.run(null:-1) Caused by [3] --> Message : com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009 Class : class com.ibm.mq.jmqi.JmqiException Stack : com.ibm.mq.jmqi.remote.impl.RemoteConnection.asyncConnectionBroken(RemoteConnection.java:3891) : com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.run(RemoteRcvThread.java:644) : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:319) : com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99) : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:343) : com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312) : com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1240) Caused by [4] --> Message : com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208: Error on receive from host '{HOST} {HOST})'. [1=-1,2=ffffffff,3=k{HOST} {HOST},4=TCP] Class : class com.ibm.mq.jmqi.JmqiException Stack : com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveBuffer(RemoteRcvThread.java:858) : com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveOneTSH(RemoteRcvThread.java:798) : com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.run(RemoteRcvThread.java:161) : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:319) : com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99) : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:343) : com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312) : com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1240) '. EXPLANATION: An attempt was made to deliver an exception to the connections exception listener but a listener has not been registered. ACTION: An exception listener must be registered with the connection to receive its exceptions.
解决建议
1. 注册IBM MQ连接异常监听器
错误栈明确提示未注册连接异常监听器,需为MQ连接添加监听器,及时捕获并处理连接断开事件,避免异常扩散。示例代码:
@Bean public MQConnectionFactory mqConnectionFactory() { MQConnectionFactory factory = new MQConnectionFactory(); // 配置主机、端口、队列管理器等基础参数 factory.setExceptionListener(new ExceptionListener() { @Override public void onException(JMSException exception) { // 记录详细日志,触发自定义重连逻辑 log.error("MQ连接异常: {}", exception.getMessage(), exception); // 可结合Spring重试机制触发连接重建 } }); return factory; }
2. 调整Spring JMS容器的重连与恢复策略
针对DefaultMessageListenerContainer配置以下参数增强容错性:
setRecoveryInterval(5000):设置连接恢复间隔(单位毫秒)setMaxMessagesPerTask(1):避免长时间无消息时连接闲置setIdleTaskExecutionLimit(10):限制闲置任务执行次数,触发连接检查setCacheLevel(CACHE_CONNECTION):缓存连接,减少重复创建开销
示例配置:
@Bean public DefaultMessageListenerContainer jmsListenerContainer(ConnectionFactory connectionFactory, MessageListener messageListener) { DefaultMessageListenerContainer container = new DefaultMessageListenerContainer(); container.setConnectionFactory(connectionFactory); container.setDestinationName("YOUR_TOPIC_NAME"); container.setMessageListener(messageListener); container.setRecoveryInterval(5000); container.setMaxMessagesPerTask(1); container.setIdleTaskExecutionLimit(10); container.setCacheLevel(DefaultMessageListenerContainer.CACHE_CONNECTION); return container; }
3. 配置IBM MQ的心跳机制
启用MQ客户端心跳,防止网络中间件(防火墙、负载均衡)因长时间无流量断开连接:
- 设置客户端重连选项:
setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT) - 配置心跳间隔:
setHeartbeatInterval(30)(单位秒,建议30-60秒)
示例代码:
factory.setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT); factory.setHeartbeatInterval(30);
4. 检查网络层面的超时设置
- 确认防火墙、负载均衡器的超时时间,确保大于MQ心跳间隔,避免无消息时被主动断开
- 调整TCP层面的保活参数,可通过JVM参数配置:
-Dsun.net.client.defaultReadTimeout=300000 -Dsun.net.client.defaultConnectTimeout=300000
5. 验证MQ主题的订阅配置
- 确认订阅为持久化订阅,避免无消息时订阅被MQ服务器自动清理
- 检查MQ服务器的
SUBEXP(订阅过期时间)参数,设置足够大的阈值,防止闲置订阅被回收
6. 启用MQ客户端详细日志
开启IBM MQ客户端DEBUG日志,捕获连接断开前的细节,定位根本原因:
在日志配置文件(如logback.xml)中添加:
<logger name="com.ibm.msg.client" level="DEBUG"/> <logger name="com.ibm.mq" level="DEBUG"/>
内容的提问来源于stack exchange,提问作者ahmetyavuzoruc
相关产品推荐
相关产品推荐

