如何用JGroups发现配置WildFly连接远程ActiveMQ Artemis集群

问题 #

我们正尝试将WildFly实例连接至远程ActiveMQ Artemis集群。当前已手动配置指向集群各节点的remote-connector，但希望通过JGroups实现集群节点自动发现，无需逐个添加节点条目。

目前基于JNDI查找的配置可正常工作，但为pooled-connection-factory配置jgroups-discovery-group的方案无法生效，需要指导正确配置方式。

当前可用的JNDI配置 #

<subsystem xmlns="urn:jboss:domain:naming:2.0">
  <bindings>
    <external-context name="java:global/remoteContext" module="org.apache.activemq.artemis" class="javax.naming.InitialContext">
      <environment>
        <property name="java.naming.factory.initial" value="org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory"/>
        <property name="java.naming.provider.url" value="${env.ACTIVEMQ_URL,application.activemq.url}"/>
        <property name="connectionFactory.ConnectionFactory" value="${env.ACTIVEMQ_URL,application.activemq.url}"/>
      </environment>
    </external-context>
  </bindings>
  <remote-naming/>
</subsystem>

Artemis集群的jgroups-ping.xml配置 #

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd"
>
  <TCP bind_addr="${jgroups.bind_addr:site_local}"
       bind_port="${jgroups.bind_port:7800}"
       external_addr="${jgroups.external_addr}"
       external_port="${jgroups.external_port}"
       thread_pool.min_threads="0"
       thread_pool.max_threads="200"
       thread_pool.keep_alive_time="30000"/>
  <RED/>

  <dns.DNS_PING
      dns_query="message-broker-ha-ping-svc"
      dns_record_type="${DNS_RECORD_TYPE:A}"/>

  <MERGE3 min_interval="10000"
          max_interval="30000"/>
  <FD_SOCK2/>
  <FD_ALL3 timeout="40000" interval="5000"/>
  <VERIFY_SUSPECT2 timeout="1500"/>
  <pbcast.NAKACK2 use_mcast_xmit="false"/>
  <UNICAST3/>
  <pbcast.STABLE desired_avg_gossip="50000"
                 max_bytes="4M"/>
  <pbcast.GMS print_local_addr="true" join_timeout="2000"/>
  <UFC max_credits="2M"
       min_threshold="0.4"/>
  <MFC max_credits="2M"
       min_threshold="0.4"/>
  <FRAG2 frag_size="60K"/>
</config>

无效的pooled-connection-factory配置 #

<subsystem xmlns="urn:jboss:domain:messaging-activemq:13.0">
  <server name="default">
    <security elytron-domain="ApplicationDomain"/>
    <http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
    <jgroups-discovery-group name="remote-broker" refresh-timeout="10000" jgroups-channel="activemq_broadcast_channel"/>
    <pooled-connection-factory name="remote-broker" entries="java:/RemoteJmsXA java:jboss/RemoteJMSXAConnectionFactory java:jboss/DefaultJMSConnectionFactory" discovery-group="remote-broker" thread-pool-max-size="80" transaction="xa" user="${env.ACTIVEMQ_USER,application.activemq.user}" password="${env.ACTIVEMQ_PASSWORD,application.activemq.password}" max-pool-size="80" enlistment-trace="true" statistics-enabled="true"/>
    <pooled-connection-factory name="remote-broker-non-xa" entries="java:/RemoteJms java:jboss/RemoteJMSConnectionFactory" discovery-group="remote-broker" thread-pool-max-size="80" user="${env.ACTIVEMQ_USER,application.activemq.user}" password="${env.ACTIVEMQ_PASSWORD,application.activemq.password}" max-pool-size="80" enlistment-trace="true" statistics-enabled="true"/>
  </server>
</subsystem>

解决方案 #

• 对齐JGroups通道与堆栈配置
  WildFly的jgroups-discovery-group指定的通道必须和Artemis集群使用的通道名称一致（默认是activemq_broadcast_channel）。同时需要在WildFly的jgroups子系统中定义与Artemis兼容的堆栈，确保PING协议匹配：

  <subsystem xmlns="urn:jboss:domain:jgroups:11.0">
    <channels default="activemq_broadcast_channel">
      <channel name="activemq_broadcast_channel" stack="artemis-dns-ping"/>
    </channels>
    <stacks>
      <stack name="artemis-dns-ping">
        <transport type="TCP" socket-binding="jgroups-tcp"/>
        <protocol type="RED"/>
        <protocol type="dns.DNS_PING">
          <property name="dns_query">message-broker-ha-ping-svc</property>
          <property name="dns_record_type">A</property>
        </protocol>
        <protocol type="MERGE3"/>
        <protocol type="FD_SOCK2" socket-binding="jgroups-tcp-fd"/>
        <protocol type="FD_ALL3"/>
        <protocol type="VERIFY_SUSPECT2"/>
        <protocol type="pbcast.NAKACK2"/>
        <protocol type="UNICAST3"/>
        <protocol type="pbcast.STABLE"/>
        <protocol type="pbcast.GMS"/>
        <protocol type="UFC"/>
        <protocol type="MFC"/>
        <protocol type="FRAG2"/>
      </stack>
    </stacks>
  </subsystem>
  

  对应添加socket绑定：

  <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
    <!-- 保留原有绑定，新增以下内容 -->
    <socket-binding name="jgroups-tcp" port="7800"/>
    <socket-binding name="jgroups-tcp-fd" port="57600"/>
    <socket-binding name="artemis-remote" port="61616"/>
  </socket-binding-group>
  

• 修正pooled-connection-factory配置
  需明确指定Artemis集群名称、关联匹配的远程连接器，确保发现组能正确识别节点：

  <subsystem xmlns="urn:jboss:domain:messaging-activemq:13.0">
    <server name="default">
      <security elytron-domain="ApplicationDomain"/>
      <!-- 添加匹配Artemis端口的远程连接器 -->
      <remote-connector name="artemis-remote" socket-binding="artemis-remote"/>
      <!-- 关联自定义JGroups堆栈 -->
      <jgroups-discovery-group name="remote-broker" refresh-timeout="10000" jgroups-channel="activemq_broadcast_channel" stack="artemis-dns-ping"/>
      <!-- 配置连接工厂，指定集群名称与连接器 -->
      <pooled-connection-factory name="remote-broker" entries="java:/RemoteJmsXA java:jboss/RemoteJMSXAConnectionFactory java:jboss/DefaultJMSConnectionFactory" discovery-group="remote-broker" cluster="my-artemis-cluster" connector-name="artemis-remote" thread-pool-max-size="80" transaction="xa" user="${env.ACTIVEMQ_USER,application.activemq.user}" password="${env.ACTIVEMQ_PASSWORD,application.activemq.password}" max-pool-size="80" enlistment-trace="true" statistics-enabled="true"/>
      <pooled-connection-factory name="remote-broker-non-xa" entries="java:/RemoteJms java:jboss/RemoteJMSConnectionFactory" discovery-group="remote-broker" cluster="my-artemis-cluster" connector-name="artemis-remote" thread-pool-max-size="80" user="${env.ACTIVEMQ_USER,application.activemq.user}" password="${env.ACTIVEMQ_PASSWORD,application.activemq.password}" max-pool-size="80" enlistment-trace="true" statistics-enabled="true"/>
    </server>
  </subsystem>
  

  注意：cluster属性值需与Artemis集群配置中的cluster-name一致。

• 验证连通性
  启动WildFly后，检查日志中JGroups通道加入记录，或通过CLI命令查看连接工厂运行状态：

  /subsystem=messaging-activemq/server=default/pooled-connection-factory=remote-broker:read-resource(include-runtime=true)
  

  确认输出中包含发现的Artemis节点列表。

内容的提问来源于stack exchange，提问作者Moritz Schmitz v. Hülst