gRPC微服务中Hazelcast集群成员无法相互连接问题求助

问题概述 #

两个通过gRPC通信的微服务应用启用Hazelcast缓存时，gRPC服务端的Hazelcast节点无法连接作为gRPC客户端的Hazelcast节点，报错Connection refused: no further information to address /127.0.0.1:9702。

应用配置文件 #

gRPC服务端应用的hazelcast.yml #

hazelcast:
    cluster-name: banks_data
    network:
      port:
        auto-increment: true
        port-count: 20
        port: 9701
      join:
        multicast:
          enabled: false
        tcp-ip:
          enabled: true
          connection-timeout-seconds: 60000
          member-list:
            - 127.0.0.1:9701
            - 127.0.0.1:9702

gRPC客户端应用的hazelcast.yml #

hazelcast:
  cluster-name: banks_data
  network:
    public-address: 127.0.0.1:9702
    port:
      port: 9702
    join:
      multicast:
        enabled: false
      tcp-ip:
        enabled: true

关键错误日志 #

2023-05-29 17:04:08.417  INFO 24828 --- [cached.thread-3] c.h.internal.nio.tcp.TcpIpConnector      : [127.0.0.1]:9701 [banks_data] [4.0.2] Connecting to /127.0.0.1:9702, timeout: 10000, bind-any: true
2023-05-29 17:04:08.418  INFO 24828 --- [cached.thread-3] c.h.internal.nio.tcp.TcpIpConnector      : [127.0.0.1]:9701 [banks_data] [4.0.2] Could not connect to: /127.0.0.1:9702. Reason: SocketException[Connection refused: no further information to address /127.0.0.1:9702]
2023-05-29 17:04:08.419  INFO 24828 --- [cached.thread-3] c.h.internal.cluster.impl.TcpIpJoiner    : [127.0.0.1]:9701 [banks_data] [4.0.2] [127.0.0.1]:9702 is added to the blacklist.
2023-05-29 17:04:09.415  INFO 24828 --- [           main] c.h.internal.cluster.ClusterService      : [127.0.0.1]:9701 [banks_data] [4.0.2] 
Members {size:1, ver:1} [
    Member [127.0.0.1]:9701 - 88660304-9ec4-461a-a580-8a4d52cad7d0 this
]

解决方案 #

1. 修正客户端Hazelcast配置 #

客户端的TCP-IP连接配置缺少member-list，导致Hazelcast无法识别集群中的其他节点。修改客户端配置，添加成员列表并关闭端口自动递增（因已指定固定端口）：

hazelcast:
  cluster-name: banks_data
  network:
    public-address: 127.0.0.1:9702
    port:
      port: 9702
      auto-increment: false
    join:
      multicast:
        enabled: false
      tcp-ip:
        enabled: true
        member-list:
          - 127.0.0.1:9701
          - 127.0.0.1:9702

2. 调整应用启动顺序 #

服务端启动时会立即尝试连接客户端的9702端口，如果客户端尚未完成启动绑定端口，就会出现连接拒绝。先启动客户端应用，待其Hazelcast节点完全启动后，再启动服务端应用。

3. 验证端口可用性 #

确保9702端口未被其他进程占用：

• Windows系统执行：netstat -ano | findstr :9702
• Linux/Mac系统执行：lsof -i :9702
  若端口被占用，需终止占用进程或修改客户端的Hazelcast端口。

4. 确认连接超时设置 #

服务端配置的connection-timeout-seconds: 60000（60秒）足够覆盖常规应用启动时间，若客户端启动较慢，可保持该配置，确保客户端在超时前完成Hazelcast节点初始化。

内容的提问来源于stack exchange，提问作者Sanjay Naik