GCP环境Docker Swarm部署Selenium Grid节点数量波动故障

问题详情 #

同一Docker Compose配置的Selenium Grid，在Azure虚拟机的Docker Swarm环境可正常运行，但在GCP环境中出现以下异常：

• 部署时所有节点能正常启动，但测试运行后节点数量持续减少，最终导致测试失败
• 测试报错（会话丢失）：

01:17:29.140 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 438df7f8fbf108115a19cdee31d96494
Build info: version: '4.2.2', revision: '683ccb65d6'
System info: host: '8fb32b33e907', ip: '10.0.1.8', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1031-gcp', java.version: '11.0.15'
Driver info: driver.version: unknown
01:17:29.140 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 438df7f8fbf108115a19cdee31d96494
Build info: version: '4.2.2', revision: '683ccb65d6'
System info: host: '8fb32b33e907', ip: '10.0.1.8', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1031-gcp', java.version: '11.0.15'
Driver info: driver.version: unknown
        at org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)
        at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)
        at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
        at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

• 节点日志报错（主机名解析失败）：

java.net.UnknownHostException: selenium-hub: Name or service not known
        at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
        at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
        at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519)
        at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
        at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
        at java.base/java.net.InetAddress.getByName(InetAddress.java:1252)
        at org.openqa.selenium.events.zeromq.UnboundZmqEventBus.isSubAddressIPv6(UnboundZmqEventBus.java:170)
        at org.openqa.selenium.events.zeromq.UnboundZmqEventBus.lambda$new$6(UnboundZmqEventBus.java:125)
        at dev.failsafe.Functions.lambda$toCtxSupplier$9(Functions.java:228)
        at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
        at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
        at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:193)
        at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
        at dev.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:220)
        at org.openqa.selenium.events.zeromq.UnboundZmqEventBus.<init>(UnboundZmqEventBus.java:122)
        at org.openqa.selenium.events.zeromq.ZeroMqEventBus.create(ZeroMqEventBus.java:53)
        at org.openqa.selenium.events.zeromq.ZeroMqEventBus.create(ZeroMqEventBus.java:79)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.openqa.selenium.grid.config.ClassCreation.callCreateMethod(ClassCreation.java:50)
        at org.openqa.selenium.grid.config.MemoizedConfig.lambda$getClass$4(MemoizedConfig.java:100)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
        at org.openqa.selenium.grid.config.MemoizedConfig.getClass(MemoizedConfig.java:95)
        at org.openqa.selenium.grid.server.EventBusOptions.createBus(EventBusOptions.java:51)
        at org.openqa.selenium.grid.server.EventBusOptions.getEventBus(EventBusOptions.java:41)
        at org.openqa.selenium.grid.node.httpd.NodeServer.createHandlers(NodeServer.java:117)
        at org.openqa.selenium.grid.node.httpd.NodeServer.asServer(NodeServer.java:183)
        at org.openqa.selenium.grid.node.httpd.NodeServer.execute(NodeServer.java:231)
        at org.openqa.selenium.grid.TemplateGridCommand.lambda$configure$4(TemplateGridCommand.java:129)
        at org.openqa.selenium.grid.Main.launch(Main.java:83)
        at org.openqa.selenium.grid.Main.go(Main.java:57)
        at org.openqa.selenium.grid.Main.main(Main.java:42)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.openqa.selenium.grid.Bootstrap.runMain(Bootstrap.java:77)
        at org.openqa.selenium.grid.Bootstrap.main(Bootstrap.java:70)
12:03:53.760 WARN [UnboundZmqEventBus.lambda$new$3] - Connecting to tcp://selenium-hub:4442 and tcp://selenium-hub:4443 failed
12:04:03.520 WARN [UnboundZmqEventBus.lambda$new$4] - Failure #1. Retrying.

• 已尝试操作：更新Docker镜像、修复DNS解析、重启Grid；节点重启后仅短暂在线，随后数量逐渐减少
• 使用配置：Docker Compose文件基于官方Swarm示例，通过docker service scale -c docker_compose sel命令扩容节点

问题根源 #

核心原因是GCP环境中Selenium节点无法解析selenium-hub服务名，导致节点无法连接Hub的ZeroMQ事件总线（4442/4443端口），持续重试失败后被Swarm调度移除，进而引发测试会话丢失错误。Azure与GCP的Docker Swarm网络DNS解析策略存在差异，GCP默认配置对Swarm服务名的解析支持不足。

解决步骤 #

1. 强制指定Overlay网络的DNS别名 #

在Docker Compose文件中为selenium-hub服务添加明确的网络别名，确保节点能稳定解析：

services:
  selenium-hub:
    image: selenium/hub:4.2.2
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    networks:
      selenium-grid:
        aliases:
          - selenium-hub
    deploy:
      replicas: 1
      placement:
        constraints: [node.role == manager]

  chrome-node:
    image: selenium/node-chrome:4.2.2
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    networks:
      - selenium-grid
    deploy:
      replicas: 5

networks:
  selenium-grid:
    driver: overlay
    attachable: true

2. 配置Docker Daemon的DNS解析 #

在GCP虚拟机上修改Docker daemon配置，添加内置DNS服务器：

• 编辑/etc/docker/daemon.json（不存在则创建）：

{
  "dns": ["127.0.0.11", "8.8.8.8"]
}

• 重启Docker daemon：

systemctl restart docker

127.0.0.11是Docker内置DNS服务器，负责解析Swarm服务名；8.8.8.8作为备用公共DNS。

3. 检查GCP防火墙规则 #

确保VPC防火墙允许以下通信：

• UDP端口4789（Overlay网络VXLAN通信）
• TCP端口4442、4443、4444（Selenium Grid通信）
• Swarm节点间的内部IP段通信

4. 调整Selenium节点超时配置 #

延长节点注册超时时间，避免因网络延迟被判定为失败：

chrome-node:
  environment:
    - SE_EVENT_BUS_HOST=selenium-hub
    - SE_EVENT_BUS_PUBLISH_PORT=4442
    - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    - SE_NODE_REGISTER_TIMEOUT=300000  # 5分钟注册超时
    - SE_NODE_HEARTBEAT_INTERVAL=30000  # 30秒心跳间隔

5. 验证服务名解析 #

在任意节点容器内执行解析测试，确认服务名能正常解析：

docker exec -it <chrome-node-container-id> nslookup selenium-hub

内容的提问来源于stack exchange，提问作者user22965033