Docker Swarm跨节点容器互通异常：A/B与C节点容器无法通信求助

Hey there, let's dig into this overlay network issue you're facing. Cross-region Swarm clusters with mixed OS nodes can have tricky edge cases, but we can narrow this down with targeted checks:

1. Verify Docker Version Compatibility #

First off, your Ubuntu nodes are running Docker 18.03.0-ce while the CoreOS node uses 17.12.0-ce. Even minor version gaps can cause overlay network inconsistencies—especially around VXLAN tunnel behavior, which is critical for cross-node container communication.

• Try upgrading Docker on Node C to match the 18.03.0-ce version used on A and B. If upgrading CoreOS's Docker feels risky, at least move to the latest patch release of the 17.12.x series to rule out known bugs.
• After upgrading, restart Docker on Node C, rejoin the swarm, and retest container connectivity.

2. Adjust MTU for Cross-Region Networks #

Cross-region public networks often have smaller MTU limits than local networks, and Swarm's overlay network defaults to 1500. Mismatched MTU causes silent packet drops, which present as connection timeouts.

• Check the MTU of each node's public network interface:

  ip link show
  

• Create a new overlay network with an MTU matching the lowest value across your nodes (e.g., 1450 is a safe common value):

  docker network create --driver overlay --opt com.docker.network.driver.mtu=1450 cross-region-overlay
  

• Redeploy your containers to this new network and test communication.

3. Rule Out Pipework Interference #

Even if you didn't configure pipework manually, the pre-installed tool on Ubuntu nodes can modify iptables rules or network interfaces—conflicting with Swarm's overlay network setup.

• Stop any running pipework services:

  systemctl stop pipework  # If using systemd
  

• Disable it from starting on boot to avoid future conflicts:

  systemctl disable pipework
  

• Restart Docker on nodes A and B, then check if the overlay network starts working between all nodes.
• Compare iptables rules between Node A and Node C (use iptables -L -n -v) to spot any pipework-generated rules that might block VXLAN traffic.

4. Validate VXLAN Tunnel Connectivity #

Swarm overlay networks rely on VXLAN tunnels (UDP port 4789) between nodes—you mentioned testing 7946, but 4789 is equally critical.

• On Node C, start a tcpdump to capture VXLAN traffic:

  tcpdump -i any port 4789
  

• From a container on Node A, attempt to ping a container on Node C. If you don't see any packets in the tcpdump output, the VXLAN tunnel isn't establishing.
  • Double-check that your cloud provider allows UDP port 4789 through their firewall (some block it by default even if you opened it on the node).
• Verify the VXLAN interface on each node:

  ip link show | grep vxlan
  

  The interface (usually vxlan0) should list remote endpoints pointing to the public IPs of other swarm nodes.

5. Check CoreOS-Specific Network Restrictions #

CoreOS uses systemd-networkd for network management, which can have stricter default rules than Ubuntu's network setup.

• Inspect CoreOS's iptables rules to ensure UDP port 4789 and overlay network traffic isn't blocked:

  iptables-save
  

• Look at network configuration files in /etc/systemd/network/ to confirm there's no configuration limiting VXLAN interfaces or inter-node traffic.
• Check Docker's startup parameters on Node C to ensure inter-container communication isn't disabled:

  ps aux | grep dockerd
  

  If you see --icc=false, that's blocking container communication—remove it and restart Docker.

If none of these steps resolve the issue, share the output of docker network inspect <your-overlay-network> and docker node ls across all nodes, plus any relevant iptables snippets. That'll help narrow things down further.

内容的提问来源于stack exchange，提问作者Sebastian S