如何在Docker Swarm v1.18上搭建3节点（1主2工作节点）Elasticsearch v6集群？

Got it, let's walk through setting up a 3-node Elasticsearch v6 cluster on Docker Swarm v1.18 with 1 manager and 2 worker nodes. I’ve done this a few times, so here’s a step-by-step guide that should work smoothly:

First, make sure you have these sorted on all nodes:

• Docker CE 17.06+ (since Docker Swarm v1.18 corresponds to this version range)
• Network connectivity between all nodes (no firewalls blocking required ports)
• Sufficient memory (at least 1GB per node, more if you plan to index large datasets)

Before deploying ES, we need to tweak some system settings and set up the Swarm cluster:

1. Adjust kernel parameter for ES: Elasticsearch requires a higher virtual memory limit. Run this on every node:

   sysctl -w vm.max_map_count=262144
   

   To make this permanent, add vm.max_map_count=262144 to /etc/sysctl.conf and run sysctl -p.

2. Initialize Docker Swarm: On your chosen manager node, run:

   docker swarm init --advertise-addr <your-manager-node-ip>
   

   Copy the docker swarm join command from the output, then run it on both worker nodes to add them to the cluster.

3. Verify Swarm status: On the manager node, check that all nodes are listed:

   docker node ls
   

   You should see 1 manager marked as Leader and 2 workers.

ES nodes need to communicate with each other reliably. Create an attachable overlay network on the manager node:

docker network create --driver overlay --attachable es-cluster-net

Create a docker-compose.yml file on the manager node with the following content. I’ve included comments to explain key parts:

version: '3.3'

services:
  # Manager node (acts as both master and data node)
  es-manager:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.8.23
    deploy:
      placement:
        # Ensure this runs only on the Swarm manager node
        constraints: [node.role == manager]
      replicas: 1
    environment:
      - cluster.name=es-cluster
      - node.name=es-manager
      - node.master=true
      - node.data=true
      # List all cluster nodes for discovery
      - discovery.zen.ping.unicast.hosts=es-manager,es-worker-1,es-worker-2
      # Prevent split-brain: require at least 2 master-eligible nodes to form a cluster
      - discovery.zen.minimum_master_nodes=2
      # Lock memory to avoid swapping (critical for performance)
      - bootstrap.memory_lock=true
      # Adjust heap size based on your node's memory; 512m is safe for small nodes
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    # Persist data to a named volume
    volumes:
      - es-manager-data:/usr/share/elasticsearch/data
    networks:
      - es-cluster-net
    # Expose ports for external access
    ports:
      - "9200:9200"
      - "9300:9300"

  # First worker node (data-only)
  es-worker-1:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.8.23
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
    environment:
      - cluster.name=es-cluster
      - node.name=es-worker-1
      - node.master=false
      - node.data=true
      - discovery.zen.ping.unicast.hosts=es-manager,es-worker-1,es-worker-2
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es-worker1-data:/usr/share/elasticsearch/data
    networks:
      - es-cluster-net

  # Second worker node (data-only)
  es-worker-2:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.8.23
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
    environment:
      - cluster.name=es-cluster
      - node.name=es-worker-2
      - node.master=false
      - node.data=true
      - discovery.zen.ping.unicast.hosts=es-manager,es-worker-1,es-worker-2
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es-worker2-data:/usr/share/elasticsearch/data
    networks:
      - es-cluster-net

# Named volumes for persistent storage
volumes:
  es-manager-data:
  es-worker1-data:
  es-worker2-data:

# Use the overlay network we created earlier
networks:
  es-cluster-net:
    external: true

On the manager node, deploy the stack with:

docker stack deploy -c docker-compose.yml es-cluster

Wait a minute or two for the nodes to start and form the cluster, then run these checks:

1. Check service status:

   docker stack services es-cluster
   

   All three services should show 1/1 under REPLICAS.

2. Check cluster health:

   curl http://<manager-node-ip>:9200/_cluster/health?pretty
   

   A status of yellow is normal here (since we don’t have replica shards for our single data nodes), but green is ideal if you adjust shard settings later. red means something’s broken.

3. List cluster nodes:

   curl http://<manager-node-ip>:9200/_cat/nodes?v
   

   You should see all three nodes listed, with the manager marked as a master (* in the master column).

• Nodes can’t join the cluster: Make sure firewalls allow traffic on ports 9200, 9300 (ES), and 2377/tcp, 7946/tcp/udp, 4789/udp (Docker Swarm).
• ES fails to start: Check the service logs with docker service logs es-cluster_es-manager. Common issues include missing vm.max_map_count setting or insufficient memory.
• Cluster status is red: Verify that all data volumes have correct permissions (the ES container runs as user 1000, so the volume should be writable by that user) or check if any nodes are offline.

内容的提问来源于stack exchange，提问作者mosalaki