使用Docker Compose搭建跨主机多节点Elasticsearch集群：主机配置与部署方式咨询

I'm new to Docker and want to set up an Elasticsearch cluster using Docker. I have three machines on the same LAN with Docker daemon running, their IPs are ip1, ip2, ip3, and I plan to build a cluster with these three machines. The Elasticsearch official website provides the following Docker Compose configuration file:

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic
volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local
networks:
  elastic:
    driver: bridge

I have the following questions:

1. How to configure these three machines in this Docker Compose file?
2. Should I fill in the full URL, IP address, or other content in the node.name field?
3. Do I need to execute the docker-compose up command on all three machines?

Great question! The default Compose file from Elastic is built for a single-host cluster (all three nodes running on one machine). To spread your cluster across three separate hosts, you'll need to tweak the config for each machine and understand a few key Elasticsearch + Docker concepts. Let's break this down step by step:

1. What to put in node.name #

The node.name is just a logical identifier for your Elasticsearch node—you don't need to use IP addresses or URLs here. Feel free to keep using es01, es02, es03 like the original config, just make sure each node has a unique name across the cluster. This name is used for internal cluster identification, not network routing.

2. Modify the Compose file for each host #

Since each machine will run exactly one Elasticsearch node, you'll create a customized Compose file for ip1, ip2, and ip3 respectively. Here are the key changes needed:

Cross-host communication adjustments:

• Expose the cluster port: Elasticsearch uses port 9300 for inter-node cluster traffic. You need to map this port on each host so nodes can reach each other over the LAN.
• Update discovery.seed_hosts: Replace container names (which only work on the same Docker network/host) with the actual LAN IPs + 9300 port of the other two hosts. This tells each node where to find the rest of the cluster.
• Set network.host=0.0.0.0: This allows the Elasticsearch node to listen on all network interfaces inside the container, so external nodes can connect to it.

Example Compose file for ip1:

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=ip2:9300,ip3:9300  # Use LAN IPs of the other two hosts
      - cluster.initial_master_nodes=es01,es02,es03  # Keep all node names here
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - network.host=0.0.0.0  # Allow external connections to the node
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200  # HTTP API port
      - 9300:9300  # Cluster communication port (critical for cross-host)
    networks:
      - elastic
volumes:
  data01:
    driver: local
networks:
  elastic:
    driver: bridge

Compose file for ip2:

Swap the node name to es02, update discovery.seed_hosts to ip1:9300,ip3:9300, and use data02 as the volume name.

Compose file for ip3:

Swap node name to es03, set discovery.seed_hosts to ip1:9300,ip2:9300, and use data03 as the volume name.

3. Do I need to run docker-compose up on all three machines? #

Absolutely! Each host will run its own Elasticsearch node, so you need to:

1. Copy the tailored Compose file to each respective machine.
2. On each machine, run docker-compose up -d (the -d flag runs containers in the background, so you don't have to keep the terminal open).

Critical Pre-Requisites #

Before starting the cluster, make sure:

• The 9200 and 9300 ports are open on all three hosts (check firewall rules to allow traffic between the LAN IPs).
• You set the kernel parameter vm.max_map_count to at least 262144 on each host (required by Elasticsearch to handle large memory mappings):

  sudo sysctl -w vm.max_map_count=262144
  

  To make this permanent, add vm.max_map_count=262144 to /etc/sysctl.conf and run sudo sysctl -p.

Once all nodes are up, you can verify the cluster health by sending a request to any host's 9200 port:

curl http://ip1:9200/_cluster/health?pretty

You should see all three nodes listed in the response if everything is working correctly.

内容的提问来源于stack exchange，提问作者nadavgam