Docker Compose部署ELK集群数据不持久化问题求助

Alright, let's figure out why your ELK cluster is losing data on restart—there are a few key issues in your Docker Compose configuration that are causing this:

1. Elasticsearch Data Volume Mount Path Is Incorrect #

The official Elasticsearch 6.7.0 image stores its data in /usr/share/elasticsearch/data by default, but your config mounts the volume to /root/elasticKibana/data inside the container. This means Elasticsearch isn't actually writing data to your bound host directory—it's using ephemeral container storage that gets wiped when the container restarts.

2. Kibana Has No Persistence Configured #

Kibana stores dashboards, index patterns, and all custom user configurations in /usr/share/kibana/data. Your current setup doesn't mount this directory to the host, so every time Kibana restarts, it resets to a fresh state.

3. Potential Permission Issues #

Even if you fix the mount paths, the host directories might not have the correct permissions for the Elasticsearch/Kibana container users (both run as UID 1000 in 6.x versions). This can prevent the services from writing data to the bound volumes.

Fixed Docker Compose Configuration #

Here's the corrected config that addresses all these issues:

version: '3.3'
services:
  logstash:
    container_name: logstash
    image: docker.elastic.co/logstash/logstash:6.7.0
    command: bash -c 'bin/logstash -e "input { http { port => 5044 } } filter { split {} csv { separator => \",\" columns => [\"Job\", \"BuildId\", \"Start\", \"Start_date\", \"Start_time\", \"Stop_time\", \"Stop_date\", \"Stop_time\", \"Execution_time\", \"Queue_time\", \"Executor\", \"Result\", \"Parent\", \"ChangeId\", \"Repo\", \"User\"] convert => { \"Start_time\" => \"date\" \"Stop_time\" => \"date\" } } mutate { convert => { \"Execution_time\" => \"float\" \"Queue_time\" => \"float\"}} date { match => [ \"Start time\", \"yyyy-MM-dd HH:mm:ss\" ] } } output { elasticsearch { hosts => [\"elasticsearch:9200\"] index => \"job-executions\" } }"'
    networks:
      - elastic-net
    ports:
      - 5044:5044

  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # Adjust based on your host's memory
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data # Corrected to ES's default data path
    networks:
      - elastic-net
    ports:
      - "9200:9200"

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:6.7.0
    volumes:
      - kibanadata:/usr/share/kibana/data # Added Kibana data persistence
    networks:
      - elastic-net
    ports:
      - "5601:5601" # Expose Kibana port for access
    depends_on:
      - elasticsearch # Ensure Kibana starts after Elasticsearch

networks:
  elastic-net:
    driver: bridge

volumes:
  esdata01:
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/root/elasticKibana/data'
  kibanadata: # New volume for Kibana persistence
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/root/elasticKibana/kibana_data'

Post-Configuration Steps #

1. Clean up old containers and volumes:

   docker-compose down -v
   

2. Create the Kibana host directory:

   mkdir -p /root/elasticKibana/kibana_data
   

3. Fix directory permissions to match the container users:

   chown -R 1000:1000 /root/elasticKibana/data
   chown -R 1000:1000 /root/elasticKibana/kibana_data
   

4. Start the cluster:

   docker-compose up -d
   

After these changes, your Elasticsearch data and Kibana configurations will persist across container restarts.

内容的提问来源于stack exchange，提问作者RaistlinMolina