Windows Docker环境下Fluentd日志问题：启动报错且日志无法转发

问题现象 #

macOS Docker环境中Fluentd日志采集正常，但Windows Docker环境运行同一项目时，Fluentd容器启动阶段出现Errno::ENOENT错误，提示找不到缓冲文件或目录，无法清理缓冲块，导致日志无法转发。更换Fluentd镜像后问题依旧。

错误日志 #

fluentd           | 2023-06-23 23:02:06 +0000 [error]: #0 fluent/log.rb:402:error: failed to purge buffer chunk chunk_id="5fec8dc19520e0488b378ed1aa9dcf26" error_class=Errno::ENOENT error=#<Errno::ENOENT: No such file or directory @ apply2files - /fluentd/log/nginx/buffers/buffer.q5fec8dc19520e0488b378ed1aa9dcf26.log.meta>
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer/file_chunk.rb:161:in `unlink'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer/file_chunk.rb:161:in `purge'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer.rb:601:in `block in purge_chunk'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/3.1.0/monitor.rb:202:in `synchronize'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/3.1.0/monitor.rb:202:in `mon_synchronize'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer.rb:592:in `purge_chunk'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1110:in `commit_write'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1229:in `try_flush'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1538:in `flush_thread_run'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:510:in `block (2 levels) in start'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
fluentd           | 2023-06-23 23:02:06 +0000 [error]: #0 fluent/log.rb:402:error: failed to purge buffer chunk chunk_id="5fec8dc200df70d262d50e3912a1d713" error_class=Errno::ENOENT error=#<Errno::ENOENT: No such file or directory @ apply2files - /fluentd/log/frontend/buffers/buffer.q5fec8dc200df70d262d50e3912a1d713.log.meta>
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer/file_chunk.rb:161:in `unlink'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer/file_chunk.rb:161:in `purge'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer.rb:601:in `block in purge_chunk'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/3.1.0/monitor.rb:202:in `synchronize'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/3.1.0/monitor.rb:202:in `mon_synchronize'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/buffer.rb:592:in `purge_chunk'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1110:in `commit_write'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1229:in `try_flush'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:1538:in `flush_thread_run'
fluentd           |   2023-06-23 23:02:06 +0000 [error]: #0 3.1.0/monitor.rb:202:mon_synchronize: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin/output.rb:510:in `block (2 levels) in start'

相关配置 #

Docker Compose配置 #

version: "3.8"
services:
  api:
    build: .
    depends_on:
      - db
      - db_test
    container_name: api
    volumes:
      - ./files:/files:rw
      - ./src:/app:rw
    ports:
      - 8000:8000
    env_file: .env
    command: >
      bash -c "wait_for db:5432 db_test:5432 && sleep 30 && entrypoint.sh"
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        fluentd-async-connect: "true"
        fluentd-retry-wait: 1s
        fluentd-max-retries: 200
        tag: api.api

  fluentd:
    container_name: fluentd
    image: fluentd:v1.16-1
    ports:
      - 24224:24224
      - 24224:24224/udp
    volumes:
      - ./fluent.conf:/fluentd/etc/fluent.conf:ro
      - ./src/logs:/fluentd/log:rw
    command: >
      /usr/bin/fluentd -c /fluentd/etc/fluent.conf -v
    networks:
      - default
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        fluentd-async-connect: "true"
        fluentd-retry-wait: 1s
        fluentd-max-retries: 200
        tag: api.fluentd

Fluentd配置（fluent.conf） #

<source>
  @type  forward
  @id    input1
  @label @mainstream
  port  24224
</source>

<label @mainstream>
  <match api.api>
    @type file
    path         /fluentd/log/api/api
    time_slice_format %Y-%m-%d
    time_slice_wait   1m
    time_format       %Y%m%dT%H%M%S%z
    buffer_chunk_limit 10K
    buffer_path /fluentd/log/api/buffers
    append true
    <buffer>
      path /fluentd/log/api/buffers
      chunk_limit_size 30K
      flush_mode interval
      flush_interval 5s
      flush_thread_count 2

    </buffer>
  </match>

  <match api.fluentd>
    @type file
    path         /fluentd/log/fluentd/api-fluentd
    time_slice_format %Y-%m-%d
    time_slice_wait   1m
    time_format       %Y%m%dT%H%M%S%z
    buffer_chunk_limit 10K
    buffer_path /fluentd/log/fluentd/buffers
    append true
    <buffer>
      path /fluentd/log/fluentd/buffers
      chunk_limit_size 30K
      flush_mode interval
      flush_interval 5s
      flush_thread_count 2

    </buffer>
  </match>
  
</label>

解决方案 #

1. 手动创建缓冲目录 #

Windows Docker文件系统兼容性问题导致Fluentd无法自动生成嵌套缓冲目录，在宿主机执行以下命令创建所需目录：

• Linux/macOS终端：

mkdir -p src/logs/api/buffers src/logs/fluentd/buffers src/logs/nginx/buffers src/logs/frontend/buffers

• Windows命令行：

mkdir src\logs\api\buffers
mkdir src\logs\fluentd\buffers
mkdir src\logs\nginx\buffers
mkdir src\logs\frontend\buffers

2. 修正Docker网络配置 #

API容器和Fluentd容器在同一Docker网络中，需用服务名fluentd替代localhost才能正常通信，同时避免Fluentd自身日志驱动循环依赖：
修改docker-compose.yml中API和Fluentd的logging配置：

# API容器部分
logging:
  driver: "fluentd"
  options:
    fluentd-address: fluentd:24224
    fluentd-async-connect: "true"
    fluentd-retry-wait: 1s
    fluentd-max-retries: 200
    tag: api.api

# Fluentd容器部分
logging:
  driver: "json-file"

3. 优化Fluentd缓冲配置 #

移除<match>块中重复的buffer_path配置（仅保留<buffer>内的路径），添加重试策略增强容错性：
修改后的<match api.api>示例：

<match api.api>
  @type file
  path         /fluentd/log/api/api
  time_slice_format %Y-%m-%d
  time_slice_wait   1m
  time_format       %Y%m%dT%H%M%S%z
  append true
  <buffer>
    path /fluentd/log/api/buffers
    chunk_limit_size 30K
    flush_mode interval
    flush_interval 5s
    flush_thread_count 2
    retry_type exponential_backoff
    retry_wait 1s
    retry_max_interval 30s
  </buffer>
</match>

4. 清理损坏的缓冲文件 #

删除宿主机./src/logs下的旧缓冲文件，避免残留文件导致错误：

rm -rf src/logs/*/buffers/*

内容的提问来源于stack exchange，提问作者hfc