Docker部署Apache NiFi卡在FlowFile Repository循环求助

在两台Ubuntu服务器（18.04搭配Docker 24.0.2、24.04搭配Docker 27.4.0）上部署Apache NiFi时，进程持续输出FlowFile Repository检查点日志，无法完成启动。尝试过多个NiFi版本及多种docker-compose配置，最简配置如下：

循环日志内容 #

2024-12-19 09:58:15,319 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:58:15,319 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-12-19 09:58:35,319 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:58:35,319 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-12-19 09:58:55,320 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:58:55,320 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-12-19 09:59:10,333 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive
2024-12-19 09:59:10,334 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 141.27 GB, bytes free = 278.44 GB, capacity = 419.71 GB
2024-12-19 09:59:15,320 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:59:15,320 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-12-19 09:59:35,321 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:59:35,321 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-12-19 09:59:55,321 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-12-19 09:59:55,321 INFO [Checkpoint FlowFile Repository] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
tail: '/opt/nifi/nifi-current/logs/nifi-app.log' has been replaced;  following new file
2024-12-19 10:00:10,361 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive
2024-12-19 10:00:10,363 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 141.27 GB, bytes free = 278.44 GB, capacity = 419.71 GB
...

最简docker-compose配置 #

services:
  nifi_test:
    image: apache/nifi:latest
    container_name: nifi_test
    ports:
      - "8089:8080/tcp" # HTTP interface
      - "8449:8443/tcp" # HTTPS interface
    restart: unless-stopped

解决方案 #

• 指定稳定NiFi版本：避免使用latest标签，选择成熟稳定版本，例如apache/nifi:2.0.0-M3或官方推荐的2.x正式版
• 完善docker-compose配置：添加必要环境变量与持久化卷，确保NiFi权限与存储正常：

  services:
    nifi_test:
      image: apache/nifi:2.0.0-M3
      container_name: nifi_test
      ports:
        - "8089:8080/tcp"
        - "8449:8443/tcp"
      environment:
        - NIFI_WEB_HTTP_PORT=8080
        - NIFI_CLUSTER_IS_NODE=false
        - NIFI_SENSITIVE_PROPS_KEY=your_custom_secret_key # 替换为自定义密钥
      volumes:
        - nifi_data:/opt/nifi/nifi-current/data
        - nifi_logs:/opt/nifi/nifi-current/logs
      restart: unless-stopped
  volumes:
    nifi_data:
    nifi_logs:
  

• 分配足够内存：NiFi启动需要充足内存资源，添加内存限制配置：

  deploy:
    resources:
      limits:
        memory: 4G
      reservations:
        memory: 2G
  

• 清理残留数据：彻底删除旧容器与数据卷，避免遗留问题：

  docker-compose down -v
  docker system prune -f
  

• 调整主机文件描述符限制：NiFi需要较高的文件描述符上限，临时调整执行：

  ulimit -n 65535
  

  永久配置需编辑/etc/security/limits.conf：

  * soft nofile 65535
  * hard nofile 65535
  

内容的提问来源于stack exchange，提问作者Jean Vieille