You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Docker环境下Spring Boot Kafka生产者报错排查与配置优化咨询

avatar
安全阿岳
2026-6-30
问题描述

我正在尝试创建一个Spring Boot生产者应用并在Docker中运行，已经编写了包含PostgreSQL数据库与Kafka镜像的Docker Compose配置文件。启动容器后，Broker和生产者应用分别出现错误，附上我的docker-compose.yml和Dockerfile，希望得到问题排查方案和配置优化建议（刚接触Docker）。

Broker错误日志 

[2024-02-10 07:41:13,201] INFO [Controller id=1] Skipping preferred replica election for partitions due to topic deletion:  (kafka.controller.KafkaController) 2024-02-10T07:41:13.202837919Z [2024-02-10 07:41:13,201] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker broker:9092 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread) 2024-02-10T07:41:13.202843669Z java.io.IOException: Connection to broker:9092 (id: 1 rack: null) failed. 2024-02-10T07:41:13.202845419Z    at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70) 2024-02-10T07:41:13.202854544Z    at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:298) 2024-02-10T07:41:13.202856336Z    at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251) 2024-02-10T07:41:13.202857794Z     at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)

生产者应用错误日志 

2024-02-10T07:41:51.769527256Z 2024-02-10T07:41:51.768Z  INFO 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-1] Node -1 disconnected.
2024-02-10T07:41:51.769663798Z 2024-02-10T07:41:51.769Z  WARN 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
2024-02-10T07:41:52.803914007Z

现有配置文件

docker-compose.yml 

version: "2"

services:
  producer:
    image: "spring-boot-kafka-producer:latest"
    build:
      context: .
    container_name: producer
    depends_on:
      - database
      - broker
      - zookeeper
    env_file: .env
    ports:
      - "8080:8080"
    environment:
      - SPRING_DATASOURCE_URL=${SPRING_DATASOURCE_URL}
      - SPRING_DATASOURCE_USERNAME=${POSTGRESDB_USER}
      - SPRING_DATASOURCE_PASSWORD=${POSTGRESDB_ROOT_PASSWORD}
      - SPRING_JPA_HIBERNATE_DDL_AUTO=update
      - spring.kafka.producer.bootstrap-servers=broker:9092
      - spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer
      - spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
    volumes:
      - .m2:/root/.m2
    tty: true
    stdin_open: true
  database:
    image: 'postgres:13.1-alpine'
    container_name: database
    restart: unless-stopped
    env_file:
      - .env
    environment:
      - POSTGRES_USER=${POSTGRESDB_USER}
      - POSTGRES_PASSWORD=${POSTGRESDB_ROOT_PASSWORD}
      - POSTGRES_DB=${POSTGRESDB_DATABASE}
    ports:
      - "5432:5432"

  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_HOST: zookeeper
      ports:
        "2181:2181"

  broker:
    image: confluentinc/cp-enterprise-kafka:latest
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "29092:29092"
      - "29093:29093"
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://broker:9092,EXTERNAL_SAME_HOST://localhost:29092,EXTERNAL_DIFFERENT_HOST://157.245.80.232:29093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT,EXTERNAL_DIFFERENT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL_SAME_HOST://:29092,EXTERNAL_DIFFERENT_HOST://:29093
    restart: unless-stopped

Dockerfile 

FROM azul/zulu-openjdk-alpine:17-jre-latest 
LABEL description="Spring boot producer application to explore the various funtionalities spring offers for kakfa"
ARG JAR_FILE=*.jar
COPY ${JAR_FILE} application.jar

ENTRYPOINT ["java", "-jar", "application.jar"]

HEALTHCHECK --interval=5m --timeout=3s \
  CMD curl -f http://localhost/ || exit 1
问题排查与优化建议

1. 错误原因分析

Broker连接失败

核心问题是Zookeeper配置的语法错误：ports配置被错误嵌套在environment块内，导致Zookeeper无法正常启动，进而Kafka Broker无法连接Zookeeper，触发控制器连接失败的错误。

生产者连接错误

虽然docker-compose中配置了spring.kafka.producer.bootstrap-servers=broker:9092，但日志显示生产者尝试连接localhost:9092，大概率是以下两种情况：

  • Spring Boot应用的配置文件（如application.yml/properties）中硬编码了bootstrap-servers=localhost:9092，覆盖了环境变量配置
  • 环境变量未正确注入到容器中，可进入容器执行printenv | grep kafka验证

2. 配置修复方案

修复Zookeeper配置

将ports移出environment块，修正后的Zookeeper配置如下：

zookeeper:
  image: confluentinc/cp-zookeeper:latest
  environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000
    ZOOKEEPER_HOST: zookeeper
  ports:
    - "2181:2181"

确认生产者Kafka配置

  • 检查应用配置文件，删除硬编码的bootstrap-servers配置
  • 进入生产者容器执行printenv | grep kafka，确认spring.kafka.producer.bootstrap-servers=broker:9092已正确注入

优化容器启动顺序（可选）

depends_on仅保证容器启动顺序，不保证服务就绪。添加健康检查确保依赖服务就绪后再启动生产者：

  • 给Zookeeper添加健康检查：
zookeeper:
  # ... 其他配置
  healthcheck:
    test: ["CMD", "zkServer.sh", "status"]
    interval: 10s
    timeout: 5s
    retries: 5
  • 给Broker添加健康检查：
broker:
  # ... 其他配置
  healthcheck:
    test: ["CMD", "kafka-topics.sh", "--list", "--bootstrap-server", "localhost:9092"]
    interval: 10s
    timeout: 5s
    retries: 5
  • 修改生产者的depends_on：
producer:
  # ... 其他配置
  depends_on:
    database:
      condition: service_healthy
    broker:
      condition: service_healthy
    zookeeper:
      condition: service_healthy

3. Docker配置优化建议

版本与镜像选择

  • 将docker-compose版本升级到3.8或更高，支持更多特性（如健康检查依赖）
  • 替换Kafka企业版镜像为社区版confluentinc/cp-kafka:latest，减少资源占用
  • 升级PostgreSQL镜像到最新稳定版（如postgres:16-alpine），提升性能与安全性

添加资源限制

为每个容器设置资源上限，避免单个容器耗尽主机资源：

services:
  producer:
    # ... 其他配置
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M
  database:
    # ... 其他配置
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M
  zookeeper:
    # ... 其他配置
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M
  broker:
    # ... 其他配置
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 1G

数据持久化

添加数据卷避免容器销毁后数据丢失：

  • 为PostgreSQL和Kafka添加数据卷：
database:
  # ... 其他配置
  volumes:
    - postgres-data:/var/lib/postgresql/data

broker:
  # ... 其他配置
  volumes:
    - kafka-data:/var/lib/kafka/data
  • 在docker-compose.yml末尾定义卷：
volumes:
  postgres-data:
  kafka-data:

生产者容器优化

  • 移除tty: true和stdin_open: true，生产环境不需要交互模式
  • 修改健康检查端点：如果应用没有根路径接口，改为Spring Boot Actuator的健康检查端点（需添加Actuator依赖），示例：
HEALTHCHECK --interval=5m --timeout=3s \
  CMD curl -f http://localhost:8080/actuator/health || exit 1

内容的提问来源于stack exchange，提问作者Aravindraj Ganesan

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分)， 限时 9.9 元起

产品介绍相关文档
相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分)， 限时 9.9 元起

产品介绍相关文档
火山引擎 最新活动
全面开放
Doubao-Seedance-2.5全新模型升级

模型再升级,30秒超长叙事, 模态参考扩容

HOT
方舟Codingplan

模型自由，工具不限，最新支持 Deepseek-V4 系列、GLM-5.3 系列

HOT
方舟Agent plan

超全模态模型 × Harness 升级，最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列

HOT
新用户特惠专场

大模型19元起，Al应用9.9元畅享，新人首购爆款尽享优惠

最近更新时间：2026.06.30 12:42:32