容器内Spring Boot应用无法连接Kafka问题求助

问题复现步骤 #

• 克隆项目
• 使用Maven构建listener和publisher项目
• 各项目内通过Dockerfile构建镜像，运行docker-compose.yml
• 在listener项目中查看错误日志

背景说明 #

Spring Boot应用在容器内与RabbitMQ配合运行正常，添加Kafka配置到docker-compose.yml，并在application.properties中配置同时连接RabbitMQ和Kafka的参数后，应用无法连接Kafka。

应用侧错误日志 #

org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-myId-1, groupId=myId] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.

Kafka侧错误日志 #

2024-12-16 17:38:08 [2024-12-16 09:38:08,360] INFO [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
2024-12-16 17:38:08 [2024-12-16 09:38:08,361] WARN [Controller id=1, targetBrokerId=1] Connection to node 1 (kafka/172.18.0.4:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
2024-12-16 17:38:08 [2024-12-16 09:38:08,362] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker kafka:9092 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
2024-12-16 17:38:08 java.io.IOException: Connection to kafka:9092 (id: 1 rack: null) failed.
2024-12-16 17:38:08     at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:71)
2024-12-16 17:38:08     at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:299)
2024-12-16 17:38:08     at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:252)
2024-12-16 17:38:08     at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:135)

相关配置文件 #

application.properties #

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=orderGroup
spring.kafka.consumer.auto-offset-reset=earliest

Dockerfile #

FROM openjdk:21-jdk-slim
VOLUME /tmp
COPY target/listener-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

docker-compose.yml #

version: '3'
services:
  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"
      - "15672:15672"
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
      interval: 5s
      timeout: 5s
      retries: 5

  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    healthcheck:
      test: [ "CMD", "echo", "ruok", "|", "nc", "localhost", "2181", "|", "grep", "imok" ]
      interval: 10s
      timeout: 5s
      retries: 5

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    healthcheck:
      test: [ "CMD", "kafka-broker-api-versions", "--bootstrap-server", "localhost:9092" ]
      interval: 10s
      timeout: 5s
      retries: 5

  listener:
    build: ./listener
    ports:
      - "8081:8081"
    depends_on:
      rabbitmq:
        condition: service_healthy
      kafka:
        condition: service_healthy
    restart: unless-stopped

求助说明 #

无法定位问题所在，曾尝试修改application.properties和docker-compose.yml，期望Kafka能像RabbitMQ一样正常运行，但未获得有效解决思路，恳请协助排查。

内容的提问来源于stack exchange，提问作者EthanAlef