Kogito Data Index配置Kafka地址后仍连接localhost问题求助

我搭建了一个Kogito Quarkus应用，需要与多个服务交互，为此编写了包含zookeeper、kafka、infinispan、management-console、task-console、keycloak和data-index这7个服务的docker-compose文件。目前除data-index服务外，其余服务均正常运行，但data-index始终无法连接Kafka。

我的docker-compose文件内容如下：

version: '2.1'

services:

  infinispan:
    image: infinispan/server:14.0.5.Final
    container_name: infinispan
    ports:
        - "11222:11222"
    command: "-c infinispan-demo.xml"
    volumes:
      - ./infinispan/infinispan.xml:/opt/infinispan/server/conf/infinispan-demo.xml:z
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:11222/rest/v2/cache-managers/default/health/status" ]
      interval: 1s
      timeout: 1s
      retries: 50

  zookeeper:
    container_name: zookeeper
    image: strimzi/kafka:0.20.1-kafka-2.6.0
    command: [
      "sh", "-c",
      "bin/zookeeper-server-start.sh config/zookeeper.properties"
    ]
    ports:
      - "2181:2181"
    environment:
      LOG_DIR: "/tmp/logs"

  kafka:
    image: strimzi/kafka:0.20.1-kafka-2.6.0
    container_name: kafka
    command: [
      "sh", "-c",
      "bin/kafka-server-start.sh config/server.properties --override inter.broker.listener.name=$${KAFKA_INTER_BROKER_LISTENER_NAME} --override listener.security.protocol.map=$${KAFKA_LISTENER_SECURITY_PROTOCOL_MAP} --override listeners=$${KAFKA_LISTENERS} --override advertised.listeners=$${KAFKA_ADVERTISED_LISTENERS} --override zookeeper.connect=$${KAFKA_ZOOKEEPER_CONNECT}"
    ]
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 0
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://kafka:9092
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      LOG_DIR: "/tmp/logs"

  keycloak:
    container_name: keycloak
    image: quay.io/keycloak/keycloak:legacy
    ports:
      - "8480:8080"
    volumes:
      - ./keycloak/kogito-realm.json:/tmp/kogito-realm.json
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:8080/auth/realms/kogito" ]
      interval: 1s
      timeout: 1s
      retries: 50
    environment:
      KEYCLOAK_USER: admin
      KEYCLOAK_PASSWORD: admin
      KEYCLOAK_IMPORT: /tmp/kogito-realm.json

  data-index:
    container_name: data-index
    image: quay.io/kiegroup/kogito-data-index-infinispan:1.41
    ports:
      - "8180:8080"
    depends_on:
      kafka:
        condition: service_started
      infinispan:
        condition: service_healthy
    volumes:
      - ./persistence/:/home/kogito/data/protobufs/
    environment:
      QUARKUS_INFINISPAN_CLIENT_USE_AUTH: "false"
      QUARKUS_INFINISPAN_CLIENT_HOSTS: infinispan:11222
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
      QUARKUS_HTTP_CORS_ORIGINS: "/.*/"
      KOGITO_DATA_INDEX_PROPS: -Dkogito.protobuf.folder=/home/kogito/data/protobufs/

  management-console:
    container_name: management-console
    image: quay.io/kiegroup/kogito-management-console:1.41
    ports:
      - 8280:8080
    depends_on:
      data-index:
        condition: service_started
      keycloak:
        condition: service_healthy
    volumes:
      - ./svg/:/home/kogito/data/svg/
    environment:
      KOGITO_DATAINDEX_HTTP_URL: http://${DOCKER_GATEWAY_HOST:-host.docker.internal}:8180/graphql
      QUARKUS_HTTP_CORS_ORIGINS: "/.*/"
      KOGITO_MANAGEMENT_CONSOLE_PROPS: -Dkogito.consoles.keycloak.config.url=http://localhost:8480/auth -Dkogito.consoles.keycloak.config.health-check-url=http://localhost:8480/auth/realms/kogito/.well-known/openid-configuration -Dkogito.svg.folder.path=/home/kogito/data/svg

  task-console:
    container_name: task-console
    image: quay.io/kiegroup/kogito-task-console:1.41
    ports:
      - 8380:8080
    depends_on:
      data-index:
        condition: service_started
      keycloak:
        condition: service_healthy
    environment:
      KOGITO_DATAINDEX_HTTP_URL: http://${DOCKER_GATEWAY_HOST:-host.docker.internal}:8180/graphql
      QUARKUS_HTTP_CORS_ORIGINS: "/.*/"
      KOGITO_TASK_CONSOLE_PROPS: -Dkogito.consoles.keycloak.config.url=http://localhost:8480/auth -Dkogito.consoles.keycloak.config.health-check-url=http://localhost:8480/auth/realms/kogito/.well-known/openid-configuration

data-index服务的错误日志：

[org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-kogito-processinstances-events, groupId=kogito-data-index-processinstances] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

已排查情况：

• 在data-index的环境变量中配置了KAFKA_BOOTSTRAP_SERVERS: kafka:9092，但服务尝试连接localhost而非kafka容器
• 在data-index容器终端执行getent hosts kafka，返回结果为172.30.0.5 kafka，说明容器间DNS解析正常

问题根源 #

Kafka的advertised.listeners配置导致连接异常：当data-index通过kafka:9092连接到Kafka broker后，broker会返回EXTERNAL://localhost:9092作为客户端后续通信的地址，而data-index容器内部无法将localhost解析到Kafka容器，因此连接失败。

修复步骤 #

1. 修改Kafka服务的监听器配置
   更新Kafka的环境变量，调整监听器和公告地址：

   KAFKA_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://0.0.0.0:9092
   KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:29092,EXTERNAL://localhost:9092
   

   保持原有的KAFKA_LISTENER_SECURITY_PROTOCOL_MAP不变：

   KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
   

2. 调整data-index的Kafka连接地址
   将data-index的KAFKA_BOOTSTRAP_SERVERS改为使用Kafka的内部监听器地址：

   KAFKA_BOOTSTRAP_SERVERS: kafka:29092
   

   这样data-index连接后，Kafka会返回INTERNAL://kafka:29092作为通信地址，容器内部可以正常解析kafka域名。

验证方法 #

修改配置后重启所有服务，查看data-index日志是否出现成功连接Kafka的信息；也可以在data-index容器内执行kafka-topics.sh --list --bootstrap-server kafka:29092，验证是否能正常访问Kafka集群。

内容的提问来源于stack exchange，提问作者maksnurgazy