JHipster微服务Docker部署Kafka报错Broker may not be available问题咨询

根因说明 #

该错误本质是两方面配置问题共同导致：

1. Docker中Kafka服务的监听和通告地址配置错误，导致服务未正常启动或无法被访问
2. 自定义Kafka配置类的属性绑定逻辑存在问题，导致配置的bootstrap地址未生效，最终默认连接localhost:9092

修复步骤 #

1. 修正Docker的Kafka配置 #

将kafka.yml中Kafka服务的配置替换为以下内容，解决Kafka网卡绑定和内外网访问的问题：

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.5.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    networks:
      - food_default
  kafka:
    image: confluentinc/cp-kafka:5.5.0
    ports:
      - 9092:9092
      - 9093:9093
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      # 监听容器内所有网卡，9092供容器内部服务访问，9093供公网访问
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:9093
      # 对外通告的访问地址
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://206.189.178.228:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    networks:
      - food_default

networks:
  food_default:
    external: true

修改完成后重启Kafka和Zookeeper容器，执行docker logs <Kafka容器ID>确认服务无报错、正常启动。

2. 修正项目配置 #

修正Kafka配置类属性名

原配置类中bootStrapServers属性名拼写不符合驼峰转换规则，会导致yml配置绑定失效，修改如下：

@Configuration
@ConfigurationProperties(prefix = "kafka")
public class KafkaProperties {

    // 修正属性名，对应yml中的bootstrap-servers配置
    private String bootstrapServers = "kafka:9092";

    private Map<String, String> consumer = new HashMap<>();

    private Map<String, String> producer = new HashMap<>();

    public String getBootstrapServers() {
        return bootstrapServers;
    }

    public void setBootstrapServers(String bootstrapServers) {
        this.bootstrapServers = bootstrapServers;
    }

    public Map<String, Object> getConsumerProps() {
        Map<String, Object> properties = new HashMap<>(this.consumer);
        if (!properties.containsKey("bootstrap.servers")) {
            properties.put("bootstrap.servers", this.bootstrapServers);
        }
        return properties;
    }

    public void setConsumer(Map<String, String> consumer) {
        this.consumer = consumer;
    }

    public Map<String, Object> getProducerProps() {
        Map<String, Object> properties = new HashMap<>(this.producer);
        if (!properties.containsKey("bootstrap.servers")) {
            properties.put("bootstrap.servers", this.bootstrapServers);
        }
        return properties;
    }

    public void setProducer(Map<String, String> producer) {
        this.producer = producer;
    }
}

修正application.yml配置

和Kafka同属一个Docker网络的微服务直接用容器名访问即可，无需走公网IP，修改配置如下：

kafka:
  bootstrap-servers: kafka:9092
  consumer:
    key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
    value.deserializer: org.apache.kafka.common.serialization.StringDeserializer
    group.id: store-service
    auto.offset.reset: earliest
  producer:
    key.serializer: org.apache.kafka.common.serialization.StringSerializer
    value.serializer: org.apache.kafka.common.serialization.StringSerializer

删除consumer、producer层级下多余的bootstrap-servers配置，避免冲突。

3. 验证修复 #

重新打包微服务镜像部署到服务器，查看服务日志确认Kafka连接成功，无报错即可。

内容的提问来源于stack exchange，提问作者Abdalrhman Alkraien