Docker化Spring Boot应用无法通过localhost访问主机Kafka实例

我有一个Docker化的Spring Boot Web服务，需要向Kafka主题生产消息。本地Ubuntu机器上运行着带Zookeeper的Kafka，Web服务运行在同一机器的Docker容器中。已通过netstat -tlpn确认Kafka Broker在9092端口运行。

使用docker run my-image-name:latest --network="host"启动应用后，Kafka生产者无法连接到bootstrap broker。

@Component
public class KafkaService {

    private String bootstrapServerUrl;

    private String topicName;

    private KafkaProducer<String, HandleCaptureKafkaMessage> kafkaProducer;

        // 创建生产者配置
    public KafkaService(@Value("${kafka-bootstrap-server-url}") String bootstrapServerUrl, @Value("${kafka-topic-name}") String topicName) {
        this.bootstrapServerUrl = bootstrapServerUrl;
        this.topicName = topicName;
        System.out.println("SERVERURL " + bootstrapServerUrl);
        Properties properties = new Properties();
        properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServerUrl);
        properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, HandleCaptureKafkaMessageSerializer.class.getName());

        kafkaProducer = new KafkaProducer<>(properties);
    }

    public void sendMessage(HandleCaptureKafkaMessage handleCaptureKafkaMessage) {
        kafkaProducer.send(new ProducerRecord<>(topicName, null, handleCaptureKafkaMessage), null);
    }

}

2024-02-05T19:13:00.425Z  INFO 1 --- [           main] o.a.k.clients.producer.ProducerConfig    : ProducerConfig values:
        acks = -1
        auto.include.jmx.reporter = true
        batch.size = 16384
        bootstrap.servers = [localhost:9092]
        buffer.memory = 33554432
...
2024-02-05T19:13:00.498Z  INFO 1 --- [           main] o.a.k.clients.producer.KafkaProducer     : [Producer clientId=producer-1] Instantiated an idempotent producer.
2024-02-05T19:13:00.554Z  INFO 1 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version: 3.6.1
2024-02-05T19:13:00.559Z  INFO 1 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId: 5e3c2b738d253ff5
2024-02-05T19:13:00.560Z  INFO 1 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka startTimeMs: 1707160380540
2024-02-05T19:13:00.830Z  INFO 1 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Node -1 disconnected.
2024-02-05T19:13:00.832Z  WARN 1 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
2024-02-05T19:13:00.832Z  WARN 1 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

• 修正Kafka绑定与通告地址
  Kafka默认仅绑定127.0.0.1，即便用了Docker host网络，容器内的localhost指向容器自身而非宿主机。修改Kafka的server.properties配置：

  • 将listeners=PLAINTEXT://localhost:9092改为listeners=PLAINTEXT://0.0.0.0:9092，让Kafka监听所有网卡地址
  • 添加或修改advertised.listeners=PLAINTEXT://<宿主机实际IP>:9092，确保生产者能获取正确的Broker访问地址

• 替换Bootstrap地址为宿主机IP
  把应用配置中的kafka-bootstrap-server-url从localhost:9092改为宿主机的实际IP（比如192.168.1.100:9092），避免容器内localhost的歧义问题

• 验证网络连通性
  进入Docker容器内部，执行ping <宿主机IP>和telnet <宿主机IP> 9092，确认容器能正常访问宿主机的9092端口

• 检查防火墙规则
  Ubuntu的ufw防火墙可能拦截了9092端口，执行以下命令开放端口：

  sudo ufw allow 9092/tcp
  sudo ufw reload
  

内容的提问来源于stack exchange，提问作者Jesse Ford