Let's break down why you're hitting this error and walk through actionable fixes step by step:

Root Causes #

1. Incorrect Bootstrap Server Address
   You're likely using localhost:29092 for your server variable. Inside your app container, localhost refers to the container itself—not the Kafka container. The correct address for inter-container communication is kafka:29092, which matches the PLAINTEXT://kafka:29092 entry in your KAFKA_ADVERTISED_LISTENERS config.

2. Missing Service Dependencies
   Your app service doesn't have a depends_on rule for Kafka. While depends_on only guarantees container startup order (not full service readiness), it prevents your app from launching before Kafka's container even initializes.

3. Mismatched API Version
   You're manually setting api_version=(0,10,2) but using the latest Confluent Kafka image, which runs a much newer Kafka version. This can cause compatibility issues during API negotiation.

Step-by-Step Solutions #

1. Fix the Bootstrap Server Address #

Update your KafkaAdminClient initialization to use the container-internal address:

self._kafka_admin = KafkaAdminClient(
    bootstrap_servers="kafka:29092",  # Use the Kafka service name from docker-compose
    api_version_auto_timeout_ms=120000
    # Remove the manual api_version setting to let the client auto-negotiate
)

2. Add Dependencies and Network Configuration #

Update your app service in docker-compose.yml to explicitly depend on Kafka and confirm it's in the same network:

app:
  build:
    context: ./
  ports:
    - 5000:5000
  depends_on:
    - kafka  # Ensures Kafka container starts before the app
  networks:
    - default  # Joins the default docker-compose network (same as Kafka/Zookeeper)

For better organization, you could also create a dedicated kafka-network and add Zookeeper, Kafka, and your app to it—this keeps your services grouped logically.

3. Handle Kafka Service Readiness #

depends_on doesn't wait for Kafka to finish initializing. Add retry logic to your Python code to handle this:

from kafka import KafkaAdminClient, errors
import time

def get_kafka_admin_client():
    max_retries = 10
    retry_interval = 5  # Seconds between retries
    for attempt in range(max_retries):
        try:
            return KafkaAdminClient(
                bootstrap_servers="kafka:29092",
                api_version_auto_timeout_ms=120000
            )
        except errors.NoBrokersAvailable:
            print(f"Kafka not ready yet (attempt {attempt+1}/{max_retries}), retrying...")
            time.sleep(retry_interval)
    raise RuntimeError("Failed to connect to Kafka after all retries")

self._kafka_admin = get_kafka_admin_client()

Alternatively, you can add a wait script (like wait-for-it) to your app's Dockerfile to block startup until Kafka is reachable.

4. Verify Network Connectivity #

Double-check that your app container can reach the Kafka container by running:

docker exec -it <your-app-container-name> ping kafka

If this fails, you have a network misconfiguration—make sure both services are in the same Docker network.

内容的提问来源于stack exchange，提问作者Tavis