Docker环境下Confluent 3.3.0版本KSQL连接Kafka失败求助

Hey there, let's work through this KSQL-CLI connection issue step by step. First, let's complete your existing Confluent stack setup (since your Schema Registry command was incomplete) and then dive into the most common reasons for connection failures.

Step 1: Complete Your Confluent Stack Setup #

Your Zookeeper and Kafka commands are on the right track, but let's finish the Schema Registry and add the required KSQL Server (KSQL CLI connects to KSQL Server, not directly to Kafka):

Zookeeper (your existing command, formatted properly):

docker run -d -it \
  --net=host \
  --name=zookeeper \
  -e ZOOKEEPER_CLIENT_PORT=32181 \
  confluentinc/cp-zookeeper:3.3.0

Kafka (your existing command):

docker run -d \
  --net=host \
  --name=kafka \
  -e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 \
  -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \
  -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
  confluentinc/cp-kafka:3.3.0

Schema Registry (complete command):

docker run -d \
  --net=host \
  --name=schema-registry \
  -e SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=localhost:32181 \
  -e SCHEMA_REGISTRY_HOST_NAME=localhost \
  -e SCHEMA_REGISTRY_LISTENERS=http://localhost:8081 \
  confluentinc/cp-schema-registry:3.3.0

KSQL Server (required for CLI to connect):

docker run -d \
  --net=host \
  --name=ksql-server \
  -e KSQL_BOOTSTRAP_SERVERS=localhost:29092 \
  -e KSQL_SCHEMA_REGISTRY_URL=http://localhost:8081 \
  -e KSQL_LISTENERS=http://localhost:8088 \
  -e KSQL_KSQL_SERVICE_ID=ksql_service_1 \
  confluentinc/cp-ksql-server:3.3.0

KSQL CLI (connect to KSQL Server):

docker run -it \
  --net=host \
  confluentinc/cp-ksql-cli:3.3.0 \
  http://localhost:8088

Step 2: Troubleshoot Connection Failures #

If you're still seeing connection issues after setting up all services, check these common culprits:

• Verify Kafka is reachable:
  Enter the Kafka container and run a test to confirm it's working:

  docker exec -it kafka /bin/bash
  # List existing topics
  kafka-topics --zookeeper localhost:32181 --list
  # Send a test message
  kafka-console-producer --broker-list localhost:29092 --topic test_topic
  

• Check Schema Registry health:
  Run this from your host machine to confirm Schema Registry is connected to Kafka:

  curl http://localhost:8081/subjects
  

  You should get an empty array ([]) or a list of existing subjects if you have any.

• Inspect KSQL Server logs:
  Look for explicit connection errors to Kafka or Schema Registry:

  docker logs ksql-server
  

  Common issues here include incorrect bootstrap server URLs or Schema Registry addresses.

• Validate network configuration:
  Even with --net=host, ensure your Docker host's localhost is resolving correctly inside containers. If you're on a macOS or Windows machine using Docker Desktop, you might need to use host.docker.internal instead of localhost for cross-container communication.

• Confirm Kafka offsets topic exists:
  KSQL relies on Kafka's __consumer_offsets topic. If it wasn't created automatically, you can create it manually (though your Kafka command sets KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 which should handle this):

  docker exec -it kafka kafka-topics --zookeeper localhost:32181 --create --topic __consumer_offsets --partitions 50 --replication-factor 1 --config cleanup.policy=compact
  

内容的提问来源于stack exchange，提问作者Arshanvit