使用Kafka进行事件监控/微服务监控时加载CSV数据至Kafka遭遇错误的排查求助

I'm working on event/microservice monitoring with Kafka, following steps to load CSV data into Kafka. When I run this kafkacat command:

kafkacat -b kafka:29092 -t orders_spooldir_00 \
-C -o-1 -J \
-s key=s -s value=avro -r http://schema-registry:8081 | \
jq '.payload'

I hit this error from kafkacat:

% ERROR: Failed to query metadata for topic orders_spooldir_00: Local: Broker transport failure

And my Kafka Connect logs show this startup error:

kafka-connect | [2021-11-09 16:02:35,778] ERROR [source-csv-spooldir-00|worker] WorkerConnector{id=source-csv-spooldir-00} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:193)

I'm not sure if this is a Docker networking issue or a server-side Kafka/Connect problem. Could someone help me with troubleshooting directions and next steps? Thanks a lot!

Troubleshooting Directions & Next Steps #

• Verify Kafka Broker Reachability

  • From the machine/container where you're running kafkacat, test connectivity to the Kafka port with telnet kafka 29092 or nc -zv kafka 29092. If this fails, the core issue is network-related:
    • Check if all your Kafka, Connect, and schema registry containers are on the same custom Docker network (the default bridge network has limited DNS resolution capabilities).
    • If accessing Kafka from outside Docker, confirm the broker is configured to listen on an external interface (not just localhost) and that port 9092 (the typical external-facing port) is mapped correctly (29092 is usually for internal Docker communication).

• Check Kafka Broker Health

  • Pull the Kafka container logs with docker logs kafka (adjust the container name if yours is different) to look for startup failures, port binding errors, or other critical issues. Confirm the broker is actively listening on port 29092.

• Validate Topic Existence

  • Once you can reach the broker, check if the orders_spooldir_00 topic exists using Kafka's built-in tool:

    docker exec kafka kafka-topics.sh --list --bootstrap-server kafka:29092
    

  • If the topic doesn't exist, either your Spooldir connector isn't configured to auto-create topics, or you can manually create it with:

    docker exec kafka kafka-topics.sh --create --topic orders_spooldir_00 --bootstrap-server kafka:29092 --partitions 1 --replication-factor 1
    

• Dig Into Kafka Connect Configuration & Logs

  • The current Connect error is generic—look further down the Connect logs for full exception stack traces. These will usually point to root causes like:
    • Incorrect Kafka bootstrap server address in the connector config
    • Missing permissions to create topics
    • Invalid CSV source directory configuration
  • Double-check that your connector config uses the same bootstrap server (kafka:29092) as your kafkacat command.

• Confirm Schema Registry Access (For Avro)

  • While the immediate error is broker-related, once that's fixed, ensure the schema registry is reachable from your kafkacat environment:

    curl http://schema-registry:8081/subjects
    

  • If this fails, add the schema registry container to the same Docker network and verify its logs for startup issues.

内容的提问来源于stack exchange，提问作者sharon rogers