如何在Windows环境下通过Kafka Connect将Kafka消息消费至服务器文件？

Hey there! Let's get this sorted out for you. First off, I notice you're using a Linux-style command on Windows—we need to adjust that a bit, plus make sure your sink connector configuration is set up correctly to write messages to a file.

Step 1: Fix the Windows Command Syntax #

On Windows, Kafka uses .bat scripts instead of shell scripts, and path separators are backslashes (\) instead of forward slashes (/). Your original command includes both source and sink connectors, but if you only need to consume messages and write them to a file, you can omit the source configuration entirely.

Here's the corrected Windows command to run:

bin\windows\connect-standalone.bat etc\kafka\connect-standalone.properties etc\kafka\connect-file-sink.properties

Step 2: Configure the File Sink Connector #

Before executing the command, make sure your connect-file-sink.properties is properly tuned to target your Kafka topic and desired output file. Update these key settings:

• name=kafka-file-sink (a unique identifier for your sink connector)
• connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector (the built-in file sink class from Kafka)
• topics=your_target_topic (replace this with the actual topic you want to consume)
• file=C:\\kafka-output\\consumed-messages.txt (Windows file path—use double backslashes or single forward slashes, e.g., C:/kafka-output/consumed-messages.txt)
• key.converter=org.apache.kafka.connect.storage.StringConverter
• value.converter=org.apache.kafka.connect.storage.StringConverter

If you need to write to a remote server file (like a network share), use the UNC path format:

file=\\\\server-name\\shared-folder\\output-messages.txt

Step 3: Verify & Troubleshoot #

• Double-check that connect-standalone.properties has the correct bootstrap.servers pointing to your Kafka broker(s) (e.g., localhost:9092 if running locally).
• Ensure the output directory exists—Kafka Connect won't create it automatically.
• If you hit errors, check the Kafka logs in the logs/ directory (look for files starting with connect- to debug issues).

Quick Note on Your Original Command #

Your original command loads both a file source (to read from a local file into Kafka) and a file sink (to write from Kafka to a file). If you only need the sink functionality, removing the connect-file-source.properties from the command is all you need to do.

内容的提问来源于stack exchange，提问作者JordanBlairDI