如何判断Filebeat发送至Kafka的日志数据是否为明文或已加密

Hey there! Let's walk through how you can tell if your Filebeat-to-Kafka log traffic is unencrypted (plaintext) or encrypted. I'll cover the most straightforward and reliable methods below:

The first place to check is your Filebeat output.kafka settings. Encrypted traffic requires explicit SSL configuration here.

• If your config includes SSL-related fields with ssl.enabled: true, you're using encrypted transport:

  output.kafka:
    hosts: ["your-kafka-broker:9093"]  # 9093 is the common default SSL port for Kafka
    ssl.enabled: true
    ssl.certificate_authorities: ["/path/to/ca.crt"]  # Trusted CA for Kafka broker cert
    # Optional: client cert/key if Kafka requires mutual TLS
    # ssl.certificate: "/path/to/client.crt"
    # ssl.key: "/path/to/client.key"
  

• If your output.kafka only defines hosts pointing to port 9092 (Kafka's default plaintext port) and has no ssl.* fields, your traffic is almost certainly plaintext.

Next, check your Kafka broker's server.properties to confirm what listeners are enabled:

• A plaintext listener looks like this (unencrypted):

  listeners=PLAINTEXT://0.0.0.0:9092
  advertised.listeners=PLAINTEXT://your-kafka-broker:9092
  

• An SSL-encrypted listener will use the SSL protocol:

  listeners=SSL://0.0.0.0:9093,PLAINTEXT://0.0.0.0:9092  # Mixed listeners
  advertised.listeners=SSL://your-kafka-broker:9093
  ssl.keystore.location=/path/to/kafka.keystore.jks
  ssl.keystore.password=your-keystore-pass
  ssl.truststore.location=/path/to/kafka.truststore.jks
  ssl.truststore.password=your-truststore-pass
  ssl.enabled.protocols=TLSv1.2,TLSv1.3
  

If Filebeat is connecting to the SSL listener port (9093 by default), your traffic is encrypted.

For definitive proof, use a tool like tcpdump or Wireshark to inspect the actual traffic:

• To check plaintext traffic (port 9092):

  tcpdump -i any port 9092 -A
  

  If you can read clear, human-readable log content in the output, your traffic is unencrypted.

• To check encrypted traffic (port 9093):

  tcpdump -i any port 9093 -A
  

  Encrypted traffic will appear as garbled, unreadable text—you won't be able to parse log messages directly from the capture.

Filebeat logs will tell you if it successfully established an SSL connection to Kafka. Look for entries in your Filebeat log file (typically /var/log/filebeat/filebeat.log or a custom path you defined):

• Successful SSL connection logs will look like:

  INFO [kafka] kafka/client.go:524 Successfully established SSL connection to your-kafka-broker:9093

• If you don't see any SSL-related success messages, or logs mention "plaintext connection", your traffic is unencrypted.

Putting it all together: start with checking your configs (both Filebeat and Kafka) since that's the quickest method, then use network captures or log checks to confirm. That should give you a definitive answer!

内容的提问来源于stack exchange，提问作者user3013193