基于Bitnami Chart在Kubernetes集群部署的Kafka无法接收Filebeat日志问题求助

Let's break down the possible issues step by step based on your setup and logs:

1. Verify the Target Log File is Accessible and Contains Data #

First, rule out the simplest causes:

• Check if /var/log/test.log actually has new log entries. Run tail -f /var/log/test.log to confirm logs are actively being written to it.
• Ensure Filebeat has read permissions for the file. If you're running Filebeat in a container, double-check that the log file path is correctly mounted into the container, and the container user has read access (you can run ls -l /var/log/test.log inside the Filebeat container to verify).

2. Enable Debug Logging for Filebeat to Get More Context #

Your current logs only show basic scan info—enable debug logging to see details about file detection, event processing, and Kafka delivery attempts:
Add this to your Filebeat config:

logging.level: debug
logging.selectors: ["*"]

Restart Filebeat and look for key logs like:

• "Successfully opened file" (confirms Filebeat can read the log)
• "Publish event" (confirms events are being sent to the pipeline)
• Kafka-specific errors (like authentication failures or connection timeouts)

3. Check Kafka Authentication and Permissions #

Bitnami Kafka often enables ACLs by default. Even though Filebeat created the topic, it might not have write permissions:

• If you set up Kafka with default Bitnami credentials, add the following to your output.kafka section (replace with your actual credentials from the Bitnami Kafka deployment):

output.kafka:
  hosts: ["ip-172-31-26-181:30092"]
  topic: "logs-topic"
  codec.json:
    pretty: false
  username: "admin"
  password: "your-kafka-admin-password"

• You can test write permissions directly using the Kafka console producer:

kafka-console-producer.sh --broker-list ip-172-31-26-181:30092 --topic logs-topic --producer.config /path/to/client.properties

If this fails, it confirms a permission or network issue with Kafka.

4. Validate Kafka Network Connectivity #

Ensure the Filebeat instance can reach the Kafka NodePort:

• Run telnet ip-172-31-26-181 30092 from the Filebeat host/container. If it fails, check firewall rules, Kubernetes NodePort configuration, or if the Kafka service is healthy.
• Alternatively, use nc -zv ip-172-31-26-181 30092 to test connectivity.

5. Check Filebeat's Registry and Offset Tracking #

Filebeat tracks which parts of files it has processed in a registry file. If it already read /var/log/test.log before, it won't reprocess old entries:

• Locate the registry file (default path: /var/lib/filebeat/registry).
• Search for entries related to /var/log/test.log—if found, delete the registry file (or the specific entry) and restart Filebeat. This will force Filebeat to re-scan the file from the beginning.

6. Temporarily Disable Processors to Rule Out Filtering #

Your processors (add_host_metadata, add_kubernetes_metadata, etc.) might be inadvertently dropping events. Try commenting them out temporarily:

# processors:
#   - add_host_metadata:
#       when.not.contains.tags: forwarded
#   - add_cloud_metadata: ~
#   - add_docker_metadata: ~
#   - add_kubernetes_metadata: ~

Restart Filebeat and check if logs start flowing into the Kafka topic. If they do, debug each processor to see which one is causing the issue.

内容的提问来源于stack exchange，提问作者lemahdois