Kubernetes及独立Docker容器内非标准日志转至Elasticsearch方案咨询

Got it, let's break this down—you've got logs trapped inside Kubernetes Pods and standalone Docker containers that aren't routed to stdout/stderr, and they aren't mounted to the host filesystem. No worries, here are solid, production-ready solutions for both environments:

The go-to approach here is using a sidecar container—it's clean, non-intrusive, and keeps log collection separate from your application logic.

核心思路：共享卷 + Sidecar 采集器 #

Since containers in the same Pod share the same pod-level resources, we can use an emptyDir volume to share the app's log directory with a sidecar that handles log shipping. Here's how to set it up with Filebeat (you can swap this for Fluentd, Logstash, etc.):

Step 1: Update your Deployment YAML #

Add an emptyDir volume, mount it to your app container's log directory, and attach a Filebeat sidecar that reads from the same volume:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-business-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-business-app
  template:
    metadata:
      labels:
        app: my-business-app
    spec:
      volumes:
        # Shared volume for logs
        - name: app-log-storage
          emptyDir: {}
        # Config volume for Filebeat
        - name: filebeat-config
          configMap:
            name: filebeat-sidecar-config
      containers:
        - name: app-container
          image: your-app-image:latest
          volumeMounts:
            - name: app-log-storage
              mountPath: /app/internal/logs  # Path where your app writes logs
        - name: filebeat-sidecar
          image: docker.elastic.co/beats/filebeat:8.10.0
          volumeMounts:
            - name: app-log-storage
              mountPath: /var/log/app-logs  # Path Filebeat will read from
            - name: filebeat-config
              mountPath: /usr/share/filebeat/filebeat.yml
              subPath: filebeat.yml
          env:
            - name: ELASTIC_USERNAME
              valueFrom:
                secretKeyRef:
                  name: elastic-credentials
                  key: username
            - name: ELASTIC_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: elastic-credentials
                  key: password

Step 2: Create the Filebeat ConfigMap #

Define how Filebeat reads logs and ships them to Elasticsearch:

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-sidecar-config
data:
  filebeat.yml: |
    filebeat.inputs:
    - type: filestream
      paths:
        - /var/log/app-logs/*.log  # Match all log files in the shared volume
      fields:
        app: my-business-app  # Add a label for easier filtering in ES
    output.elasticsearch:
      hosts: ["elasticsearch-cluster:9200"]
      username: "${ELASTIC_USERNAME}"
      password: "${ELASTIC_PASSWORD}"

Alternative: Temporary ad-hoc collection (for debugging only) #

If you can't modify the Deployment right now, you can use kubectl exec to pull logs manually or via a script. This isn't for production, but it works for quick checks:

kubectl exec <pod-name> -- cat /app/internal/logs/error.log | curl -X POST -H "Content-Type: application/json" -d @- http://elasticsearch:9200/logs/_doc

For standalone containers, you've got two main production-grade options—both avoid modifying your app code.

Option 1: Bind Mount Log Directory to Host #

Mount the container's log directory to your host filesystem, then run a log collector (like Filebeat) on the host to ship logs:

# Start your app container with a bind mount
docker run -d \
  --name my-docker-app \
  -v /host/machine/logs:/app/internal/logs \
  your-docker-image:latest

# Run Filebeat on the host (or as a separate container) to read from /host/machine/logs
docker run -d \
  --name filebeat-host \
  -v /host/machine/logs:/var/log/app-logs \
  -v /host/machine/filebeat.yml:/usr/share/filebeat/filebeat.yml \
  docker.elastic.co/beats/filebeat:8.10.0

Option 2: Shared Volume + Sidecar Container (No Host Mount) #

If you don't want to clutter your host filesystem, use a Docker-managed volume to share logs between your app container and a sidecar collector:

# Create a shared volume
docker volume create app-log-volume

# Start your app container, mounting the volume to its log directory
docker run -d \
  --name my-docker-app \
  -v app-log-volume:/app/internal/logs \
  your-docker-image:latest

# Start Filebeat sidecar, mounting the same volume
docker run -d \
  --name filebeat-docker-sidecar \
  -v app-log-volume:/var/log/app-logs \
  -v /host/machine/filebeat.yml:/usr/share/filebeat/filebeat.yml \
  docker.elastic.co/beats/filebeat:8.10.0

• Log Rotation: Always configure log rotation (e.g., logrotate in containers, or app-level rotation) to prevent logs from eating up disk space.
• Permissions: Ensure your log collector has read access to the log files—run the collector container with a user that has the right permissions, or adjust file permissions in your app container.
• Performance: For high-volume logs, tune your collector's buffer settings (e.g., Filebeat's queue.mem.events) to avoid overwhelming your centralized log system or impacting app performance.

内容的提问来源于stack exchange，提问作者Abhizer Saifee