Docker中Datadog Agent日志无法发送:TLS握手失败(未知证书)
问题分析
从process-agent.log中的x509: certificate signed by unknown authority错误可以确定,Datadog Agent容器无法验证Datadog US5站点的SSL证书,导致TLS握手失败,进而阻断了日志发送流程——虽然日志已被Agent处理(LogsProcessed=222),但发送环节完全失败,因此LogsSent=0。
解决方案
1. 挂载主机CA证书到Agent容器
在datadog-agent服务的volumes配置中添加主机根证书目录的只读挂载,让Agent容器能使用系统信任的CA证书库:
volumes: # 保留原有挂载配置 - /etc/ssl/certs/:/etc/ssl/certs/:ro
此操作可让Agent直接读取主机上的可信CA证书,完成对Datadog站点证书的合法性验证。
2. 验证Agent镜像与证书配置
如果使用自定义环境证书,需将证书文件挂载到容器的/usr/local/share/ca-certificates/目录,并触发证书更新。可通过修改datadog-agent的启动命令实现:
command: ["sh", "-c", "update-ca-certificates && agent run"]
若使用官方Agent镜像,默认已包含常用CA证书,优先采用挂载主机证书目录的方式即可。
3. 确认站点与API Key正确性
检查DD_SITE=us5.datadoghq.com配置是否匹配实际使用的Datadog站点,同时确保DD_API_KEY无拼写错误、权限有效。
4. 测试容器网络连通性
进入Agent容器,测试与Datadog站点的SSL连通性,排查是否存在网络或证书问题:
docker exec -it dd-agent curl -v https://agent-http-intake.logs.us5.datadoghq.com
若返回证书相关错误,说明证书验证问题仍存在;若出现超时,需检查防火墙是否允许Agent访问443端口。
相关配置与日志信息
Docker-Compose配置
version: "3.9" # optional since v1.27.0 services: web: build: context: . dockerfile: Dockerfile image: my_app container_name: app_container labels: com.datadoghq.ad.init_configs: '[{}]' com.datadoghq.ad.instances: '[{"host":"%%host%%", "port": "%%port%%"}]' com.datadoghq.ad.logs: '[{"type": "file", "path": "/mnt/log/datadog_my_app.log", "service": "my_app", "source": "python", "sourcecategory": "sourcecode"}]' com.datadoghq.tags.service: 'my_app' environment: - DD_AGENT_HOST=datadog-agent volumes: - /tmp/logs/:/mnt/log/ restart: always datadog-agent: image: gcr.io/datadoghq/agent:7 container_name: dd-agent restart: always links: - web environment: - DD_API_KEY=<API_KEY> - DD_SITE=us5.datadoghq.com - DD_HOSTNAME=my_app - DD_APM_ENABLED=true - DD_LOGS_ENABLED=true - DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true - DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true - DD_LOGS_CONFIG_DOCKER_CONTAINER_FORCE_USE_FILE=true - DD_CONTAINER_EXCLUDE=image:gcr.io/datadoghq/agent* - DD_CONTAINER_EXCLUDE_METRICS=image:gcr.io/datadoghq/agent* - DD_CONTAINER_EXCLUDE_LOGS=image:gcr.io/datadoghq/agent* - DD_AC_INCLUDE=name:app_container - DD_LOGS_CONFIG_USE_HTTP=true volumes: - /var/run/docker.sock:/var/run/docker.sock - /proc/:/host/proc/:ro - /sys/fs/cgroup/:/host/sys/fs/cgroup:ro - /var/lib/docker/containers:/var/lib/docker/containers:ro - /tmp/logs/:/mnt/log/:ro
Agent日志状态
========== Logs Agent ========== Reliable: Sending compressed logs in HTTPS to agent-http-intake.logs.us5.datadoghq.com on port 443 BytesSent: 818556 EncodedBytesSent: 13542 LogsProcessed: 222 LogsSent: 0 docker ------ - Type: file Identifier: 77a47e6c4b5e89d00414ac38b1923c290bb69fe347cbcdc1739f80bf0e439dd5 Path: /mnt/log/datadog_my_app.log Service: my_app Source: python Status: OK Inputs: /mnt/log/datadog_my_app.log Bytes Read: 37207 Pipeline Latency: Average Latency (ms): 0 24h Average Latency (ms): 0 Peak Latency (ms): 0 24h Peak Latency (ms): 0
Process-Agent错误日志
2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:10 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:30 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/default_forwarder.go:679 in func3) | timed out waiting for responses, received 0/1 2023-06-21 05:59:30 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:191 in process) | Too many errors for endpoint 'https://process.us5.datadoghq.com/api/v1/container': retrying later 2023-06-21 05:59:35 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/transaction/transaction.go:109 in func4) | TLS Handshake failure: x509: certificate signed by unknown authority 2023-06-21 05:59:35 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/worker.go:195 in process) | Error while processing transaction: error while sending transaction, rescheduling it: Post "https://process.us5.datadoghq.com/api/v1/discovery": x509: certificate signed by unknown authority
内容的提问来源于stack exchange,提问作者Rajat Suneja
相关产品推荐
相关产品推荐

