Docker容器部署下OTEL Collector无法连接Jaeger问题求助

环境配置 #

Jaeger启动命令 #

docker run --rm --name jaeger -e COLLECTOR_OTLP_ENABLED=true -p 16686:16686 -p 5000:4317 jaegertracing/all-in-one:latest

OTEL Collector启动命令 #

docker run --rm --name otelcollector --env COLLECTOR_OTLP_ENABLED=true -p 127.0.0.1:4317:4317 -p 127.0.0.1:55679:55679 -v "C:\OpenTelemetryCollector\otel-collector-config.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector --config=/etc/otelcol/config.yaml

OTEL Collector配置（原） #

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"

processors:
  batch:

exporters:
  logging:
    verbosity: detailed
  otlp:
    endpoint: "localhost:5000"
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

Web API追踪代码 #

traceProviderBuilder.AddAspNetCoreInstrumentation()
.AddOtlpExporter(options =>
{
    options.Endpoint = new Uri("http://localhost:4317");
});

错误信息 #

2024-05-16T12:06:58.501Z        info    exporterhelper/retry_sender.go:118      Exporting failed. Will retry the request after interval.        {"kind": "exporter", "data_type": "logs", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:5000: connect: connection refused\"", "interval": "21.931525309s"}
2024-05-16T12:07:13.343Z        error   exporterhelper/queue_sender.go:101      Exporting failed. Dropping data.       {"kind": "exporter", "data_type": "logs", "name": "otlp", "error": "no more retries left: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:5000: connect: connection refused\"", "dropped_items": 6}
go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1
        go.opentelemetry.io/collector/exporter@v0.100.0/exporterhelper/queue_sender.go:101
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume
        go.opentelemetry.io/collector/exporter@v0.100.0/internal/queue/bounded_memory_queue.go:52
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1
        go.opentelemetry.io/collector/exporter@v0.100.0/internal/queue/consumers.go:43
2024-05-16T12:07:20.435Z        error   exporterhelper/queue_sender.go:101      Exporting failed. Dropping data.       {"kind": "exporter", "data_type": "logs", "name": "otlp", "error": "no more retries left: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:5000: connect: connection refused\"", "dropped_items": 5}
go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1
        go.opentelemetry.io/collector/exporter@v0.100.0/exporterhelper/queue_sender.go:101
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume
        go.opentelemetry.io/collector/exporter@v0.100.0/internal/queue/bounded_memory_queue.go:52
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1
        go.opentelemetry.io/collector/exporter@v0.100.0/internal/queue/consumers.go:43

解决方案 #

1. 统一容器网络 #

Docker默认bridge网络下，容器间无法通过名称直接访问，需创建自定义网络并将两个容器加入：

# 创建自定义网络
docker network create otel-network

# 重启Jaeger并加入自定义网络
docker run --rm --name jaeger --network otel-network -e COLLECTOR_OTLP_ENABLED=true -p 16686:16686 -p 5000:4317 jaegertracing/all-in-one:latest

# 重启OTEL Collector并加入自定义网络
docker run --rm --name otelcollector --network otel-network --env COLLECTOR_OTLP_ENABLED=true -p 127.0.0.1:4317:4317 -p 127.0.0.1:55679:55679 -v "C:\OpenTelemetryCollector\otel-collector-config.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector --config=/etc/otelcol/config.yaml

2. 修正OTLP Exporter端点 #

Jaeger容器内部的OTLP gRPC端口为4317，本地5000是映射到该端口的外部端口，容器间通信需直接使用内部端口。修改配置文件的exporter部分：

exporters:
  logging:
    verbosity: detailed
  otlp:
    endpoint: "jaeger:4317"  # 改为Jaeger容器名+内部OTLP端口
    tls:
      insecure: true

3. 验证 #

重启OTEL Collector后，调用Web API接口，检查Collector日志是否消除连接错误，同时访问Jaeger UI（http://localhost:16686）确认追踪数据是否正常接收。

内容的提问来源于stack exchange，提问作者Saurav