.NET对接OpenTelemetry Collector Contrib追踪数据转发异常排查

试图搭建OpenTelemetry追踪系统，将追踪数据导出至Jaeger和Datadog。所有容器运行正常且无控制台报错，但将.NET应用的OtelUrl从http://jaeger:4317改为http://otel-collector:4317后，数据无法发送到otel-collector，且Collector日志和Jaeger中都看不到追踪数据。直接对接Jaeger时可以正常看到数据。

当前使用的配置： #

docker-compose.yml

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.95.0
    ports:
      - 1888:1888 # pprof extension
      - 8888:8888 # Prometheus metrics exposed by the Collector
      - 8889:8889 # Prometheus exporter metrics
      - 13133:13133 # health_check extension
      - 4317:4317 # OTLP gRPC receiver
      - 4318:4318 # OTLP http receiver
      - 55679:55679 # zpages extension
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    environment:
      DD_SITE: <url>
      DD_API_KEY: <key>
        
  jaeger:
    container_name: jaeger
    ports:
      - 16686:16686
      - 43017:4317
    image: jaegertracing/opentelemetry-all-in-one
    environment:
      COLLECTOR_ZIPKIN_HOST_PORT: 9411
      COLLECTOR_OTLP_ENABLED: true
      
  my-api:
    image: myapi
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 8080:80
      - 8081:443
    depends_on:
      - jaeger
    environment:
      OtelUrl: http://jaeger:4317
      ASPNETCORE_ENVIRONMENT: Production
      DD_LOGS_INJECTION: true
      DD_SERVICE: MyAPI

otel-collector-config.yaml

receivers:
  otlp:
    protocols:
      http:
        endpoint: "localhost:4318"
      grpc:
        endpoint: "localhost:4317"

processors:

exporters:
  otlp/jaeger: # Jaeger supports OTLP directly. The default port for OTLP/gRPC is 4317
    endpoint: http://jaeger:4317
    tls:
      insecure: true
  datadog:
    api:
      site: ${env:DD_SITE}
      key: ${env:DD_API_KEY}
    traces:
      trace_buffer: 500

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: []
      exporters: [datadog]
    traces:
      receivers: [otlp]
      processors: []
      exporters: [datadog, otlp/jaeger]
    logs:
      receivers: [otlp]
      processors: []
      exporters: [datadog]

Program.cs 配置

services.AddOpenTelemetry()
                .ConfigureResource(resource => resource.AddService(AppTracing.TelemetryServiceName))
                .WithTracing(tracing => tracing
                    .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(AppTracing.TelemetryServiceName))
                    .AddSource(AppTracing.SourceName)
                    .AddAspNetCoreInstrumentation()
                    .AddHttpClientInstrumentation()
                    .SetSampler(new AlwaysOnSampler())
                    .AddOtlpExporter(otlp =>
                    {
                        otlp.Endpoint = otlpEndpoint;
                    })
                    //.AddConsoleExporter()
                );

测试接口

[HttpPost("test-datadog")]
public void SendBatchTraces()
{
    var session = Random.Shared.Next(0, 100);
    foreach (var index in Enumerable.Range(0, 1000))
    {
        using var activity = AppTracing.Source.StartActivity($"Sending trace number: {index}");
        activity?.SetTag("Session", session);
    }
}

1. 修复Otel Collector接收器监听问题 #

你的Collector配置中，OTLP接收器的endpoint设为localhost:4317/localhost:4318，这会导致Collector仅监听容器内部的localhost接口，其他容器（如my-api）无法访问。需修改为监听所有网卡：

receivers:
  otlp:
    protocols:
      http:
        endpoint: "0.0.0.0:4318"
      grpc:
        endpoint: "0.0.0.0:4317"

2. 更新.NET应用的目标地址 #

将my-api的OtelUrl环境变量改为指向otel-collector：

my-api:
  environment:
    OtelUrl: http://otel-collector:4317
    # 其他环境变量保持不变

同时确保Program.cs中正确读取该环境变量：

var otlpEndpoint = new Uri(Environment.GetEnvironmentVariable("OtelUrl") ?? "http://localhost:4317");

3. 开启详细日志排查 #

.NET应用侧 #

• 在appsettings.json中添加OpenTelemetry调试日志：

{
  "Logging": {
    "LogLevel": {
      "OpenTelemetry": "Debug",
      "OpenTelemetry.Exporter.OpenTelemetryProtocol": "Debug"
    }
  }
}

• 临时启用ConsoleExporter，验证追踪数据是否正常生成：

.WithTracing(tracing => tracing
    // 其他配置不变
    .AddOtlpExporter(otlp =>
    {
        otlp.Endpoint = otlpEndpoint;
    })
    .AddConsoleExporter() // 临时添加，直接在控制台查看数据
);

Otel Collector侧 #

• 修改Collector配置，开启debug级日志：

service:
  telemetry:
    logs:
      level: debug
  # 原有pipelines配置不变

• 或者在docker-compose的command中添加日志级别参数：

otel-collector:
  command: ["--config=/etc/otel-collector-config.yaml", "--log-level=debug"]

通过docker logs <otel-collector-container-name>查看详细日志，确认是否接收到请求、导出是否成功。

4. 验证网络连通性 #

在my-api容器内执行命令，测试与otel-collector的网络连通性：

docker exec -it <my-api-container-id> curl http://otel-collector:4318/v1/traces

若返回405 Method Not Allowed，说明网络正常（只是HTTP接口不支持GET请求）；若超时，需检查docker网络配置（默认同一compose下的容器在同一网络，无需额外配置）。

内容的提问来源于stack exchange，提问作者Vladyslav Ishchuk