如何移除OpenTelemetry导入Loki的重复traceId/spanId字段？

已通过OpenTelemetry采集Spring Boot应用日志，并用OpenTelemetry Collector将日志同时导入Loki和VictoriaMetrics-Logs做对比。Collector配置如下：

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  zipkin: # just for troubleshooting

processors:
  attributes:
    actions:
      - key: team
        action: insert
        value: martinahrer

exporters:
  debug:
    verbosity: detailed

  otlphttp/victoria-logs:
    compression: gzip
    endpoint: http://victoria-logs:9428/insert/opentelemetry
    tls:
      insecure: true

  otlphttp/victoria-metrics:
    compression: gzip
    endpoint: http://victoria-metrics:8428/opentelemetry/
    tls:
      insecure: true

  otlphttp/tempo:
    endpoint: http://tempo:4318
    tls:
      insecure: true

  prometheusremotewrite:
    endpoint: http://prometheus:9090/api/v1/write

  otlphttp/loki:
    endpoint: http://loki:3100/otlp

service:
  telemetry:
    logs:
      level: DEBUG

  extensions:
    - zpages
  pipelines:
    logs:
      receivers: [otlp]
      processors: []
      exporters: [debug, otlphttp/victoria-logs, otlphttp/loki]
    metrics:
      receivers: [otlp]
      processors: [attributes]
      exporters: [debug, otlphttp/victoria-metrics, prometheusremotewrite]
    traces:
      receivers: [otlp, zipkin]
      exporters: [debug, otlphttp/tempo]

extensions:
  zpages:
    endpoint: "0.0.0.0:55679"

Loki使用默认配置运行，在Grafana中查询Loki数据源时，发现spanId、traceID字段分别以span_id、trace_id的形式重复出现，而VictoriaMetrics-Logs无此问题。重复的结构化元数据示例：

S trace_id d4631c7e0ac527b81064282c3523a39f
S traceId d4631c7e0ac527b81064282c3523a39f

查看OpenTelemetry Collector的日志输出，仅能看到spanId和traceId属性。这是Loki的特性吗？如何避免字段重复，或者至少移除Loki中的重复属性？尝试过Loki文档中的limits_config但无效。

这不是Loki的特性，而是Loki的OTLP接收器在处理OpenTelemetry日志时，会自动将日志关联的Trace上下文（traceId、spanId）转换为Loki标准的蛇形命名字段（trace_id、span_id），同时保留了原日志中的驼峰命名字段，导致重复。

可以通过两种方式解决：

方式一：在OpenTelemetry Collector中移除重复字段 #

修改Collector的日志Pipeline，添加attributes处理器，在导出到Loki前移除原有的traceId和spanId字段：

processors:
  attributes:
    actions:
      - key: team
        action: insert
        value: martinahrer
  attributes/remove-trace-fields:
    actions:
      - key: traceId
        action: delete
      - key: spanId
        action: delete

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [attributes/remove-trace-fields] # 添加这个处理器
      exporters: [debug, otlphttp/victoria-logs, otlphttp/loki]

如果VictoriaMetrics-Logs需要保留原traceId和spanId字段，可以为Loki单独创建一个日志Pipeline，只在导出到Loki的分支中移除这些字段：

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: []
      exporters: [debug, otlphttp/victoria-logs]
    logs/loki:
      receivers: [otlp]
      processors: [attributes/remove-trace-fields]
      exporters: [otlphttp/loki]

方式二：在Loki中配置处理规则，过滤重复字段 #

修改Loki的配置，添加processing_rules来移除重复的驼峰命名字段：

limits_config:
  # 保留原有配置
processing_rules:
  - action: drop_label
    label: traceId
  - action: drop_label
    label: spanId

保存配置后重启Loki即可生效。

内容的提问来源于stack exchange，提问作者Martin Ahrer