OpenTelemetry Kafka Streams手动埋点Span未导出问题排查

在SpringBoot应用中同时使用OpenTelemetry Java Agent与OpenTelemetry SDK，对Kafka Streams的业务逻辑进行手动埋点，但手动创建的Span始终无法导出。

相关配置与代码 #

build.gradle.kts依赖 #

implementation("io.opentelemetry:opentelemetry-api")
implementation("io.opentelemetry:opentelemetry-sdk")
implementation("io.opentelemetry:opentelemetry-exporter-otlp")

手动埋点代码示例 #

override fun assignDemographic(transaction: Transaction): Either<DemographicAssignmentError, RawCustomerDemographic> {
        val span = tracer.spanBuilder("assignDemographic")
            .setSpanKind(SpanKind.CONSUMER)
            .startSpan()
        span.makeCurrent()

        val spanAttributes = Attributes.of(
            stringKey("transaction"), transaction.transaction,
            stringKey("customerId"), transaction.customerId)
        span.addEvent("Assigning Demographics", spanAttributes)

        val result = transactionValidatorService.checkIsValidTransaction(transaction)

        val validationSpanAttributes = Attributes.of(booleanKey("validationResult"), result)
        span.addEvent("Checked to see if transaction is valid/supported", validationSpanAttributes)

        return when (result) {
            true -> {
                val assignedDemographic = demographicAssignerService.assignDemographic(transaction).right()
                assignedDemographic.map {
                    val resultAttributes = Attributes.of(stringArrayKey("result"), it.demographics)
                    span.setStatus(StatusCode.OK)
                    span.addEvent("Successfully assigned demograhics to customer", resultAttributes)
                }
                assignedDemographic
            }
            else -> {
                val error = DemographicAssignmentError().left()

                error.mapLeft {
                    span.setStatus(StatusCode.ERROR)
                    span.addEvent("Could not successfully assign demographics to the customer")
                }
                error
            }
        }
    }

应用启动配置 #

export JAVA_TOOL_OPTIONS="-javaagent:./opentelemetry-javaagent.jar"
export OTEL_SERVICE_NAME="app-name"
export OTEL_JAVAAGENT_LOGGING=application
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT="https://backend-endpoint"

java \
  -jar \
  -Dotel.instrumentation.kafka.experimental-span-attributes=true \
  ./build/libs/$APP_NAME.jar

• 未正确结束Span：OpenTelemetry的Span必须调用span.end()才会标记为完成并进入导出流程。当前代码仅启动Span但未结束，需用try-finally包裹业务逻辑，确保无论成功失败都能执行结束操作：

  override fun assignDemographic(transaction: Transaction): Either<DemographicAssignmentError, RawCustomerDemographic> {
      val span = tracer.spanBuilder("assignDemographic")
          .setSpanKind(SpanKind.CONSUMER)
          .startSpan()
      span.makeCurrent()
      return try {
          // 原有业务逻辑代码
      } finally {
          span.end()
      }
  }
  

• Tracer实例来源冲突：同时使用Agent和SDK时，Agent会替换默认TracerProvider。若手动创建的tracer是通过SDK自定义初始化的（而非GlobalOpenTelemetry.getTracer("your-instrumentation-name")获取），会导致手动Span脱离Agent的导出管道。需统一通过GlobalOpenTelemetry获取Tracer实例。

• 未关联父Span：Kafka Streams Agent会自动生成消费Span，若手动Span未关联到该父Span，可能被视为孤立Span无法导出。需在构建Span时添加setParent(Context.current())，确保继承当前上下文的父Span：

  val span = tracer.spanBuilder("assignDemographic")
      .setSpanKind(SpanKind.CONSUMER)
      .setParent(Context.current())
      .startSpan()
  

• SDK与Agent配置冲突：手动初始化SDK的TracerProvider或Exporter会覆盖Agent的配置，导致手动Span无法进入Agent的导出流程。应避免手动配置SDK核心组件，完全依赖Agent的环境变量配置，仅通过OpenTelemetry API进行埋点。

• 导出环节异常：可查看应用日志（通过OTEL_JAVAAGENT_LOGGING=application配置输出），排查是否存在网络连接超时、后端端点认证失败等问题，这类异常会导致所有Span（包括手动创建的）无法正常导出。

内容的提问来源于stack exchange，提问作者austiine