Grafana中Prometheus数据源不显示已导出指标问题

环境：Spring Boot 3.2.0、JDK 17，Docker部署Grafana Stack
放弃JavaAgent方案，采用Spring Boot手动埋点推送指标与追踪数据。Collector已生成Prometheus导出日志，但Prometheus指标查询下拉框无内容，且无目标显示（Actuator因JWT锁定未开启采集）。

应用日志 #

14:13:04.035 [main] INFO  i.m.c.i.push.PushMeterRegistry - publishing metrics for OtlpMeterRegistry every 10s

Collector日志 #

debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.threads.states{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: executor.active{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: executor.active{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: executor.queue.remaining{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.gc.live.data.size{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: executor.completed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.buffer.memory.used{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: spring.integration.channels{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.threads.states{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: rabbitmq.acknowledged{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.used{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.buffer.count{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: logback.events{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: rabbitmq.rejected{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: tasks.scheduled.execution{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.max{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.threads.live{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.classes.unloaded{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.memory.committed{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.classes.loaded{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.threads.peak{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: application.started.time{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: rabbitmq.not_acknowledged_published{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: rabbitmq.failed_to_publish{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: tomcat.sessions.rejected{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: jvm.gc.memory.promoted{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
debug prometheusexporter@v0.91.0/accumulator.go:79accumulating metric: disk.total{"kind": "exporter", "data_type": "metrics", "name": "prometheus"}
info MetricsExporter{"kind": "exporter", "data_type": "metrics", "name": "logging", "resource metrics": 1, "metrics": 109, "data points": 109}
info ResourceMetrics #0
 Resource SchemaURL: 
 Resource attributes:
      -> telemetry.sdk.name: Str(io.micrometer)
      -> telemetry.sdk.language: Str(java)
      -> telemetry.sdk.version: Str(1.12.0)
      -> service.name: Str(host-validation)

Collector配置 #

exporters:
  logging:
    verbosity: detailed

  #Metrics to Prometheus
  prometheus:
    endpoint: "0.0.0.0:8889"
    enable_open_metrics: true
    const_labels:
      label1: osint
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  loki:
    endpoint: "http://loki:3100/loki/api/v1/push"

# The Collector pipeline.
service:

  extensions: [health_check, pprof, zpages]
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus,logging]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo,logging]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [loki,logging]

额外信息 #

• Tempo可正常查看链路，Loki可正常查看日志
• 启动时出现如下异常，但无明显影响：

java.lang.Throwable: null
    at io.opentelemetry.context.ContextStorageWrappers.addWrapper(ContextStorageWrappers.java:32)
    at io.opentelemetry.context.ContextStorage.addWrapper(ContextStorage.java:89)
    at org.springframework.boot.actuate.autoconfigure.tracing.OpenTelemetryAutoConfiguration.otelCurrentTraceContext(OpenTelemetryAutoConfiguration.java:159)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)

• 补全Prometheus采集配置：确保Prometheus的scrape配置中添加Collector的otel-collector:8889端点（默认路径/metrics），示例配置：

  scrape_configs:
    - job_name: 'otel-collector'
      static_configs:
        - targets: ['otel-collector:8889']
  

• 验证Collector端点可用性：在Docker网络内访问http://otel-collector:8889/metrics，确认能返回指标数据，排查端口映射或网络连通性问题。
• 兼容指标格式：若存在Prometheus不支持的指标标签，可在Collector中添加transform或filter处理器调整指标名称、标签。
• 忽略启动异常：该异常是Spring Boot 3.2.x与OpenTelemetry的兼容性小问题，无功能影响，也可升级OpenTelemetry依赖至适配版本。

内容的提问来源于stack exchange，提问作者B Randall