Minikube环境下无法在Grafana中添加Prometheus数据源

环境与操作步骤 #

• 使用MiniKube 1.31，执行命令部署OpenTelemetry Collector Operator：

  kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml
  

• 在default命名空间部署OpenTelemetryCollector实例，YAML配置如下：

  apiVersion: opentelemetry.io/v1beta1
  kind: OpenTelemetryCollector
  metadata:
    name: simplest
  spec:
    config:
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317
            http:
              endpoint: 0.0.0.0:4318
      processors:
        memory_limiter:
          check_interval: 1s
          limit_percentage: 75
          spike_limit_percentage: 15
        batch:
          send_batch_size: 10000
          timeout: 10s
  
      exporters:
        debug: {}
  
      service:
        pipelines:
          traces:
            receivers: [otlp]
            processors: [memory_limiter, batch]
            exporters: [debug]
  

  部署成功后自动生成ClusterIP类型服务simplest-collector。
• 在monitoring命名空间部署Grafana，Deployment配置（省略PVC）：

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: grafana
    namespace: monitoring
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: grafana
    template:
      metadata:
        labels:
          app: grafana
    spec:
      containers:
        - name: grafana
          image: grafana/grafana:latest
          ports:
            - containerPort: 3000
          volumeMounts:
            - name: grafana-storage
              mountPath: /var/lib/grafana
      volumes:
        - name: grafana-storage
          persistentVolumeClaim:
            claimName: grafana-pvc
  

• 对应Grafana的Service配置：

  apiVersion: v1
  kind: Service
  metadata:
    name: grafana-service
    namespace: monitoring
  spec:
    type: NodePort
    ports:
      - port: 3000
        targetPort: 3000
    selector:
      app: grafana
  

问题现象 #

在Grafana中配置Prometheus数据源时，使用地址http://simplest-collector.default.svc.cluster.local:4317无法连接，返回错误：

Post "http://simplest-collector.default.svc.cluster.local:4317/api/v1/query": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00" - There was an error returned querying the Prometheus API.

尝试过用minikube service simplest-collector暴露服务，以及NodePort方式，均未成功，全程未使用HTTPS。

内容的提问来源于stack exchange，提问作者FourZeroFive