如何配置单个Solr Exporter抓取Kubernetes集群多命名空间下所有Solr StatefulSet实例的指标？

Yes, this is absolutely feasible—and a smart way to cut down on redundant resource usage in your cluster. The exact approach depends on how your Solr instances are managed, but I’ll walk you through the two most common solutions, along with modified configurations and key considerations.

Approach 1: Use ZooKeeper Discovery (Recommended for ZK-Managed Clusters) #

If all your Solr StatefulSets register to the same ZooKeeper ensemble (which your existing config already leverages with the -z flag), the Solr Exporter can automatically discover all Solr nodes across namespaces. You just need to ensure:

• The exporter has network access to your ZooKeeper cluster and all Solr pods.
• All Solr clusters are registered to the same ZK ensemble.

Modified Deployment YAML for Cluster-Wide Scrapping #

Here’s how to adapt your existing setup to work as a single cluster-wide exporter:

apiVersion: v1
kind: Service
metadata:
  name: cluster-wide-solr-exporter
  labels:
    app: cluster-wide-solr-exporter
spec:
  ports:
  - port: 9983
    name: metrics
  selector:
    app: cluster-wide-solr-exporter
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: cluster-wide-solr-exporter
  name: cluster-wide-solr-exporter
  namespace: monitoring  # Use a dedicated observability namespace for clarity
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-wide-solr-exporter
  template:
    metadata:
      labels:
        app: cluster-wide-solr-exporter
    spec:
      containers:
      - command:
        - /opt/solr/contrib/prometheus-exporter/bin/solr-exporter
        - -p 9983
        - -z zk-cs.default.svc.cluster.local:2181  # Replace with your full ZK FQDN
        - -n 7  # Adjust node discovery interval as needed
        - -f /opt/solr/contrib/prometheus-exporter/conf/solr-exporter-config.xml
        image: solr:8.1.1
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /metrics
            port: 9983
            scheme: HTTP
          initialDelaySeconds: 120  # Shorter delay since we rely on ZK discovery
          periodSeconds: 5
        name: solr-exporter
        ports:
        - containerPort: 9983
          protocol: TCP
        readinessProbe:
          failureThreshold: 2
          httpGet:
            path: /metrics
            port: 9983
            scheme: HTTP
          initialDelaySeconds: 60
          periodSeconds: 5
        resources:
          limits:
            cpu: 1000m  # Increase to handle metrics from multiple Solr nodes
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 128Mi
        volumeMounts:
        - mountPath: /opt/solr/contrib/prometheus-exporter/conf/solr-exporter-config.xml
          name: solr-exporter-config
          readOnly: true
          subPath: solr-exporter-config.xml
      volumes:
      - configMap:
          defaultMode: 420
          name: cluster-wide-solr-exporter-config
        name: solr-exporter-config

Key changes from your original config:

• Renamed resources to avoid conflicts with old per-Solr exporters
• Moved to a monitoring namespace (standard for observability tools)
• Adjusted resource limits to handle metrics from multiple Solr instances
• Removed the init container (ZK discovery eliminates the need to wait for a single Solr instance)

Approach 2: Scrape Explicit Solr Endpoints (For Standalone/Separate Clusters) #

If your Solr instances don’t share a ZooKeeper cluster, you can configure the exporter to scrape explicit Solr endpoints across namespaces by updating the solr-exporter-config.xml file.

Example ConfigMap for Explicit Endpoints #

Modify your ConfigMap to list all Solr pod URLs (use their full Kubernetes FQDNs):

<config>
  <solrUrls>
    <list>
      <item>http://solr-0.solr.default.svc.cluster.local:8983/solr</item>
      <item>http://solr-0.solr.team-a.svc.cluster.local:8983/solr</item>
      <item>http://solr-0.solr.team-b.svc.cluster.local:8983/solr</item>
      <!-- Add all your Solr StatefulSet pod URLs here -->
    </list>
  </solrUrls>
  <!-- Keep your existing metrics collection config here -->
</config>

Then update the exporter command to remove the ZK flag and use the explicit config:

/opt/solr/contrib/prometheus-exporter/bin/solr-exporter -p 9983 -f /opt/solr/contrib/prometheus-exporter/conf/solr-exporter-config.xml

Critical Considerations #

• Network Access: Ensure the exporter’s namespace has access to all Solr namespaces. Most clusters allow cross-namespace traffic by default, but if you use network policies, add rules to permit traffic from the exporter to Solr pods/services.
• Prometheus Update: Adjust your Prometheus scrape config to target this single exporter instead of multiple instances:

  scrape_configs:
    - job_name: 'solr'
      static_configs:
        - targets: ['cluster-wide-solr-exporter.monitoring.svc.cluster.local:9983']
  

• Resource Monitoring: Keep an eye on the exporter’s CPU/memory usage as you add more Solr nodes—you may need to tweak limits if you see throttling or latency.
• High Availability: For production, run two replicas of the exporter with a service to avoid downtime if one pod fails.

Final Steps #

1. Delete your old per-Solr exporter deployments and services
2. Deploy the cluster-wide exporter using the modified YAML
3. Update Prometheus to scrape the new exporter
4. Verify metrics for all Solr instances are appearing in the Prometheus UI

内容的提问来源于stack exchange，提问作者Paul Joseph