Minikube集群Heapster API无法获取文件系统指标求助

Hey there! Let's work through why you can't pull filesystem metrics via the Heapster API, even though CPU and memory metrics are working perfectly. Here are actionable steps to diagnose and fix the issue:

1. Check Heapster's Collector Configuration #

Heapster uses specific collectors to fetch different metric types, and filesystem metrics might not be enabled by default. Let's verify your deployment settings:

• Edit the Heapster deployment with this command:

  kubectl edit deployment heapster -n kube-system
  

• Look for the args section in the container definition. You should see a line specifying enabled collectors, like this:

  args:
  - --source=kubernetes.summary_api:''
  - --collectors=filesystem,cpu,memory,network
  

  If filesystem is missing from the --collectors list, add it, save your changes, and restart the Heapster pod to apply the update:

  kubectl delete pod -l k8s-app=heapster -n kube-system
  

2. Verify Kubelet's Summary API is Reporting Filesystem Data #

Heapster pulls metrics directly from the Kubelet's Summary API, so we need to confirm this endpoint is returning filesystem stats. Connect to your Minikube node and run:

minikube ssh
curl http://localhost:10255/stats/summary

Scan the output for fs sections under node or pod entries. If you don't see any filesystem-related data here:

• Check the Kubelet service config (usually at /etc/systemd/system/kubelet.service.d/10-kubeadm.conf) to ensure no flags are disabling stats collection.
• Confirm there are no restrictive security policies (like AppArmor or SELinux) blocking the Kubelet from accessing filesystem metrics.

3. Validate Heapster-to-InfluxDB Metric Delivery #

Even if Heapster collects filesystem metrics, it might not be pushing them to InfluxDB correctly:

• Check Heapster's logs for sink-related errors:

  kubectl logs -l k8s-app=heapster -n kube-system
  

  Look for lines indicating failures to write filesystem metrics, or confirm metrics are being sent successfully.
• Query InfluxDB directly to check if filesystem metrics exist. First, access the InfluxDB pod:

  kubectl exec -it <influxdb-pod-name> -n kube-system -- influx
  

  Then run these commands in the InfluxDB shell:

  USE k8s;
  SHOW MEASUREMENTS;
  

  Look for measurements like filesystem or node_filesystem. If they're missing, circle back to Heapster's collector configuration.

4. Check Grafana Dashboard Setup (If Relevant) #

If you're also checking metrics via Grafana (not just the API), ensure your dashboard is configured to pull filesystem metrics from InfluxDB. Default dashboards often don't include filesystem panels, so you might need to import a dashboard that supports these metrics or create custom panels using the relevant InfluxDB measurements.

内容的提问来源于stack exchange，提问作者Mufeed