Kubernetes Metrics Server无响应致HPA自动扩缩容失效求助

Hey there, let's break down why your HPA stopped scaling and fix those annoying Metrics Server TLS errors. The key clues here are the FailedGetResourceMetric error in your HPA status and the TLS handshake failures popping up in your Metrics Server logs. Let's work through this step by step:

1. Verify Core Metrics Server & APIService Health #

First, let's confirm the foundational components are in place:

• Check if the Metrics Server pod is running properly:

  kubectl get pods -n kube-system | grep metrics-server
  

  If it's crashing or restarting, run kubectl describe pod <metrics-server-pod-name> -n kube-system to dig into startup errors or event logs.
• Check if the metrics API is registered with the cluster:

  kubectl get apiservice v1beta1.metrics.k8s.io
  

  The STATUS column should show Available=True. If not, the Kubernetes API Server can't reach the Metrics Server—this is exactly why your HPA can't fetch metrics.

2. Fix TLS Handshake Mismatches #

The tls: first record does not look like a TLS handshake error means there's a protocol mismatch between what Metrics Server expects and the requests it's getting. Here's how to resolve this:

a. Update Metrics Server Startup Flags #

Your Metrics Server v0.2.1 is using default settings that clash with Kubernetes 1.10.3. Add these critical flags to its deployment:

• --kubelet-preferred-address-types=InternalIP: Ensures Metrics Server uses node internal IPs to connect to kubelets (avoids DNS or public IP routing issues)
• --kubelet-insecure-tls: Skips kubelet TLS certificate validation (use this for testing; for production, configure trusted cluster certificates instead)
• --tls-cert-file and --tls-private-key-file: Replace the auto-generated self-signed cert with a cluster-trusted certificate (prevents the API Server from rejecting Metrics Server requests)

To edit the deployment:

kubectl edit deployment metrics-server -n kube-system

Add the flags to the args section under spec.template.spec.containers[0].

b. Check Kubelet Port Configuration #

Your Metrics Server logs show it's targeting kubelet port 10255, but Kubernetes 1.10+ often disables this unauthenticated HTTP read-only port by default. Verify if your kubelets have it enabled:
On any worker node, run:

ps aux | grep kubelet

Look for the --read-only-port=10255 flag. If it's missing, you have two options:

1. Enable the port by adding --read-only-port=10255 to your kubelet configuration (restart kubelet after making changes)
2. Configure Metrics Server to use the secure kubelet port (10250) instead by adjusting the --source flag to kubernetes.summary_api:https://<kubelet-ip>:10250 (requires proper TLS certs for authentication)

3. Validate HPA & Target Resource Config #

Double-check your HPA and its target deployment to ensure scaling thresholds are properly set:

• Ensure your target deployment has CPU requests defined in its pod template (HPA can't calculate utilization without this):

  spec:
    template:
      spec:
        containers:
        - name: my-app
          resources:
            requests:
              cpu: "100m"
  

• Confirm your HPA manifest correctly targets CPU utilization (example snippet):

  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 70
  

4. Restart & Test #

After making changes:

• Restart the Metrics Server pod to apply new config: kubectl delete pod <metrics-server-pod-name> -n kube-system
• Wait 5-10 minutes, then check HPA status: kubectl describe hpa <your-hpa-name>
• Check Metrics Server logs again to confirm TLS errors are gone: kubectl logs <metrics-server-pod-name> -n kube-system

内容的提问来源于stack exchange，提问作者macintoshPrime