为何GKE不建议搭配托管IG自动扩缩容？生产实践求答疑

Great question—let’s break down why Google explicitly warns against this combination, even if your current DaemonSet-heavy setup seems to work smoothly. The risks go beyond just "system-critical pods being deleted" and touch on core Kubernetes and GKE integration gaps:

1. Kubernetes Cluster State Desync #

Compute Engine Autoscaler operates entirely outside of Kubernetes’ awareness. It only monitors VM-level metrics (like CPU/memory usage or custom external metrics) and has no visibility into:

• Pod scheduling status
• Node readiness in the Kubernetes control plane
• Pending pods waiting for resources

For example:

• When scaling down, it might delete a VM before Kubernetes has a chance to detect the node is gone and reschedule pods. This causes abrupt pod termination, even with replicas, leading to transient service outages (especially for slow-starting or stateful workloads).
• When scaling up, it might provision a VM that’s "Ready" in Compute Engine but still initializing in Kubernetes (waiting for kubelet registration, CNI setup, or node labeling). The autoscaler could stop scaling prematurely, leaving pending pods stuck even though new VMs exist.

2. No Pod Evacuation or Drain Logic #

Cluster Autoscaler follows Kubernetes best practices: before deleting a node, it runs kubectl drain to gracefully evict pods, ensuring they’re rescheduled to healthy nodes first. Compute Engine Autoscaler skips this entirely—it deletes VMs directly, forcing Kubernetes to reactively detect node failures and restart pods.

Even with 3 replicas of critical pods, this leads to:

• Unnecessary pod restarts and service disruption during scale-down
• Potential race conditions where pods can’t be rescheduled fast enough, leaving your cluster underprovisioned temporarily

3. Misaligned Scaling Triggers #

Compute Engine Autoscaler’s metrics are VM-centric, while Kubernetes manages resources at the pod level. This creates mismatches:

• If you add non-DaemonSet workloads (like Deployments or Jobs) later, the autoscaler won’t account for their resource requests/limits. It might scale up too late (when VM usage spikes) or scale down too early (when pod usage drops but VM usage is still high).
• DaemonSet pods have fixed per-node resource footprints, but if your cluster has variable workloads, the autoscaler’s VM-level metrics won’t reflect the actual scheduling pressure in Kubernetes.

4. Ignorance of Kubernetes Scheduling Constraints #

Cluster Autoscaler considers Kubernetes’ scheduling rules when adding nodes: node affinities, taints/tolerations, pod priorities, and custom node labels. Compute Engine Autoscaler doesn’t— it just provisions VMs based on your MIG template.

This can lead to:

• New nodes missing required labels or taints, making them unusable for your DaemonSet or other workloads
• Wasted resources on nodes that can’t run any pods, while other nodes are overloaded
• Failed pod scheduling if your workloads require specific node configurations the autoscaler doesn’t replicate

5. Long-Term Support and Compatibility Risks #

Google explicitly states this setup is unsupported. That means:

• If you hit issues (like nodes failing to register with Kubernetes, or scaling loops), GCP support won’t assist you.
• Future GKE updates (new node initialization workflows, scheduling features, or control plane changes) could break your setup unexpectedly. Compute Engine Autoscaler isn’t designed to keep up with Kubernetes ecosystem changes.

6. Resource Request vs. Actual Usage Mismatch #

Cluster Autoscaler uses pod resource requests to calculate cluster capacity, ensuring it scales to meet the requested resources (not just actual usage). Compute Engine Autoscaler only looks at current VM usage, which can lead to:

• Under-scaling: If pods request more resources than the VM is currently using, Kubernetes can’t schedule them even if the VM has free capacity.
• Over-scaling: If VM usage is high but pod requests are low, the autoscaler might add unnecessary nodes.

While your DaemonSet-focused cluster might work for now, these risks will become more pronounced as your workloads grow, change, or as GKE evolves. The initial setup overhead of Cluster Autoscaler + HPA pays off in stability, compatibility, and alignment with Kubernetes’ native resource management.

内容的提问来源于stack exchange，提问作者Jordan Pittier