ACS Kubernetes集群续期后VMSS扩容节点无法获取新服务主体密钥

Hey there, let's work through this issue together. The core problem here is that manually updating /etc/kubernetes/azure.json on existing nodes only affects those running instances—it doesn't update the underlying Virtual Machine Scale Set (VMSS) configuration that's used to spin up new nodes during scaling. Here's how to fix this properly:

Step 1: Update the VMSS Cloud-Init Custom Data #

ACS Kubernetes uses cloud-init to inject configuration (including service principal details) into nodes when they're provisioned. You need to update this source template with your new key:

• First, pull the current cloud-init script from your VMSS (replace placeholders with your resource group and VMSS name):

  az vmss show --resource-group <your-resource-group> --name <your-vmss-name> --query virtualMachineProfile.osProfile.customData -o tsv | base64 --decode
  

• Look for the write_files block in the output that generates /etc/kubernetes/azure.json. Replace the old service principal key in this section with your new one.
• Encode the modified script back to base64:

  echo -n "<your-modified-cloud-init-script>" | base64
  

• Push the updated custom data back to the VMSS:

  az vmss update --resource-group <your-resource-group> --name <your-vmss-name> --set virtualMachineProfile.osProfile.customData=<base64-encoded-string>
  

Step 2: Validate New Node Provisioning #

When you scale out the VMSS now, new nodes will use the updated cloud-init script to generate a correct /etc/kubernetes/azure.json with your new service principal key.

Step 3: Verify the Fix #

After scaling out, SSH into the new node and run this command to confirm the key is present:

cat /etc/kubernetes/azure.json

You should see your updated service principal key in the output. Also, check that the node successfully joins the cluster and can access Azure resources (like storage or load balancers) to ensure the service principal is working as expected.

Key Note for Future Changes #

Avoid manually modifying configuration files on individual nodes for settings that need to persist across scaling. Always update the VMSS's custom data or template—this is the single source of truth for all new nodes in your ACS Kubernetes cluster.

内容的提问来源于stack exchange，提问作者Vineesha.C