升级EKS至1.25后出现kube-apiserver-kubelet-client授权错误

问题现象 #

升级EKS到1.25版本后，执行kubectl logs <pod-name> -n <namespace>时触发以下授权错误：

Internal error occurred: Authorization error (user=kube-apiserver-kubelet-client, verb=get, resource=nodes, subresource=proxy)

已排查操作 #

• 检查aws-auth ConfigMap、ClusterRole和RoleBinding，未发现明显配置问题（这些资源已存在2年）
• 尝试为kube-apiserver-kubelet-client绑定system:kubelet-api-admin ClusterRole，问题未解决
• 对比新部署的EKS实例，发现当前集群缺少CSR（证书签名请求）资源

相关配置 #

aws-auth ConfigMap #

apiVersion: v1
data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::<some-number>:role/eksctl-<xyz-abs>-nodegrou-NodeInstanceRole-DMQXBTLLXHNU
      username: system:node:{{EC2PrivateDNSName}}
  mapUsers: |
    - userarn: arn:aws:iam::043519645107:user/kube-developer
      username: kube-developer
      groups:
       - kube-developer
kind: ConfigMap
metadata:
  creationTimestamp: "2020-07-03T16:55:08Z"
  name: aws-auth
  namespace: kube-system
  resourceVersion: "104191269"
  uid: 844f189d-b3d6-4204-bf85-7b789c0ee91a

自定义ClusterRole与ClusterRoleBinding #

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-developer-cr
rules:
- apiGroups: ["*"]
  resources:
    - configmaps
    - endpoints
    - events
    - ingresses
    - ingresses/status
    - services
  verbs:
    - create
    - get
    - list
    - update
    - watch
    - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-developer-crb
subjects:
- kind: Group
  name: kube-developer
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: kube-developer-cr
  apiGroup: rbac.authorization.k8s.io

尝试创建的ClusterRoleBinding #

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-apiserver
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:kubelet-api-admin
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: kube-apiserver-kubelet-client

解决步骤 #

1. 修正RBAC绑定对象类型
   你之前创建的ClusterRoleBinding有误，kube-apiserver-kubelet-client实际是集群内置组而非用户，正确配置如下：

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: kube-apiserver-kubelet-client
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: system:kubelet-api-admin
   subjects:
     - apiGroup: rbac.authorization.k8s.io
       kind: Group
       name: system:kube-apiserver-kubelet-client
   

   应用配置后，验证权限是否生效：

   kubectl auth can-i get nodes/proxy --as=system:kube-apiserver-kubelet-client
   

2. 检查并处理CSR资源
   EKS 1.25强化了kubelet证书管理，旧集群可能缺失自动签名的CSR。执行命令查看CSR状态：

   kubectl get csr
   

   如果存在未批准的CSR，手动批准：

   kubectl certificate approve <csr-name>
   

   若没有CSR，需确认集群certificates.k8s.io API已启用，可通过更新节点组kubelet配置开启自动证书轮换。

3. 验证节点组权限
   检查system:nodes组是否拥有nodes/proxy访问权限，默认system:node ClusterRole应包含该权限，执行命令确认：

   kubectl describe clusterrole system:node | grep -A5 -B5 "nodes/proxy"
   

   若缺失，补充对应规则到system:node ClusterRole，或绑定额外ClusterRole到system:nodes组。

4. 触发kube-apiserver滚动重启
   权限变更后，可通过更新集群配置触发kube-apiserver组件滚动重启，确保新权限生效。

内容的提问来源于stack exchange，提问作者cloudviz