AKS使用AGIC作为Ingress无变更出现权限与资源NotFound错误求助

问题重现 #

E0109 08:27:56.481514 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.20.0-beta.1/tools/cache/reflector.go:167: Failed to watch *v1beta1.AzureApplicationGatewayRewrite: failed to list *v1beta1.AzureApplicationGatewayRewrite: azureapplicationgatewayrewrites.appgw.ingress.azure.io is forbidden: User "system:serviceaccount:kube-system:ingress-appgw-sa" cannot list resource "azureapplicationgatewayrewrites" in API group "appgw.ingress.azure.io" at the cluster scope: Azure does not have opinion for this user.
E0109 09:38:27.684411 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.20.0-beta.1/tools/cache/reflector.go:167: Failed to watch *v1beta1.AzureApplicationGatewayRewrite: failed to list *v1beta1.AzureApplicationGatewayRewrite: the server could not find the requested resource (get azureapplicationgatewayrewrites.appgw.ingress.azure.io)

已尝试删除并重建AGIC Pod，但错误仍存在。

解决步骤 #

1. 验证CRD是否存在并兼容 #

• 先检查集群中是否存在目标CRD：

  kubectl get crd azureapplicationgatewayrewrites.appgw.ingress.azure.io
  

• 如果返回NotFound，说明CRD未安装或版本不匹配。获取AGIC官方发布的对应v1beta1版本CRD文件，执行以下命令安装：

  kubectl apply -f <path-to-crd-file>
  

2. 修复Service Account的集群权限 #

• 检查当前AGIC使用的ClusterRole是否包含azureapplicationgatewayrewrites资源的权限：

  kubectl describe clusterrole ingress-appgw-role
  

• 如果权限缺失，创建或更新ClusterRole：
  保存以下内容为agic-clusterrole.yaml：

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: ingress-appgw-role
  rules:
  - apiGroups: ["appgw.ingress.azure.io"]
    resources: ["azureapplicationgatewayrewrites"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  

  执行更新：

  kubectl apply -f agic-clusterrole.yaml
  

• 确认ClusterRoleBinding已正确绑定到ingress-appgw-sa：

  kubectl describe clusterrolebinding ingress-appgw-crb
  

  如果绑定不存在，创建agic-clusterrolebinding.yaml：

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: ingress-appgw-crb
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: ingress-appgw-role
  subjects:
  - kind: ServiceAccount
    name: ingress-appgw-sa
    namespace: kube-system
  

  执行安装：

  kubectl apply -f agic-clusterrolebinding.yaml
  

3. 重启AGIC部署 #

• 仅删除Pod无法确保加载更新后的CRD和权限，需重启整个部署：

  kubectl rollout restart deployment ingress-appgw -n kube-system
  

• 验证Pod状态：

  kubectl get pods -n kube-system -l app=ingress-appgw
  

内容的提问来源于stack exchange，提问作者Vikram