GitLab CI/CD部署Rancher K8s集群时Secrets权限报错求助
我正尝试通过GitLab CI/CD流水线将Web应用部署到Rancher Kubernetes集群,使用的相关.gitlab-ci.yaml配置如下:
.deploy: stage: deploy variables: IMAGE_TAG: $CI_COMMIT_SHA NAMESPACE: webapp-${ENV} script: - > helm upgrade --install -n ${NAMESPACE} --create-namespace webapp ./helm -f helm/values.yaml --wait --set backend.image="${CI_REGISTRY_IMAGE}/backend" --set backend.tag="${IMAGE_TAG}" --set frontend.image="${CI_REGISTRY_IMAGE}/frontend" --set frontend.tag="${IMAGE_TAG}" .helm-apply: stage: deploy image: name: alpine/helm:3.13.3 entrypoint: [""] before_script: - mkdir -p ~/.kube/ - echo "$KUBECONFIG_B64" | base64 -d > ~/.kube/config - chmod 600 -R ~/.kube tags: - docker helm-lint-dev: extends: - .shared-helm-lint needs: - build:branch helm-template-dev: extends: - .shared-helm-template needs: - build:branch deploy-helm-dev: extends: - .dev - .helm-apply - .deploy rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != 'schedule' - when: manual needs: - helm-lint-dev - helm-template-dev
运行流水线时出现如下报错:
Error: query: failed to query with labels: secrets is forbidden: User "system:serviceaccount:gitlab-runner:default" cannot list resource "secrets" in API group "" in the namespace "webapp-dev"
已在GitLab CI/CD变量中配置了包括Base64编码的Kubeconfig文件在内的所有变量,且在本地Minikube环境中测试部署完全正常,推测Helm Chart、Kubeconfig及Rancher集群均无问题。请问该权限错误的原因是什么,如何解决?
原因
报错明确显示GitLab Runner使用的system:serviceaccount:gitlab-runner:default服务账号,在webapp-dev命名空间下没有列出Secrets的权限。Helm执行upgrade --install时需要查询集群内的资源(包括Helm自身存储release信息的Secrets),当前服务账号权限不足导致操作失败。
本地测试正常是因为你本地Kubeconfig对应的账号权限足够,但CI流水线使用的是集群内GitLab Runner的服务账号,两者权限范围完全不同。
解决方法
1. 定义命名空间级权限(推荐)
创建Role来声明部署Web应用所需的所有权限,以下是适配webapp-dev命名空间的配置:
apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: webapp-deploy-role namespace: webapp-dev rules: # 基础资源权限 - apiGroups: [""] resources: ["secrets", "pods", "services", "configmaps"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # 应用资源权限 - apiGroups: ["apps"] resources: ["deployments", "statefulsets"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # 网络资源权限(按需添加) - apiGroups: ["networking.k8s.io"] resources: ["ingresses"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
2. 绑定权限到GitLab Runner服务账号
创建RoleBinding将上面的Role绑定到gitlab-runner命名空间下的default服务账号:
apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: webapp-deploy-binding namespace: webapp-dev subjects: - kind: ServiceAccount name: default namespace: gitlab-runner roleRef: kind: Role name: webapp-deploy-role apiGroup: rbac.authorization.k8s.io
3. 应用配置到集群
将上述两个配置保存为webapp-rbac.yaml,执行以下命令生效:
kubectl apply -f webapp-rbac.yaml
可选:测试环境快速授权(不推荐生产)
如果是测试环境,可直接授予edit集群角色(包含大部分常用资源的读写权限):
kubectl create rolebinding webapp-dev-edit --clusterrole=edit --serviceaccount=gitlab-runner:default --namespace=webapp-dev
内容的提问来源于stack exchange,提问作者DevThiman

