Crossplane Provider升级故障:仅允许一个OwnerReference设Controller=true
我们在kind集群中通过ArgoCD部署Crossplane环境,Crossplane本身以ArgoCD Application形式安装,Crossplane Provider同样通过ArgoCD Application部署,对应的ArgoCD配置如下:
apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: "provider-aws" namespace: "argocd" labels: crossplane.jonashackt.io: "crossplane" finalizers: - "resources-finalizer.argocd.argoproj.io" spec: project: "default" source: repoURL: "https://github.com/jonashackt/crossplane-argocd" targetRevision: "HEAD" path: "upbound/provider-aws/provider" destination: namespace: "default" server: "https://kubernetes.default.svc" syncPolicy: automated: prune: true retry: limit: 5 backoff: duration: "5s" factor: 2 maxDuration: "1m"
对应的Crossplane Provider定义如下:
apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: "provider-aws-s3" spec: package: "xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.1" packagePullPolicy: "Always" revisionActivationPolicy: "Automatic" revisionHistoryLimit: 1
当新版本Crossplane Provider provider-aws-ec2:v1.2.1发布后,Provider进入Degraded状态,事件报错信息:
cannot apply package revision: cannot create object: ProviderRevision.pkg.crossplane.io "provider-aws-ec2-150095bdd614" is invalid: metadata.ownerReferences: Invalid value: []v1.OwnerReference{v1.OwnerReference{APIVersion:"pkg.crossplane.io/v1", Kind:"Provider", Name:"provider-aws-ec2", UID:"30bda236-6c12-412c-a647-b96368eff8b6", Controller:(*bool)(0xc02afeb38c), BlockOwnerDeletion:(*bool)(0xc02afeb38d)}, v1.OwnerReference{APIVersion:"pkg.crossplane.io/v1", Kind:"Provider", Name:"provider-aws-ec2", UID:"ee890f53-7590-4957-8f81-e92b931c4e8d", Controller:(*bool)(0xc02afeb38e), BlockOwnerDeletion:(*bool)(0xc02afeb38f)}}: Only one reference can have Controller set to true. Found "true" in references for Provider/provider-aws-ec2 and Provider/provider-aws-ec2
执行kubectl get providerrevisions可见新版本自动安装,旧版本健康状态异常:
kubectl get providerrevisions NAME HEALTHY REVISION IMAGE STATE DEP-FOUND DEP-INSTALLED AGE provider-aws-ec2-3d66ea2d7903 Unknown 1 xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.1 Active 1 1 5m31s provider-aws-ec2-3d66ea2d7903 Unknown 1 xpkg.upbound.io/upbound/provider-aws-ec2:v1.2.1 Active 1 1 5m31s upbound-provider-family-aws-7cc64a779806 True 1 xpkg.upbound.io/upbound/provider-family-aws:v1.2.1 Active 30m
要避免Provider升级时破坏现有部署,可从以下几个方面调整配置:
1. 禁用自动版本激活,改为手动控制升级
将Provider资源的revisionActivationPolicy从Automatic改为Manual,新版本Provider发布后不会自动激活,需手动确认后再切换激活状态:
apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: "provider-aws-s3" spec: package: "xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.1" packagePullPolicy: "Always" revisionActivationPolicy: "Manual" # 修改为手动激活 revisionHistoryLimit: 1
后续需要升级时,先通过kubectl get providerrevisions确认新版本已拉取,再执行命令激活:
kubectl patch providerrevision <新版本ProviderRevision名称> -p '{"spec":{"active":true}}' --type=merge
2. 固定Provider版本,避免自动拉取最新版
将Provider的package字段指定固定版本,同时将packagePullPolicy改为IfNotPresent,防止Crossplane自动拉取未指定的新版本:
apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: "provider-aws-s3" spec: package: "xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.1" # 固定版本号 packagePullPolicy: "IfNotPresent" # 仅本地无镜像时拉取 revisionActivationPolicy: "Automatic" revisionHistoryLimit: 1
需要升级时,手动修改package字段的版本号,再通过ArgoCD同步即可。
3. 调整ArgoCD同步策略,避免意外替换资源
修改ArgoCD Application的syncPolicy,添加资源保留配置并关闭自修复,防止ArgoCD强制覆盖手动调整的状态:
spec: syncPolicy: automated: prune: true selfHeal: false # 关闭自修复 preserveResourcesOnDeletion: true retry: limit: 5 backoff: duration: "5s" factor: 2 maxDuration: "1m"
4. 临时修复当前Degraded状态
针对已出现的冲突问题,先清理旧Provider资源再重新同步:
# 删除冲突的旧Provider资源(操作前请备份配置) kubectl delete provider provider-aws-ec2 # 重新同步ArgoCD Provider应用 kubectl apply -f <你的Provider ArgoCD Application配置文件路径>
内容的提问来源于stack exchange,提问作者jonashackt

