K8S Ingress API迁移后Helm升级报错：找不到extensions/v1beta1资源

问题描述 #

正在将Kubernetes Ingress的API版本从extensions/v1beta1迁移至networking.k8s.io/v1，迁移前后的Helm模板如下：

迁移前的helm/templates/ingress.yaml #

{{- if .Values.ingress.enabled -}}
{{- $serviceName := include "sonarqube.fullname" . -}}
{{- $servicePort := .Values.service.externalPort -}}
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ template "sonarqube.fullname" . }}
  labels:
    app: {{ template "sonarqube.name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.ingress.labels }}
{{ .Values.ingress.labels | toYaml | trimSuffix "\n"| indent 4 -}}
{{- end}}
{{- if .Values.ingress.annotations}}
  annotations:
    {{- range $key, $value := .Values.ingress.annotations }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
{{- end }}
spec:
  rules:
    {{- range .Values.ingress.hosts }}
    {{- $path := default "/" .path }}
    - host: {{ .name }}
      http:
        paths:
          - path: {{ $path }}
            backend:
              serviceName: {{ $serviceName }}
              servicePort: {{ $servicePort }}
          - path: {{ printf "%s/*" (trimSuffix "/" $path) }}
            backend:
              serviceName: {{ $serviceName }}
              servicePort: {{ $servicePort }}

迁移后的helm/templates/ingress.yaml #

{{- if .Values.ingress.enabled -}}
{{- $serviceName := include "sonarqube.fullname" . -}}
{{- $servicePort := .Values.service.externalPort -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ template "sonarqube.fullname" . }}
  labels:
    app: {{ template "sonarqube.name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.ingress.labels }}
{{ .Values.ingress.labels | toYaml | trimSuffix "\n"| indent 4 -}}
{{- end}}
{{- if .Values.ingress.annotations}}
  annotations:
    {{- range $key, $value := .Values.ingress.annotations }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
{{- end }}
spec:
  rules:
    {{- range .Values.ingress.hosts }}
    {{- $path := default "/" .path }}
    - host: {{ .name }}
      http:
        paths:
          - backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
            path: {{ $path }}
            pathType: ImplementationSpecific
          - backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
            path: {{ printf "%s/*" (trimSuffix "/" $path) }}
            pathType: ImplementationSpecific
    {{- end -}}
  {{- if .Values.ingress.tls }}
  tls:
{{ toYaml .Values.ingress.tls | indent 4 }}
  {{- end -}}
{{- end -}}

执行Helm升级时出现报错：

Error: UPGRADE FAILED: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "myname-myname" namespace: "" from "": no matches for kind "Ingress" in version "extensions/v1beta1" ensure CRDs are installed first

但集群中运行的Ingress实际已为networking.k8s.io/v1版本：

abhi@mySystem:$ kubectl get ing myname-myname -n sonarqube -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    meta.helm.sh/release-name: sonarqube
    meta.helm.sh/release-namespace: sonarqube
    nginx.ingress.kubernetes.io/proxy-body-size: 200m
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.org/client-max-body-size: 200m
  creationTimestamp: "2021-11-16T14:34:25Z"
  generation: 1
  labels:
    app: sonarqube
    app.kubernetes.io/managed-by: Helm
    chart: sonarqube-3.10.5
    heritage: Helm
    release: sonarqube
  name: myname-myname
  namespace: sonarqube
  resourceVersion: "111111111"
  uid: 11a1a11a-1111-11aa-a11a-aa1a111aa111
spec:
  rules:
  - host: sonarqube-dev.ad.xyz.com
    http:
      paths:
      - backend:
          service:
            name: myname-myname
            port:
              number: 9000
        path: /
        pathType: ImplementationSpecific
      - backend:
          service:
            name: myname-myname
            port:
              number: 9000
        path: /*
        pathType: ImplementationSpecific
status:
  loadBalancer:
    ingress:
    - ip: 10.110.110.110

环境信息：Helm 3.5.1 | Kubectl客户端v1.25.3 | Kubectl服务端v1.23.5

报错原因 #

Helm执行升级时，会读取存储在集群etcd中的旧版本release manifest（其中Ingress的API版本仍是extensions/v1beta1），并尝试与新模板资源对比。由于K8s 1.22+已废弃extensions/v1beta1版本的Ingress API，Helm无法解析旧manifest中的资源，从而抛出错误。即使实际集群中的Ingress资源已升级为networking.k8s.io/v1，Helm的内部release记录仍保留旧API版本信息。

解决方法 #

方法1：更新Helm的release manifest记录 #

1. 导出当前release的manifest文件：

   helm get manifest sonarqube -n sonarqube > old-manifest.yaml
   

2. 编辑old-manifest.yaml，将所有apiVersion: extensions/v1beta1替换为apiVersion: networking.k8s.io/v1
3. 使用修改后的manifest强制升级，覆盖旧记录：

   # 先执行预检查验证
   helm upgrade sonarqube ./your-chart-directory -n sonarqube --force --reuse-values --dry-run
   # 预检查通过后执行实际升级
   helm upgrade sonarqube ./your-chart-directory -n sonarqube --force --reuse-values
   

方法2：升级Helm版本 #

Helm 3.5.1对K8s新API版本的兼容性有限，建议升级到3.8及以上版本，新版本能更好地处理API版本迁移场景，避免此类解析错误。

方法3：删除旧release后重新安装（谨慎操作） #

若上述方法无效，可考虑先卸载旧release再重新安装新版本（注意备份数据，避免业务中断）：

helm uninstall sonarqube -n sonarqube
helm install sonarqube ./your-chart-directory -n sonarqube -f your-values.yaml

内容的提问来源于stack exchange，提问作者Abhinav Kumar