升级Ingress Controller至1.11.5后移除Snippet配置遇403错误求助

问题背景 #

我们将NGINX Ingress Controller从1.7.0版本升级至1.11.5版本。原配置依赖auth-snippet和configuration-snippet注解，新版本出于安全与可维护性考虑建议禁用这些注解，目前临时通过ConfigMap设置allow-snippet-annotations=true过渡。在替换为官方支持的注解与规则过程中，出现OPTIONS请求返回403状态码的问题。

原Ingress配置（使用Snippets） #

ingress:
  enabled: true
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/auth-url: http://auth-api.auth-service-dev.svc.cluster.local/auth
    nginx.ingress.kubernetes.io/auth-method: HEAD
    nginx.ingress.kubernetes.io/auth-response-headers: Authorization
    nginx.ingress.kubernetes.io/auth-snippet: |
      if ( $request_uri ~* "^(/api/docs|/api/public/.*|/api/$)" ) {
        return 200;
      }
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/x-forwarded-prefix: "/api"
    nginx.ingress.kubernetes.io/whitelist-source-range: |
     <Ips>
    nginx.ingress.kubernetes.io/configuration-snippet: |
      if ($http_origin ~* (.*\.dcloud.com)|(https://cloud2-local.test.com:3000)) {
        set $cors "true";
      }
      if ($request_method = 'OPTIONS') {
        set $cors "${cors}options";
      }
      if ($cors = "true") {
        more_set_headers "Access-Control-Allow-Origin: $http_origin";
        more_set_headers "Access-Control-Allow-Credentials: true";
        more_set_headers "Access-Control-Expose-Headers: etag";
      }
      if ($cors = "trueoptions") {
        more_set_headers "Access-Control-Allow-Origin: $http_origin";
        more_set_headers "Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS";
        more_set_headers "Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization,If-Match";
        more_set_headers "Access-Control-Max-Age: 1728000";
        more_set_headers "Content-Type: text/plain charset=UTF-8";
        more_set_headers "Content-Length: 0";
        return 204;
      }
    hosts:
      - host: expo-dev.dev.ciscodcloud.com
        paths:
          - /api/(.*)
ingressInternal:
  enabled: false

修改后Ingress配置（无Snippets） #

ingress:
  enabled: true
  className: nginx
  # Shared annotations for both public and protected Ingresses
  commonAnnotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/rewrite-target: "/$1"
    nginx.ingress.kubernetes.io/x-forwarded-prefix: /api
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
    nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*.cloud.com, dcloud2-local.test.com:3000"
    nginx.ingress.kubernetes.io/cors-expose-headers: "etag"
    nginx.ingress.kubernetes.io/cors-max-age: "1728000"
    nginx.ingress.kubernetes.io/whitelist-source-range: |
      <Ips>
  # Public Ingress (no auth)
  publicAnnotations: {}
  publicHosts:
      - host: expo-dev.dev.ciscodcloud.com
        paths:
          - path: /api/docs
            pathType: Exact
          - path: /api/public/(.*)
            pathType: ImplementationSpecific
          - path: /api
            pathType: Exact
  protectedAnnotations:
    nginx.ingress.kubernetes.io/auth-url: http://auth-api.auth-service-dev.svc.cluster.local/auth
    nginx.ingress.kubernetes.io/auth-method: HEAD
    nginx.ingress.kubernetes.io/auth-response-headers: Authorization
  protectedHosts:
      - host: expo-dev.dev.ciscodcloud.com
        paths:
          - path: /api
            pathType: ImplementationSpecific
ingressInternal:
  enabled: false

Ingress Controller错误日志 #

64.103.40.22 - - [01/May/2025:09:24:14 +0000] "GET /api/public/feature-toggles HTTP/2.0" 200 228 "https://expo-ui-dev.dev.ciscodcloud.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36" 455 0.020 [expo-dev-kapua-expo-api-80] [] 100.96.129.124:8080 239 0.020 200 4e0c976b55f7966953d4976ebdc0c820
64.103.40.22 - - [01/May/2025:09:24:42 +0000] "GET /api/public/feature-toggles HTTP/2.0" 200 228 "https://expo-ui-dev.dev.ciscodcloud.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36" 455 0.009 [expo-dev-kapua-expo-api-80] [] 100.96.141.149:8080 239 0.009 200 95d4609657586726709902e7934d8523
64.103.40.22 - - [01/May/2025:09:24:42 +0000] "OPTIONS /api/expos?sort=start,DESC HTTP/2.0" 403 20 "https://expo-ui-dev.dev.ciscodcloud.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36" 108 0.009 [expo-dev-kapua-expo-api-80] [] 100.96.129.124:8080 31 0.009 403 722424ba0b1a6d1a1a954651435a2e14
{"transaction":{"client_ip":"64.103.40.22","time_stamp":"Thu May  1 09:24:42 2025","server_id":"38a9d8fcec30381e69ef31c13cddc877c2122312","client_port":53804,"host_ip":"100.96.140.183","host_port":443,"unique_id":"174609148230.805598","request":{"method":"OPTIONS","http_version":2.0,"uri":"/api/expos?sort=start,DESC","headers":{"user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36","sec-fetch-site":"same-site","origin":"https://expo-ui-dev.dev.ciscodcloud.com","access-control-request-method":"GET","access-control-request-headers":"authorization","cache-control":"no-cache","accept":"*/*","pragma":"no-cache","referer":"https://expo-ui-dev.dev.ciscodcloud.com/","sec-fetch-dest":"empty","sec-fetch-mode":"cors","host":"expo-dev.dev.ciscodcloud.com","accept-encoding":"gzip, deflate, br, zstd","accept-language":"en-US,en;q=0.9","priority":"u=1, i"}},"response":{"body":"","http_code":403,"headers":{"Vary":"Origin","Vary":"Access-Control-Request-Method","Vary":"Access-Control-Request-Headers","X-Frame-Options":"DENY","Expires":"0","X-Content-Type-Options":"nosniff","Pragma":"no-cache","Cache-Control":"no-cache, no-store, max-age=0, must-revalidate","Strict-Transport-Security":"max-age=31536000; includeSubDomains","X-XSS-Protection":"1; mode=block","Connection":"close","Date":"Thu, 01 May 2025 09:24:42 GMT","Server":"","Server":""}},"producer":{"modsecurity":"ModSecurity v3.0.14 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"DetectionOnly","components":["OWASP_CRS/4.10.0\"]}},"messages":[]}}

问题分析与解决建议 #

核心问题 #

1. OPTIONS预检请求被认证拦截：原配置通过configuration-snippet直接对OPTIONS请求返回204，跳过认证流程；修改后的配置中，受保护路径的Ingress启用了auth-url，但未排除OPTIONS请求，导致预检请求因未携带认证信息被auth服务返回403。
2. CORS配置不完整：修改后的cors-allow-headers缺少原配置中的Authorization、Content-Type、If-Match字段，无法匹配请求中携带的access-control-request-headers；同时cors-allow-origin中的本地域名缺少https://前缀，可能导致origin匹配失败。

修复步骤 #

1. 让OPTIONS请求跳过认证

在protectedAnnotations中添加官方注解，排除OPTIONS请求的认证检查：

protectedAnnotations:
  nginx.ingress.kubernetes.io/auth-url: http://auth-api.auth-service-dev.svc.cluster.local/auth
  nginx.ingress.kubernetes.io/auth-method: HEAD
  nginx.ingress.kubernetes.io/auth-response-headers: Authorization
  nginx.ingress.kubernetes.io/auth-skip-options: "true"  # 新增：OPTIONS请求跳过认证

2. 补全CORS允许的请求头

更新commonAnnotations中的cors-allow-headers，与原配置保持一致：

nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization,If-Match"

3. 修正CORS允许的Origin格式

补全本地域名的https://前缀，确保origin匹配正确：

nginx.ingress.kubernetes.io/cors-allow-origin: "*.cloud.com, https://dcloud2-local.test.com:3000"

4. 验证路径匹配优先级

确认publicHosts中的路径（Exact类型）优先级高于protectedHosts的ImplementationSpecific路径，确保公开路径不会被认证拦截。

验证修复 #

应用修改后的Ingress配置后，重新发送OPTIONS请求，检查是否返回204状态码；同时验证正常业务请求的认证逻辑和CORS响应头是否符合预期。

内容的提问来源于stack exchange，提问作者Gowthamakanthan Gopal