自建Nginx Ingress LoadBalancer无法获取Let's Encrypt证书排查

问题概述 #

自建Nginx Ingress LoadBalancer，尝试通过Let's Encrypt自动签发SSL证书失败，acme挑战请求返回404，推测nginx.conf配置存在问题但未定位到。已创建本地目录/var/www/letsencrypt。

关键日志信息 #

• Nginx访问日志：

"GET /.well-known/acme-challenge/Yop3zwchpFCM-h_cchYjiPwQ0LfINMwy4j0rNugMrmM HTTP/1.1" 404 162 "-" "cert-manager-challenges/v1.14.2 (linux/amd64) cert-manager/306e329365989f205185024a86de9b9d4bad10a5"

• Cert-manager Pod日志：

err="wrong status code '404', expected '200'"

• Nginx错误日志无有效报错，仅显示信号进程启动：

[notice] 3129#3129: signal process started

相关配置文件 #

ClusterIssuer配置 #

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: bayarkhuu.srs@gmail.com
    privateKeySecretRef:
      name: letsencrypt
    solvers:
      - http01:
          ingress:
            class: nginx

Ingress配置 #

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  tls:
    - hosts:
        - longhorn.medsoft.care
      secretName: longhorn.medsoft.care
  rules:
    - host: longhorn.medsoft.care
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: longhorn-frontend
                port:
                  number: 80

自建Nginx的nginx.conf配置 #

# nginx.
http {
    ssl_protocols TLSv1.2 TLSv1.3;

    upstream kubernetes_https {
        server 10.20.30.82:443; # master01
        server 10.20.30.83:443; # master02
        server 10.20.30.84:443; # worker01
        server 10.20.30.85:443; # worker02
        # Add more servers as necessary

        # Health check configuration for Kubernetes API backend
        # tulburtei nginx-d bdag module bololtoi
        # health_check interval=5 fails=3 passes=2 uri=/healthz;
    }

    upstream kubernetes_http {
        server 10.20.30.82:80; # master01
        server 10.20.30.83:80; # master02
        server 10.20.30.84:80; # worker01
        server 10.20.30.85:80; # worker02
        # Add more servers as necessary

        # Health check configuration for Kubernetes node backend
        # tulburtei nginx-d bdag module bololtoi
        # health_check interval=5 fails=3 passes=2 uri=/healthz;
    }

    server {
        listen 443 ssl;

        ssl_certificate /etc/nginx/ssl/nginx.crt;
        ssl_certificate_key /etc/nginx/ssl/nginx.key;

        location ^~ /.well-known/acme-challenge/ {
            allow all;
            default_type "text/plain";
            root /var/www/letsencrypt;
        }

        location / {
            proxy_pass https://kubernetes_https;
            proxy_ssl_verify off;
            proxy_ssl_session_reuse on;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Original-URI $request_uri;
            try_files $uri $uri/index.html $uri.html =404;
        }
    }

    server {
        listen 80;

        location / {
            proxy_pass http://kubernetes_http;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Original-URI $request_uri;
        }
    }
}

问题分析 #

1. 端口映射缺失：acme的http-01挑战默认通过80端口发起，但当前nginx.conf仅在443端口的server块配置了acme挑战路径的本地映射，80端口的请求直接代理到Kubernetes集群内部，导致挑战请求被转发后找不到对应资源，返回404。
2. Ingress Controller不兼容：cert-manager的http01 solver配置依赖官方nginx-ingress controller的Ingress规则处理逻辑，但你使用的是自建Nginx直接代理到Kubernetes API Server，无法识别cert-manager创建的临时Ingress规则，导致挑战请求无法正确路由。

解决方案 #

方案一：调整自建Nginx配置，直接处理80端口的acme挑战 #

修改nginx.conf中80端口的server块，优先处理acme挑战请求：

server {
    listen 80;

    # 优先处理acme挑战请求
    location ^~ /.well-known/acme-challenge/ {
        allow all;
        default_type "text/plain";
        root /var/www/letsencrypt;
    }

    location / {
        proxy_pass http://kubernetes_http;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Original-URI $request_uri;
    }
}

• 确保/var/www/letsencrypt目录在自建Nginx所在服务器上存在，多节点环境需配置共享存储（如NFS），保证cert-manager能将挑战文件写入该目录。
• 重启Nginx服务使配置生效。

方案二：改用官方nginx-ingress controller替换自建Nginx #

官方nginx-ingress controller可自动处理cert-manager的临时Ingress规则，无需手动配置Nginx：

1. 卸载自建Nginx LoadBalancer
2. 安装官方nginx-ingress controller
3. 确保ClusterIssuer和Ingress配置中的ingress.class与controller匹配（默认值为nginx）
4. 删除现有证书Secret后重新触发签发

注意事项 #

• 确保域名longhorn.medsoft.care可正常解析到Ingress LoadBalancer的公网IP
• 检查防火墙/安全组是否开放80、443端口，允许外部访问
• 确认cert-manager Pod运行正常，具备创建临时资源的权限

内容的提问来源于stack exchange，提问作者Bayarkhuu