Istio集群:内部mTLS正常,外部Let's Encrypt证书HTTPS访问失败
Istio 内部mTLS+外部Let's Encrypt证书访问问题排查与解决
环境与目标
- 核心需求:Kubernetes集群中,Istio实现内部服务自动mTLS通信,外部通过Let's Encrypt证书HTTPS访问
- 已生效基础配置:
istio-system命名空间配置严格模式的PeerAuthentication,内部服务通信正常- 所有服务Pod已添加
sidecar.istio.io/inject: "true"标签
已配置的Ingress资源
1. Ingress Gateway(istio-ingress命名空间)
apiVersion: networking.istio.io/v1 kind: Gateway metadata: name: gateway namespace: istio-ingress spec: selector: istio: gateway servers: - port: name: http number: 80 protocol: HTTP hosts: - "*.customer.ocs.nu" tls: httpsRedirect: true - port: name: https number: 443 protocol: HTTPS hosts: - "*.customer.ocs.nu" tls: credentialName: "istio-ingress/star-customer-ocs-nu-crt" mode: SIMPLE
2. VirtualService(customer-application命名空间)
apiVersion: networking.istio.io/v1 kind: VirtualService metadata: name: application namespace: customer-application spec: gateways: - istio-ingress/gateway - mesh hosts: - application.customer.ocs.nu http: - match: - uri: prefix: / route: - destination: host: application.customer-application.svc.cluster.local port: number: 8000
当前问题现象
HTTP访问正常,自动重定向到HTTPS:
# curl -kv http://application.customer.ocs.nu/ * Host application.customer.ocs.nu:80 was resolved. * IPv6: (none) * IPv4: IP * Trying IP:80... * Connected to application.customer.ocs.nu (IP) port 80 > GET / HTTP/1.1 > Host: application.customer.ocs.nu > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off < HTTP/1.1 301 Moved Permanently < location: https://application.customer.ocs.nu/ < date: Fri, 31 Jan 2025 11:39:09 GMT < server: istio-envoy < content-length: 0对应日志:
[2025-01-31T11:39:09.561Z] "GET / HTTP/1.1" 301 - direct_response - "-" 0 0 0 - "10.244.0.165" "curl/8.7.1" "b05ac233-eea7-46d6-9fee-e9f9c9cf8bb8" "application.customer.ocs.nu" "-" - - 10.244.0.200:80 10.244.0.165:22533 - -HTTPS访问失败,无日志输出:
# curl -kv https://application.customer.ocs.nu/ * Host application.customer.ocs.nu:443 was resolved. * IPv6: (none) * IPv4: IP * Trying IP:443... * Connected to application.customer.ocs.nu (IP) port 443 * ALPN: curl offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to application.customer.ocs.nu:443 * Closing connection curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to application.customer.ocs.nu:443HTTP访问443端口返回空响应,日志显示
filter_chain_not_found:# curl -kv http://application.customer.ocs.nu:443/ * Host application.customer.ocs.nu:443 was resolved. * IPv6: (none) * IPv4: IP * Trying IP:443... * Connected to application.customer.ocs.nu (IP) port 443 > GET / HTTP/1.1 > Host: application.customer.ocs.nu:443 > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off * Empty reply from server * Closing connection curl: (52) Empty reply from server对应日志:
[2025-01-31T11:43:01.227Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.244.0.200:443 10.244.0.165:44729 - -仅使用HTTP配置时,所有访问正常。
已完成的排查步骤
istioctl analyze -A未发现重大问题,仅提示部分网格外服务名称非法、部分命名空间无注入注解。- 初始检查网关Pod证书状态:Let's Encrypt证书处于
WARMING状态且无效:# istioctl pc secret istio-gateway-76676d4954-l5498.istio-ingress RESOURCE NAME TYPE STATUS VALID CERT SERIAL NUMBER NOT AFTER NOT BEFORE kubernetes://istio-ingress/star-customer-ocs-nu-crt WARMING false default Cert Chain ACTIVE true 12c998930e47b4c9df3f5ae259fb1a92 2025-02-01T03:04:23Z 2025-01-31T03:02:23Z ROOTCA CA ACTIVE true c6b587095c06abdabc53c84b1af924d3 2035-01-18T12:59:47Z 2025-01-20T12:59:47Z - 确认Certbot DNS验证颁发的证书已就绪:
# kubectl get certificate -n istio-ingress NAME READY SECRET AGE star-customer-ocs-nu True star-customer-ocs-nu-crt 23h # kubectl get certificaterequest -n istio-ingress NAME APPROVED DENIED READY ISSUER REQUESTER AGE star-customer-ocs-nu-1 True True letsencrypt-prod system:serviceaccount:default:cert-manager 23h - 修改Gateway的
credentialName,移除命名空间前缀(从istio-ingress/star-customer-ocs-nu-crt改为star-customer-ocs-nu-crt),重启网关Pod后证书状态变为ACTIVE且有效,但HTTPS访问问题仍存在:# istioctl pc secret istio-gateway-76676d4954-8hhjl.istio-ingress RESOURCE NAME TYPE STATUS VALID CERT SERIAL NUMBER NOT AFTER NOT BEFORE default Cert Chain ACTIVE true 8101cda2556b2fd7c31872f9d013d72f 2025-02-01T12:31:02Z 2025-01-31T12:29:02Z kubernetes://star-customer-ocs-nu-crt Cert Chain ACTIVE true 4c8a2f7ccab5ff0c7aa61dd2a46aa9bef0b 2025-04-30T11:56:26Z 2025-01-30T11:56:27Z ROOTCA CA ACTIVE true c6b587095c06abdabc53c84b1af924d3 2035-01-18T12:59:47Z 2025-01-20T12:59:47Z
解决方案
1. 校验证书与Gateway Hosts匹配
检查证书的SAN(主题备用名称)是否包含*.customer.ocs.nu或application.customer.ocs.nu:
kubectl get secret star-customer-ocs-nu-crt -n istio-ingress -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
若不匹配,重新申请包含对应域名的证书。
2. 补充完整证书链
Let's Encrypt证书需要包含中间链才能被浏览器信任,更新Secret:
# 下载Let's Encrypt中间证书 curl -o letsencrypt-ca.pem https://letsencrypt.org/certs/lets-encrypt-r3.pem # 合并证书(将中间证书追加到主证书后) cat star-customer-ocs-nu.crt letsencrypt-ca.pem > fullchain.crt # 更新Secret kubectl create secret tls star-customer-ocs-nu-crt --cert=fullchain.crt --key=star-customer-ocs-nu.key -n istio-ingress --dry-run=client -o yaml | kubectl apply -f -
3. 为网关命名空间配置宽松模式PeerAuthentication
全局严格mTLS会强制网关与内部服务通信使用mTLS,但需要允许外部无mTLS访问网关。在istio-ingress命名空间添加PeerAuthentication:
apiVersion: security.istio.io/v1 kind: PeerAuthentication metadata: name: gateway-peer-auth namespace: istio-ingress spec: mtls: mode: PERMISSIVE
4. 检查网关网络配置
- 确认网关Service的443端口已正确映射到Pod的443端口:
kubectl get svc istio-gateway -n istio-ingress - 检查是否有网络策略阻止外部访问443端口,或阻止网关Pod读取证书Secret:
kubectl get networkpolicy -n istio-ingress
5. 验证网关FilterChain配置
确认HTTPS端口的FilterChain已正确关联证书:
istioctl pc listener istio-gateway-<pod-name>.istio-ingress -o json | jq '.[] | select(.address.portValue == 443)'
检查输出中是否存在匹配*.customer.ocs.nu的FilterChain,且transportSocket配置指向正确的证书Secret。
验证
完成上述调整后,重新测试HTTPS访问:
curl -kv https://application.customer.ocs.nu/
若成功,会返回应用响应内容,同时内部服务间的mTLS通信保持正常。
内容的提问来源于stack exchange,提问作者klafbang
相关产品推荐
相关产品推荐

