Traefik IngressRouteTCP无法处理HTTP2的问题排查求助

我在Azure Kubernetes Service（AKS）集群（部署于Azure负载均衡器后）配置了Traefik的IngressRouteTCP，目标是基于SNI而非主机头进行路由，证书由Cloudflare为test.example.com生成。应用本身支持HTTP2，但当前配置下HTTP2无法正常工作——只有将TLSOption的alpnProtocols设置为http/1.1时才能正常运行。我不确定故障出在Traefik、curl还是应用本身，使用以下命令测试时出现HTTP2错误：

curl -svk --connect-to test.example.com:443:my-azure-load-balancer.cloudapp.azure.com:443 https://test.example.com

Traefik版本为2.9.1。

测试日志 #

* Connecting to hostname: my-azure-load-balancer.cloudapp.azure.com
* Connecting to port: 443
*   Trying x.x.x.x:443...
* TCP_NODELAY set
* Connected to my-azure-load-balancer.cloudapp.azure.com (x.x.x.x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: O=CloudFlare, Inc.; OU=CloudFlare Origin CA; CN=CloudFlare Origin Certificate
*  start date: Sep 20 12:40:00 2022 GMT
*  expire date: Sep 16 12:40:00 2037 GMT
*  issuer: C=US; O=CloudFlare, Inc.; OU=CloudFlare Origin SSL Certificate Authority; L=San Francisco; ST=California
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x558d30129860)
> GET / HTTP/2
> Host: test.example.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* http2 error: Remote peer returned unexpected data while we expected SETTINGS frame.  Perhaps, peer does not support HTTP/2 properly.
* Connection #0 to host my-azure-load-balancer.cloudapp.azure.com left intact

Kubernetes配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
spec:
  replicas: 2
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.6.0
          imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
    - port: 80
      name: whoami
  selector:
    app: whoami
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: whoami
spec:
  entryPoints:
    - websecure
  routes:
    - match: HostSNI(`test.example.com`)
      services:
        - name: whoami
          port: 80
  tls:
    secretName: cloudflare-cert

Traefik日志 #

traefik-c757597b9-2xv65 time="2022-10-31T08:52:59Z" level=debug msg="Handling connection from 10.9.3.227:61988 to 10.9.3.73:80"
traefik-c757597b9-2xv65 time="2022-10-31T08:52:59Z" level=debug msg="Error during connection: read tcp 10.9.3.58:34400->10.9.3.73:80: read: connection reset by peer"

相关IP说明 #

• 10.9.3.227：Kubernetes节点
• 10.9.3.58：Traefik实例
• 10.9.3.73：whoami服务的ClusterIP

排查结论与修复方案 #

1. 核心问题定位
   从日志看，Traefik和客户端协商成功使用HTTP2（ALPN返回h2），但Traefik将HTTP2流量直接转发给了仅支持HTTP1.1的后端服务（whoami v1.6.0默认只支持HTTP1.1），导致后端重置连接，进而引发客户端的HTTP2错误。

2. 修复步骤

   • 方案一：升级后端服务支持HTTP2
     将whoami镜像版本升级到traefik/whoami:v1.8.0及以上，新版本默认支持HTTP2，无需额外配置即可适配HTTP2流量。
   • 方案二：配置Traefik降级HTTP2到HTTP1.1转发
     若无法升级后端，创建TLSOption并在IngressRouteTCP中引用，让Traefik将客户端的HTTP2请求转为HTTP1.1转发给后端：

     apiVersion: traefik.containo.us/v1alpha1
     kind: TLSOption
     metadata:
       name: http2-to-http1
     spec:
       alpnProtocols:
         - h2
         - http/1.1
       minVersion: VersionTLS12
     

     修改后的IngressRouteTCP：

     apiVersion: traefik.containo.us/v1alpha1
     kind: IngressRouteTCP
     metadata:
       name: whoami
     spec:
       entryPoints:
         - websecure
       routes:
         - match: HostSNI(`test.example.com`)
           services:
             - name: whoami
               port: 80
       tls:
         secretName: cloudflare-cert
         options:
           name: http2-to-http1
     

   • 验证：重新部署配置后，再次执行测试命令，确认HTTP2连接正常建立。

3. 额外说明
   之前设置alpnProtocols为http/1.1能工作，是因为客户端和Traefik协商使用HTTP1.1，Traefik转发HTTP1.1流量给后端，后端能正常处理。而默认情况下Traefik会将HTTP2流量直接透传给后端，若后端不支持则会触发连接重置。

内容的提问来源于stack exchange，提问作者Joel