配置带SSL的NGINX连接.NET gRPC服务器遇502错误排查

我运行着一个.NET 8 gRPC服务器，处于测试阶段，配置为同时在HTTP端口5000和HTTPS端口12345接收gRPC请求，本地运行一切正常。但将服务器与NGINX部署在Docker容器中后，仅HTTP端口可用，测试HTTPS端口时出现502 Bad Gateway错误，本地和远程访问的错误日志有所不同：

本地访问错误日志 #

2024/11/29 01:31:27 [error] 30#30: *1 upstream prematurely closed connection while reading response header from upstream, client: 172.19.0.1, server: grpcgateway, request: "POST /xxxx/Register HTTP/2.0", upstream: "grpc://172.19.0.4:12345", host: "localhost:12345
172.19.0.1 - - [29/Nov/2024:01:31:27 +0000] "POST /xxxx/Register HTTP/2.0" 502 157 "-" "grpc-node-js/1.8.10"

远程访问错误日志 #

2024/11/29 02:02:43 [error] 33#33: *31 upstream prematurely closed connection while reading response header from upstream, client: 172.25.88.146, server: grpcgateway, request: "POST /xxxx/Register HTTP/2.0", upstream: "grpc://172.19.0.4:12345", host: "172.25.88.91:12345"
172.25.88.146 - - [29/Nov/2024:02:02:43 +0000] "POST /xxxx/Register HTTP/2.0" 502 157 "-" "grpc-node-js/1.11.0-postman.1"
172.25.88.146 - - [29/Nov/2024:02:02:44 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xB23'iA\xF6\x98FxqZ\xD1\xC8xo\xD0\xA8\x91\xCC\xF4\xB7\xA9\xE4\x85\xE9\xF9L\xF6x\xCD\x0C, \x866\xC7\xC2\x93\xA7\xA7\xD9\xB4\xA8\xD3\xD1\x0E\x85R2\x5Cc\xA7\x89j\xAB\xA1h\xD9\xF7\xB6\xF4\xE8\xDE=\xAF\x00$\x13\x01\x13\x02\x13\x03\xC0/\xC0+\xC00\xC0,\xC0'\xCC\xA9\xCC\xA8\xC0\x09\xC0\x13\xC0" 400 157 "-" "-"
172.25.88.146 - - [29/Nov/2024:02:02:44 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03" 400 157 "-" "-"

当前配置 #

.NET服务器配置 #

var serverCert = X509Certificate2.CreateFromPemFile("Credentials/nginx.crt", "Credentials/nginx.key");
option.ListenAnyIP(5000, o =>
{
    o.Protocols = Microsoft.AspNetCore.Server.Kestrel.Core.HttpProtocols.Http2;
});
option.ListenAnyIP(12345, o => o.UseHttps(httpsOptions =>
    {
        httpsOptions.ServerCertificate = serverCert;
    }
));

NGINX配置 #

worker_processes auto;
 
events {
    worker_connections 1024;
}
 
http {
    include       mime.types;
    default_type  application/octet-stream;
    
    ssl_certificate      /etc/nginx/nginx.crt;
    ssl_certificate_key  /etc/nginx/nginx.key;
    
    server {
        listen 12345 ssl http2;
        server_name grpcgateway;
        default_type application/grpc;

        ssl_session_cache builtin:1000 shared:SSL:10m;
        ssl_session_timeout  5m;

        location / {
           grpc_set_header X-Forwarded-For $remote_addr;
           grpc_pass grpc://abc-server:12345;
        }
    }

    server {
        listen 5000 http2;
        server_name grpcgateway;
        default_type application/grpc;

        location / {
           grpc_set_header X-Forwarded-For $remote_addr;
           grpc_pass grpc://abc-server:5000;
        }
    }
}

附：证书不重要，测试阶段甚至希望绕过证书。

核心问题 #

NGINX配置中，HTTPS端口的grpc_pass使用了grpc://协议，但后端.NET服务器的12345端口是HTTPS加密端口，NGINX需要用grpcs://协议才能与加密的gRPC后端通信。当前用grpc://直接连接HTTPS端口，会导致连接握手失败，触发upstream prematurely closed connection错误。

远程日志中的乱码是客户端直接向NGINX的HTTPS端口发送了TLS握手请求（而非HTTP/2请求），虽不是主因，但也说明请求路径的TLS层处理需要匹配。

修复步骤 #

1. 修改NGINX的HTTPS服务器配置 #

将HTTPS服务器块中的grpc_pass协议从grpc://改为grpcs://：

server {
    listen 12345 ssl http2;
    server_name grpcgateway;
    default_type application/grpc;

    ssl_session_cache builtin:1000 shared:SSL:10m;
    ssl_session_timeout  5m;

    location / {
       grpc_set_header X-Forwarded-For $remote_addr;
       # 替换为grpcs://，匹配后端的HTTPS端口
       grpc_pass grpcs://abc-server:12345;
    }
}

2. 测试阶段绕过证书验证（可选） #

如果需要绕过证书校验（仅测试环境使用），在NGINX的HTTPS服务器块中添加证书关闭验证配置：

location / {
   grpc_set_header X-Forwarded-For $remote_addr;
   grpc_pass grpcs://abc-server:12345;
   # 关闭证书验证
   grpc_ssl_verify off;
   # 可选：指定信任的证书文件
   grpc_ssl_trust_certificate /etc/nginx/nginx.crt;
}

3. 验证.NET服务器的证书可用性 #

确保Docker容器中.NET服务器能正确读取到证书文件Credentials/nginx.crt和Credentials/nginx.key，路径权限无问题，避免因证书加载失败导致后端端口无法正常提供服务。

额外说明 #

• HTTP端口配置是正确的：后端5000端口是明文HTTP/2，NGINX用grpc://连接没问题，所以可以正常工作。
• 远程日志中的400错误是客户端直接发送TLS握手包到NGINX的HTTPS端口，但NGINX已经处理了TLS层，后续请求应为HTTP/2格式，修复主问题后即可正常访问。

内容的提问来源于stack exchange，提问作者Po-Sen