Nginx配置resolver+proxy_pass+变量后出现404问题排查

问题背景 #

为避免上游DNS未就绪时Nginx服务崩溃，通过set指令引用环境变量FOO_SERVER_SOCKET_IO_URL并添加resolver配置后，Nginx能正常启动，但访问/foo-server/socket.io路径时返回404错误。

原配置（环境变量未定义时会崩溃） #

server {
  listen 80;

  gzip on;
  gzip_disable "msie6";

  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_min_length 0;
  gzip_types text/plain application/javascript text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype;

  location / {
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    try_files $uri $uri/ /index.html;
  }

  location /foo-server/socket.io {
    proxy_pass ${FOO_SERVER_SOCKET_IO_URL}/socket.io;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }

  ...

  error_page 500 502 503 504 /50x.html;

  location = /50x.html {
    root  /usr/share/nginx/html;
  }
}

修改后的配置（启动正常但返回404） #

server {
  listen 80;
  resolver 127.0.0.11 valid=60s ipv6=off;
  resolver_timeout 10s;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_min_length 0;
  gzip_types text/plain application/javascript text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype;

location / {
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    try_files $uri $uri/ /index.html;
  }

  location /foo-server/socket.io {
    set $foo_server_socket_io $FOO_SERVER_SOCKET_IO_URL;
    proxy_pass $foo_server_socket_io/socket.io;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }

  ...

  error_page 500 502 503 504 /50x.html;

  location = /50x.html {
    root  /usr/share/nginx/html;
  }
}

相关Nginx日志 #

2024-02-13 16:05:35 192.168.65.1 - - [13/Feb/2024:15:05:35 +0000] "GET /foo-server/socket.io/?EIO=4&transport=polling&t=OsZ086l HTTP/1.1" 404 147 "http://localhost:3000/foo-path" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" "-"

原因分析 #

当proxy_pass使用变量时，Nginx不会自动截断当前location的匹配路径，而是会把完整的请求URI（包括/foo-server/socket.io）拼接到代理地址后面。

举个例子：如果FOO_SERVER_SOCKET_IO_URL是http://foo.example.com，原配置中proxy_pass ${FOO_SERVER_SOCKET_IO_URL}/socket.io会把请求代理到http://foo.example.com/socket.io/?xxx；但修改后用变量的情况下，实际代理的地址会变成http://foo.example.com/socket.io/foo-server/socket.io/?xxx，上游服务器找不到该路径，因此返回404。

解决方案 #

需要手动剔除location匹配的前缀，调整proxy_pass的写法，推荐修改location块如下：

location /foo-server/socket.io {
  set $foo_server_socket_io $FOO_SERVER_SOCKET_IO_URL;
  # 剔除/foo-server前缀，只保留/socket.io及后续路径
  rewrite ^/foo-server(.*)$ $1 break;
  proxy_pass $foo_server_socket_io;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection 'upgrade';
  proxy_set_header Host $host;
  proxy_cache_bypass $http_upgrade;
}

或者直接构造正确的代理路径（注意FOO_SERVER_SOCKET_IO_URL末尾不要带/，避免出现重复的//）：

location /foo-server/socket.io {
  set $foo_server_socket_io $FOO_SERVER_SOCKET_IO_URL;
  proxy_pass $foo_server_socket_io$request_uri;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection 'upgrade';
  proxy_set_header Host $host;
  proxy_cache_bypass $http_upgrade;
}

内容的提问来源于stack exchange，提问作者Balint Laszlo