向CKAN上传超2GB文件时出现Nginx 502错误的求助

问题背景 #

• CKAN源码部署的生产环境，通过API或Web界面上传文件时，2.5GB以内的文件可正常上传，但上传4GB文件时Nginx返回502 Bad Gateway错误
• 暂初步排除Nginx为问题根源（临时将Nginx代理到自建测试应用可成功传输大文件，但未100%确认）

已修改的配置 #

CKAN主配置（ckan.ini） #

ckan.max_resource_size = 100000

相关错误日志 #

Nginx日志（nginx.log） #

2023/02/28 09:58:51 [error] 27692#27692: *1684 upstream prematurely closed connection while reading response header from upstream, client: 111.111.111.111, server: my-domain-name.de, request: "POST /dataset/test-data/resource/new HTTP/1.1", upstream: "http://127.0.0.1:8080/dataset/test-data/resource/new", host: "my-domain-name.de", referrer: "https://my-domain-name.de/dataset/test-data/resource/new"

uWSGI日志（uwsgi.stderr.log） #

Tue Feb 28 09:58:50 2023 - *** HARAKIRI ON WORKER 1 (pid: 25434, try: 1) ***
Tue Feb 28 09:58:50 2023 - HARAKIRI !!! worker 1 status !!!
Tue Feb 28 09:58:50 2023 - HARAKIRI [core 0] 127.0.0.1 - POST /dataset/test-data/resource/new since 1677578279
Tue Feb 28 09:58:50 2023 - HARAKIRI !!! end of worker 1 status !!!
DAMN ! worker 1 (pid: 25434) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 28408)

当前Nginx完整配置 #

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m max_size=500m;
proxy_temp_path /tmp/nginx_proxy 1 2;

server {
    client_max_body_size 10000M;
    client_header_timeout 10080;
    client_body_timeout 10080;

    server_name my-domain-name.de;
    location / {
        proxy_pass http://127.0.0.1:8080/;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_cache cache;
        proxy_cache_bypass $cookie_auth_tkt;
        proxy_no_cache $cookie_auth_tkt;
        proxy_cache_valid 30m;
        proxy_cache_key $host$scheme$proxy_host$request_uri;

    proxy_connect_timeout 10080s;
    proxy_send_timeout 10080;
    proxy_read_timeout 10080;
    uwsgi_read_timeout 20m;
    proxy_request_buffering off;
    proxy_buffering off;

        # In emergency comment out line to force caching
        # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/my-domain-name.de/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/my-domain-name.de/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = my-domain-name.de) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    server_name my-domain-name.de;
    return 404; # managed by Certbot
}

已尝试的无效操作 #

• 增大ckan-uwsgi.ini中的harakiri超时值：uWSGI日志不再报错，但Nginx 502错误依旧
• 关闭Nginx的post_buffering配置：问题未解决

解决思路建议 #

1. 排查uWSGI进程资源限制：uWSGI worker被signal 9杀死大概率是系统OOM Killer触发，需检查：

   • /etc/security/limits.conf中uWSGI运行用户的内存限制设置
   • uWSGI配置中的limit-as参数，确保分配足够内存处理大文件上传
   • 系统日志（/var/log/syslog或/var/log/messages）中的OOM Killer记录

2. 优化CKAN大文件上传机制：

   • CKAN默认会将上传文件加载到内存，建议配置流式上传或直接写入本地存储，避免内存占用过高
   • 确认ckan.storage_path指向的目录有足够磁盘空间，且权限正确
   • 检查是否启用了外部存储插件（如S3），将大文件直接存储到外部服务

3. 调整uWSGI缓冲与超时配置：

   • 设置post-buffering = 65536（或更大值），优化大文件上传的缓冲处理
   • 调整socket-timeout、request-timeout参数，匹配大文件上传的耗时需求
   • 确认uWSGI的workers、threads配置是否足够支撑并发上传

4. 系统层面验证：

   • 检查磁盘IO性能，大文件上传时写入缓慢可能导致超时
   • 临时关闭swap或增加swap空间，缓解内存压力
   • 确认系统的文件描述符限制是否足够，避免上传过程中出现资源耗尽

5. Nginx细节补全：

   • 验证proxy_temp_path目录的权限与磁盘空间，确保临时文件可正常写入
   • 移除无效的uwsgi_read_timeout配置（当前为proxy模式，该参数不生效），确保proxy_read_timeout设置足够长（如30分钟）

内容的提问来源于stack exchange，提问作者FrostyFrog