Django中1-2GB DCM文件后台持续上传实现方案咨询

一、前端改造：用 Service Worker 接管上传进程 #

当前 Dropzone 上传依赖页面上下文，切换页面会中断 XHR 请求。Service Worker 是浏览器后台线程，页面切换后仍能持续运行，可完美解决这个问题。

1. 注册 Service Worker #

在页面初始化时完成注册：

if ('serviceWorker' in navigator) {
  window.addEventListener('load', async () => {
    try {
      await navigator.serviceWorker.register('/sw-upload.js');
    } catch (err) {
      console.error('Service Worker 注册失败:', err);
    }
  });
}

2. 修改 Dropzone 配置，委托上传给 Service Worker #

取消默认 XHR 发送，将分片数据传给 Service Worker 处理：

autoProcessQueue: false,
url: url,
maxFiles: 100000000,
maxFilesize: 99999,
acceptedFiles: '.dcm',
timeout: 9999999999,
chunking: true,
chunkSize: 2000000,
clickable: "#dicomDropzone",
parallelChunkUploads: false,
retryChunks: true,
retryChunksLimit: 5,
forceChunking: true,
init: function() {  
    $('#CBCTUpload').click(function(){      
        myDropzone2.processQueue();
    })
    this.on("addedfile", function(file) {
        totalAddedFiles++;
    });
    this.on("sending", function(file, xhr, formData) {
        // 终止默认XHR请求
        xhr.abort();
        // 给Service Worker发送分片数据
        navigator.serviceWorker.controller.postMessage({
            type: 'UPLOAD_CHUNK',
            payload: {
                file: file,
                slug: parametre,
                chunkIndex: formData.get('dzchunkindex'),
                totalChunks: formData.get('dztotalchunkcount'),
                chunkByteOffset: formData.get('dzchunkbyteoffset'),
                filename: file.name
            }
        });
        // 阻止Dropzone继续执行默认流程
        return false;
    });
}

3. 实现 Service Worker 上传逻辑 #

新建 sw-upload.js 文件，处理分片上传与重试：

self.addEventListener('message', async (event) => {
  if (event.data.type === 'UPLOAD_CHUNK') {
    const { file, slug, chunkIndex, totalChunks, chunkByteOffset, filename } = event.data.payload;
    // 截取当前分片
    const chunk = file.slice(chunkByteOffset, chunkByteOffset + 2000000);
    // 构建与后端兼容的FormData
    const formData = new FormData();
    formData.append('file', chunk);
    formData.append('dzchunkindex', chunkIndex);
    formData.append('dztotalchunkcount', totalChunks);
    formData.append('dzchunkbyteoffset', chunkByteOffset);
    formData.append('slug', slug);
    formData.append('dzfilename', filename);
    
    try {
      const response = await fetch('/your-upload-endpoint/', {
        method: 'POST',
        body: formData,
        credentials: 'include'
      });
      if (!response.ok) throw new Error('分片上传失败');
      // 通知页面上传进度
      event.source.postMessage({
        type: 'CHUNK_UPLOADED',
        payload: { chunkIndex, totalChunks, filename }
      });
    } catch (err) {
      // 重试逻辑，间隔2秒后重新发送
      setTimeout(() => {
        self.postMessage(event.data);
      }, 2000);
    }
  }
});

二、后端改造：Celery 异步合并分片 #

当前后端直接处理分片，改用 Celery 可在所有分片上传完成后后台执行合并，避免阻塞请求。

1. 配置 Celery #

确保项目已配置 Celery（以 Redis 为消息代理为例）：

# celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings')
app = Celery('your_project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

2. 修改上传接口，存储分片并触发合并任务 #

调整视图逻辑，收到最后一个分片时启动 Celery 任务：

from django.http import HttpResponse
from .tasks import merge_dcm_chunks
import os

def upload_chunk(request):
    if request.method == 'POST':
        if 'file' not in request.FILES:
            return HttpResponse(status=500)
        file = request.FILES['file']
        current_chunk = int(request.POST['dzchunkindex']) + 1
        total_chunks = int(request.POST['dztotalchunkcount'])
        folder_slug = request.POST.get('slug')
        filename = request.POST.get('dzfilename')
        
        # 创建临时分片存储目录
        temp_dir = f'/tmp/dcm_uploads/{folder_slug}/{filename}/'
        os.makedirs(temp_dir, exist_ok=True)
        
        # 保存当前分片
        chunk_path = os.path.join(temp_dir, f'chunk_{current_chunk}')
        with open(chunk_path, 'wb+') as destination:
            for chunk in file.chunks():
                destination.write(chunk)
        
        # 最后一个分片上传完成，触发合并任务
        if current_chunk == total_chunks:
            merge_dcm_chunks.delay(folder_slug, filename, total_chunks)
        
        return HttpResponse(status=200)

3. 实现 Celery 合并任务 #

在 tasks.py 中编写分片合并逻辑：

from celery import shared_task
import os
import shutil

@shared_task
def merge_dcm_chunks(folder_slug, filename, total_chunks):
    temp_dir = f'/tmp/dcm_uploads/{folder_slug}/{filename}/'
    final_path = f'/path/to/your/storage/{folder_slug}/{filename}'
    
    # 创建最终文件存储目录
    os.makedirs(os.path.dirname(final_path), exist_ok=True)
    
    # 合并所有分片
    with open(final_path, 'wb') as final_file:
        for i in range(1, total_chunks + 1):
            chunk_path = os.path.join(temp_dir, f'chunk_{i}')
            with open(chunk_path, 'rb') as chunk_file:
                shutil.copyfileobj(chunk_file, final_file)
            # 删除已合并的分片文件
            os.remove(chunk_path)
    
    # 删除临时目录
    os.rmdir(temp_dir)
    
    # 可添加后续逻辑：如数据库记录上传完成、发送通知等
    return f'文件 {filename} 合并完成'

三、简化替代方案：无状态分片上传 + 前端轮询 #

若不想引入 Service Worker，可让每个分片上传为独立无状态请求，前端在页面加载时检查未完成的上传任务，继续上传剩余分片。但此方案仍依赖页面存在，可靠性不如 Service Worker 方案。

内容的提问来源于stack exchange，提问作者Ahmet Resul KURU