使用blueimp File Uploader直传AWS S3时分片覆盖问题求助

I’ve run into this exact same problem before—small files upload perfectly, but large chunked uploads end up as just the final chunk in S3. The root cause is that you’re not leveraging AWS S3’s Multipart Upload API correctly. Let’s break down how to fix this step by step:

Why This Happens #

Small files use a single PUT request to S3, which works directly. But for chunked uploads, if you don’t initialize a dedicated Multipart Upload session and track each chunk’s part number, every new chunk overwrites the same S3 object instead of being added as a separate piece of the whole file. That’s why you only see the last 10MB chunk.

Step-by-Step Fix #

1. Enable Multipart Support in the Plugin #

First, update your plugin config to enable multipart uploads and disable iframe transport (which doesn’t support chunked requests properly):

$('#fileupload').fileupload({
    url: 'YOUR_S3_BUCKET_BASE_URL',
    method: 'PUT',
    maxChunkSize: 10 * 1024 * 1024, // 10MB chunks
    multipart: true,
    forceIframeTransport: false,
    sequentialUploads: true, // Critical: upload chunks one at a time to track part numbers
    acceptFileTypes: /(\.|\/)(jpg|jpeg|png|gif|pdf)$/i // Adjust to your file types
});

2. Initialize a Multipart Upload Session #

Before uploading any chunks, you need to create a Multipart Upload session with S3 to get an UploadId. This tells S3 to expect multiple chunks for a single file.

Add an add event handler to trigger this initialization when a file is added:

$('#fileupload').on('fileuploadadd', function(e, data) {
    // Call your backend to generate the Multipart Upload ID (uses S3's CreateMultipartUpload API)
    $.ajax({
        url: '/your-backend-endpoint/create-s3-multipart',
        method: 'POST',
        data: {
            fileName: data.files[0].name,
            contentType: data.files[0].type
        },
        success: function(response) {
            // Store the critical values in the upload data object
            data.uploadId = response.uploadId;
            data.s3ObjectKey = response.objectKey;
            // Start uploading the chunks
            data.submit();
        }
    });
});

Your backend should interact with S3 to create the session and return the UploadId and target object key.

3. Attach UploadId and Part Number to Each Chunk Request #

Each chunk needs to include the UploadId and PartNumber (starting at 1) so S3 knows which piece of the file it is. Use the beforesend event to modify the request URL:

$('#fileupload').on('fileuploadbeforesend', function(e, data, settings) {
    if (data.uploadId) {
        const partNumber = data.chunkNumber || 1;
        // Update the URL to include S3's multipart parameters
        settings.url = `${settings.url}/${data.s3ObjectKey}?uploadId=${data.uploadId}&partNumber=${partNumber}`;
        // Ensure the request uses PUT for chunk uploads
        settings.type = 'PUT';
    }
});

4. Complete the Multipart Upload After All Chunks Are Done #

Once all chunks are uploaded, you need to tell S3 to merge them into a single file. Use the fileuploaddone event to trigger this final step:

$('#fileupload').on('fileuploaddone', function(e, data) {
    if (data.uploadId) {
        // Collect ETags for each chunk (S3 returns these in the response headers)
        const parts = data.result.map((chunkResult, index) => ({
            PartNumber: index + 1,
            ETag: chunkResult.headers['ETag']
        }));

        // Send the complete request to your backend (uses S3's CompleteMultipartUpload API)
        $.ajax({
            url: '/your-backend-endpoint/complete-s3-multipart',
            method: 'POST',
            data: {
                objectKey: data.s3ObjectKey,
                uploadId: data.uploadId,
                parts: parts
            },
            success: function() {
                console.log('File uploaded successfully!');
            },
            error: function() {
                // Clean up failed uploads to avoid leftover chunks in S3
                $.ajax({
                    url: '/your-backend-endpoint/abort-s3-multipart',
                    method: 'POST',
                    data: { objectKey: data.s3ObjectKey, uploadId: data.uploadId }
                });
            }
        });
    }
});

Critical S3 CORS Configuration #

Make sure your bucket’s CORS policy allows the methods needed for multipart uploads:

<CORSConfiguration>
    <CORSRule>
        <AllowedOrigin>YOUR_APP_DOMAIN</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedMethod>DELETE</AllowedMethod>
        <AllowedHeader>*</AllowedHeader>
        <ExposeHeader>ETag</ExposeHeader> <!-- Required to retrieve chunk ETags -->
    </CORSRule>
</CORSConfiguration>

Final Tips #

• Ensure your pre-signed URLs (if using them) have permissions for s3:PutObject, s3:CreateMultipartUpload, s3:UploadPart, and s3:CompleteMultipartUpload.
• Abort failed Multipart Uploads to avoid incurring storage costs for leftover chunks.
• Test with different chunk sizes to find the optimal balance for your use case.

内容的提问来源于stack exchange，提问作者Abhishek