如何在Serverless+Node.js环境下通过API Gateway上传>5MB大文件？

Hey there, let's work through this together—your hunch about API Gateway (and likely Lambda) payload limits is almost certainly the root cause here. Let's break down why this happens and the best ways to fix it.

First, Confirm the Root Cause #

API Gateway REST APIs have a default maximum request size limit of 10MB, but if you're using Lambda proxy integration (which is common with Serverless + Express), Lambda has a stricter 6MB payload limit for synchronous invocations. That's why files under ~5MB work, but anything larger gets rejected.

To confirm this quickly:

• Test your upload logic locally (bypassing API Gateway/Lambda) with a >5MB file. If it works, the issue is definitely the cloud-side limits.

Recommended Solution: Use S3 Pre-Signed URLs (For Any File Size) #

This is AWS's official recommended approach for large file uploads, since it lets clients upload directly to S3 without going through your API Gateway/Lambda entirely. This bypasses all payload limits (S3 supports files up to 5TB) and reduces load on your backend.

Step 1: Add an API Endpoint to Generate Pre-Signed URLs #

Update your Express app to include a route that generates a time-limited S3 pre-signed URL for uploads:

const AWS = require('aws-sdk');
const s3 = new AWS.S3();
const express = require('express');
const app = express();

// Generate pre-signed URL for direct S3 upload
app.get('/generate-upload-url', async (req, res) => {
  const { fileName, fileType } = req.query;
  
  if (!fileName || !fileType) {
    return res.status(400).json({ error: 'fileName and fileType are required' });
  }

  const params = {
    Bucket: process.env.S3_BUCKET_NAME,
    Key: fileName,
    ContentType: fileType,
    Expires: 3600 // URL stays valid for 1 hour
  };

  try {
    const uploadUrl = await s3.getSignedUrlPromise('putObject', params);
    res.json({ uploadUrl });
  } catch (err) {
    console.error('Error generating pre-signed URL:', err);
    res.status(500).json({ error: 'Failed to generate upload URL' });
  }
});

Step 2: Update Serverless Config for Permissions #

Make sure your Lambda function has permission to generate pre-signed URLs (which requires s3:PutObject access):

service: your-service-name

custom:
  s3Bucket: your-s3-bucket-name

functions:
  generateUploadUrl:
    handler: handler.generateUploadUrl # Point to your Express handler
    events:
      - http:
          path: generate-upload-url
          method: get
    iamRoleStatements:
      - Effect: Allow
        Action:
          - s3:PutObject
        Resource: arn:aws:s3:::${self:custom.s3Bucket}/*

Step 3: Client-Side Upload Flow #

1. Your client calls /generate-upload-url with the file name and type.
2. It receives the pre-signed URL, then sends a PUT request to that URL with the file data.
3. S3 handles the upload directly—no more payload limits!

Alternative: Adjust Limits (Only For Files <10MB) #

If you only need to handle files up to 10MB, you can tweak API Gateway and Lambda settings (note: Lambda's 6MB limit still applies for proxy integration):

1. Increase API Gateway Request Size #

Update your Serverless HTTP event to set a higher maxSize (max allowed is 10MB = 10485760 bytes):

functions:
  uploadFile:
    handler: handler.uploadFile
    events:
      - http:
          path: upload
          method: post
          request:
            maxSize: 10485760 # 10MB in bytes

2. Fix Multer's Storage Setup #

By default, Multer stores files in memory, which will crash Lambda for large files. Switch to disk storage using Lambda's /tmp directory:

const multer = require('multer');
const fs = require('fs');

// Configure Multer to use disk storage
const storage = multer.diskStorage({
  destination: (req, file, cb) => {
    cb(null, '/tmp'); // Lambda's temporary storage directory
  },
  filename: (req, file, cb) => {
    cb(null, `${Date.now()}-${file.originalname}`); // Unique filename
  }
});
const upload = multer({ storage: storage });

// Upload route using Multer
app.post('/upload', upload.array('files'), async (req, res) => {
  try {
    for (const file of req.files) {
      await s3.putObject({
        Bucket: process.env.S3_BUCKET_NAME,
        Key: file.filename,
        Body: fs.createReadStream(file.path)
      }).promise();
    }
    res.json({ success: true, uploadedFiles: req.files.map(f => f.filename) });
  } catch (err) {
    console.error('Upload error:', err);
    res.status(500).json({ error: 'Failed to upload files' });
  }
});

Final Notes #

• If you need to handle files larger than 6MB, pre-signed URLs are the only viable long-term solution—you can't get around Lambda's 6MB payload limit for synchronous invocations.
• Pre-signed URLs also improve performance and reduce costs, since you're not paying for Lambda execution time for file uploads.

内容的提问来源于stack exchange，提问作者Luis V.