求通过API Gateway接收文件并上传至S3的实现示例

Got it, let's break down how to set up API Gateway to receive files and upload them to S3—this is a super common pattern, and I'll walk you through two practical approaches with step-by-step examples.

This is ideal if you don't need to process the file before storing it—API Gateway passes the file straight to S3, keeping things lightweight.

Step 1: Prepare Your S3 Bucket #

First, create an S3 bucket (stick to the same region as you'll use for API Gateway) and update its CORS configuration to allow uploads from your client:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT", "POST"],
    "AllowedOrigins": ["*"], // Restrict this to your actual domain in production!
    "ExposeHeaders": ["ETag"]
  }
]

Step 2: Configure API Gateway for Binary Files #

API Gateway needs explicit setup to handle binary data (like images, PDFs, etc.):

1. Create a new REST API in the API Gateway console.
2. Go to Settings > Binary Media Types and add */* (or specific types like image/png, application/pdf if you want to restrict uploads).
3. Save the changes.

Step 3: Set Up the POST Method with S3 Integration #

1. Create a new resource (e.g., /upload) in your API.
2. Add a POST method to this resource:
   • For Integration Type, select AWS Service.
   • Pick your AWS region, set AWS Service to S3, and HTTP Method to PUT.
   • Under Action Type, choose Path override and enter /{bucket}/{key} (replace {bucket} with your actual bucket name; {key} will be the filename you want to use).
3. Configure the Integration Request:
   • Expand Mapping Templates, add a template for multipart/form-data (for form-based uploads) or application/octet-stream (for raw binary uploads).
   • For multipart/form-data, use this template to extract the file content:

     #set($boundary = $input.params('boundary'))
     #set($body = $input.body)
     #set($start = $body.indexOf("--$boundary") + $boundary.length() + 4)
     #set($end = $body.indexOf("--$boundary--", $start))
     $body.substring($start, $end)
     

4. Attach an IAM role to API Gateway that allows s3:PutObject on your bucket. The policy should look like this:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3:::your-bucket-name/*"
       }
     ]
   }
   

Step 4: Deploy and Test #

Deploy your API to a stage, then test with curl:

curl -X POST https://your-api-id.execute-api.region.amazonaws.com/prod/upload \
  -F "file=@/path/to/your/file.jpg"

If you need to validate file types, resize images, or modify the file before storing, use Lambda as a bridge between API Gateway and S3.

Step 1: Create the Lambda Function #

Here's a Python example that parses a multipart/form-data request, extracts the file, and uploads it to S3:

import boto3
import os
import base64
from werkzeug.datastructures import FileStorage
from io import BytesIO

s3 = boto3.client('s3')
BUCKET_NAME = os.environ['BUCKET_NAME']

def lambda_handler(event, context):
    # Parse the multipart form data
    content_type = event['headers']['Content-Type']
    body = event['body']
    is_base64_encoded = event['isBase64Encoded']
    
    # Convert body to BytesIO for parsing
    if is_base64_encoded:
        body_stream = BytesIO(base64.b64decode(body))
    else:
        body_stream = BytesIO(body.encode('utf-8'))
    
    # Use werkzeug to extract the file from form data
    fs = FileStorage(stream=body_stream, content_type=content_type)
    fs.seek(0)
    uploaded_file = fs.files['file']  # 'file' is the form field name from your client
    
    # Upload to S3
    s3.put_object(
        Bucket=BUCKET_NAME,
        Key=uploaded_file.filename,
        Body=uploaded_file.stream,
        ContentType=uploaded_file.content_type
    )
    
    return {
        'statusCode': 200,
        'body': f"File {uploaded_file.filename} uploaded successfully!"
    }

Note: You'll need to add the werkzeug library to your Lambda layer or package it with your deployment zip.

Step 2: Configure API Gateway with Lambda Integration #

1. Create a REST API, enable binary media types as in Approach 1.
2. Add a POST method, set Integration Type to Lambda Function, and select your Lambda function.
3. In Integration Request, set Passthrough Behavior to WHEN_NO_MATCH to pass the raw request body to Lambda.

Step 3: Set Lambda Permissions #

Make sure your Lambda execution role has s3:PutObject permission for your bucket, using the same policy structure as in Approach 1.

Step 4: Test #

Use the same curl command as before to test the upload—Lambda will process the file and send it to S3.

• Binary Media Types: Always configure these in API Gateway, otherwise your files will get converted to Base64 and become corrupted.
• CORS: Restrict AllowedOrigins to your actual domain in production (don't leave it as * for security).
• Permissions: Double-check that API Gateway/Lambda has the necessary S3 write permissions—missing permissions are one of the most common roadblocks here.
• Large Files: For files over 10MB, consider using presigned URLs (API Gateway returns a pre-signed S3 URL, and the client uploads directly to S3) to avoid API Gateway payload limits.

内容的提问来源于stack exchange，提问作者Logan Knott