Python Flask中Boto3 SSE-C上传S3时MD5哈希不匹配问题的排查与解决

Hey there, let's work through this SSE-C MD5 mismatch issue you've been stuck on—totally get how frustrating it is to hit this error after debugging for days.

First, Let's Break Down the Problem #

You're getting an InvalidArgument error when using SSE-C encryption with boto3's upload_fileobj(): the calculated MD5 hash of your customer key doesn't match what you provided. Your original code tried generating the MD5 manually and passing it via SSECustomerKeyMD5 in ExtraArgs, but there were two key issues here.

What Was Wrong With Your Original Implementation? #

Let's look at your MD5 code first:

md5_hash = hashlib.md5()
md5_hash.update(file)
digest = md5_hash.hexdigest()
base64_encoded_md5 = base64.b64encode(digest.encode("utf-8"))
base64_encoded_md5_string = str(base64_encoded_md5.decode("utf-8"))

1. You're encoding the wrong thing: S3 expects the Base64 encoding of the raw 16-byte MD5 digest of your customer key—not the Base64 of a hex string representation of that digest. Your code converts the MD5 to a 32-character hex string first, then encodes that string, which gives a completely different value than what S3 calculates.
2. You don't need to pass SSECustomerKeyMD5 at all: Here's a big one—boto3's underlying botocore library automatically calculates the correct MD5 hash of your SSECustomerKey and adds it to the request headers right before sending the request. Manually passing this parameter is redundant and error-prone.

On top of that, your original customer key was only 16 characters long, but SSE-C requires a valid AES key (either 16 bytes for AES-128, represented as a 32-character hex string, or 32 bytes for AES-256, represented as a 64-character hex string).

The Fixes You Need to Apply #

1. Ditch the SSECustomerKeyMD5 Parameter #

Remove it entirely from your ExtraArgs—let boto3 handle this part for you. Your upload call should look like this:

s3_client.upload_fileobj(
    your_file_object,
    Bucket="your-bucket-name",
    Key="your-target-object-key",
    ExtraArgs={
        "SSECustomerAlgorithm": "AES256",  # Or AES128 if using a 16-byte key
        "SSECustomerKey": your_corrected_key
    }
)

2. Generate a Valid Customer Key #

Make sure your SSECustomerKey is a valid AES key. Here's a secure way to generate one in Python:

import secrets

# For AES-256 (64-character hex string = 32 bytes)
sse_customer_key = secrets.token_hex(32)

# For AES-128 (32-character hex string = 16 bytes)
# sse_customer_key = secrets.token_hex(16)

Bonus: How to Calculate the Correct MD5 Manually (For Debugging) #

If you ever want to verify the MD5 that boto3 is sending, here's the right way to compute it for your customer key:

import hashlib
import base64

# Compute MD5 of the customer key's raw bytes
md5_hash = hashlib.md5(sse_customer_key.encode('utf-8'))
# Base64 encode the raw 16-byte digest
correct_base64_md5 = base64.b64encode(md5_hash.digest()).decode('utf-8')

This will match what boto3 generates automatically.

Quick Recap #

• Stop passing SSECustomerKeyMD5—boto3 does this for you.
• Use a valid AES key (32-character hex for AES-128, 64-character for AES-256) as your SSECustomerKey.

内容的提问来源于stack exchange，提问作者pranshu