如何使用Boto 3通过签名URL复制外部S3存储桶对象至自有存储桶？

Short Answer #

You can't directly use the pre-signed URL's X-Amz-S* query parameters with Boto3's s3.Client.copy() method. That method relies on server-side replication, which requires your AWS account to have explicit permissions to access the source bucket/object (not just a pre-signed URL). Instead, you'll need to perform a client-side copy: download the object via the pre-signed URL and then upload it to your own bucket.

Why copy() Doesn't Work with Pre-Signed URLs #

The copy() method triggers a server-side copy operation within AWS. For this to work, your AWS credentials (used by Boto3) need to be allowed to read the source object via IAM policies or the source bucket's access policy. Pre-signed URLs are designed for temporary, client-side access—they contain signature parameters (X-Amz-Signature, X-Amz-Date, etc.) that authenticate a specific client request, not AWS's internal server-to-server requests. The S3 service won't recognize these parameters when processing a server-side copy.

Solution: Client-Side Copy with Pre-Signed URL #

Here's a practical implementation using Boto3 and requests to handle the download/upload flow:

1. First, install requests if you haven't already:

   pip install requests
   

2. Use this code to fetch the object from the pre-signed URL and upload it to your bucket:

   import boto3
   import requests
   
   # Initialize your S3 resource
   s3 = boto3.resource('s3')
   destination_bucket = "your-personal-bucket-name"
   destination_file_path = "folder/your-target-file-name.ext"
   source_pre_signed_url = "THE_PRE_SIGNED_URL_YOU_RECEIVED"
   
   # Download the object from the pre-signed URL
   try:
       download_response = requests.get(source_pre_signed_url)
       download_response.raise_for_status()  # Throw error if download fails
   except requests.exceptions.RequestException as e:
       print(f"Failed to download object: {str(e)}")
       exit(1)
   
   # Upload the content to your own bucket
   try:
       s3.Bucket(destination_bucket).put_object(
           Key=destination_file_path,
           Body=download_response.content
       )
       print(f"Success! Object copied to s3://{destination_bucket}/{destination_file_path}")
   except Exception as e:
       print(f"Failed to upload to your bucket: {str(e)}")
       exit(1)
   

Alternative: If You Have Direct Source Bucket Permissions #

If you do have explicit access to the source bucket (e.g., the bucket owner granted your AWS account s3:GetObject permission), you can use the standard copy() method as you initially referenced—no pre-signed URL needed:

import boto3
s3 = boto3.resource('s3')
copy_source = {
    'Bucket': 'external-source-bucket',
    'Key': 'path/to/source-object'
}
s3.meta.client.copy(copy_source, 'your-personal-bucket-name', 'destination-object-path')

内容的提问来源于stack exchange，提问作者bcattle