使用Python Lambda跨AWS账户复制S3数据遇权限错误求助

问题背景 #

需将AWS account1（region1）中S3桶的数据复制到account2（region2）的S3桶，通过account1中的Lambda函数实现，Lambda由SNS触发（当account1的源桶有新文件上传时执行），但执行时出现AccessDenied错误，报错指向CreateMultipartUpload操作。

现有配置 #

1. account2目标桶桶策略 #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::destination-bucket/*",
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                },
                "ForAllValues:StringNotEquals": {
                    "s3:TlsVersion": [
                        "1.2",
                        "1.3"
                    ]
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<account1-id>:role/cross-account-file-share-role"
                ]
            },
            "Action": [
                "s3:Get*",
                "s3:Put*",
                "s3:List*",
                "s3:AbortMultipartUpload",
                "s3:Delete*"
            ],
            "Resource": [
                "arn:aws:s3:::destination-bucket",
                "arn:aws:s3:::destination-bucket/*"
            ]
        }
    ]
}

2. account1中cross-account-file-share-role信任关系 #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "lambda.amazonaws.com",
                    "ec2.amazonaws.com",
                    "storagegateway.amazonaws.com",
                    "s3.amazonaws.com"
                ],
                "AWS": "arn:aws:iam::<account1-id>:role/FullResourceAccessforEC2"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

3. Lambda函数内联策略（附加到AWSLambdaBasicExecutionRole） #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::account1-source-bucket/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::account2-destination-bucket/*"
        }
    ]
}

4. Lambda函数代码 #

import json
import boto3
import urllib

TARGET_BUCKET = 'account2-destination-bucket'

def lambda_handler(event, context):
    # Get incoming bucket and key
    source_bucket = json.loads(event['Records'][0]['Sns']['Message'])['Records'][0]['s3']['bucket']['name']
    source_key = urllib.parse.unquote_plus(json.loads(event['Records'][0]['Sns']['Message'])['Records'][0]['s3']['object']['key'])
    print("source_bucket :", source_bucket)
    print("source_key :", source_key)
    
    # Copy object to different bucket
    s3_resource = boto3.resource('s3')
    copy_source = {
        'Bucket': source_bucket,
        'Key': source_key
    }
    target_key = source_key    
    s3_resource.Bucket(TARGET_BUCKET).Object(target_key).copy(copy_source, ExtraArgs={'ACL': 'bucket-owner-full-control'})

错误信息 #

{
  "errorMessage": "An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied",
  "errorType": "ClientError",
  "requestId": "d37ad461-8c9a-409b-bd53-0bc11a5c2263",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 25, in lambda_handler\n    s3_resource.Bucket(TARGET_BUCKET).Object(target_key).copy(copy_source, ExtraArgs={'ACL': 'bucket-owner-full-control'})\n",
    "  File \"/var/runtime/boto3/s3/inject.py\", line 565, in object_copy\n    return self.meta.client.copy(\n",
    "  File \"/var/runtime/boto3/s3/inject.py\", line 444, in copy\n    return future.result()\n",
    "  File \"/var/runtime/s3transfer/futures.py\", line 103, in result\n    return self._coordinator.result()\n",
    "  File \"/var/runtime/s3transfer/futures.py\", line 266, in result\n    raise self._exception\n",
    "  File \"/var/runtime/s3transfer/tasks.py\", line 139, in __call__\n    return self._execute_main(kwargs)\n",
    "  File \"/var/runtime/s3transfer/tasks.py\", line 162, in _execute_main\n    return_value = self._main(**kwargs)\n",
    "  File \"/var/runtime/s3transfer/tasks.py\", line 348, in _main\n    response = client.create_multipart_upload(\n",
    "  File \"/var/runtime/botocore/client.py\", line 530, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 960, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

问题分析与解决方案 #

核心问题 #

当前Lambda使用自身的执行角色（包含AWSLambdaBasicExecutionRole和自定义内联策略）访问account2的目标桶，但目标桶的桶策略仅允许arn:aws:iam::<account1-id>:role/cross-account-file-share-role这个角色执行S3操作，Lambda并未扮演该角色，因此触发权限拒绝。

解决步骤 #

1. 给Lambda执行角色添加AssumeRole权限
   编辑Lambda的执行角色，添加以下内联策略，允许其扮演cross-account-file-share-role：

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": "sts:AssumeRole",
               "Resource": "arn:aws:iam::<account1-id>:role/cross-account-file-share-role"
           }
       ]
   }
   

2. 修改Lambda代码，扮演指定角色后再操作S3
   修改代码，先通过STS获取cross-account-file-share-role的临时凭证，再用该凭证创建S3资源客户端：

   import json
   import boto3
   import urllib
   
   TARGET_BUCKET = 'account2-destination-bucket'
   CROSS_ACCOUNT_ROLE_ARN = 'arn:aws:iam::<account1-id>:role/cross-account-file-share-role'
   
   def lambda_handler(event, context):
       # 解析源桶和对象键
       sns_message = json.loads(event['Records'][0]['Sns']['Message'])
       source_bucket = sns_message['Records'][0]['s3']['bucket']['name']
       source_key = urllib.parse.unquote_plus(sns_message['Records'][0]['s3']['object']['key'])
       print("source_bucket :", source_bucket)
       print("source_key :", source_key)
       
       # 扮演跨账号角色
       sts_client = boto3.client('sts')
       assumed_role = sts_client.assume_role(
           RoleArn=CROSS_ACCOUNT_ROLE_ARN,
           RoleSessionName='S3CrossAccountCopySession'
       )
       
       # 使用临时凭证创建S3资源
       s3_resource = boto3.resource(
           's3',
           aws_access_key_id=assumed_role['Credentials']['AccessKeyId'],
           aws_secret_access_key=assumed_role['Credentials']['SecretAccessKey'],
           aws_session_token=assumed_role['Credentials']['SessionToken']
       )
       
       # 执行复制操作
       copy_source = {
           'Bucket': source_bucket,
           'Key': source_key
       }
       target_key = source_key    
       s3_resource.Bucket(TARGET_BUCKET).Object(target_key).copy(copy_source, ExtraArgs={'ACL': 'bucket-owner-full-control'})
   

3. 验证配置细节

   • 确保所有配置中的<account1-id>替换为实际的account1账号ID
   • 确认目标桶名称destination-bucket与Lambda代码中的TARGET_BUCKET一致
   • 检查Lambda执行角色的信任关系是否允许lambda.amazonaws.com服务（默认AWSLambdaBasicExecutionRole已包含此配置）

内容的提问来源于stack exchange，提问作者djm