跨账号跨区域S3对象复制遇Access Denied错误求助

尝试将us-west-2区域的demo-bucket-a中的S3对象跨账号复制到us-east-1区域的demo-bucket-b，无论使用哪种S3复制方法，均收到错误：An error occurred (AccessDenied) when calling the CopyObject operation: Access Denied。

代码打印结果 #

Copying file1.json
{SOURCE_BUCKET}/{obj.key}---->demo-bucket-a/year=2022/month=12/day=21/file1.json
cp_source---->{'Bucket': 'demo-bucket-a', 'Key': 'year=2022/month=12/day=21/file1.json'}
TARGET_BUCKET----->demo-bucket-b
obj.key----->year=2022/month=12/day=21/file1.json

Lambda代码 #

import json
import boto3
import urllib
import uuid
import os
import logging
from datetime import datetime, timedelta

logger = logging.getLogger()
logger.setLevel(logging.INFO)
# boto3.set_stream_logger('', logging.DEBUG)

# lambda function to copy file from 1 s3 to another s3
def lambda_handler(event, context):
    
    dt_day = ''
    dt_month = ''
    dt_year = ''
    
    execution_time = datetime.now() - timedelta(3)
    logger.info(f'Start lambda: {execution_time}')
    year = execution_time.year
    month = execution_time.month
    day = execution_time.day
    
    SOURCE_BUCKET = 'demo-bucket-a'
    SOURCE_PATH = f"year={year}/month={month:02}/day={day:02}/"
    

    # Target Bucket Details 
    TARGET_BUCKET = 'demo-bucket-b'
    TARGET_PATH = f"sub-folder-b/"
    # TARGET_LOC = f"year={year}/month={month:02}/day={day:02}/"
    
    kd = 'Detail/'
    ko = 'Owned/'
    ks = 'Summary/'
    
    s3_client = boto3.client('s3')
    s3_resource = boto3.resource('s3')

    bucket = s3_resource.Bucket(SOURCE_BUCKET)
    
    fname = ''
    # For loop prints the file names located in the S3 bucket and source path
    for obj in s3_resource.Bucket(name=SOURCE_BUCKET).objects.filter(Prefix=SOURCE_PATH):
        filename = obj.key.split('/')[-1]
        print(filename)
        
        target_key = obj.key[obj.key.rfind('/')+1:]
        
        if 'summary' in filename:
            fname = ks
        elif 'Owned' in filename: 
            fname = ko
        else:
            fname = kd
        
        cp_source = {'Bucket':SOURCE_BUCKET, 'Key': obj.key}
        
        print('Copying', target_key)
        print("{SOURCE_BUCKET}/{obj.key}---->" + f"{SOURCE_BUCKET}/{obj.key}")
        print("cp_source---->" + str(cp_source))
        print("TARGET_BUCKET----->" + str(TARGET_BUCKET))
        print("obj.key----->" + str(obj.key))
        
        # ACCESS DENIED ERROR for below code -------->
        # s3_resource.Object(TARGET_BUCKET, TARGET_PATH + fname + SOURCE_PATH + target_key).copy({'Bucket':SOURCE_BUCKET, 'Key': obj.key})
        
        # ACCESS DENIED ERROR for below code -------->
        # s3_resource.Object(TARGET_BUCKET, TARGET_PATH + fname + SOURCE_PATH + target_key).copy_from(CopySource=f"{SOURCE_BUCKET}/{obj.key}")
        
        # ACCESS DENIED ERROR for below code -------->
        # s3_client.copy_object(CopySource=cp_source, Bucket=TARGET_BUCKET, Key=obj.key)
        
        s3_resource.meta.client.copy(cp_source, TARGET_BUCKET, f"fname/{obj.key}")
    
    s3_resource.Bucket(name=TARGET_BUCKET).put_object(Body='', Key=TARGET_PATH + 'Manifest/' + SOURCE_PATH + str(uuid.uuid4()) + '.manifest', ACL = 'bucket-owner-full-control')
    
    return {
        'statusCode': 200,
        'body': json.dumps('Executed S3-to-S3-Copy-Lambda Successfully !!!')
    }

已配置权限 #

源桶demo-bucket-a桶级策略 #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::xxxxxxxxx:role/iam-s3-copy-lambda-role"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectTagging"
            ],
            "Resource": [
                "arn:aws:s3:::demo-bucket-a",
                "arn:aws:s3:::demo-bucket-a/*"
            ]
        }
    ]
}

Lambda角色iam-s3-copy-lambda-role权限 #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:us-east-1:xxxxxxxxx:key/aws-kms-key",
            "Effect": "Allow"
        },
        {
            "Action": [
                "kms:GenerateDataKey*",
                "kms:Decrypt"
            ],
            "Resource": "arn:aws:kms:us-east-1:xxxxxxxxx:key/cmk-kms-key",
            "Effect": "Allow"
        },
        {
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "s3:GetBucketAcl",
                "s3:GetBucketLocation",
                "s3:GetBucketNotification",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectTagging",
                "s3:ListBucket",
                "s3:CopyObject",
                "s3:HeadObject",
                "s3:List*",
                "s3:PutObject",
                "s3:PutObjectTagging",
                "s3:PutObjectAcl",
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::demo-bucket-b",
                "arn:aws:s3:::demo-bucket-b/*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectTagging",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::demo-bucket-a",
                "arn:aws:s3:::demo-bucket-a/*"
            ],
            "Effect": "Allow"
        }
    ]
}

目标桶demo-bucket-b桶级策略 #

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxxx:role/iam-s3-copy-lambda-role"
            },
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging",
                "s3:PutObjectVersionAcl"
            ],
            "Resource": [
                "arn:aws:s3:::demo-bucket-b",
                "arn:aws:s3:::demo-bucket-b/*"
            ]
        }
    ]
}

排查方向 #

1. 代码路径错误：
   当前copy操作的目标Key写为f"fname/{obj.key}"，这里的fname是字符串字面量，而非变量，应改为f"{fname}/{obj.key}"，否则会尝试写入不存在权限的fname/前缀路径，直接触发权限错误。

2. 跨区域KMS权限缺失：

   • 若源桶启用了KMS加密，Lambda角色需要拥有us-west-2区域对应KMS密钥的kms:Decrypt权限，当前角色权限仅包含us-east-1的KMS密钥，需补充源区域的KMS权限。
   • 目标桶若启用KMS加密，需确认角色权限中的us-east-1 KMS密钥ARN与实际使用的密钥一致。

3. 跨账号所有者权限：
   跨账号复制时，需在copy操作中指定ACL='bucket-owner-full-control'，确保目标桶所有者拥有对象权限，修改代码如下：

   s3_resource.meta.client.copy(cp_source, TARGET_BUCKET, f"{fname}/{obj.key}", ExtraArgs={'ACL': 'bucket-owner-full-control'})
   

4. 源桶策略账号ID验证：
   检查源桶策略中Principal的IAM角色ARN是否正确，确保账号ID为Lambda角色所属的目标账号ID，而非源桶所属账号ID。

5. 开启DEBUG日志：
   取消注释代码中的boto3.set_stream_logger('', logging.DEBUG)，查看更详细的API调用日志，定位具体的权限缺失项。

内容的提问来源于stack exchange，提问作者Bhavesh Bendale