跨账号S3复制遇DstMultipartUploadNotPermitted错误求助

问题概述 #

我在两个不同AWS账号的不同区域各有一个S3桶，需要将bucket-a（已有存储对象）中的所有文件（含历史文件）复制到bucket-b（新建空桶）。配置复制规则并启动批量复制任务后，63%的对象复制失败，错误为DstPutObjectNotPermitted或DstMultipartUploadNotPermitted，无更多详细信息，剩余对象复制正常。

后续更新：已为bucket-b的客户托管KMS密钥添加允许复制角色的权限，DstPutObjectNotPermitted错误已消除，但DstMultipartUploadNotPermitted错误仍存在。用新桶复现问题未成功，推测该错误与bucket-a中旧文件的存储方式有关。

现有配置信息 #

bucket-a配置 #

• 已启用版本控制
• 使用AWS托管默认KMS密钥加密
• 开启ACL
• 桶策略：

{
    "Version": "2008-10-17",
    "Id": "NoBucketDelete",
    "Statement": [
        {
            "Sid": "NoBucketDeleteStatement",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:DeleteBucket",
            "Resource": "arn:aws:s3:::bucket-a"
        },
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::bucket-a/*",
                "arn:aws:s3:::bucket-a"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        }
    ]
}

bucket-b配置 #

• 已启用版本控制与ACL
• 使用客户托管KMS密钥加密
• 桶策略：

{
    "Version": "2012-10-17",
    "Id": "Policy1644945280205",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::bucket-b/*",
                "arn:aws:s3:::bucket-b"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        },
        {
            "Sid": "Stmt1644945277847",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::12345:role/bucket-replication-role"
            },
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateTags",
                "s3:ObjectOwnerOverrideToBucketOwner",
                "s3:ReplicateDelete"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        }
    ]
}

• KMS密钥策略：

{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Allow access through S3 for all principals in the account that are authorized to use S3",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "kms:CallerAccount": "12345",
                    "kms:ViaService": "s3.us-west-2.amazonaws.com"
                }
            }
        },
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::12345:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::12345:user/root",
                    "arn:aws:iam::12345:user/user"
                ]
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::12345:user/user"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::12345:user/user"
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

IAM复制角色（账号a中的bucket-replication-role） #

• 信任关系策略：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

• 附加权限策略：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ReplicateDelete"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:GenerateDataKey"
            ],
            "Resource": [
                "arn:aws:kms:us-east-1:12345:key/[account-a-kms-key-id]"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:GenerateDataKey",
                "kms:Encrypt"
            ],
            "Resource": [
                "arn:aws:kms:us-west-2:12345:key/[account-b-kms-key-id]"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-a"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-a/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateTags"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        }
    ]
}

• 复制规则已关联该角色，批量操作为默认选项
• 待复制对象为小型png、json、html等常规文件

问题分析与解决建议 #

错误原因推测 #

DstMultipartUploadNotPermitted错误表明复制角色在向bucket-b执行分段上传操作时缺少权限。虽然已配置s3:ReplicateObject，但对于原始通过分段上传方式存入bucket-a的对象，S3复制时会沿用分段上传的方式向bucket-b写入，此时需要额外的权限支持。

另外，bucket-a中的旧对象可能存在特殊的存储属性（如分段上传遗留的未完成分片、对象版本的ACL限制），导致复制时触发分段上传权限检查。

解决步骤 #

1. 更新bucket-b的桶策略：为复制角色添加分段上传相关的S3权限，在现有允许的Action列表中加入：

   • s3:AbortMultipartUpload
   • s3:ListMultipartUploadParts
   • s3:CreateMultipartUpload
     示例更新后的Statement：

   {
       "Sid": "Stmt1644945277847",
       "Effect": "Allow",
       "Principal": {
           "AWS": "arn:aws:iam::12345:role/bucket-replication-role"
       },
       "Action": [
           "s3:ReplicateObject",
           "s3:ReplicateTags",
           "s3:ObjectOwnerOverrideToBucketOwner",
           "s3:ReplicateDelete",
           "s3:AbortMultipartUpload",
           "s3:ListMultipartUploadParts",
           "s3:CreateMultipartUpload"
       ],
       "Resource": "arn:aws:s3:::bucket-b/*"
   }
   

2. 更新复制角色的权限策略：同样在角色的权限策略中添加上述分段上传相关权限，确保角色拥有完整的分段上传操作权限：
   在现有s3:ReplicateObject所在的Statement中加入：

   "s3:AbortMultipartUpload",
   "s3:ListMultipartUploadParts",
   "s3:CreateMultipartUpload"
   

3. 检查bucket-a中失败对象的存储属性：

   • 使用AWS CLI查看失败对象的版本历史，确认是否为分段上传创建的对象：

     aws s3api list-object-versions --bucket bucket-a --prefix <失败对象的键>
     

   • 对于存在未完成分段上传的对象，先清理未完成的分片，再重新触发复制：

     aws s3api abort-multipart-upload --bucket bucket-a --key <对象键> --upload-id <上传ID>
     

4. 重新触发批量复制任务：删除之前的失败批量任务，创建新的批量复制任务，针对失败对象进行重试。

内容的提问来源于stack exchange，提问作者hhaslam11