AWS S3 Javascript SDK V2调用deleteObjects()批量删除失败求助

错误日志 #

0923a739-1498-4c61-89c0-30cbb70c8008  INFO  500: null
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/s3.js:710:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 500,
  region: null,
  time: 2022-10-28T13:29:37.291Z,
  requestId: null,
  extendedRequestId: undefined,
  cfId: undefined,
  statusCode: 500,
  retryable: true
}

实现代码 #

const keys = listObjectsV2Output?.Contents.map((content) => content.Key) as [];
console.log(`Deleting ${keys?.length} number of keys inside ${bucket}/${Key}. Keys to delete: ${JSON.stringify(keys)}`);

// Bulk delete upto 1000 keys in one requestBody
let objectIdentifierList: ObjectIdentifierList = [];
for (let key of keys) {
  const objectIdentifier: ObjectIdentifier = {
    Key: key,
  };
  objectIdentifierList.push(objectIdentifier);
}
const deleteKeys: Delete = {
  Objects: objectIdentifierList,
  Quiet: true,
};
const deleteObjectsRequest: DeleteObjectsRequest = {
  Bucket: bucket,
  Delete: deleteKeys,
};
console.log(`S3 deleteObjects deleteObjectsRequest : ${JSON.stringify(deleteObjectsRequest)}`);

const deleteObjectsOutput: DeleteObjectsOutput = await s3Client.deleteObjects(deleteObjectsRequest).promise();
console.log(`S3 bulk deleteObjects response : ${JSON.stringify(deleteObjectsOutput)}`);
console.log(`Errors during delete : ${JSON.stringify(deleteObjectsOutput.Errors)}`);
allKeysDeleted = deleteObjectsOutput.Errors?.length == 0;

请求示例 #

{
    "Bucket": "mybucket",
    "Delete": {
        "Objects": [
            {
                "Key": "clients/C1/sites/G100/camear/MP4_H264-1660553709-1660554009 - Copy.mp4"
            },
            {
                "Key": "clients/C1/sites/G100/camear/MP4_H264-1660554009-1660554309 - Copy (2).mp4"
            },
            {
                "Key": "clients/C1/sites/G100/camear/"
            },
            {
                "Key": "clients/C1/sites/G100/siteInfo"
            },
            {
                "Key": "clients/C1/sites/G100/"
            }
        ],
        "Quiet": true
    }
}

排查与解决方法 #

1. 移除前缀（目录）Key #

从请求示例可以看到，你尝试删除以斜杠结尾的Key（如clients/C1/sites/G100/camear/），但S3中这些不是实际存储的对象，只是对象的前缀路径。deleteObjects()接口只能删除实际存在的对象，无法删除前缀，这是导致500错误的核心原因。

修改代码过滤掉以斜杠结尾的Key：

for (let key of keys) {
  // 跳过以斜杠结尾的前缀路径
  if (key.endsWith('/')) continue;
  const objectIdentifier: ObjectIdentifier = {
    Key: key,
  };
  objectIdentifierList.push(objectIdentifier);
}

2. 检查权限配置 #

确保执行删除操作的IAM实体（用户/角色）拥有s3:DeleteObject权限，且权限策略覆盖目标Bucket及所有待删除的对象路径：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:DeleteObject",
            "Resource": "arn:aws:s3:::mybucket/*"
        }
    ]
}

3. 开启SDK调试日志 #

如果问题仍存在，开启SDK调试日志，查看更详细的请求/响应细节：

const s3Client = new AWS.S3({
  // 原有配置
  logger: console,
  apiVersion: '2006-03-01',
  // 开启调试日志
  paramValidation: false,
  computeChecksums: true,
});

4. 配置重试策略 #

错误返回中retryable: true，说明这是可重试的错误。可以为SDK配置自动重试规则：

const s3Client = new AWS.S3({
  retryDelayOptions: { base: 1000 }, // 初始重试延迟1秒
  maxRetries: 3, // 最多重试3次
});

内容的提问来源于stack exchange，提问作者Jay