S3对象上传触发Lambda偶现NoSuchKey错误排查求助

搭建了一套流程：用户将文件上传至S3存储桶后，触发Lambda函数对该文件进行处理。但目前遇到偶现的NoSuchKey错误，多次上传同一文件时，有时能正常运行得到预期结果，有时则报错。请问这是否是Lambda在文件未完全上传时就尝试读取导致的？或是函数超时引发的？

相关代码 #

import {
  S3Client,
  GetObjectCommand,
} from "@aws-sdk/client-s3";

export const handler = async (event) => {
  console.log('Event -'+JSON.stringify(event));
  const s3Client = new S3Client({
      region: process.env.REGION, 
      credentials: {
        accessKeyId: process.env.ACCESS_KEY_ID,
        secretAccessKey: process.env.SECRET_ACCESS_KEY,
    },});


  // Read the object.
  const { Body } = await s3Client.send(
    new GetObjectCommand({
      Bucket: event.Records[0].s3.bucket.name,
      Key: event.Records[0].s3.object.key
    })
  );
  console.log(await Body.transformToString());
}

错误示例 #

{
    "errorType": "NoSuchKey",
    "errorMessage": "The specified key does not exist.",
    "name": "NoSuchKey",
    "$fault": "client",
    "$metadata": {
        "httpStatusCode": 404,
        "requestId": "FJ4VQ3NWCX5TKPJB",
        "extendedRequestId": "EPS8NoiKhdjQdPEWUvC4mtyOYwMAKzLoaglVEeZPZz/G5ZCViAZ0eV4AekjPTIs0k/GUL6peTME=",
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "Code": "NoSuchKey",
    "Key": "Parts+List.txt",
    "RequestId": "FJ4VQ3NWCX5TKPJB",
    "HostId": "hostId",
    "message": "The specified key does not exist.",
    "stack": [
        "NoSuchKey: The specified key does not exist.",
        "    at de_NoSuchKeyRes (/var/runtime/node_modules/@aws-sdk/client-s3/dist-cjs/protocols/Aws_restXml.js:6082:23)",
        "    at de_GetObjectCommandError (/var/runtime/node_modules/@aws-sdk/client-s3/dist-cjs/protocols/Aws_restXml.js:4327:25)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
        "    at async /var/runtime/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24",
        "    at async /var/runtime/node_modules/@aws-sdk/middleware-signing/dist-cjs/awsAuthMiddleware.js:14:20",
        "    at async /var/runtime/node_modules/@aws-sdk/middleware-retry/dist-cjs/retryMiddleware.js:27:46",
        "    at async /var/runtime/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/flexibleChecksumsMiddleware.js:58:20",
        "    at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:7:26",
        "    at async Runtime.handler (file:///var/task/index.mjs:25:20)"
    ]
}

原因分析 #

1. Lambda在文件未完全上传时触发：这是最可能的诱因。S3采用最终一致性模型，新上传的对象可能先触发Lambda事件，但此时对象尚未完全同步到所有存储节点，导致GetObject请求返回NoSuchKey。这种情况在大文件上传、跨区域复制场景下更容易出现。
2. 排除函数超时：从错误日志看，请求仅尝试1次且无重试延迟，直接返回404，并非超时。超时错误会明确显示Task timed out，因此可以排除该可能。

解决方案 #

• 通过EventBridge触发Lambda：将S3事件目标改为EventBridge，而非直接触发Lambda。EventBridge会等待对象完全上传后再推送事件，确保对象已存在于S3中。
• 添加重试机制：利用AWS SDK内置的重试功能，增加GetObject请求的重试次数，应对临时的一致性问题。修改S3客户端配置示例：

  const s3Client = new S3Client({
      region: process.env.REGION, 
      credentials: {
        accessKeyId: process.env.ACCESS_KEY_ID,
        secretAccessKey: process.env.SECRET_ACCESS_KEY,
      },
      retryMode: 'adaptive',
      maxAttempts: 3
  });
  

• 前置对象存在性检查：在读取对象前，调用HeadObjectCommand验证对象是否存在，若不存在则延迟一段时间后重试。

内容的提问来源于stack exchange，提问作者Sumchans