CodePipeline集成本地Bitbucket Server时S3上传报错排查

按照AWS官方示例步骤将AWS CodePipeline与本地Bitbucket Server集成，推送代码至仓库时触发Lambda执行报错，错误信息如下：

File downloaded successfully: <ref *2> IncomingMessage {    
  _events: 
    close: [Function: onclose],
    error: [Function: onerror],
    data: undefined,
    end: [ [Function: responseOnEnd], [Function: onend] ],
    readable: undefined,
    finish: [Function: onfinish]
  
WARN    Are you using a Stream of unknown length as the Body of a PutObject request? Consider using Upload instead from @aws-sdk/lib-storage.
WARN    An error was encountered in a non-retryable streaming request.
INFO    Exiting with error NotImplemented: A header you provided implies functionality that is not implemented
    at throwDefaultError (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:838:20)
    at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:847:5
    at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4756:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /var/runtime/node_modules/@aws-sdk/middleware-signing/dist-cjs/index.js:225:18
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /var/runtime/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/index.js:173:18
    at async /var/runtime/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:97:20
    at async /var/runtime/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:120:14 {
  '$fault': 'client',
  '$metadata': 
    httpStatusCode: 501,
    requestId: 'G64RMGHEPWVDE',
    extendedRequestId: 'FTZNy/uEqwgkrWH3XP/QIIiCIZAc5f+ZxCG3tFJp7QhFrCopRunG5ScmqfMxbpWus3o6cuXoC1M=',
    cfId: undefined
  
  Code: 'NotImplemented',
  Header: 'Transfer-Encoding',
  RequestId: 'G64RMGHYPWVDE',
  HostId: 'FTZNy/uEqwgkrWH3XP/QIIiCIZAc5f3tFJp7QhFrCopRunG5ScmqfMxbpWus3o6cuXoC1M='
}

已配置的Bucket Policy：

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxxx:role/lambda-executionrole"
            },
            "Action": [
                "s3:Get*",
                "s3:List*",
                "s3:Put*",
                "s3:Delete*"
            ],
            "Resource": [
                "arn:aws:s3:::bucketname",
                "arn:aws:s3:::bucketname/*"
            ]
        }
    ]
}

Lambda IAM角色已附加S3全访问和AWSLambdaVPCAccessExecutionRole托管策略，自定义权限：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetBucket*",
                "s3:ListBucket*"
            ],
            "Resource": [
                "arn:aws:s3:::bucketname"
            ],
            "Effect": "Allow",
            "Sid": "S3BucketPolicy"
        },
        {
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:GetObject*",
                "s3:PutObject*",
                "s3:DeleteObject*",
                "s3:RestoreObject",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::bucketname/*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "kms:decrypt"
            ],
            "Resource": [
                "arn:aws:kms:us-east-1:xxxxxxxxx:key/xxxxxxxxxxxxx9"
            ],
            "Effect": "Allow"
        }
    ]
}

Lambda核心代码片段：

// Download the repository package from Bitbucket Server
const file = await downloadFile(repoConfig, proxy);
console.log('File downloaded successfully:', file);

// Create a PutObjectCommand with the zipped repository package key and file
const uploadCommand = new PutObjectCommand({
    Bucket: process.env.S3BUCKET,
    Key: `${repoConfig.projectName}/${repoConfig.repoName}/${repoConfig.branch}.zip`,
    Body: file,
    ServerSideEncryption: "AES256"
});

// Upload the repository package to S3 bucket
const s3Upload = await s3Client.send(uploadCommand);

报错核心原因：使用PutObjectCommand上传未知长度的流（Axios返回的IncomingMessage流无Content-Length头），S3不支持该场景的Transfer-Encoding: chunked上传，因此抛出NotImplemented错误。官方警告已明确提示改用@aws-sdk/lib-storage的Upload类处理。

具体修复步骤：

1. 添加依赖包 #

在Lambda部署包中安装@aws-sdk/lib-storage，用于支持未知长度流的分块上传。

2. 修改Lambda代码 #

替换原有的PutObjectCommand上传逻辑，改用Upload类处理：

// 替换原S3导入，新增Upload类
import {S3Client} from "@aws-sdk/client-s3";
import {Upload} from "@aws-sdk/lib-storage";
import axios from "axios";
import crypto from 'node:crypto';

// ... 保留原有工具函数不变 ...

export const handler = async (event) => {
    try {
        // ... 保留原有签名验证、仓库信息获取等逻辑 ...

        // 下载仓库压缩包
        const file = await downloadFile(repoConfig, proxy);
        console.log('File downloaded successfully:', file);

        // 配置上传参数
        const uploadParams = {
            Bucket: process.env.S3BUCKET,
            Key: `${repoConfig.projectName}/${repoConfig.repoName}/${repoConfig.branch}.zip`,
            Body: file,
            ServerSideEncryption: "AES256"
        };

        // 使用Upload类执行分块上传
        const upload = new Upload({
            client: s3Client,
            params: uploadParams
        });

        // 完成上传
        const s3Upload = await upload.done();

        console.log(s3Upload);
        console.log('Exiting successfully');
        return responseToApiGw(200, 'success');
    }
    catch (err) {
        console.log('Exiting with error', err);
        return responseToApiGw(500, err.message);
    }
}

3. 权限验证 #

当前IAM角色已包含s3:PutObject和s3:AbortMultipartUpload权限，满足分块上传要求，无需额外调整。

补充说明 #

@aws-sdk/lib-storage的Upload类会自动处理：

• 未知长度流的分块上传逻辑
• 上传中断后的断点续传
• 规避S3对Transfer-Encoding的限制

内容的提问来源于stack exchange，提问作者InteligentCloud