使用AWS Lambda流式生成Excel并上传S3时遇NotImplemented错误求助

问题场景 #

从数据库读取数据后，用ExcelJS流式生成Excel文件，再通过AWS Lambda上传到S3时触发如下错误：

NotImplemented: A header you provided implies functionality that is not implemented
...
Code: 'NotImplemented',
Header: 'Transfer-Encoding'

错误原因 #

AWS SDK在处理无长度的流（比如PassThrough）时，会默认添加Transfer-Encoding: chunked请求头，但S3的PutObject接口不支持分块编码上传，因此返回501 NotImplemented错误。

解决方案 #

方案1：先将流转为Buffer再上传（适合小数据量） #

把流式生成的Excel数据先缓存到Buffer中，再通过PutObjectCommand上传，这样SDK能自动计算Content-Length，避免分块编码：

const date = new Date().toISOString().split('T')[0];
const fileName = `reports/${date}/applications.xlsx`;

const chunks = [];
const passThrough = new stream.PassThrough();

// 收集流数据到chunks数组
passThrough.on('data', (chunk) => chunks.push(chunk));
passThrough.on('end', async () => {
  const buffer = Buffer.concat(chunks);
  
  const uploadCommand = new PutObjectCommand({
    Bucket: BUCKET_NAME,
    Key: fileName,
    Body: buffer,
    ContentType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
  });
  await s3.send(uploadCommand);
});

// 流式生成Excel
const workbook = new ExcelJS.stream.xlsx.WorkbookWriter({
  stream: passThrough,
  useStyles: true
});

const worksheet = workbook.addWorksheet('Applications');
const headers = Object.keys(applications[0]);
worksheet.addRow(headers);

for (let application of applications) {
  const row = headers.map(header => {
    if (hyperlinkHeaders.includes(header)) {
      return { text: application[header], hyperlink: application[header] };
    }
    return application[header];
  });
  worksheet.addRow(row);
}
await workbook.commit();

方案2：使用Multipart Upload（适合大数据量） #

用AWS SDK提供的@aws-sdk/lib-storage库中的Upload类，它会自动处理流式上传的分块逻辑，兼容S3的Multipart接口：

1. 先安装依赖：

npm install @aws-sdk/lib-storage

2. 修改上传代码：

const { Upload } = require("@aws-sdk/lib-storage");
const date = new Date().toISOString().split('T')[0];
const fileName = `reports/${date}/applications.xlsx`;

const passThrough = new stream.PassThrough();

// 流式生成Excel
const workbook = new ExcelJS.stream.xlsx.WorkbookWriter({
  stream: passThrough,
  useStyles: true
});

const worksheet = workbook.addWorksheet('Applications');
const headers = Object.keys(applications[0]);
worksheet.addRow(headers);

for (let application of applications) {
  const row = headers.map(header => {
    if (hyperlinkHeaders.includes(header)) {
      return { text: application[header], hyperlink: application[header] };
    }
    return application[header];
  });
  worksheet.addRow(row);
}
await workbook.commit();

// 用Multipart方式上传流
const upload = new Upload({
  client: s3,
  params: {
    Bucket: BUCKET_NAME,
    Key: fileName,
    Body: passThrough,
    ContentType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
  }
});

await upload.done();

内容的提问来源于stack exchange，提问作者jos j