AWS SDK v3 JavaScript GetObjectCommand未返回预期PDF二进制内容

问题原因 #

在AWS SDK for JavaScript v3中，GetObjectCommand返回的response.Body是Node.js可读流（IncomingMessage），而非直接的二进制数据，所以直接await response.Body无法得到预期内容。

解决方案 #

需要将可读流转换为Buffer（二进制内容），以下是几种可行实现方式：

方法1：使用stream.promises的pipeline（推荐） #

利用Node.js内置模块处理流，自带错误处理，代码更简洁：

const { S3Client, GetObjectCommand } = require("@aws-sdk/client-s3");
const { pipeline } = require('stream/promises');
const { Writable } = require('stream');

let client = new S3Client({
    region: 'xxxxx',
    credentials: {
        accessKeyId: 'xxxxx',
        secretAccessKey: 'xxxxx'
    }
});

const getObjectFromS3 = async (pdfFileName) => {
    const command = new GetObjectCommand({
        Bucket: "xxxxxxx",
        Key: pdfFileName,
    });
    const response = await client.send(command);
    
    // 将可读流转换为Buffer
    const chunks = [];
    await pipeline(
        response.Body,
        new Writable({
            write(chunk, _, callback) {
                chunks.push(chunk);
                callback();
            }
        })
    );
    const content = Buffer.concat(chunks);
    console.log(content); // 此时为PDF的二进制Buffer
    return content;
};

方法2：手动监听流事件 #

通过监听流的data/end/error事件收集数据块，最终合并为Buffer：

const { S3Client, GetObjectCommand } = require("@aws-sdk/client-s3");

let client = new S3Client({
    region: 'xxxxx',
    credentials: {
        accessKeyId: 'xxxxx',
        secretAccessKey: 'xxxxx'
    }
});

const streamToBuffer = (stream) => {
    return new Promise((resolve, reject) => {
        const chunks = [];
        stream.on('data', (chunk) => chunks.push(chunk));
        stream.on('end', () => resolve(Buffer.concat(chunks)));
        stream.on('error', reject);
    });
};

const getObjectFromS3 = async (pdfFileName) => {
    const command = new GetObjectCommand({
        Bucket: "xxxxxxx",
        Key: pdfFileName,
    });
    const response = await client.send(command);
    const content = await streamToBuffer(response.Body);
    console.log(content);
    return content;
};

方法3：兼容旧版Node.js的实现 #

如果Node.js版本低于15.x，用util.promisify包装传统stream.pipeline：

const { S3Client, GetObjectCommand } = require("@aws-sdk/client-s3");
const { pipeline } = require('stream');
const { promisify } = require('util');
const pipelineAsync = promisify(pipeline);

let client = new S3Client({
    region: 'xxxxx',
    credentials: {
        accessKeyId: 'xxxxx',
        secretAccessKey: 'xxxxx'
    }
});

const getObjectFromS3 = async (pdfFileName) => {
    const command = new GetObjectCommand({
        Bucket: "xxxxxxx",
        Key: pdfFileName,
    });
    const response = await client.send(command);
    
    const chunks = [];
    await pipelineAsync(
        response.Body,
        (source) => {
            source.on('data', (chunk) => chunks.push(chunk));
            return source;
        }
    );
    const content = Buffer.concat(chunks);
    console.log(content);
    return content;
};

验证方式 #

转换后的content是Buffer类型，可通过以下方式验证：

const fs = require('fs/promises');
await fs.writeFile('local.pdf', content);

打开生成的local.pdf，若能正常显示则转换成功。

内容的提问来源于stack exchange，提问作者vapdev