如何用Jest与AWS SDK V3模拟S3 Select的SelectObjectContentCommand响应

问题描述 #

我在Node.js应用中使用S3 Select查询CSV文件并提取数据行，请问如何编写单元测试来模拟SelectObjectContentCommand的响应？该响应类型为SelectObjectContentCommandOutput。

原业务代码 #

import { SelectObjectContentCommand, S3Client, ExpressionType, FileHeaderInfo } from '@aws-sdk/client-s3';

const s3Client = new S3Client({});

export const getDataUsingS3Select = async (bucketName: string, fileName: string, query: string, params: any) => {
  
  const command = new SelectObjectContentCommand(params);
  const response = await s3Client.send(command);

  if (response.$metadata.httpStatusCode === 200) {
    if (response.Payload) {
      const data = convertDataToJson(response.Payload);
      return data;
    } else {
      console.log(`S3Select did not have payload for ${bucketName}/${fileName}`);
      return [];
    }
  } else {
    console.log(`S3Select did not receive 200 for ${bucketName}/${fileName}. 
    Metadata is ${response.$metadata}`);
    return [];
  }
}

// 假设这是处理Payload的辅助函数
export const convertDataToJson = async (payload: AsyncIterable<any>) => {
  const result = [];
  // 实际业务中会解析CSV数据，此处为示例逻辑
  for await (const chunk of payload) {
    if (chunk.Records) {
      const lines = chunk.Records.Payload.toString().split('\n').filter(Boolean);
      result.push(...lines.map(line => line.split(',')));
    }
  }
  return result;
}

单元测试实现方案 #

1. 先做代码重构（关键） #

原代码中直接实例化S3Client，不利于测试时替换为模拟对象。建议将s3Client作为参数传入函数，通过依赖注入降低耦合：

// 重构后的函数：新增s3Client参数，默认值为实例化的S3Client
export const getDataUsingS3Select = async (
  bucketName: string, 
  fileName: string, 
  query: string, 
  params: any,
  s3Client: S3Client = new S3Client({})
) => {
  // 业务逻辑不变
}

2. 使用Jest编写测试用例 #

通过模拟S3Client的send方法，构造符合SelectObjectContentCommandOutput类型的响应，覆盖所有分支场景。

测试代码示例

import { jest } from '@jest/globals';
import { S3Client, SelectObjectContentCommandOutput } from '@aws-sdk/client-s3';
import { getDataUsingS3Select, convertDataToJson } from './your-file-path';

// 模拟S3Client类及其send方法
jest.mock('@aws-sdk/client-s3', () => {
  const original = jest.requireActual('@aws-sdk/client-s3');
  return {
    ...original,
    S3Client: jest.fn().mockImplementation(() => ({
      send: jest.fn(),
    })),
  };
});

describe('getDataUsingS3Select', () => {
  let mockS3Client: jest.Mocked<S3Client>;

  beforeEach(() => {
    // 每次测试前初始化模拟的S3Client
    mockS3Client = new S3Client({}) as jest.Mocked<S3Client>;
  });

  // 场景1：成功返回解析后的CSV数据
  test('成功解析S3 Select返回的Payload', async () => {
    // 构造模拟的异步迭代器Payload，模拟S3返回的Records事件
    const mockPayload: AsyncIterable<any> = {
      [Symbol.asyncIterator]: async function* () {
        yield {
          Records: {
            Payload: Buffer.from('name,age\nAlice,30\nBob,25'),
          },
        };
      },
    };

    const mockResponse: Partial<SelectObjectContentCommandOutput> = {
      $metadata: { httpStatusCode: 200 },
      Payload: mockPayload,
    };

    // 让send方法返回模拟响应
    mockS3Client.send.mockResolvedValue(mockResponse as SelectObjectContentCommandOutput);

    const result = await getDataUsingS3Select(
      'test-bucket', 
      'test.csv', 
      'SELECT * FROM S3Object', 
      {}, 
      mockS3Client
    );

    expect(result).toEqual([['name', 'age'], ['Alice', '30'], ['Bob', '25']]);
    expect(mockS3Client.send).toHaveBeenCalledTimes(1);
  });

  // 场景2：Payload为空时返回空数组
  test('无Payload时返回空数组', async () => {
    const mockResponse: Partial<SelectObjectContentCommandOutput> = {
      $metadata: { httpStatusCode: 200 },
      Payload: undefined,
    };

    mockS3Client.send.mockResolvedValue(mockResponse as SelectObjectContentCommandOutput);

    const result = await getDataUsingS3Select(
      'test-bucket', 
      'test.csv', 
      'SELECT * FROM S3Object', 
      {}, 
      mockS3Client
    );

    expect(result).toEqual([]);
  });

  // 场景3：非200状态码时返回空数组
  test('请求失败（非200状态）返回空数组', async () => {
    const mockResponse: Partial<SelectObjectContentCommandOutput> = {
      $metadata: { httpStatusCode: 400, requestId: 'test-request-id' },
    };

    mockS3Client.send.mockResolvedValue(mockResponse as SelectObjectContentCommandOutput);

    const result = await getDataUsingS3Select(
      'test-bucket', 
      'test.csv', 
      'SELECT * FROM S3Object', 
      {}, 
      mockS3Client
    );

    expect(result).toEqual([]);
  });
});

关键注意点 #

• 模拟Payload的结构：S3 Select的Payload是异步迭代器，需要通过Symbol.asyncIterator构造可遍历的模拟数据，模拟实际返回的Records（包含CSV数据）、Stats、End等事件。
• 覆盖所有分支：测试成功解析、无Payload、非200状态三种核心场景，保证逻辑完整性。
• 避免硬依赖：通过依赖注入S3Client，让测试可以灵活替换为模拟对象，不依赖真实AWS服务。

内容的提问来源于stack exchange，提问作者Denil Parmar