能否通过单次BatchGetItem从DynamoDB获取100条以上数据？

如果你使用AWS官方SDK（比如Python的boto3、JavaScript的AWS SDK v3），完全可以借助SDK的并发能力实现高效批量获取，不用自己写递归拆分逻辑：

• Python (boto3)：利用ThreadPoolExecutor并行发送分块请求。先将键数组按BatchGetItem的上限（单请求最多100个键）拆分，再用线程池同时发起多个请求，最后合并结果。示例代码：

  import boto3
  from concurrent.futures import ThreadPoolExecutor
  
  dynamodb = boto3.client('dynamodb')
  table_name = 'YourTableName'
  all_keys = [{'id': {'S': f'key-{i}'}} for i in range(500)]  # 示例键集合
  chunk_size = 100
  key_chunks = [all_keys[i:i+chunk_size] for i in range(0, len(all_keys), chunk_size)]
  
  def fetch_chunk(chunk):
      response = dynamodb.batch_get_item(RequestItems={
          table_name: {'Keys': chunk}
      })
      return response['Responses'][table_name]
  
  with ThreadPoolExecutor(max_workers=5) as executor:
      results = list(executor.map(fetch_chunk, key_chunks))
  
  # 合并所有返回结果
  all_items = [item for chunk_result in results for item in chunk_result]
  

• JavaScript (AWS SDK v3)：用Promise.all()并行处理分块请求。同样按100个键拆分后，同时发起多个batchGetItem调用，等待全部请求完成后合并数据。示例代码：

  import { DynamoDBClient, BatchGetItemCommand } from "@aws-sdk/client-dynamodb";
  
  const client = new DynamoDBClient({});
  const tableName = "YourTableName";
  const allKeys = Array.from({length: 500}, (_, i) => ({id: {S: `key-${i}`}}));
  const chunkSize = 100;
  const keyChunks = [];
  
  for (let i = 0; i < allKeys.length; i += chunkSize) {
      keyChunks.push(allKeys.slice(i, i + chunkSize));
  }
  
  const fetchChunk = async (chunk) => {
      const command = new BatchGetItemCommand({
          RequestItems: {
              [tableName]: { Keys: chunk }
          }
      });
      const response = await client.send(command);
      return response.Responses[tableName];
  };
  
  // 并行发起所有请求
  const results = await Promise.all(keyChunks.map(fetchChunk));
  const allItems = results.flat();
  

另外，部分第三方封装SDK（比如Python的pynamodb）已经内置了批量获取的并行处理逻辑，直接调用对应方法即可，无需手动拆分键数组。

需要注意：并行请求数量建议控制在5-10个，避免触发AWS API的并发限流。

内容的提问来源于stack exchange，提问作者cafe123