DynamoDB 如何按非主键/排序键字段查询并生成指定格式结果
DynamoDB 非主键属性筛选及结果聚合实现
场景说明
- 现有DynamoDB表结构:以
user为分区键,Time为排序键 - 筛选需求:提取所有
type属性值为Product Team的记录,按指定结构聚合输出 - 已知限制:
type不属于主键或排序键,无法通过get_item方法直接按type条件定向查询
表内示例数据
原示例数据存在字段间缺失逗号的语法问题,已做修正
{ "user": "a@gmail.com", "type": "Product Team", "message": "Developer", "employeeId": "101", "message_requested":"Requested for the 192.168.1.1 access", "Time": "2021-01-08 12:09:54.986542" }, { "user": "a@gmail.com", "type": "Product Team", "message": "Developer", "employeeId": "101", "message_requested":"Requested for the 192.168.1.2 access", "Time": "2021-01-09 12:10:54.986542" }, { "user": "c@gmail.com", "type": "Ops Team", "message": "Tester", "employeeId": "102", "message_requested":"Requested for the 192.168.1.1 access", "Time": "2021-01-08 11:09:54.986542" }
目标输出结构
{ "user": "a@gmail.com", "params": [ { "message_requested": "Requested for the 192.168.1.1 access", "Time": "2021-01-08 12:09:54.986542" }, { "message_requested": "Requested for the 192.168.1.2 access", "Time": "2021-01-09 12:10:54.986542" } ] }
现有代码问题
当前代码尝试用scan方法按type值筛选记录,但未实现结果聚合逻辑,无法直接输出目标格式;同时注释掉的get_item实现存在两处错误:一是get_item仅支持通过主键查询,无法按非主键的type过滤;二是TableName参数错误传入了表对象,应当传入表名字符串。
def details(type): dynamodb = boto3.resource('dynamodb') client = boto3.client('dynamodb') table = dynamodb.Table("my_db_table_name") # 错误实现:get_item仅支持主键查询,不支持按type过滤 #res = client.get_item( # TableName=table, # Key={ # 'type': {'S': 'Product Team' # }}) res = table.scan(FilterExpression=Attr("type").eq("Product Team")) return res
可落地的聚合实现逻辑
- 导入依赖:从
boto3.dynamodb.conditions引入Attr类,支持FilterExpression语法 - 处理scan分页:
scan单次返回最多1MB数据,需循环读取分页结果,避免大表场景漏查数据 - 结果分组聚合:将筛选后的匹配记录按
user维度分组,同用户下的message_requested、Time字段归集到params数组,匹配目标输出结构
from boto3.dynamodb.conditions import Attr import boto3 def details(filter_type: str): dynamodb = boto3.resource('dynamodb') table = dynamodb.Table("my_db_table_name") scan_kwargs = { "FilterExpression": Attr("type").eq(filter_type) } matched_items = [] # 循环读取所有分页的匹配记录 while True: response = table.scan(**scan_kwargs) matched_items.extend(response.get("Items", [])) # 无后续分页时终止循环 if "LastEvaluatedKey" not in response: break scan_kwargs["ExclusiveStartKey"] = response["LastEvaluatedKey"] # 按user做分组聚合 grouped_result = {} for item in matched_items: current_user = item["user"] if current_user not in grouped_result: grouped_result[current_user] = { "user": current_user, "params": [] } grouped_result[current_user]["params"].append({ "message_requested": item.get("message_requested", ""), "Time": item.get("Time", "") }) # 返回所有匹配分组,按需求提取对应user的结果即可 return list(grouped_result.values()) # 调用示例 if __name__ == "__main__": result = details("Product Team") # 输出a@gmail.com对应的聚合结果,和目标结构一致 print(result[0])
内容的提问来源于stack exchange,提问作者user14257643
相关产品推荐
相关产品推荐

