无需Pandas:基于状态去重ID,实现AWS数据导出至Excel
问题描述
我需要把AWS账户数据导出到Excel表格,目前用GraphQL获取数据,通过jmespath.search做映射,但存储时遇到了重复ID的问题。我想把deregistered、deactivated字段合并成active或inactive的状态字段,同时要去除重复ID,每个ID仅保留active状态的数据(如果存在)。
示例输入数据:
data = [ {"id": 1, "deregistered ": True, "deactivated": True, "location": True}, {"id": 1, "deregistered ": False, "deactivated": False, "location": True}, {"id": 2, "deregistered ": False, "deactivated": False, "location": True}, ]
期望输出:
output = [ {"id": 1, "status": "active", "location": True}, {"id": 2, "status": "inactive", "location": True} ]
我尝试了以下代码,但逻辑有问题,求不使用Pandas的实现方案:
for val in data: loc_enabled = val.get("location") if loc_enabled: search = """ { "id": id, "status": ((deregistered == `true` || deactivated == `true`) && `Inactive`) || `Active`, "location":location }""" test = jmespath.search(search, val) if test: loc_enabled.append(test) print(loc_enabled)
解决方案
可以用字典按ID做去重处理,优先保留active状态的条目,具体实现如下:
import jmespath data = [ {"id": 1, "deregistered ": True, "deactivated": True, "location": True}, {"id": 1, "deregistered ": False, "deactivated": False, "location": True}, {"id": 2, "deregistered ": False, "deactivated": False, "location": True}, ] # 修正JMESPath映射表达式,确保status判断逻辑正确 search_expr = """ { "id": id, "status": (deregistered == `false` && deactivated == `false`) ? `active` : `inactive`, "location": location } """ # 用字典存储去重后的数据,优先保留active状态条目 result_dict = {} for item in data: processed_item = jmespath.search(search_expr, item) current_id = processed_item["id"] # 若ID未记录,或当前条目是active状态,则更新存储 if current_id not in result_dict or processed_item["status"] == "active": result_dict[current_id] = processed_item # 将字典值转为列表,得到最终结果 output = list(result_dict.values()) print(output)
逻辑说明
- 修正JMESPath的
status判断逻辑:仅当deregistered和deactivated均为false时,状态设为active,否则为inactive,匹配需求规则 - 利用字典键的唯一性实现去重,同时通过判断
status为active时覆盖旧数据,保证每个ID优先保留active状态的条目 - 最后将字典的值转为列表,得到符合格式要求的输出
内容的提问来源于stack exchange,提问作者akdgp
相关产品推荐
相关产品推荐

