如何使用Python按一级Key收集并重新解析JSON数据

现有一组JSON数据，每个元素包含system字段和单个data对象，需要将相同system值的元素合并，把对应的data对象汇总为数组，最终输出分组后的JSON格式。

原始输入JSON #

[{"system": "test111","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.000000","value": 0}},{"system": "test111","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.111111","value": 11}},{"system": "test111","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.222222","value": 22}},{"system": "test222","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.333333","value": 33}},{"system": "test222","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.444444","value": 44}},{"system": "test333","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.555555","value": 55}},{"system": "test333","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.666666","value": 66}},{"system": "test444","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.777777","value": 77}},{"system": "test444","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.888888","value": 88}},{"system": "test555","data": {"title": "SomeValue","time": "2022-10-22T10:36:43.999999","value": 99}}]

目标输出JSON #

[{"system": "test111","data": [{"title": "SomeValue","time": "2022-10-22T10:36:43.000000","value": 0},{"title": "SomeValue","time": "2022-10-22T10:36:43.111111","value": 11},{"title": "SomeValue","time": "2022-10-22T10:36:43.222222","value": 22}]},{"system": "test222","data": [{"title": "SomeValue","time": "2022-10-22T10:36:43.333333","value": 33},{"title": "SomeValue","time": "2022-10-22T10:36:43.444444","value": 44}]},{"system": "test333","data": [{"title": "SomeValue","time": "2022-10-22T10:36:43.555555","value": 55},{"title": "SomeValue","time": "2022-10-22T10:36:43.666666","value": 66}]},{"system": "test444","data": [{"title": "SomeValue","time": "2022-10-22T10:36:43.777777","value": 77},{"title": "SomeValue","time": "2022-10-22T10:36:43.888888","value": 88}]},{"system": "test555","data": [{"title": "SomeValue","time": "2022-10-22T10:36:43.999999","value": 99}]}]

解决方法 #

方法1：普通字典手动分组 #

手动遍历元素，用system作为键存储对应data列表，最后转换为目标格式：

import json

# 加载原始JSON为Python列表
raw_data = json.loads('你的原始JSON字符串')
grouped_dict = {}

for item in raw_data:
    system_key = item['system']
    if system_key not in grouped_dict:
        grouped_dict[system_key] = []
    grouped_dict[system_key].append(item['data'])

# 转换为目标格式
result = [{'system': k, 'data': v} for k, v in grouped_dict.items()]

# 输出格式化后的JSON
print(json.dumps(result, indent=2))

方法2：用collections.defaultdict简化代码 #

defaultdict自动初始化不存在的键对应的列表，减少判断逻辑：

import json
from collections import defaultdict

raw_data = json.loads('你的原始JSON字符串')
grouped_dict = defaultdict(list)

for item in raw_data:
    grouped_dict[item['system']].append(item['data'])

result = [{'system': k, 'data': v} for k, v in grouped_dict.items()]
print(json.dumps(result, indent=2))

方法3：用itertools.groupby分组 #

注意：groupby要求数据先按分组键排序，否则仅会连续分组相同元素：

import json
from itertools import groupby

raw_data = json.loads('你的原始JSON字符串')
# 先按system字段排序
sorted_data = sorted(raw_data, key=lambda x: x['system'])

result = []
for system_key, group in groupby(sorted_data, key=lambda x: x['system']):
    data_list = [item['data'] for item in group]
    result.append({'system': system_key, 'data': data_list})

print(json.dumps(result, indent=2))

内容的提问来源于stack exchange，提问作者Tinke