如何用Python筛选列表中共享特定键值对的对象并拆分列表

如何找出列表中共享特定键值对（此处为host和path）的对象，拆分出两个列表：一个存放合并对应ID的重复组，另一个存放无重复的对象？

示例输入 #

[
    {
        "id": "111",
        "host": "aaa",
        "path": "/b/c/d"
    },
    {
        "id": "222",
        "host": "bbb",
        "path": "/x/y/z"
    },
    {
        "id": "333",
        "host": "aaa",
        "path": "/b/c/d"
    },
    {
        "id": "444",
        "host": "aaa",
        "path": "/b/c/d"
    }
]

期望输出 #

1. 合并重复项的列表（共享host和path） #

[
    {
        "host": "aaa",
        "path": "/b/c/d",
        "ids": [
            "111",
            "333",
            "444"
        ]
    }
]

2. 无重复项的列表 #

[
    {
        "id": "222",
        "host": "bbb",
        "path": "/x/y/z"
    }
]

我的错误尝试 #

我写的代码把所有对象都放进了dups_list，完全没区分开，代码如下：

import jsonpickle
main_list = list((dict(Id="111",host="aaa",path="/b/c/d"),dict(Id="222",host="bbb",path="/x/y/z"),dict(Id="333",host="aaa",path="/b/c/d"),dict(Id="444",host="aaa",path="/b/c/d")))
dups_list = list()
non_dups_list = list()
for o in main_list:
    is_duplicate = False
    for o2 in main_list:
        if o2['host'] == o['host'] and o2['path'] == o['path']:
            is_duplicate = True
            break
    if is_duplicate:
        dups_list.append(o)
    else:
        non_dups_list.append(o)

print(jsonpickle.encode(non_dups_list, indent=4))
# 输出：[]
print(jsonpickle.encode(dups_list, indent=4))
# 输出所有原始对象

正确解决方案 #

你之前的代码逻辑错误在于：每个对象都会和自身匹配，导致is_duplicate永远为True，所有对象都被判定为重复项。正确做法是先统计每个(host, path)组合的出现次数，再根据次数拆分并合并ID。

方法：使用字典分组统计 #

from collections import defaultdict
import json

main_list = [
    {"id": "111", "host": "aaa", "path": "/b/c/d"},
    {"id": "222", "host": "bbb", "path": "/x/y/z"},
    {"id": "333", "host": "aaa", "path": "/b/c/d"},
    {"id": "444", "host": "aaa", "path": "/b/c/d"}
]

# 1. 按(host, path)分组收集对应ID
grouped = defaultdict(list)
for item in main_list:
    key = (item['host'], item['path'])
    grouped[key].append(item['id'])

# 2. 拆分重复组和非重复组
merged_duplicates = []
non_duplicates = []

for (host, path), ids in grouped.items():
    if len(ids) > 1:
        merged_duplicates.append({
            "host": host,
            "path": path,
            "ids": ids
        })
    else:
        non_duplicates.append({
            "id": ids[0],
            "host": host,
            "path": path
        })

# 打印结果
print("合并后的重复项列表：")
print(json.dumps(merged_duplicates, indent=4))
print("\n非重复项列表：")
print(json.dumps(non_duplicates, indent=4))

代码说明 #

1. 分组统计：用(host, path)作为字典的键，将对应id收集到列表中，快速统计每个组合的ID数量。
2. 拆分逻辑：遍历分组后的字典，ID列表长度大于1的为重复组，合并成指定结构；长度为1的为非重复项，还原原对象结构。

运行结果 #

合并后的重复项列表：
[
    {
        "host": "aaa",
        "path": "/b/c/d",
        "ids": [
            "111",
            "333",
            "444"
        ]
    }
]

非重复项列表：
[
    {
        "id": "222",
        "host": "bbb",
        "path": "/x/y/z"
    }
]

内容的提问来源于stack exchange，提问作者David Gard