如何基于设备名称合并两个列表并补充关联字段？

需求说明 #

现有两个数据集：

1. 设备配置列表（list）：包含设备名称、配置命令等字段
2. 设备元数据列表（json.data）：包含设备名称、ID、插件类型字段

需要生成新列表，将配置列表中的设备与元数据列表匹配，补充对应的ID和PluginKey字段，优先使用高效方法处理大规模数据。

原始数据 #

设备配置列表 #

"list": [
    {
        "DEVICE_NAME": "uscx001.net",
        "IMPLEMENTATION": "system\nInterface range Gi1/10/0/42 to Gi1/10/0/45 Gi2/10/0/32 to Gi2/10/0/34\ndefault\nport link-mode bridge\ndescription *::HP-HSP::Available::::\nport access vlan 999\nshutdown\nreturn\nsave force",
        "POST_CHECK": "dis inter brief | inc 1/10/0/4[2345]|2/10/0/3[234]",
        "PRE_CHECK": "dis inter brief | inc 1/10/0/4[2345]|2/10/0/3[234]",
        "NO": "1"
    },
    {
        "DEVICE_NAME": "uscx002.net",
        "IMPLEMENTATION": "system\nInterface range Gi1/9/0/9 to Gi1/9/0/12 Gi2/9/0/9 to Gi2/9/0/11\ndefault\nport link-mode bridge\ndescription *::HP-HSP::Available::::\nport access vlan 999\nshutdown\nreturn\nsave force",
        "POST_CHECK": "dis inter brief | inc 1/9/0/9|1/9/0/1[012]|2/9/0/9|2/9/0/1[01]",
        "PRE_CHECK": "dis inter brief | inc 1/9/0/9|1/9/0/1[012]|2/9/0/9|2/9/0/1[01]",
        "NO": "2"
    }
]

设备元数据列表 #

"json": {
    "offset": 0,
    "limit": 1000,
    "search": {},
    "filter_op": "and",
    "total": 701,
    "fields": [
        "ID",
        "Name",
        "PluginKey"
    ],
    "data": [
        {
            "ID": 54,
            "Name": "inet-fw.lab.net",
            "PluginKey": "checkpoint_gaia"
        },
        {
            "ID": 2558,
            "Name": "uscx001.net",
            "PluginKey": "hp_aseries_alt_2"
        },
        {
            "ID": 2559,
            "Name": "uscx002.net",
            "PluginKey": "hp_aseries_alt_2"
        },
        {
            "ID": 2560,
            "Name": "test1.net",
            "PluginKey": "hp_aseries_alt_2"
        }
    ]
}

高效解决方案（Python实现） #

针对大规模数据，先将元数据列表转换为设备名到元数据的映射字典，实现O(1)时间复杂度的匹配查询，再遍历配置列表补充字段，整体时间复杂度O(n+m)，避免嵌套循环的O(n*m)低效问题。

# 假设已加载两个数据集到变量：config_list 和 metadata_json
config_list = [...]  # 第一个列表的内容
metadata_data = metadata_json["data"]

# 构建设备名到元数据的映射字典
device_metadata_map = {item["Name"]: {"ID": item["ID"], "PluginKey": item["PluginKey"]} for item in metadata_data}

# 生成关联后的新列表
list2 = []
for device in config_list:
    device_name = device["DEVICE_NAME"]
    # 匹配元数据，存在则补充字段
    if device_name in device_metadata_map:
        # 合并字典，避免修改原数据
        merged_device = {**device, **device_metadata_map[device_name]}
        list2.append(merged_device)

# 输出结果
print(list2)

结果示例 #

"list2": [
    {
        "DEVICE_NAME": "uscx001.net",
        "IMPLEMENTATION": "system\nInterface range Gi1/10/0/42 to Gi1/10/0/45 Gi2/10/0/32 to Gi2/10/0/34\ndefault\nport link-mode bridge\ndescription *::HP-HSP::Available::::\nport access vlan 999\nshutdown\nreturn\nsave force",
        "POST_CHECK": "dis inter brief | inc 1/10/0/4[2345]|2/10/0/3[234]",
        "PRE_CHECK": "dis inter brief | inc 1/10/0/4[2345]|2/10/0/3[234]",
        "NO": "1",
        "ID": 2558,
        "PluginKey": "hp_aseries_alt_2"
    },
    {
        "DEVICE_NAME": "uscx002.net",
        "IMPLEMENTATION": "system\nInterface range Gi1/9/0/9 to Gi1/9/0/12 Gi2/9/0/9 to Gi2/9/0/11\ndefault\nport link-mode bridge\ndescription *::HP-HSP::Available::::\nport access vlan 999\nshutdown\nreturn\nsave force",
        "POST_CHECK": "dis inter brief | inc 1/9/0/9|1/9/0/1[012]|2/9/0/9|2/9/0/1[01]",
        "PRE_CHECK": "dis inter brief | inc 1/9/0/9|1/9/0/1[012]|2/9/0/9|2/9/0/1[01]",
        "NO": "2",
        "ID": 2559,
        "PluginKey": "hp_aseries_alt_2"
    }
]

内容的提问来源于stack exchange，提问作者Tomáš Halagan