如何更高效实现Python字典列表IP去重并累加bytes值？

输入数据如下：

[{"ip": "1.2.3.4", "bytes": 10}, 
 {"ip": "2.3.4.10", "bytes": 10}, 
 {"ip": "5.6.2.3", "bytes": 10},
 {"ip": "1.2.3.4", "bytes": 20}, 
 {"ip": "1.2.3.4", "bytes": 5}, 
 {"ip": "2.3.4.10", "bytes": 1},
 {"ip": "10.20.30.40", "bytes": 0}, 
 {"ip": "0.0.0.0", "bytes": 10}, 
 {"ip": "2.3.4.10", "bytes": 6}]

需求是：输出唯一IP地址，并对重复IP的bytes值累加，预期输出如下：

[{'ip': '0.0.0.0', 'bytes': 10}, 
 {'ip': '10.20.30.40', 'bytes': 0}, 
 {'ip': '2.3.4.10', 'bytes': 17}, 
 {'ip': '5.6.2.3', 'bytes': 10}, 
 {'ip': '1.2.3.4', 'bytes': 35}]

我已经写出了实现该功能的Python代码：

import json
logs = """[{"ip": "1.2.3.4", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 10}, {"ip": "5.6.2.3", "bytes": 10},
           {"ip": "1.2.3.4", "bytes": 20}, {"ip": "1.2.3.4", "bytes": 5}, {"ip": "2.3.4.10", "bytes": 1},
           {"ip": "10.20.30.40", "bytes": 0}, {"ip": "0.0.0.0", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 6}]"""

logs_json = json.loads(logs)

ips_unique = set(ip.get("ip") for ip in logs_json)

ip_unique_list = []
for ip in ips_unique:
        ip_dict = {"ip": ip, "bytes": 0}
        ip_unique_list.append(ip_dict)

for ip_unique_sep in ip_unique_list:
        for log in logs_json:
                if log["ip"] == ip_unique_sep["ip"]:
                        ip_unique_sep["bytes"] += log["bytes"]

print(ip_unique_list)

请问有没有更优、更高效的实现方式？

你的现有代码时间复杂度是O(n*m)（n是唯一IP数，m是日志条目数），数据量较大时效率会明显下降。下面提供几种更高效简洁的实现方式：

方式一：普通字典统计（一次遍历） #

只需要遍历一次日志数据，用IP作为键累加bytes值，最后转换为目标格式。时间复杂度O(n)，是最基础的高效实现：

import json

logs = """[{"ip": "1.2.3.4", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 10}, {"ip": "5.6.2.3", "bytes": 10},
           {"ip": "1.2.3.4", "bytes": 20}, {"ip": "1.2.3.4", "bytes": 5}, {"ip": "2.3.4.10", "bytes": 1},
           {"ip": "10.20.30.40", "bytes": 0}, {"ip": "0.0.0.0", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 6}]"""

logs_json = json.loads(logs)

# 统计每个IP的bytes总和
ip_stats = {}
for entry in logs_json:
    ip = entry["ip"]
    if ip in ip_stats:
        ip_stats[ip] += entry["bytes"]
    else:
        ip_stats[ip] = entry["bytes"]

# 转换为目标格式列表，可选排序匹配预期输出
result = [{"ip": ip, "bytes": total} for ip, total in ip_stats.items()]
result.sort(key=lambda x: x["ip"])

print(result)

方式二：用collections.defaultdict简化代码 #

借助defaultdict省去IP存在性判断，代码更简洁：

import json
from collections import defaultdict

logs = """[{"ip": "1.2.3.4", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 10}, {"ip": "5.6.2.3", "bytes": 10},
           {"ip": "1.2.3.4", "bytes": 20}, {"ip": "1.2.3.4", "bytes": 5}, {"ip": "2.3.4.10", "bytes": 1},
           {"ip": "10.20.30.40", "bytes": 0}, {"ip": "0.0.0.0", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 6}]"""

logs_json = json.loads(logs)

ip_stats = defaultdict(int)
for entry in logs_json:
    ip_stats[entry["ip"]] += entry["bytes"]

result = [{"ip": ip, "bytes": total} for ip, total in ip_stats.items()]
result.sort(key=lambda x: x["ip"])

print(result)

方式三：用collections.Counter适配累加场景 #

Counter专门用于计数/累加场景，代码最简洁：

import json
from collections import Counter

logs = """[{"ip": "1.2.3.4", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 10}, {"ip": "5.6.2.3", "bytes": 10},
           {"ip": "1.2.3.4", "bytes": 20}, {"ip": "1.2.3.4", "bytes": 5}, {"ip": "2.3.4.10", "bytes": 1},
           {"ip": "10.20.30.40", "bytes": 0}, {"ip": "0.0.0.0", "bytes": 10}, {"ip": "2.3.4.10", "bytes": 6}]"""

logs_json = json.loads(logs)

ip_stats = Counter()
for entry in logs_json:
    ip_stats[entry["ip"]] += entry["bytes"]

result = [{"ip": ip, "bytes": total} for ip, total in ip_stats.items()]
result.sort(key=lambda x: x["ip"])

print(result)

优化点说明 #

1. 效率提升：所有方案仅需一次遍历日志，时间复杂度从O(n*m)降至O(n)，数据量越大优势越明显。
2. 代码简洁性：使用defaultdict或Counter可消除冗余的存在性判断，代码更易读维护。
3. 排序可选：如果不需要匹配预期输出的IP排序顺序，可去掉sort步骤进一步提升效率。

内容的提问来源于stack exchange，提问作者Rad4