如何确定Pickle序列化对象各属性的磁盘占用以优化__getstate__？

要找出Pickle文件里占用空间最大的属性，你可以按以下步骤操作：

1. 先保存完整对象到Pickle文件 #

先把未做任何修改的完整对象序列化保存，示例代码：

import pickle

with open('full_object.pkl', 'wb') as f:
    pickle.dump(your_top_object, f)

2. 加载对象并计算各属性的内存占用 #

Pickle的磁盘占用和对象内存大小高度相关，通过计算内存大小就能定位大属性，有两种实现方式：

方法A：自定义递归计算函数 #

自己实现递归逻辑，遍历所有嵌套对象计算总内存：

import sys

def get_object_size(obj, seen=None):
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    seen.add(obj_id)
    # 处理字典类型
    if isinstance(obj, dict):
        size += sum(get_object_size(v, seen) for v in obj.values())
        size += sum(get_object_size(k, seen) for k in obj.keys())
    # 处理自定义类实例
    elif hasattr(obj, '__dict__'):
        size += get_object_size(obj.__dict__, seen)
    # 处理列表、元组等容器
    elif isinstance(obj, (list, tuple, set, frozenset)):
        size += sum(get_object_size(item, seen) for item in obj)
    return size

# 加载完整对象
with open('full_object.pkl', 'rb') as f:
    obj = pickle.load(f)

# 遍历顶层属性计算大小
for attr_name, attr_value in obj.__dict__.items():
    size_mb = get_object_size(attr_value) / 1024 / 1024
    print(f"属性 {attr_name}: {size_mb:.2f} MB")

# 深入遍历嵌套属性
nested_obj = obj.some_key.some_other
for attr_name, attr_value in nested_obj.__dict__.items():
    size_mb = get_object_size(attr_value) / 1024 / 1024
    print(f"嵌套属性 some_key.some_other.{attr_name}: {size_mb:.2f} MB")

方法B：用pympler工具简化计算 #

使用pympler的asizeof模块，它能更精准计算对象总内存（含所有嵌套引用），先安装pympler，再执行代码：

from pympler import asizeof
import pickle

with open('full_object.pkl', 'rb') as f:
    obj = pickle.load(f)

# 计算顶层属性大小
for attr_name, attr_value in obj.__dict__.items():
    size_mb = asizeof.asizeof(attr_value) / 1024 / 1024
    print(f"{attr_name}: {size_mb:.2f} MB")

# 直接计算指定嵌套属性大小
target_size = asizeof.asizeof(obj.some_key.some_other.key) / 1024 / 1024
print(f"some_key.some_other.key: {target_size:.2f} MB")

3. 修正你的__getstate__写法 #

你示例代码中del attr['some_key'].some_other.key的写法需要注意：如果some_other是自定义类实例，直接删除属性的写法是对的；但如果some_other是字典，要改成del attr['some_key']['some_other']['key']。修正后的示例：

def __getstate__(self):
    attr = self.__dict__.copy()
    # 针对自定义类实例的嵌套属性
    del attr['some_key'].some_other.key
    # 针对字典类型的嵌套属性
    # del attr['some_key']['some_other']['key']
    return attr

内容的提问来源于stack exchange，提问作者Aenaon