Python类实例化使用pickle HIGHEST_PROTOCOL加载文件避免无限递归

问题根因 #

pickle协议版本>1时，反序列化过程会主动调用类的__new__方法构造空实例，再恢复实例属性。你将pickle加载逻辑写在了__new__中，反序列化调用__new__时会再次触发pickle加载，形成无限递归。

解决方案1（最推荐，无侵入） #

不要将实例加载逻辑耦合到__new__方法中，抽成独立的类方法供外部调用，__new__保留默认实现逻辑，避免pickle反序列化时触发加载流程。
修改后的module_a.py代码：

import os
import pickle

save_path = r'C:\tests\pickle_tests\saved_instance_of_a.pkl'

def load(path):
    with open(path, 'rb') as f:
        return pickle.load(f)

def dump(x, path):
    with open(path, 'wb') as f:
        pickle.dump(
            x, f,
            protocol=pickle.HIGHEST_PROTOCOL)

class ClassA:
    def __init__(self):
        print('__init__ called')
        self.my_dict = {'pi': 3.14}
        dump(self, save_path)
    
    @classmethod
    def get_instance(cls):
        print('get_instance called')
        if os.path.isfile(save_path):
            print('The saved pickle exists: loading from file.')
            return load(save_path)
        else:
            print('The saved pickle does not exist: creating.')
            return cls()

myprogram.py调用方式调整为：

import os
import module_a

if __name__ =='__main__':
    instance_a = module_a.ClassA.get_instance()
    print(instance_a.my_dict)

该方案完全兼容最高版本pickle协议，首次运行和二次运行都不会出现递归问题，逻辑拆分清晰，后续维护成本低。

解决方案2（兼容原有ClassA()调用方式） #

如果必须保留ClassA()的调用方式，可通过给__new__加跳过加载的开关，配合pickle序列化自定义传参实现：
修改后的module_a.py代码：

import os
import pickle

save_path = r'C:\tests\pickle_tests\saved_instance_of_a.pkl'

def load(path):
    with open(path, 'rb') as f:
        return pickle.load(f)

def dump(x, path):
    with open(path, 'wb') as f:
        pickle.dump(
            x, f,
            protocol=pickle.HIGHEST_PROTOCOL)

class ClassA:
    def __new__(cls, skip_load=False):
        print('__new__ called')
        if skip_load:
            # 是pickle反序列化调用，直接创建空实例
            return super(ClassA, cls).__new__(cls)
        
        if os.path.isfile(save_path):
            print('The saved pickle exists: loading from file.')
            instance = load(save_path)
        else:
            print('The saved pickle does not exist: creating.')
            instance = super(ClassA, cls).__new__(cls)
        return instance

    def __init__(self):
        print('__init__ called')
        if not os.path.isfile(save_path):
            self.my_dict = {'pi': 3.14}
            dump(self, save_path)
    
    def __getnewargs_ex__(self):
        # 序列化时指定反序列化调用__new__需要传的参数，标记跳过加载逻辑
        return ( (True, ), {} )

myprogram.py不需要修改，直接调用ClassA()即可正常运行。

内容的提问来源于stack exchange，提问作者thomast