子类Pickle序列化时新增实例属性未被保存的原因排查

问题描述 #

对requests.Session的子类MySession进行Pickle序列化时，子类新增的details属性未被保存，反序列化后访问该属性会抛出AttributeError异常。

复现代码 #

import requests
import pickle

class MySession(requests.Session):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.details = False

data = MySession()
file = "/tmp/pickle"

with open(file, "wb") as f:
    from pprint import pprint
    print(f"Saving session {data} to {file} with {data.details}")
    pprint(data.__dict__)
    pickle.dump(data, f)
print("Validating")
with open(file, "rb") as f:
    newdata = pickle.load(f)
    pprint(newdata.__dict__)
    print(data.details)
    print(newdata.details)

运行报错信息 #

Saving session <__main__.MySession object at 0x7f0976cb9400> to /tmp/pickle with False
{'adapters': OrderedDict([('https://',
                           <requests.adapters.HTTPAdapter object at 0x7f0975dca4e0>),
                          ('http://',
                           <requests.adapters.HTTPAdapter object at 0x7f0976be9ca0>)]),
 'auth': None,
 'cert': None,
 'cookies': <RequestsCookieJar[]>,
 'details': False,
 'headers': {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate, br', 'Accept': '*/*', 'Connection': 'keep-alive'},
 'hooks': {'response': []},
 'max_redirects': 30,
 'params': {},
 'proxies': {},
 'stream': False,
 'trust_env': True,
 'verify': True}
Validating
{'adapters': OrderedDict([('https://',
                           <requests.adapters.HTTPAdapter object at 0x7f09759c1010>),
                          ('http://',
                           <requests.adapters.HTTPAdapter object at 0x7f0975a4e960>)]),
 'auth': None,
 'cert': None,
 'cookies': <RequestsCookieJar[]>,
 'headers': {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate, br', 'Accept': '*/*', 'Connection': 'keep-alive'},
 'hooks': {'response': []},
 'max_redirects': 30,
 'params': {},
 'proxies': {},
 'stream': False,
 'trust_env': True,
 'verify': True}
False
Traceback (most recent call last):
  File "/tmp/repro.py", line 22, in <module>
    print(newdata.details)
          ^^^^^^^^^^^^^^^
AttributeError: 'MySession' object has no attribute 'details'

问题原因 #

requests.Session类自定义了__getstate__和__setstate__方法，用于控制Pickle序列化与反序列化的过程：

• 序列化时，__getstate__仅返回requests.Session自身定义的属性集合，不会包含子类新增的details属性，导致该属性未被写入Pickle文件；
• 反序列化时，__setstate__只会恢复requests.Session原有的属性，不会处理子类新增的属性，因此反序列化后的对象没有details属性。

解决方案 #

重写子类的__getstate__和__setstate__方法，在父类的状态基础上添加/恢复自定义属性：

import requests
import pickle

class MySession(requests.Session):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.details = False

    def __getstate__(self):
        # 获取父类序列化状态，添加自定义属性
        state = super().__getstate__()
        state['details'] = self.details
        return state

    def __setstate__(self, state):
        # 先恢复父类状态，再恢复自定义属性
        super().__setstate__(state)
        self.details = state.get('details', False)

# 验证代码
data = MySession()
file = "/tmp/pickle"

with open(file, "wb") as f:
    pickle.dump(data, f)

with open(file, "rb") as f:
    newdata = pickle.load(f)
    print(newdata.details)  # 输出 False，无异常

内容的提问来源于stack exchange，提问作者Brian