如何在Pickle序列化Python异常后保留Traceback并延迟抛出？

我正在编写用于磁盘缓存结果的装饰器，要求无论f(x)的结果是否已存储，调用f(x)都能得到一致的结果——即使f(x)抛出异常：加载存储的异常结果时，要抛出相同的异常（添加少量上下文标识）。

目前内存缓存场景的代码已实现，但用Pickle做磁盘存储时无法正常工作：经过Pickle反序列化后的异常丢失了traceback信息。以下是完整演示代码：

import pickle, traceback, sys
import logging

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

def buggy(x):
    raise ValueError("my error")

def f(x):
    if x < 10:
        return buggy(x)
    else:
        return x+2

class DelayedException(Exception):
    def __init__(self, s):
        super().__init__(s)

for val, method in [
    (0, "no_cache"), (100, "no_cache"), #direct function call (what we want to imitate)
    (0, "memory_cached"), (100, "memory_cached"), #reraising later without saving/loading (currently good result)
    (0, "pickled_cache"), (100, "pickled_cache") #reraising later with saving/loading in between
]:
    print("\n\n")
    try:
        logger.info(f"val = {val}, method={method}")
        if method=="no_cache":
            res = f(val)
        else:
            try:
                res = f(val)
            except BaseException as e:
                # The code can be changed here
                try:
                    #Not very readable technique to append this exception in the exception stack.
                    #I'm very open to samething better
                    raise DelayedException(f"Error in {f.__name__}({val})") 
                except DelayedException as d:
                    try:
                        raise e from d
                    except BaseException as final:
                        res = final

            if method=="pickled_cache":
                #for simplicity we use a string instead of a file for demo purposes
                res = pickle.dumps(res) 
                res = pickle.loads(res)

            if isinstance(res, BaseException):
                # The code also can be changed here
                raise res

        logger.info(f"ok, res = {res}")
    except:
        logger.error(traceback.format_exc())

问题根源 #

Python原生异常对象的traceback包含底层的帧（frame）对象，这类对象无法被Pickle序列化，因此反序列化后的异常会丢失原始调用栈信息，导致抛出时无法输出完整traceback。

基于tblib的解决方案 #

tblib库专门用于序列化和反序列化Python的traceback对象，我们可以用它保存异常的完整调用栈，再和异常对象一起序列化到磁盘；加载时恢复traceback并重新构建带完整上下文的异常。

代码修改步骤

1. 先安装tblib：pip install tblib
2. 修改代码中两个标注的可修改区域：

修改后的完整代码：

import pickle, traceback, sys
import logging
from tblib import pickling_support

# 启用tblib的pickle支持，自动处理traceback的序列化/反序列化
pickling_support.install()

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

def buggy(x):
    raise ValueError("my error")

def f(x):
    if x < 10:
        return buggy(x)
    else:
        return x+2

class DelayedException(Exception):
    def __init__(self, s):
        super().__init__(s)

for val, method in [
    (0, "no_cache"), (100, "no_cache"),
    (0, "memory_cached"), (100, "memory_cached"),
    (0, "pickled_cache"), (100, "pickled_cache")
]:
    print("\n\n")
    try:
        logger.info(f"val = {val}, method={method}")
        if method=="no_cache":
            res = f(val)
        else:
            try:
                res = f(val)
            except BaseException as e:
                # --- 修改后的异常捕获逻辑 ---
                # 捕获当前异常的traceback
                tb = sys.exc_info()[2]
                # 构建带上下文的链式异常，并保存异常+traceback元组
                try:
                    raise DelayedException(f"Error in {f.__name__}({val})") from e
                except DelayedException as chained_exc:
                    res = (chained_exc, tb)

            if method=="pickled_cache":
                res = pickle.dumps(res) 
                res = pickle.loads(res)

            if isinstance(res, tuple) and len(res) == 2 and isinstance(res[0], BaseException):
                # --- 修改后的异常抛出逻辑 ---
                exc, tb = res
                # 恢复traceback并抛出，保留完整调用栈
                raise exc.with_traceback(tb)
            elif isinstance(res, BaseException):
                # 兼容内存缓存场景的直接抛出逻辑
                raise res

        logger.info(f"ok, res = {res}")
    except:
        logger.error(traceback.format_exc())

关键说明

• pickling_support.install()会自动给Pickle打补丁，让traceback可以被序列化
• 捕获异常时保存异常对象+traceback的元组，避免单独序列化异常丢失调用栈
• 加载后通过with_traceback()恢复原始调用栈，确保抛出时能输出和原异常一致的完整traceback
• 保留了对内存缓存场景的兼容性，无需额外调整

内容的提问来源于stack exchange，提问作者Serker