Python multiprocessing环境下mock.patch打补丁失效的问题及无包装实现方案探索
多进程环境下mock.patch打补丁失效问题
基础测试场景
单进程环境下mock.patch可正常完成函数打补丁逻辑,但在multiprocessing多进程环境中使用时补丁效果完全失效。测试代码存放在test_mp.py文件中,具体内容如下:
import multiprocessing import mock def inner(): return sub() def sub(): return "abc" def test_local(): assert inner()=="abc" def test_mp(): with multiprocessing.Pool() as pool: assert pool.apply(inner,args=[])=='abc' def test_mock(): with mock.patch('test_mp.sub', return_value='xxx') as xx: assert inner()=="xxx" xx.assert_called_once() def test_mp_mock(): with multiprocessing.Pool() as pool: with mock.patch('test_mp.sub', return_value='xyz') as xx: assert pool.apply(inner,args=[])=='xyz' xx.assert_called_once()
初步测试结果
test_local、test_mock两个单进程用例均可顺利通过- 组合了多进程调用与mock打补丁的
test_mp_mock用例运行失败:实际返回值为原始函数输出的abc,与打补丁后预期的xyz不匹配,触发断言错误
问题排查过程
为定位根因,引入sharedmock库的SharedMock对象扩展测试代码,扩展内容如下:
from sharedmock.mock import SharedMock def inner2(sm): with mock.patch('test_mp.sub', sm) as xx: return inner() def test_mp_smock(): with multiprocessing.Pool() as pool: sm=SharedMock() sm.return_value="xyz" with mock.patch('test_mp.sub', sm) as xx: assert pool.apply(inner2,args=[sm])=='xyz' assert xx.call_count == 1 def test_mp_mock2(): with multiprocessing.Pool() as pool: sm=mock.Mock() sm.return_value="xyz" print(f"before patch {sub}, {locals()}") with mock.patch('test_mp.sub', sm) as xx: print(f"after patch {sub}") assert pool.apply(inner2,args=[sm])=='xyz' assert xx.call_count == 1
扩展测试结果
test_mp_smock可顺利运行test_mp_mock2抛出_pickle.PicklingError,提示普通mock.Mock对象无法被序列化
待解决的核心需求
当前能让多进程下补丁生效的实现,必须新增带内部打补丁逻辑的包装函数(比如示例中的inner2),但实际业务场景中无法修改pool.apply(inner,args=[])的直接调用逻辑,需要找到无需新增包装函数,即可将父进程的mock.patch补丁状态传递到多进程工作进程,实现多进程场景下正常打补丁的方法。
内容的提问来源于stack exchange,提问作者Vitek
相关产品推荐
相关产品推荐

