Python中Multiprocessing.Value嵌套Ctypes Structure为何引发RecursionError？

问题现象 #

使用multiprocessing.Value()在多进程间共享ctypes Structure时，部分结构体初始化时会触发RecursionError，报错发生在Multiprocessing内部深层逻辑，且排除了结构体大小的影响，类似小结构体可正常工作。

示例代码 #

from ctypes import c_float, c_bool, c_int, Structure
from multiprocessing import Value

class goodStruct(Structure):
    _pack_ = 1
    _fields_ = [
        ('connected'            ,c_bool),
        ('connected_complete'   ,c_bool)
    ]

class badStruct(Structure):
    _pack_ = 1
    _fields_ = [
        ('connected'            ,c_bool),
        ('connected_complete'   ,c_bool),
        ('power_arm'            ,c_bool),
        ('power_arm_complete'   ,c_bool),
        ('slowmove'             ,c_bool),
        ('slowmove_complete'    ,c_bool),
        ('livemove'             ,c_bool),
        ('livemove_complete'    ,c_bool),
        ('end_livemove'         ,c_bool),
        ('end_livemove_complete',c_bool),
        ('error'                ,c_int),
        ('clear_error'          ,c_bool),
        ('clear_error_complete' ,c_bool),
        ('release'              ,c_bool),
        ('release_complete'     ,c_bool),
        ('stop'                 ,c_bool),
        ('stop_complete'        ,c_bool)
    ]


a = Value(goodStruct)

b = Value(badStruct)

报错信息 #

Traceback (most recent call last):
  File "C:\Users\e33965\Software\z_messing around\struct_test.py", line 36, in <module>
    b = Value(badStruct)
  File "C:\Users\e33965\Miniconda3\lib\multiprocessing\context.py", line 135, in Value
    return Value(typecode_or_type, *args, lock=lock,
  File "C:\Users\e33965\Miniconda3\lib\multiprocessing\sharedctypes.py", line 82, in Value
    return synchronized(obj, lock, ctx=ctx)
  File "C:\Users\e33965\Miniconda3\lib\multiprocessing\sharedctypes.py", line 122, in synchronized
    return scls(obj, lock, ctx)
  File "C:\Users\e33965\Miniconda3\lib\multiprocessing\sharedctypes.py", line 190, in __init__
    self.release = self._lock.release
  File "<string>", line 13, in setrelease
  File "<string>", line 7, in getrelease
  File "<string>", line 7, in getrelease
  File "<string>", line 7, in getrelease
  [Previous line repeated 989 more times]
  File "<string>", line 3, in getrelease
RecursionError: maximum recursion depth exceeded while calling a Python object

解决方案 #

重命名结构体中与synchronized类内置方法重名的字段——你的badStruct里的release字段和synchronized实例的release（锁释放方法）重名，导致属性访问时无限递归。

修改后的结构体示例：

class badStruct(Structure):
    _pack_ = 1
    _fields_ = [
        ('connected'            ,c_bool),
        ('connected_complete'   ,c_bool),
        ('power_arm'            ,c_bool),
        ('power_arm_complete'   ,c_bool),
        ('slowmove'             ,c_bool),
        ('slowmove_complete'    ,c_bool),
        ('livemove'             ,c_bool),
        ('livemove_complete'    ,c_bool),
        ('end_livemove'         ,c_bool),
        ('end_livemove_complete',c_bool),
        ('error'                ,c_int),
        ('clear_error'          ,c_bool),
        ('clear_error_complete' ,c_bool),
        ('release_cmd'          ,c_bool),  # 重命名字段，避免冲突
        ('release_complete'     ,c_bool),
        ('stop'                 ,c_bool),
        ('stop_complete'        ,c_bool)
    ]

原因说明 #

multiprocessing.Value返回的是synchronized类的实例，该类自带release方法用于释放内部锁。当结构体中存在同名字段时，动态生成的属性访问器会在访问release属性时陷入循环：尝试获取结构体字段的操作会触发锁的逻辑，而锁逻辑又会访问release方法，最终导致递归深度超出限制。

内容的提问来源于stack exchange，提问作者Nicholas Masso