Python 3.12下tp_alloc触发段错误：C-API代码适配问题排查

问题背景 #

将Python版本从3.11.7升级至3.12.0后，原本正常运行的C-API代码在tp_alloc内部触发段错误，查阅Python 3.12移植文档未找到相关破坏性变更记录。

相关代码片段 #

头文件条件编译 #

#if PY_VERSION_HEX < 0x030C0000 // 3.12
#include <structmember.h>
#endif

结构体与成员定义 #

typedef struct {
    PyObject_HEAD
    uint32_t size;
    int32_t mdb;
    uint32_t ms;
    const char *id;
    const char *wat;
} Something;

static PyMemberDef Something_members[] = {
#if PY_VERSION_HEX >= 0x030C0000
    {"size", Py_T_UINT, offsetof(Something, size), Py_READONLY, "Size in bytes"},
    {"mdb", Py_T_INT, offsetof(Something, mdb), Py_READONLY, "mdb"},
    {"ms", Py_T_UINT, offsetof(Something, ms), Py_READONLY, "ms"},
    {"id", Py_T_OBJECT_EX, offsetof(Something, id), Py_READONLY, "id"},
    {"wat", Py_T_OBJECT_EX, offsetof(Something, wat), Py_READONLY, "wat"},
#else
    {"size", T_UINT, offsetof(Something, size), READONLY, "Size in bytes"},
    {"gain_mdb", T_INT, offsetof(Something, mdb), READONLY, "mdb"},
    {"ms", T_UINT, offsetof(Something, ms), READONLY, "ms)"},
    {"id", T_OBJECT, offsetof(Something, id), READONLY, "id"},
    {"wat", T_OBJECT, offsetof(Something, wat), READONLY, "wat"},
#endif
    (NULL} // 语法错误：应为{NULL}
};

类型对象定义 #

static PyTypeObject SomethingType = {
    PyObject_HEAD_INIT(NULL)
    .tp_name = "tiny.Something",
    .tp_basicsize = sizeof(Something),
    .tp_flags = Py_TPFLAGS_DEFAULT,
    .tp_doc = "some doc",
    .tp_members = Something_members
};

触发段错误的调用代码 #

Something *py_something= (Something *)PyType_GenericAlloc(&SomethingType, 0);

Valgrind检测结果 #

2174369 1 errors in context 1 of 1:
2174369 Invalid read of size 8
2174369 at 0x4A4E771: UnknownInlinedFun (pycore_pystate.h:118)
2174369 by 0x4A4E771: UnknownInlinedFun (obmalloc.c:866)
2174369 by 0x4A4E771: _PyObject_Malloc (obmalloc.c:1563)
2174369 by 0x4A55257: UnknownInlinedFun (obmalloc.c:801)
2174369 by 0x4A55257: _PyType_AllocNoTrack (typeobject.c:1698)
2174369 by 0x4A55172: PyType_GenericAlloc (typeobject.c:1722)
2174369 by 0x16CACB31: CallbackSomethingStart (something/wat.c:1234)
... prior things

问题原因分析 #

你遗漏了两个关键问题，其中一个是Python 3.12的隐性依赖变更：

1. 语法错误导致成员数组越界
   Something_members数组的最后一行是(NULL}，属于语法错误，正确写法应为{NULL}。这个错误会让Python在遍历成员定义时读取到无效内存，3.12版本对内存访问的检查更严格，导致原本隐蔽的错误触发段错误。

2. 3.12下缺失structmember.h头文件
   代码仅在3.12以下版本包含<structmember.h>，但PyMemberDef结构体的定义（包括Py_T_UINT、Py_READONLY等宏）都依赖这个头文件。3.12下不包含该头文件会导致编译器无法正确解析PyMemberDef的内存布局，进而在类型分配过程中出现内存访问错误。

此外，Python 3.12对C-API的类型初始化流程有更严格的要求，确保类型对象必须通过PyType_Ready()完成初始化后再调用分配函数。如果之前代码依赖隐式初始化，3.12的变更可能打破这个逻辑，建议在调用PyType_GenericAlloc前添加PyType_Ready(&SomethingType)的调用，并检查返回值。

内容的提问来源于stack exchange，提问作者firmament