调用Py_Initialize()时Valgrind检测到内存错误的解决方案咨询

问题 #

我正在把Python代码嵌入C++，制作可作为C++库使用的C API包装器。程序运行正常，但Valgrind检测出大量内存错误。示例代码如下：

#include "Python.h"
#include "iostream"

using namespace std;

int main(){
    setenv("PYTHONPATH",".",1);
    Py_Initialize();
    cout <<"Hello\n";
    Py_Finalize();
}

运行代码无报错，但Valgrind检测结果显示大量错误，例如：

==140416== Memcheck, a memory error detector
==140416== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==140416== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==140416== Command: ./test_mem
==140416== Parent PID: 86216
==140416== 
==140416== Invalid read of size 4
==140416==    at 0x4AADA41: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4ACD397: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4ACE9D8: PyDict_SetItem (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x48E2CFE: PyType_Ready (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4AB8E04: _PyTypes_Init (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F3A6A: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F41A2: Py_InitializeFromConfig (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F424B: Py_InitializeEx (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x1091ED: main (in /additional_drive/nist/frvt_onnx_sequential/11/test_mem)
==140416==  Address 0x5387020 is 1,520 bytes inside a block of size 2,208 free'd
==140416==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==140416==    by 0x4ACD397: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4AC9E1F: PyDict_SetDefault (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4A62DFF: PyUnicode_InternInPlace (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4A62E90: PyUnicode_InternFromString (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4AFC4C1: PyDescr_NewMethod (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x48E34AA: PyType_Ready (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4AB8E04: _PyTypes_Init (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F3A6A: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F41A2: Py_InitializeFromConfig (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49F424B: Py_InitializeEx (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x1091ED: main (in /additional_drive/nist/frvt_onnx_sequential/11/test_mem)
==140416==  Block was alloc'd at
...
...
...
==140416==  Block was alloc'd at
==140416==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==140416==    by 0x4AB69E7: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4B20CB7: PyGrammar_AddAccelerators (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4B23184: PyParser_New (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4B26DD2: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49E79DA: PyParser_ASTFromStringObject (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x49E8387: PyRun_StringFlags (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4A2BADF: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x4ABB546: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x48D0D6C: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x48D246C: _PyEval_EvalFrameDefault (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416==    by 0x48DC06A: ??? (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==140416== 
==140416== ERROR SUMMARY: 898 errors from 107 contexts (suppressed: 0 from 0)

我需要解决这些内存错误，因为要提交验证包到竞赛，要求程序无内存错误。已尝试设置PYTHONPATH，无效；移除Py_Initialize()后Valgrind无错误，但我必须在C++中运行Python代码。

解决方案 #

1. 使用Python官方Valgrind抑制文件 #

CPython本身存在部分Valgrind会误报的内存操作（比如内部缓存、静态对象未释放），官方提供了抑制文件过滤这些误报：

• 找到Python安装目录下的valgrind-python.supp，路径通常类似/usr/lib/python3.8/config-3.8-x86_64-linux-gnu/valgrind-python.supp
• 运行Valgrind时指定该文件：

  valgrind --suppress=/path/to/valgrind-python.supp ./test_mem
  

2. 严格控制Python初始化/终止次数 #

• 整个程序生命周期内只调用一次Py_Initialize()和Py_Finalize()，多次调用会导致内存管理混乱
• 若制作库，建议在库的初始化函数中调用Py_Initialize()，销毁函数中调用Py_Finalize()，不要在每次调用Python代码时重复执行初始化和终止

3. 严格管理Python对象引用计数 #

实际项目中调用Python API时，必须遵守引用计数规则：

• 使用Py_INCREF()和Py_DECREF()手动管理对象引用，避免泄漏
• 对Python API返回的对象，确认是否需要手动释放（比如PyRun_String()返回的对象需调用Py_DECREF()）

4. 升级Python版本 #

你使用的Python 3.8存在部分已知内存管理问题，升级到3.10+版本可修复部分问题，新版本CPython对Valgrind兼容性更好

5. 使用Py_InitializeFromConfig替代默认初始化 #

手动配置初始化参数，避免默认流程中的潜在问题：

#include "Python.h"
#include "iostream"

using namespace std;

int main(){
    PyConfig config;
    PyConfig_InitPythonConfig(&config);
    // 设置PYTHONPATH
    PyConfig_SetString(&config, &config.pythonpath, ".");
    // 初始化Python
    if (Py_InitializeFromConfig(&config) < 0) {
        cerr << "Python初始化失败" << endl;
        return 1;
    }
    PyConfig_Clear(&config);
    
    cout << "Hello\n";
    
    Py_Finalize();
    return 0;
}

内容的提问来源于stack exchange，提问作者Muhammad Awais