C++向Python暴露回调接口并传回时出现内存泄漏求助

我正在为团队的C++库开发Python包装器，在暴露回调接口时遇到了内存泄漏问题。

库代码 #

// Simulate our library
namespace library {
    // Callback pure interface
    class Callback{
    public:
        virtual ~Callback() {
            // Just for test!
            std::cout << "~Callback() destructed!" << std::endl;
        } 
        virtual void onProgress(int progress) = 0;
    };

    // Some library class who gets and stores callback instance
    class Library {
        std::shared_ptr<Callback> callback;
    public:
        Library(std::shared_ptr<Callback> _callback)
        : callback { _callback } {
            std::cout << "Library() constructed!" << std::endl;
        }

        ~Library() {
            // Library is free!
            std::cout << "~Library() destructed!" << std::endl;
        }

        void func() {
            // Use callback
            if (callback != nullptr) {
                callback->onProgress(66);
            }
        }
    };

    // Show module is unloaded.
    struct Lifecycle {
        ~Lifecycle() {
            std::cout << "Lifecycle has terminated." << std::endl;
        }
    };

    Lifecycle lc;
}

其中Lifecycle对象会在模块卸载时输出提示信息。

Boost.Python模块代码 #

#include <boost/python.hpp>

namespace py = boost::python;

namespace {
    // Python wrapper for callback
    struct CallbackWrapper
        : library::Callback
        , py::wrapper<library::Callback>
        {
            virtual void onProgress(int progress) override {
                this->get_override("onProgress")(progress);
            }
        };

    // Special testing callback
    struct CallbackWrapperTest
        : library::Callback
        , py::wrapper<library::Callback>
        {
            virtual void onProgress(int progress) override {
                std::cout << "TEST: onProgress(" << progress << ")" << std::endl;
            }
        };

    // Creation function
    std::shared_ptr<library::Library> createLibrary(std::shared_ptr<library::Callback> callback) {
        return std::make_shared<library::Library>(callback);
    }
}


BOOST_PYTHON_MODULE(callback) {
    py::class_<library::Library, boost::noncopyable>(
        "Library",
        py::no_init
        )
        .def("__init__", py::make_constructor(&createLibrary))
        .def("func", &library::Library::func)
        ;
    
    py::class_<CallbackWrapper, boost::noncopyable>("Callback");

    py::class_<CallbackWrapperTest, boost::noncopyable>("CallbackTest");
}

CallbackWrapperTest类用于后续测试问题。

Python测试代码 #

import sys
sys.path.append(r"./build")
import callback

class PyCallback(callback.Callback):
    '''Python callback implementation'''
    def onProgress(self, progress):
        print("PY onProgress", progress)

cb = PyCallback()

# Replacing by this works fine but useless
# cb = callback.CallbackTest()

lib = callback.Library(cb)

lib.func()

# explicit library deletion works fine  
# del lib
print("==========================")

注意该脚本没有main函数，这一点很重要。

实际输出 #

Library() constructed!
PY onProgress 66
==========================
Lifecycle has terminated.

期望输出 #

Library() constructed!
PY onProgress 66
==========================
~Library() destructed!
~Callback() destructed!
Lifecycle has terminated.

异常现象 #

• 如果将脚本代码包裹在函数（如main()）中，析构函数会被调用。
• 如果显式执行del lib，析构函数会被调用。
• 如果不使用Python继承，改用纯C++的CallbackTest，析构函数会被调用。

只有在使用Python派生类时才会出现内存泄漏问题，猜测是引用计数的问题，但不清楚具体原因，希望得到解答。

内容的提问来源于stack exchange，提问作者Maydyk