Pybind11继承C++类与Vector数据持久化绑定问题

问题描述 #

使用Pybind11实现C与Python类绑定时，基类ICommunicationHandler及其派生类Handler_A、Handler_B中管理的std::vector<uint8_t>类型buffer成员，在C方法中完成填充修改后，Python实例中的buffer仍为空，未同步更新。不确定是否需要使用trampoline类解决，求解决思路与建议。

调试输出 #

[DEBUG PY] Buffer content before fill, in python script (size: 0):
[]

[DEBUG CPP] Buffer content, in CPP method (size: 13):
1 0 2 0 0 0 0 1 1 8 f1 10 13 

[INFO CPP] Fill buffer with data
Status: success

[DEBUG PY] Buffer content after fill, in python script (size: 0):
[]

相关代码 #

填充buffer的C++方法 #

void DataHandler::fillBuffer(Data& data, std::vector<uint8_t>& buffer)

C++类定义 #

class ICommunicationHandler
{
    public:
        virtual ~ICommunicationHandler() {}

        std::vector<uint8_t> buffer;

        virtual int init() = 0;
        virtual int send() = 0;
        virtual int receive() = 0;
        virtual void reset() = 0;
};

class Handler_A: public ICommunicationHandler
{
    public:
        Handler_A(const std::string_view& portName);
        ~Handler_A();
    
        int init() override;
        int send() override;
        int receive() override;
        void reset() override;

    private:
        /* private members and methods */
};

class Handler_B: public ICommunicationHandler
{
    public:
        Handler_B(const std::string_view& portName);
        ~Handler_B();
    
        int init() override;
        int send() override;
        int receive() override;
        void reset() override;

    private:
        /* private members and methods */
};

Pybind11绑定代码 #

PYBIND11_MODULE(PyLibrary, m)
{
    py::class_<Library::DataHandler>(m, "DataHandler")
        .def_static("fillBuffer", &Library::DataHandler::fillBuffer)

    py::class_<Library::ICommunicationHandler>(m, "ICommunicationHandler")
        .def_readwrite("buffer", &Library::ICommunicationHandler::buffer)

    #ifdef MODE_A
        py::class_<Library::Handler_A, Library::ICommunicationHandler>(m, "Handler_A")
            .def(py::init<const std::string_view&>())
            .def("init", &Library::Handler_A::init)
            .def("send", &Library::Handler_A::send)
            .def("receive", &Library::Handler_A::receive)
            .def("reset", &Library::Handler_A::reset);
    #endif

    #ifdef MODE_B
        py::class_<Library::Handler_B, Library::ICommunicationHandler>(m, "Handler_B")
            .def(py::init<const std::string_view&>())
            .def("init", &Library::Handler_B::init)
            .def("send", &Library::Handler_B::send)
            .def("receive", &Library::Handler_B::receive)
            .def("reset", &Library::Handler_B::reset);
    #endif

    /* other classes */
}

解决思路与建议 #

1. 排除trampoline类需求：trampoline类仅用于Python继承C++虚类并覆写虚方法的场景，本次问题是成员变量同步问题，和虚函数重写无关，不需要使用trampoline类。

2. 检查Python调用方式：确保调用fillBuffer时直接传入C++实例的buffer成员，而非其副本。例如：

   # 正确：直接传原buffer引用
   DataHandler.fillBuffer(data, handler.buffer)
   # 错误：传的是副本，修改不会同步到原对象
   DataHandler.fillBuffer(data, list(handler.buffer))
   

3. 验证C++方法实现：确认DataHandler::fillBuffer确实对传入的buffer引用进行修改，比如通过buffer.push_back()、buffer.assign()等操作直接修改参数，而非操作局部vector变量。

4. 显式绑定参数（可选）：在绑定fillBuffer时显式标注参数，确保Pybind11正确识别引用类型，避免隐式转换导致的副本传递：

   py::class_<Library::DataHandler>(m, "DataHandler")
       .def_static("fillBuffer", &Library::DataHandler::fillBuffer,
           py::arg("data"), py::arg("buffer"));
   

5. 隔离问题排查：可以在C++的receive方法中直接修改this->buffer，然后在Python端调用handler.receive()后查看buffer内容，以此排查是fillBuffer的调用/实现问题，还是绑定层面的问题。

内容的提问来源于stack exchange，提问作者Mitsun0bu