I understand your issue: when you create a Python-derived class from a C++ pure virtual base class (A) and return it via a callback registered in your C++ factory, calling the virtual function go() triggers a runtime error because it tries to invoke the pure virtual A::go() instead of your Python override PyDerivedA.go().

Root Cause #

The core problem here is that the Python object you return from your callback may be getting garbage collected before the C++ code calls go(), or the automatic conversion from the Python object to std::shared_ptr<A> isn't properly retaining a reference to the Python instance. When the trampoline class PyA tries to dispatch the call to the Python method, it can't find the live Python object, so it falls back to the pure virtual function (or throws the error you see).

Another way to look at it: the std::function callback mechanism in pybind11 might not be handling the reference counting correctly for Python-derived objects, leading to the Python instance being destroyed prematurely.

Solution #

To fix this, we need to explicitly manage the conversion from the Python object to std::shared_ptr<A> to ensure the Python instance stays alive. Here's how to adjust your code:

Step 1: Modify the Callback Type in C++

Change the CreationEvaluatorCallback to return a py::object instead of std::shared_ptr<A>. This lets us explicitly handle the conversion to std::shared_ptr<A> while retaining ownership of the Python object:

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <memory>
#include <unordered_map>
#include <iostream>
#include <string>
#include <functional>
namespace py = pybind11;

class A {
public:
    A() { std::cout << "A" << std::endl; }
    virtual void go() = 0;
    virtual ~A()=default;
};

// Change callback to return py::object
using CreationEvaluatorCallback = std::function<py::object(const std::string&)>;

class PyA : public A {
public:
    using A::A;
    void go() override {
        PYBIND11_OVERLOAD_PURE(
            void,
            A,
            go
        );
    }
};

class MyFactory {
public:
    static MyFactory& Instance() {
        static MyFactory instance;
        return instance;
    }
    void Registry(const std::string &name, CreationEvaluatorCallback callback) {
        callback_registry_[name] = callback;
    }

    std::shared_ptr<A> Create(const std::string &evaluator_name) {
        auto iter = callback_registry_.find(evaluator_name);
        if (iter != callback_registry_.end()) {
            py::object obj = iter->second(evaluator_name);
            // Explicitly cast to shared_ptr with take_ownership to keep Python object alive
            return py::cast<std::shared_ptr<A>>(obj, py::return_value_policy::take_ownership);
        }
        return nullptr;
    }

private:
    MyFactory() = default;
    ~MyFactory() = default;
    MyFactory(const MyFactory&) = delete;
    MyFactory& operator=(const MyFactory&) = delete;
    std::unordered_map<std::string, CreationEvaluatorCallback> callback_registry_;
};

class MyRun {
public:
    MyRun(){}
    ~MyRun(){}
    void Run(const std::string &evaluator_name){
        auto eval = MyFactory::Instance().Create(evaluator_name);
        eval->go();
    }
};

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 包装 MyFactory 和 MyRun 示例";

    py::class_<A, PyA, std::shared_ptr<A>>(m, "A")
        .def(py::init<>())
        .def("go", &A::go);

    py::class_<MyFactory, std::unique_ptr<MyFactory, py::nodelete>>(m, "MyFactory")
        .def_static("instance", &MyFactory::Instance, py::return_value_policy::reference)
        .def("registry", &MyFactory::Registry)
        .def("create", &MyFactory::Create);

    py::class_<MyRun>(m, "MyRun")
        .def(py::init<>())
        .def("run", &MyRun::Run);
}

Step 2: Keep Your Python Code Unchanged

Your Python code doesn't need any modifications—just ensure you're returning the PyDerivedA instance as before:

import example
class PyDerivedA(example.A):
    def __init__(self):
        super().__init__()
        print("111111111111111111")
        
    def go(self):
        print("PyDerivedA go called!")

def create_derived(name):
    return PyDerivedA()

factory = example.MyFactory.instance()
factory.registry("derived", create_derived)

runner = example.MyRun()
runner.run("derived")  # Now outputs "PyDerivedA go called!"

Why This Works #

By changing the callback to return a py::object, we can explicitly cast it to std::shared_ptr<A> using py::return_value_policy::take_ownership. This ensures that the Python object's reference count is incremented, preventing it from being garbage collected while the C++ std::shared_ptr<A> is alive. The trampoline class PyA can now correctly dispatch the go() call to your Python override.

Alternative Fix (Python-Side) #

If you prefer not to modify the C++ code, you can retain a global reference to the created Python objects to prevent garbage collection:

import example
class PyDerivedA(example.A):
    def __init__(self):
        super().__init__()
        print("111111111111111111")
        
    def go(self):
        print("PyDerivedA go called!")

# Keep global references to prevent garbage collection
created_objects = []

def create_derived(name):
    obj = PyDerivedA()
    created_objects.append(obj)
    return obj

factory = example.MyFactory.instance()
factory.registry("derived", create_derived)

runner = example.MyRun()
runner.run("derived")

However, this approach is less clean as it can lead to memory leaks if you don't manage the global list properly. The C++-side fix is more robust.

备注：内容来源于stack exchange，提问作者fangchao