I'm having a problem with Pybind11 when trying to link C++ classes with Python.
I have a base class ICommunicationHandler and two derived classes Handler_A and Handler_B. These classes manage a std::vector<uint8_t> called buffer. However, changes made to buffer in C++ methods are not reflected in the Python instance. For example, after calling a C++ method that fill the buffer, I find that it remains empty and does not contain the expected modifications on the Python side.
Here's the debug output illustrating the issue:
[DEBUG PY] Buffer content before fill, in python script (size: 0):
[]
[DEBUG CPP] Buffer content, in CPP method (size: 13):
1 0 2 0 0 0 0 1 1 8 f1 10 13
[INFO CPP] Fill buffer with data
Status: success
[DEBUG PY] Buffer content after fill, in python script (size: 0):
[]
The buffer is supposed to be filled by the following C++ method:
void DataHandler::fillBuffer(Data& data, std::vector<uint8_t>& buffer)
Here are the relevant class definitions:
class ICommunicationHandler
{
public:
virtual ~ICommunicationHandler() {}
std::vector<uint8_t> buffer;
virtual int init() = 0;
virtual int send() = 0;
virtual int receive() = 0;
virtual void reset() = 0;
};
class Handler_A: public ICommunicationHandler
{
public:
Handler_A(const std::string_view& portName);
~Handler_A();
int init() override;
int send() override;
int receive() override;
void reset() override;
private:
/* private members and methods */
};
class Handler_B: public ICommunicationHandler
{
public:
Handler_B(const std::string_view& portName);
~Handler_B();
int init() override;
int send() override;
int receive() override;
void reset() override;
private:
/* private members and methods */
};
My Pybind11 link code looks like this:
PYBIND11_MODULE(PyLibrary, m)
{
py::class_<Library::DataHandler>(m, "DataHandler")
.def_static("fillBuffer", &Library::DataHandler::fillBuffer)
py::class_<Library::ICommunicationHandler>(m, "ICommunicationHandler")
.def_readwrite("buffer", &Library::ICommunicationHandler::buffer)
#ifdef MODE_A
py::class_<Library::Handler_A, Library::ICommunicationHandler>(m, "Handler_A")
.def(py::init<const std::string_view&>())
.def("init", &Library::Handler_A::init)
.def("send", &Library::Handler_A::send)
.def("receive", &Library::Handler_A::receive)
.def("reset", &Library::Handler_A::reset);
#endif
#ifdef MODE_B
py::class_<Library::Handler_B, Library::ICommunicationHandler>(m, "Handler_B")
.def(py::init<const std::string_view&>())
.def("init", &Library::Handler_B::init)
.def("send", &Library::Handler_B::send)
.def("receive", &Library::Handler_B::receive)
.def("reset", &Library::Handler_B::reset);
#endif
/* other classes */
}
I might be handling the binding incorrectly. I have started exploring some leads in the Pybind11 documentation about using 'trampoline classes' but I'm not sure if it's the right approach for this specific issue. Any insights, suggestions, or help would be greatly appreciated. Thanks !
I solved my problem by referring to Pybind11's documentation on STL containers (https://pybind11.readthedocs.io/en/stable/advanced/cast/stl.html#stl-containers).
The problem was that changes made to a std::vector<uint8_t> in the C++ code were not visible in the Python instance. This is due to Pybind11's internal conversion mechanism, which creates a copy of the STL container instead of passing by reference.
To solve this problem, as indicated in the documentation, I used :
PYBIND11_MAKE_OPAQUE(std::vector<uint8_t>) to make the type opaque, which disables Pybind11's template-based conversion mechanism.
And these lines of code in my binding file:
py::class_<std::vector<uint8_t>>(m, "uIntVector")
.def(py::init<>())
.def("clear", &std::vector<uint8_t>::clear)
.def("pop_back", &std::vector<uint8_t>::pop_back)
.def("push_back", (void (std::vector<uint8_t>::*)(const uint8_t &)) &std::vector<uint8_t>::push_back)
.def("__len__", [](const std::vector<uint8_t> &v) { return v.size(); })
.def("__iter__", [](std::vector<uint8_t> &v) { return py::make_iterator(v.begin(), v.end());}, py::keep_alive<0, 1>());