I've run into an edge case with Boost.Python that seems like it should work but doesn't.
What I have is a Base and a Derived class that I am storing in std::shared_ptr's on the python side. What I would like to do is pass a Derived type shared_ptr to a function that accepts a Base shared_ptr by reference.
I've done some research and have learned about implicitly_convertible and have attempted to employ it to fix the problem but without success (although it does help in some other situations). Passing a Derived to function that accepts a Base& works with this but if they're wrapped in shared_ptr then it fails.
What I get currently is the message below:
Boost.Python.ArgumentError: Python argument types in
test_bed_bindings.acceptBaseSharedPtrRef(Derived) did not match C++ signature:
acceptBaseSharedPtrRef(std::shared_ptr<(anonymous namespace)::Base> {lvalue})
See below for example code:
C++ Binding code
#define BOOST_PYTHON_STATIC_LIB
#define BOOST_PYTHON_USE_GCC_SYMBOL_VISIBILITY 1
#include <boost/optional.hpp>
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>
#include <iostream>
#include <memory>
namespace
{
class Base
{
};
class Derived : public Base
{
};
std::shared_ptr<Base> getBaseSharedPtr()
{
auto retVal = std::make_shared<Base>();
std::cout << "Creating Base shared_ptr - " << retVal.get() << std::endl;
return retVal;
}
std::shared_ptr<Derived> getDerivedSharedPtr()
{
auto retVal = std::make_shared<Derived>();
std::cout << "Creating Derived shared_ptr - " << retVal.get() << std::endl;
return retVal;
}
void acceptBaseSharedPtrRef(std::shared_ptr<Base>& base)
{
std::cout << "acceptBaseSharedPtrRef() with " << base.get() << std::endl;
}
void acceptBaseSharedPtrConstRef(const std::shared_ptr<Base>& base)
{
std::cout << "acceptBaseSharedPtrConstRef() with " << base.get() << std::endl;
}
void acceptBaseSharedPtrCopy(std::shared_ptr<Base> base)
{
std::cout << "acceptBaseSharedPtrCopy() with " << base.get() << std::endl;
}
//
void acceptBaseRef(Base base)
{
}
} // namespace
namespace bindings
{
BOOST_PYTHON_MODULE(test_bed_bindings)
{
PyEval_InitThreads();
Py_Initialize();
using namespace boost::python;
def("getBaseSharedPtr", &::getBaseSharedPtr);
def("getDerivedSharedPtr", &::getDerivedSharedPtr);
def("acceptBaseSharedPtrRef", &::acceptBaseSharedPtrRef);
def("acceptBaseSharedPtrConstRef", &::acceptBaseSharedPtrConstRef);
def("acceptBaseSharedPtrCopy", &::acceptBaseSharedPtrCopy);
def("acceptBaseRef", &::acceptBaseRef);
class_<Base, std::shared_ptr<Base> >("Base")
.def(init<>())
;
class_<Derived, bases<Base>, std::shared_ptr<Derived> >("Derived")
.def(init<>())
;
implicitly_convertible<Derived, Base>();
implicitly_convertible<std::shared_ptr<Derived>, std::shared_ptr<Base>>();
} // BOOST_PYTHON
} // namespace bindings
Python execution code
import test_bed_bindings
baseObj = test_bed_bindings.Base()
derivedObj = test_bed_bindings.Derived()
test_bed_bindings.acceptBaseRef( baseObj )
test_bed_bindings.acceptBaseRef( derivedObj )
baseSharedPtr = test_bed_bindings.getBaseSharedPtr()
derivedSharedPtr = test_bed_bindings.getDerivedSharedPtr()
test_bed_bindings.acceptBaseSharedPtrCopy( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrCopy( derivedSharedPtr )
test_bed_bindings.acceptBaseSharedPtrConstRef( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrConstRef( derivedSharedPtr )
test_bed_bindings.acceptBaseSharedPtrRef( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrRef( derivedSharedPtr )
Sample Output
Creating Base shared_ptr - 0x276fdb8
Creating Derived shared_ptr - 0x276fde8
acceptBaseSharedPtrCopy() with 0x276fdb8
acceptBaseSharedPtrCopy() with 0x276fde8
acceptBaseSharedPtrConstRef() with 0x276fdb8
acceptBaseSharedPtrConstRef() with 0x276fde8
acceptBaseSharedPtrRef() with 0x276fdb8
Traceback (most recent call last):
File "test_script.py", line 21, in <module>
test_bed_bindings.acceptBaseSharedPtrRef( derivedSharedPtr )
Boost.Python.ArgumentError: Python argument types in
test_bed_bindings.acceptBaseSharedPtrRef(Derived)
did not match C++ signature:
acceptBaseSharedPtrRef(std::shared_ptr<(anonymous namespace)::Base> {lvalue})
This is intentional. To reduce the chance of a dangling reference and provide explicit directionality between the languages, Boost.Python will pass the temporary object resulting from an rvalue conversion by const reference to functions. The implicit_convertible<Source, Target> function registers an rvalue from-Python conversion. As the result of the converter is an rvalue, one can only accept it by either value or constant reference.
When a class is registered via boost::python::class_<T, HeldType, Bases> and HeldType is wrapping T:
HeldTypeT to an instance of the Python classTHeldType to Python objectHeldTypeBases, registers lvalue from-Python converter for instances of the Python class to instances of T in the base (not the base's HeldType)Bases, registers a to-Python converter from instances of T held by a base to the Python classWith the following setup:
class base {};
class derived: public base {};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base, std::shared_ptr<base>>("Base");
python::class_<derived, python::bases<base>,
std::shared_ptr<derived>>("Derived");
python::implicitly_convertible<std::shared_ptr<derived>,
std::shared_ptr<base>>();
}
The following lvalue from-Python conversions are possible because the Python object holds an instance of the C++ object:
example.Base to base, base&, const base&, std::shared_ptr<base>, std::shared_ptr<base>&, and const std::shared_ptr<base>&example.Derived to base, base&, const base&, derived, derived&, const derived&, std::shared_ptr<derived>, std::shared_ptr<derived>&, and const std::shared_ptr<derived>&The following to-Python conversions are possible:
base or std::shared_ptr<base> to example.Basederived or std::shared_ptr<derived to example.DerivedIf base was polymorphic, then the following to-Python conversions would be possible:
derived* and static type of base* to example.Derivedstd::shared_ptr<base> holding an instance of derived to example.DerivedThe following rvalue conversions is possible due to explicit registration via implicitly_convertible:
example.Derived to std::shared_ptr<base> and const std::shared_ptr<base>&The difference between an lvalue and an rvalue conversion is whether or not the target C++ object already exists and is being held in a Python object. For example, an lvalue conversion of example.Derived to base& is possible because example.Derived holds an instance of derived which is-a base. On the other hand, an lvalue conversion from example.Derived to std::shared_ptr<base>& is not possible because example.Derived holds an instance of std::shared_ptr<derived>, which does not inherit from std::shared_ptr<base>. Hence, a std::shared_ptr<base> with an unspecified lifetime is constructed and passed as an rvalue argument to the exposed function.
Here is a complete example demonstrating these conversions:
#include <boost/python.hpp>
#include <memory> // std::shared_ptr
class base {};
class derived: public base {};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base, std::shared_ptr<base>>("Base");
python::class_<derived, python::bases<base>,
std::shared_ptr<derived>>("Derived");
python::implicitly_convertible<std::shared_ptr<derived>,
std::shared_ptr<base>>();
python::def("base_value", +[](base){});
python::def("base_ref", +[](base&){});
python::def("base_cref", +[](const base&){});
python::def("shared_base_value", +[](std::shared_ptr<base>){});
python::def("shared_base_ref", +[](std::shared_ptr<base>&){});
python::def("shared_base_cref", +[](const std::shared_ptr<base>&){});
python::def("derived_value", +[](derived){});
python::def("derived_ref", +[](derived&){});
python::def("derived_cref", +[](const derived&){});
python::def("shared_derived_value", +[](std::shared_ptr<derived>){});
python::def("shared_derived_ref", +[](std::shared_ptr<derived>&){});
python::def("shared_derived_cref", +[](const std::shared_ptr<derived>&){});
}
Interactive usage:
>>> base = example.Base()
>>> example.base_value(base)
>>> example.base_ref(base)
>>> example.base_cref(base)
>>> example.shared_base_value(base)
>>> example.shared_base_ref(base)
>>> example.shared_base_cref(base)
>>>
>>> derived = example.Derived()
>>> example.base_value(derived)
>>> example.base_ref(derived)
>>> example.base_cref(derived)
>>> example.shared_base_value(derived)
>>> try:
... got_exception = False
... example.shared_base_ref(derived)
... except TypeError:
... got_exception = True
... finally:
... assert(got_exception)
...
>>> example.shared_base_cref(derived)
>>> example.derived_value(derived)
>>> example.derived_ref(derived)
>>> example.derived_cref(derived)
>>> example.shared_derived_value(derived)
>>> example.shared_derived_ref(derived)
>>> example.shared_derived_cref(derived)