Is there a way to cast between classes where only the interface is known in boost python?
Consider the following example:
// ---------- Module A ----------
class A_ifc
{
public:
virtual int foo() = 0;
};
class A_impl : public A_ifc
{
public:
int foo() {...}
};
BOOST_PYTHON_MODULE(Module_A)
{
class_<A_impl, boost::noncopyable>("A", no_init)
.def("foo", &A_impl::foo)
;
}
// ---------- Module B ----------
class B_ifc
{
public:
virtual int cross_module_operation(A_ifc* a) = 0;
};
class B_impl : public B_ifc
{
public:
int cross_module_operation(A_ifc* a) {...}
};
BOOST_PYTHON_MODULE(Module_B)
{
class_<B_impl, boost::noncopyable>("B", no_init)
.def("cross_module_operation", &B_impl::cross_module_operation)
;
}
In my C code none of the classes know anything of each other except for their respective interfaces (here 'A_ifc' and 'B_ifc'). All cross-module operations pass only interface pointers.
When I want to achieve the same behavior in Python however this wouldn't work, because for Boost Python the whole class definition must be known for every referenced class.
I have already tried opaque pointers (see http://www.boost.org/doc/libs/1_55_0/libs/python/doc/v2/return_opaque_pointer.html), but whenever I compile any Boost code that contains "boost::python::return_value_policy", the application crashes with an access violation on startup.
I also looked into custom converters (see Boost.Python custom converter), but I have no idea how to apply this concept to my problem.
Any help is greatly appreciated
As it turns out, casting to a an abstract base class is indeed possible as long as as run-time type information is enabled.
In other words this will work: objects::register_conversion<A_impl, A_ifc>();
I feel a bit silly for not having tried such an obvious solution earlier.
Related
I have include a minimal working example below - it can be compiled using the typical pybind11 instructions (I use cmake).
I have an abstract base class, Abstract, which is pure virtual. I can easily wrap this in pybind11 using a "trampoline class" (this is well documented by pybind11).
Further, I have a concrete implementation of Abstract, ToBeWrapped, that is also wrapped using pybind11.
My issue is that I have some client code which accepts an arbitrary PyObject* (or, in the case of this example, pybind11's wrapper py::object) and expects to cast this to Abstract*.
However, as illustrated in my example, I am unable to cast the py::object to Abstract*.
I have no problem casting to ToBeWrapped* and then storing that as an Abstract*', however this would require my client code to know ahead of time what kind ofAbstract*` the python interpreter is sending, which defeats the purpose of the abstract base class.
TL;DR
Is it possible to modify this code such that the client accessMethod is able to arbitrarily handle an Abstract* passed from the python interpreter?
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;
// abstract base class - cannot be instantiated on its own
class Abstract
{
public:
virtual ~Abstract() = 0;
virtual std::string print() const = 0;
};
Abstract::~Abstract(){}
// concrete implementation of Abstract
class ToBeWrapped : public Abstract
{
public:
ToBeWrapped(const std::string& msg = "heh?")
: myMessage(msg){};
std::string print() const override
{
return myMessage;
}
private:
const std::string myMessage;
};
// We need a trampoline class in order to wrap this with pybind11
class AbstractPy : public Abstract
{
public:
using Abstract::Abstract;
std::string print() const override
{
PYBIND11_OVERLOAD_PURE(
std::string, // return type
Abstract, // parent class
print, // name of the function
// arguments (if any)
);
}
};
// I have client code that accepts a raw PyObject* - this client code base implements its
// own python interpreter, and calls this "accessMethod" expecting to convert the python
// object to its c++ type.
//
// Rather than mocking up the raw PyObject* method (which would be trivial) I elected to
// keep this minimal example 100% pybind11
void accessMethod(py::object obj)
{
// runtime error: py::cast_error
//Abstract* casted = obj.cast<Abstract*>();
// this works
Abstract* casted = obj.cast<ToBeWrapped*>();
}
PYBIND11_MODULE(PyMod, m)
{
m.doc() = R"pbdoc(
This is a python module
)pbdoc";
py::class_<Abstract, AbstractPy>(m, "Abstract")
.def("print", &Abstract::print)
;
py::class_<ToBeWrapped>(m, "WrappedClass")
.def(py::init<const std::string&>())
;
m.def("access", &accessMethod, "This method will attempt to access the wrapped type");
}
You need to declare the hierarchy relationship, so this:
py::class_<ToBeWrapped>(m, "WrappedClass")
should be:
py::class_<ToBeWrapped, Abstract>(m, "WrappedClass")
I'm trying to use pybind11 to bind a struct that looks like this
struct myStruct {
int na;
int nb;
double* a;
double* b;
}
I'm not sure the right way to go about it. The examples in the pybind11 documentation show how to attach buffer protocol semantics to an object, but not to a class member.
I don't have the luxury of changing the interface of myStruct to contain std::vectors either which would allow me to use the usual .def_readwrite().
I've tried doing something like this
py::class_<myStruct>(m, "myStruct")
.def_property("a",
[](myStruct &s) {return py::array<double>({s.na}, {sizeof(double), s.a};)},
[](myStruct &s, py::array_t<double> val) {std::copy((double*) val.request().ptr, (double*) val.request().ptr + s.na, s.a);)}
)
Which compiles, but in python I don't see changes persist in the underlying data
print(my_struct.a[0]) # prints 0.0
my_struct.a[0] = 123.0
print(my_struct.a[0]) # still prints 0.0
Hey most likely not the most elegant answer, but maybe it gives you a starting point and temporary solution. I think what you need to do is use shared pointers.
Under https://github.com/pybind/pybind11/issues/1150 someone asked something similar but I was not able to adapt it to your example and only got the same result to yours with no changes to the data.
What worked for me in your specific example was using the shared_ptr and defining setter and getter functions for the pointers with a simple def_property for the pybin11 class.
class class_DATA{
public:
int na;
std::shared_ptr<double> a;
void set_a(double a){*class_DATA::a = a; };
double get_a(void){return *class_DATA::a; };
};
PYBIND11_MODULE(TEST,m){
m.doc() = "pybind11 example plugin";
//the costum class
py::class_<class_DATA>(m, "class_DATA", py::dynamic_attr())
.def(py::init<>()) //needed to define constructor
.def_readwrite("na", &class_DATA::na)
.def_property("a", &class_DATA::get_a, &class_DATA::set_a, py::return_value_policy::copy);
}
I'm exposing an API to Python, written in C++ that I have no access to change, using Boost Python.
I have successfully exposed methods returning references to a std:map where the key,value pairs are value types - eg:
class_< std::map<std::string, std::string> >("StringMap")
.def(map_indexing_suite< std::map<std::string, std::string>, true >());
This works seamlessly. But when trying to achieve a similar result where the map values are pointers to classes I've exposed within the API doesn't work:
struct X_wrap : X, wrapper<X>
{
X_wrap(int i): X(i) {}
// virtual methods here, omitted for brevity - as unlikely to be the issue
}
BOOST_PYTHON_MODULE(my_py_extension)
{
class_< std::map<std::string, X*> >("XPtrMap")
.def(map_indexing_suite< std::map<std::string, X*> >());
class_<X_wrap, boost::noncopyable, bases<XBase> >("X", init<int>())
// other definitions omitted
}
Error seen in g++ 7.3.0:
/usr/include/boost/python/detail/caller.hpp:100:98: error: ‘struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<X*>’ has no member named ‘get_pytype’
I understand why the compiler is complaining - the X* in the map needs to be wrapped in a call policy so that it can be returned to Python, just like with a basic method that returns a raw pointer.
My question is what is the best way to do this?
From Googling it strikes that I can perhaps specify a DerivedPolicies child class of map_indexing_suite that will overload the necessary parts to wrap the X* in an appropriate return_value_policy. However so far I've be unsuccessful in putting anything together that the compiler doesn't bawk at!
I also suspect I can literally copy-and-paste the whole map_indexing_suite and rename it, and make the changes therein to produce a new indexing_suite with the right return_value_policy, but this seems ugly compared to the solution using DerviedPolicies - assuming I'm right that DeriviedPolicies can be used at all!
Any help, pointers, or examples gratefully received!
EDIT
I have proved that the cut-and-paste option works with a single trivial change of is_class to is_pointer. It's curious that is_pointer is not allowed in the original as the target policy can handle pointers. I'm yet to convince myself that it's an object lifetime restriction that means pointers are not allowed in the original?
The whole class is public so I suspect it's possible to avoid the full cut-and-paste by simply inheriting from map_indexing_suite or perhaps by using the mysterious DerivedPolicies parameter?
extension_def(Class& cl)
{
// Wrap the map's element (value_type)
std::string elem_name = "mapptr_indexing_suite_";
object class_name(cl.attr("__name__"));
extract<std::string> class_name_extractor(class_name);
elem_name += class_name_extractor();
elem_name += "_entry";
typedef typename mpl::if_<
mpl::and_<is_pointer<data_type>, mpl::bool_<!NoProxy> >
, return_internal_reference<>
, default_call_policies
>::type get_data_return_policy;
class_<value_type>(elem_name.c_str())
.def("__repr__", &DerivedPolicies::print_elem)
.def("data", &DerivedPolicies::get_data, get_data_return_policy())
.def("key", &DerivedPolicies::get_key)
;
}
EDIT 2
Now see answer
Slightly cleaner implementation from the cut-and-paste is to inherit map_indexing_suite - a few tweaks are needed to make this work.
This seems reasonably sensible - if someone chimes in with a neater solution or can better explain DerivedPolicies then great, otherwise I'll accept the below as the answer in a few days or so...
using namespace boost;
using namespace boost::python;
//Forward declaration
template <class Container, bool NoProxy, class DerivedPolicies>
class mapptr_indexing_suite;
template <class Container, bool NoProxy>
class final_mapptr_derived_policies
: public mapptr_indexing_suite<Container,
NoProxy, final_mapptr_derived_policies<Container, NoProxy> > {};
template <
class Container,
bool NoProxy = false,
class DerivedPolicies
= final_mapptr_derived_policies<Container, NoProxy> >
class mapptr_indexing_suite
: public map_indexing_suite<
Container,
NoProxy,
DerivedPolicies
>
{
public:
// Must be explicit if the compiler is
// going to take from the base class
using typename map_indexing_suite<
Container,NoProxy,DerivedPolicies>::data_type;
using typename map_indexing_suite<
Container,NoProxy,DerivedPolicies>::value_type;
// Only one class needs to be overridden from the base
template <class Class>
static void
extension_def(Class& cl)
{
// Wrap the map's element (value_type)
std::string elem_name = "mapptr_indexing_suite_";
object class_name(cl.attr("__name__"));
extract<std::string> class_name_extractor(class_name);
elem_name += class_name_extractor();
elem_name += "_entry";
// use of is_pointer here is the only
// difference to the base map_indexing_suite
typedef typename mpl::if_<
mpl::and_<std::is_pointer<data_type>, mpl::bool_<!NoProxy> >
, return_internal_reference<>
, default_call_policies
>::type get_data_return_policy;
class_<value_type>(elem_name.c_str())
.def("__repr__", &DerivedPolicies::print_elem)
.def("data", &DerivedPolicies::get_data, get_data_return_policy())
.def("key", &DerivedPolicies::get_key)
;
}
};
I have an existing c++ code library that uses a struct with std::vector, which should be exposed to python.
in the header:
struct sFOO
{
unsigned int start = 3 ;
double foo = 20.0 ;
};
in the cpp:
namespace myName
{
myfoo::myfoo(){
sFOO singlefoo;
std::vector<sFOO> foos;
}
sFOO singlefoo;
std::vector<sFOO>* myfoo::get_vector(){
return &foos;
}
}
and for boost::python snippet:
using namespace boost::python;
class dummy3{};
BOOST_PYTHON_MODULE(vStr)
{
scope myName = class_<dummy3>("myName");
class_<myName::sFOO>("sFOO")
.add_property("start",&myName::sFOO::start)
.add_property("foo",&myName::sFOO::foo)
;
class_<myName::myfoo>("myfoo", no_init)
.def(init<>())
.def("checkfoo",&myName::myfoo::checkfoo)
.add_property("foos",&myName::myfoo::foos)
.add_property("singlefoo",&myName::myfoo::singlefoo)
}
(FYI, fictitious class dummy3 is used to simulate namespace, and using scope is therefore not an option.)
A compilation and importing processes are OK, and I can access singlefoo, but whenever I try to access vector foos, I encounter the error message below.
Python class registered for C++ class std::vector<myName::sFOO,
std::allocator<myName::sFOO> >
To circumvent this issue,
I've firstly tried vector_indexing_suite, but it didn't help exposing pre-defined vector of struct.
I also assumed that there should be a solution related to exposing pointer to python, so I have tried to get a pointer by following:
.def("get_vector",&myName::myfoo::get_vector)
which produces compile Error.
Since I am quite a novice to both C++ and Boost, any comments including solution, tips for search, and a suggestion to suitable reference would be greatly appreciated.
Thanks in advance!
Method .def("get_vector",&myName::myfoo::get_vector) is not working because it returns a pointer to a vector, so it's necessary to inform the policy that defines how object ownership should be managed:
class_<myName::myfoo>("myfoo", no_init)
// current code
.def("get_vector", &myfoo::get_vector, return_value_policy<reference_existing_object>())
;
In order to use vector_indexing_suite, it is necessary to implement the equal to operator to the class that it holds:
struct sFOO
{
unsigned int start = 3 ;
double foo = 20.0 ;
bool operator==(const sFOO& rhs)
{
return this == &rhs; //< implement your own rules.
}
};
then you can export the vector:
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
class_<std::vector<sFOO>>("vector_sFOO_")
.def(vector_indexing_suite<std::vector<sFOO>>())
;
I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?
The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))