I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?
The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))
Related
Is it possible to store variables/arrays/objects in a C++ Object to store and modify such data through Python? For example, I want to store arrays/vectors of points/polygons/voxels/etc in a C++ Object in Python. And sometimes, I want to do something with them (change/add/remove).
Here is my simple code, but I've got an Exception:
C++:
class Foo{
public:
std::vector<openvdb::Vec3s> vertices_test;
void generateArray(std::vector<openvdb::Vec3s> ar){
// JUST DO SOMETHING GENERIC
for (int i = 0; i < 100; i++) {
ar.push_back(openvdb::Vec3s(0.3, 0.1, 0.2));
}
}
};
extern "C" {
__declspec(dllexport) Foo* Foo_new(){ return new Foo(); }
__declspec(dllexport) void Foo_generateArray(Foo* foo){ foo->generateArray(foo->vertices_test); }
}
Python part:
class Foo(object):
def __init__(self):
self.obj = lib.Foo_new()
def generateArray(self):
lib.Foo_generateArray(self.obj)
f = Foo()
f.generateArray()
But when I run Python I get an exception:
OSError: exception: access violation reading 0x0000000058ED7D28
As you can see I tried to store std::vectoropenvdb::Vec3s vertices_test object as a local variable in the C++ Object. But it looks like something goes wrong.
Also, is it possible to return float/int in the generateArray() function instead of void?
Thanks you.
From the ctypes documentation:
By default functions are assumed to return the C int type. Other return types can be specified by setting the restype attribute of the function object.
Your question implies that you're on 64-bit Windows, and on 64-bit Windows, pointers are 8 bytes and ints are only 4 bytes, so the upper 4 bytes of your pointer are being lost.
In cpp code that I can't modify, we have custom class and custom pointers.
This class reads an object and depending on kwargs might also write it.
I need to make pybind11 bindings for it.
Depending on the arguments passed, we get const or non-const pointer to the class.
Pybind complaints about inconsistent types for lambda return type.
It all goes well for making init() function that works only for const or non-const objects. Unfortunately, need to support both cases.
What would be the best way to make python binding to cpp code in this case?
Error
In lambda function:
error: inconsistent types 'std::shared_ptr<myClass>' and 'std::shared_ptr<const myClass>'
deduced for lambda return type
return *const_p;
How could the code below be modified to support both cases?
Using Cpp 11 compiler.
// those defined elsewhere
typedef std::shared_ptr< myClass > my_ptr;
typedef std::shared_ptr< const myClass > const_my_ptr;
//init function for pybind11
.def(py::init([](py::kwargs kwargs) {
bool object_writable = py::bool_(kwargs["rw"]);
int cache = py::bool_(kwargs["cache"]);
std::string path = py::str(kwargs["path"]);
if (object_writable){
//returns non const
my_ptr p = myClass::read_write(path)
return *p;
}
else{
//returns const
const_my_ptr const_p = myClass::read(path, cache)
return *const_p;
}
}))
The recommended way to expose a non-pure virtual function in boost python is to wrap it as shown below.
boost python doc
struct Base
{
virtual ~Base() {}
virtual int f() { return 0; }
};
struct BaseWrap : Base, wrapper<Base>
{
int f()
{
if (override f = this->get_override("f"))
return f(); // *note*
return Base::f();
}
int default_f() { return this->Base::f(); }
};
Finally exposing:
class_<BaseWrap, boost::noncopyable>("Base")
.def("f", &Base::f, &BaseWrap::default_f)
;
The document explains
Take note that we expose both &Base::f and &BaseWrap::default_f. Boost.Python needs to keep track of 1) the dispatch function f and 2) the forwarding function to its default implementation default_f. There's a special def function for this purpose.
What does this special def function actually do, and what is the difference below?
.def("f", &Base::f, &BaseWrap::default_f)
.def("f", &BaseWrap::default_f)
From the source code in class.hpp::def_impl,
I only see default_f as a overload function add to namespace too as &Base::f.
because of the same signature, default_f would replace the &Base::f.
Is there any thing I missed? Any suggestions or examples would be a huge help!
I have include a minimal working example below - it can be compiled using the typical pybind11 instructions (I use cmake).
I have an abstract base class, Abstract, which is pure virtual. I can easily wrap this in pybind11 using a "trampoline class" (this is well documented by pybind11).
Further, I have a concrete implementation of Abstract, ToBeWrapped, that is also wrapped using pybind11.
My issue is that I have some client code which accepts an arbitrary PyObject* (or, in the case of this example, pybind11's wrapper py::object) and expects to cast this to Abstract*.
However, as illustrated in my example, I am unable to cast the py::object to Abstract*.
I have no problem casting to ToBeWrapped* and then storing that as an Abstract*', however this would require my client code to know ahead of time what kind ofAbstract*` the python interpreter is sending, which defeats the purpose of the abstract base class.
TL;DR
Is it possible to modify this code such that the client accessMethod is able to arbitrarily handle an Abstract* passed from the python interpreter?
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;
// abstract base class - cannot be instantiated on its own
class Abstract
{
public:
virtual ~Abstract() = 0;
virtual std::string print() const = 0;
};
Abstract::~Abstract(){}
// concrete implementation of Abstract
class ToBeWrapped : public Abstract
{
public:
ToBeWrapped(const std::string& msg = "heh?")
: myMessage(msg){};
std::string print() const override
{
return myMessage;
}
private:
const std::string myMessage;
};
// We need a trampoline class in order to wrap this with pybind11
class AbstractPy : public Abstract
{
public:
using Abstract::Abstract;
std::string print() const override
{
PYBIND11_OVERLOAD_PURE(
std::string, // return type
Abstract, // parent class
print, // name of the function
// arguments (if any)
);
}
};
// I have client code that accepts a raw PyObject* - this client code base implements its
// own python interpreter, and calls this "accessMethod" expecting to convert the python
// object to its c++ type.
//
// Rather than mocking up the raw PyObject* method (which would be trivial) I elected to
// keep this minimal example 100% pybind11
void accessMethod(py::object obj)
{
// runtime error: py::cast_error
//Abstract* casted = obj.cast<Abstract*>();
// this works
Abstract* casted = obj.cast<ToBeWrapped*>();
}
PYBIND11_MODULE(PyMod, m)
{
m.doc() = R"pbdoc(
This is a python module
)pbdoc";
py::class_<Abstract, AbstractPy>(m, "Abstract")
.def("print", &Abstract::print)
;
py::class_<ToBeWrapped>(m, "WrappedClass")
.def(py::init<const std::string&>())
;
m.def("access", &accessMethod, "This method will attempt to access the wrapped type");
}
You need to declare the hierarchy relationship, so this:
py::class_<ToBeWrapped>(m, "WrappedClass")
should be:
py::class_<ToBeWrapped, Abstract>(m, "WrappedClass")
I have a legacy code in C++ (which would be a huge pain to edit) and I need to use it in Python 2 for speed reasons.
I have two classes. One is responsible for loading huge amount of data from memory, in a form of std::string and converting it to internal representation MiddleClass. Second one is converting it from internal representation MiddleClass back to std::string.
class Load {
Load(const std::string & data) { ... };
MiddleClass load() { ... };
};
class Save {
Save(std::string & data) { .... };
void save(const MiddleClass & middleclass) { ... };
};
My goal is, to use this setup in Python 2 like this:
import datahandler # my lib
import requests
request = request.get("url-to-data")
loader = datahandler.Load(request.content) # my C++ class Load
internal_representation = loader.load()
.
.
.
result_variable = str() # or None or something not important
saver = datahandler.Save(result_variable) # my C++ class Save
saver.save(internal_representation)
How can I achieve this?
I've run into trouble, right from the start.
Simple variant:
BOOST_PYTHON_MODULE(datahandler)
{
class_<MiddleClass>("MiddleClass");\
// some .defs - not important
class <Load>("Load", init<const std::string &>())
.def("load". &Load::load);
class <Save>("Save", init<std::string &>())
.def("save". &Save::save);
}
Will compile, no worries, but data which are loaded are somehow mangled, which leads me to thinking, that I am doing it terribly wrongly.
Also I found this bit offtopic SO question, which told me, that I can't have std::string &, because Python strings are immutable.
So conclusion: I have no idea what to do now :( Can anyone here help me? Thanks.
Take as reference this working example.
Define your C++ classes. For instance:
class MiddleClass {
public:
explicit MiddleClass(const std::string& data) : parent_data_(data) {}
void print() {
std::cout << parent_data_ << std::endl;
}
private:
std::string parent_data_;
};
class Loader {
public:
explicit Loader(const std::string& data) :
data_(data){
};
MiddleClass load() {
return MiddleClass(data_);
};
private:
std::string data_;
};
Create the boost bindings
boost::python::class_<MiddleClass>("MiddleClass",
boost::python::init<const std::string&>(boost::python::arg("data"), ""))
.def("print_data", &MiddleClass::print);
boost::python::class_<Loader>("Loader",
boost::python::init<const std::string&>(boost::python::arg("data"), ""))
.def("load", &Loader::load);
Install your library in the right python site-package.
Enjoy it in python:
from my_cool_package import MiddleClass, Loader
example_string = "whatever"
loader = Loader(data=example_string)
# Get the middle class
middle_class = loader.load()
# Print the data in the middle class
middle_class.print_data()
The expected output:
whatever
So, I have found a solution. Prove me wrong, but I think, that what am I trying to achieve is impossible.
Python has immutable strings, so passing a "reference" of string to function and expecting ability to change it from inside a function is simply not valid.
Take this code as an example:
variable = "Hello"
def changer(var):
var = "Bye"
changer(variable)
print(variable)
Prints "Hello". In Python, you can't make it work differently. (although to be exact, it is still being passed as a reference, but when you modify Python string, you just create a new one and a new reference).
So, how to get arround this?
Simple! Create a C++ wrapper, that will handle passing reference on std::string and return copy of resulting string. Not very effective, but you probably can't make it better.
Sample code of SaveWrapper class:
class SaveWrapper {
public:
// some constructor
std::string save(MiddleClass & value) {
std::string result;
Save saver(result);
saver.save(value);
return result;
}
};
Which can be easily "ported" to Python!