I am trying to expose the C++ class with name aliasing to python using boost python.
struct Foo
{
void hi() const { std::cout << "hi" << std::endl; }
};
BOOST_PYTHON_MODULE(Example)
{
typedef Foo Bar;
class_<Foo>("Foo")
.def("hi", &Foo::hi)
;
class_<Bar>("Bar")
.def("hi", &Bar::hi)
;
}
The code works as expected except the annoying RuntimeWarning.
RuntimeWarning: to-Python converter for Foo already registered; second conversion method ignore
Adding Bar = Foo in python also works. But I need to keep the definitions in the same module. Is there a better way to achieve this?
I'd go with the "C++ equivalent to the Python Bar = Foo" approach that Ulrich mentions.
You can use boost::python::scope to get access to the current module and its attributes.
#include <boost/python.hpp>
#include <iostream>
namespace bp = boost::python;
struct Foo
{
void hi() const { std::cout << "hi" << std::endl; }
};
BOOST_PYTHON_MODULE(Example)
{
bp::class_<Foo>("Foo")
.def("hi", &Foo::hi)
;
bp::scope().attr("Bar") = bp::scope().attr("Foo");
}
Since typedef only introduces an alias, your code just registers the same class under different names.
Suggestions:
Why would you want that anyway? Just register it once under its real name. As you mentioned, creating an alias in Python (again, why?) is easy.
If you just declared a baseclass and derived both Foo and Bar from it, you would have different types and the warning would vanish, too.
You could probably also write a C++ equivalent to the Python Bar = Foo, i.e. a simple assignment of an object to a name in the module namespace.
Given the feedback below that it's required to support legacy code, here's what I would do:
// same as above
struct Foo { ... };
// For legacy reasons, it is mandatory that Foo is exported
// under two names. In order to introduce new C++ types, we
// just derive from the original Foo. The approach using a
// typedef doesn't work because it only creates an alias but
// not an existing type.
struct FooType: Foo {};
struct BarType: Foo {};
BOOST_PYTHON_MODULE(Example)
{
class_<FooType>("Foo")
.def("hi", &FooType::hi)
;
class_<BarType>("Bar")
.def("hi", &BarType::hi)
;
}
Related
I am wrapping C++ classes with boost-python and I am wondering is there is a better way to do it than what I am doing now.
The problem is that the classes have getters/setters that have the same name and there doesn't seem to be a painless way to wrap this with boost-python.
Here is a simplified version of the problem. Given this class:
#include <boost/python.hpp>
using namespace boost::python;
class Foo {
public:
double
x() const
{
return _x;
}
void
x(const double new_x)
{
_x = new_x;
}
private:
double _x;
};
I would like to do something like:
BOOST_PYTHON_MODULE(foo)
{
class_<Foo>("Foo", init<>())
.add_property("x", &Foo::x, &Foo::x)
;
}
This doesn't work because boost-python can't figure out which version of the function to use.
In fact, you can't even do
.def("x", &Foo::x)
for the same reason.
I was re-reading the tutorial at boost.org and the section on overloading seemed super promising. Unfortunately it doesn't seem to be what I'm looking for.
In the overloading section, it mentions a BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro that works like this:
if there were another member function in Foo that took defaulted arguments:
void z(int i=42)
{
std::cout << i << "\n";
}
you can then use the macro:
BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(z_member_overloads, z, 0, 1)
and then in the BOOST_PYTHON_MODULE:
.def("z", &Foo::z, z_member_overloads())
z_member_overloads lets you call def once and it will expose methods to python for both 0 arguments and 1 argument.
I was hoping that this would work for my x() and x(double val) getter/setter, but it doesn't work.
doing:
BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(x_member_overloads, x, 0, 1)
...
.def("x", &Foo::x, x_member_overloads())
doesn't compile:
error: no matching member function for call to 'def'
.def("x", &Foo::x, x_member_overloads())
~^~~
Question:
So, is there another macro or something that can make this work?
For completeness, this is how I'm currently handling cases like this:
.add_property(
"x",
make_function(
[](Foo& foo) {
return foo.x();
},
default_call_policies(),
boost::mpl::vector<double, Foo&>()
),
make_function(
[](Foo& foo, const double val) {
foo.x(val);
},
default_call_policies(),
boost::mpl::vector<void, Foo&, double>()
)
)
You can do this by casting to appropriate overload (untested):
class_<Foo>("Foo", init<>())
.add_property("x",
static_cast< double(Foo::*)() const >(&Foo::x), // getter
static_cast< void(Foo::*)(const double) >(&Foo::x)) // setter
;
If I want to print information about an object in C++, I'll use the outstream operator <<:
class Foo
{
public:
friend std::ostream& operator<<(std::ostream& out, const Foo& foo);
private:
double bar = 7;
};
inline std::ostream& operator<<(std::ostream& out, const Foo& foo)
{
return out << foo.bar;
}
Then, I can do Foo foo; std::cout << foo << std::endl;. Something equivalent in Python would be implementing __str__ and then saying print(foo). But since the operator is not really a member of Foo, I don't know how to do this in SWIG.
What would I have to write in my interface file to reuse my implementation of the outstream operator for use in print()?
Additionally, is it possible to let SWIG do an automatic redirect of shared_ptr of an object, so that if I somewhere return std::shared_ptr<Foo>, I can still call print(sharedPtrToFoo) and it will call the __str__ or operator<< of the pointed to object?
Something like this ought to work, assuming that you're not using `-builtin':
%extend Foo {
std::string __str__() const {
std::ostringstream out;
out << *$self;
return out.str();
}
}
Note that this is substantially similar to this answer, albeit with the recommendation to use std::string instead of const char * removing a subtle bug.
With regards to shared_ptr it should be that every method of Foo is exposed with shared_ptr<Foo> transparently so I'd expect that to just work.
I'm trying to wrap a C++ api and I'm hitting a roadblock on some char* class members. It seems that boost-python will auto convert char const * and std::string types into python objects (based on this answer), but it balks at char* types. This is the error I get (in python):
TypeError: No to_python (by-value) converter found for C++ type: char*
It turns out that these particular char * members probably should have been declared as char const * since the strings are never altered.
I'm new to boost-python so maybe there is an obvious answer, but I'm not having much luck googling this one.
Is there an easy way to tell boost-python to auto convert these char* members?
(Unfortunately I can't change the declarations of char * to char const * since the API I am wrapping is not under my control.)
UPDATE:
Ok so I think that I need to add a custom converter to handle the char* members. I started writing one:
/** to-python convert for char* */
struct c_char_p_to_python_str
{
static PyObject* convert(char* s) {
return incref(object(const_cast<const char*>(s)).ptr());
}
};
// register the QString-to-python converter
to_python_converter<char*, c_char_p_to_python_str>();
Unfortunately this does not work. This is the error:
error: expected unqualified-id
to_python_converter<char*, c_char_p_to_python_str>();
^
Looking at the docs I can see that the template args have this signature:
template <class T, class Conversion, bool convertion_has_get_pytype_member=false>
Since char* isn't a class I'm guessing that's why this didn't work. Anyone have some insight?
UPDATE2:
Nope. Turns out to_python_converter needs to get called inside of the BOOST_PYTHON_MODULE call.
I got the to_python_converter working (with some modifications). I also wrote a function to convert form python and registered it with converter::registry::push_back. I can see my to_python code running, but the from_python code never seems to run.
Let's assume we're wrapping some third-party API, and set aside the awfulness of having those pointers exposed and mucking with them from the outside.
Here's a short proof of concept:
#include <boost/python.hpp>
namespace bp = boost::python;
class example
{
public:
example()
{
text = new char[1];
text[0] = '\0';
}
~example()
{
delete[] text;
}
public:
char* text;
};
char const* get_example_text(example* e)
{
return e->text;
}
void set_example_text(example* e, char const* new_text)
{
delete[] e->text;
size_t n(strlen(new_text));
e->text = new char[n+1];
strncpy(e->text, new_text, n);
e->text[n] = '\0';
}
BOOST_PYTHON_MODULE(so02)
{
bp::class_<example>("example")
.add_property("text", &get_example_text, &set_example_text)
;
}
Class example owns text, and is responsible for managing the memory.
We provide an external getter and setter function. The getter is simple, it just provides read access to the string. The setter frees the old string, allocates new memory of appropriate size, and copies the data.
Here's a simple test in python interpreter:
>>> import so02
>>> e = so02.example()
>>> e.text
''
>>> e.text = "foobar"
>>> e.text
'foobar'
Notes:
set_example_text() could perhaps take std::string or bp::object so that we have the lenght easily available, and potentially allow assignment from more than just strings.
If there are many member variables to wrap and the getter/setter pattern is similar, generate the code using templates, or even just few macros.
There may be a way to do this with the converters, I'll have a look into that tomorrow. However, as we're dealing with memory management here, i'd personally prefer to handle it this way, as it's much more obvious what's happening.
This expands on Dan's answer. I wrote some macro definitions which generate lambda expressions. The benefits of this approach are that it is not tied to a particular type or member name.
In the API I am wrapping, I have a few hundred classes to wrap. This allows me to make a single macro call for every char* class member.
Here is a modified version of Dan's example code:
#include <boost/python.hpp>
namespace bp = boost::python;
#define ADD_PROPERTY(TYPE, ATTR) add_property(#ATTR, SET_CHAR_P(TYPE, ATTR), \
GET_CHAR_P(TYPE, ATTR))
#define SET_CHAR_P(TYPE, ATTR) +[](const TYPE& e){ \
if (!e.ATTR) return ""; \
return (const char*)e.ATTR; \
}
#define GET_CHAR_P(TYPE, ATTR) +[](TYPE& e, char const* new_text){ \
delete[] e.ATTR; \
size_t n(strlen(new_text)); \
e.ATTR = new char[n+1]; \
strncpy(e.ATTR, new_text, n); \
e.ATTR[n] = '\0'; \
}
class example
{
public:
example()
{
text = new char[1];
text[0] = '\0';
}
~example()
{
delete[] text;
}
public:
char* text;
};
BOOST_PYTHON_MODULE(topics)
{
bp::class_<example>("example")
.ADD_PROPERTY(example, text);
}
I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?
The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))
I have problem wrapping an Enum for Python using Boost-Python.
Initially I intended to do something like the following in the try-catch (I've inserted my whole code below) statement:
main_namespace["Motion"] = enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
Everything was fine and compilation was done. At run time I got this error (which makes no sense to me):
AttributeError: 'NoneType' object has no attribute 'Motion'
Afterwards I decided to write a Python module using BOOST_PYTHON_MODULE in my code.
After initializing Python interpreter I wanted to use this module right away but didn't know how(?). The following is my whole code:
#include <boost/python.hpp>
#include <iostream>
using namespace std;
using namespace boost::python;
BOOST_PYTHON_MODULE(test)
{
enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
}
int main()
{
Py_Initialize();
try
{
object pyMainModule = import("__main__");
object main_namespace = pyMainModule.attr("__dict__");
//What previously I intended to do
//main_namespace["Motion"] = enum_<TestClass::Motion>("Motion")
// .value("walk", TestClass::walk)
// .value("bike", TestClass::bike)
//;
//I want to use my enum here
//I need something like line below which makes me able to use the enum!
exec("print 'hello world'", main_namespace, main_namespace);
}
catch(error_already_set const&)
{
PyErr_Print();
}
Py_Finalize();
return 0;
}
Anything useful to know about wrapping and using Enums in Python will be appreciated!
Thanks in advance
The AttributeError is the result of trying to create a Python extension type without first setting scope. The boost::python::enum_ constructor states:
Constructs an enum_ object holding a Python extension type derived from int which is named name. The named attribute of the current scope is bound to the new extension type.
When embedding Python, to use a custom Python module, it is often easiest to use PyImport_AppendInittab, then import the module by name.
PyImport_AppendInittab("example", &initexample);
...
boost::python::object example = boost::python::import("example");
Here is a complete example showing two enumerations being exposed through Boost.Python. One is included in a separate module (example) that is imported by main, and the other is exposed directly in main.
#include <iostream>
#include <boost/python.hpp>
/// #brief Mockup class with a nested enum.
struct TestClass
{
/// #brief Mocked enum.
enum Motion
{
walk,
bike
};
// #brief Mocked enum.
enum Color
{
red,
blue
};
};
/// #brief Python example module.
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
}
int main()
{
PyImport_AppendInittab("example", &initexample); // Add example to built-in.
Py_Initialize(); // Start interpreter.
// Create the __main__ module.
namespace python = boost::python;
try
{
python::object main = python::import("__main__");
python::object main_namespace = main.attr("__dict__");
python::scope scope(main); // Force main scope
// Expose TestClass::Color as Color
python::enum_<TestClass::Color>("Color")
.value("red", TestClass::red)
.value("blue", TestClass::blue)
;
// Print values of Color enumeration.
python::exec(
"print Color.values",
main_namespace, main_namespace);
// Get a handle to the Color enumeration.
python::object color = main_namespace["Color"];
python::object blue = color.attr("blue");
if (TestClass::blue == python::extract<TestClass::Color>(blue))
std::cout << "blue enum values matched." << std::endl;
// Import example module into main namespace.
main_namespace["example"] = python::import("example");
// Print the values of the Motion enumeration.
python::exec(
"print example.Motion.values",
main_namespace, main_namespace);
// Check if the Python enums match the C++ enum values.
if (TestClass::bike == python::extract<TestClass::Motion>(
main_namespace["example"].attr("Motion").attr("bike")))
std::cout << "bike enum values matched." << std::endl;
}
catch (const python::error_already_set&)
{
PyErr_Print();
}
}
Output:
{0: __main__.Color.red, 1: __main__.Color.blue}
blue enum values matched.
{0: example.Motion.walk, 1: example.Motion.bike}
bike enum values matched.