Develop Reference

Calling Python in C++

I'm learning C++, and in particular C interface to Python. Right now, my focus is on calling or importing python objects from C++ main program.
I've been studying the following link but couldn't understand some concepts. (https://www.codeproject.com/Articles/820116/Embedding-Python-program-in-a-C-Cplusplus-code)
Following is the sections of the tutorial that I can't understand fully.
My questions are:
calling of module:
Is it correct for me to assume "CPyObject pModule = PyImport_Import(pName)" is doing this job?
importing of objects:
i. Is it correct for me to assume "CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger")" is doing this job?
ii.If I want to import a dataframe from python to C++ as a CPyObject, how can I manipulate this object in C++. I'm asking because there is no equivalent object to dataframe in C++.
3) Is there anything else I need to do to make sure my Python module file is visible and callable to C++ ? Such as saving them in the same folder?
Consider the following Python program, stored in pyemb3.py:
def getInteger():
print('Python function getInteger() called')
c = 100*50/30
return c
Now we want to call the function getInteger() from the following C++ code and print the value returned this function. This is the client C++ code:
#include <stdio.h>
#include <Python.h>
#include <pyhelper.hpp>
int main()
{
CPyInstance hInstance;
CPyObject pName = PyUnicode_FromString("pyemb3");
CPyObject pModule = PyImport_Import(pName);
if(pModule)
{
CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger");
if(pFunc && PyCallable_Check(pFunc))
{
CPyObject pValue = PyObject_CallObject(pFunc, NULL);
printf_s("C: getInteger() = %ld\n", PyLong_AsLong(pValue));
}
else
{
printf("ERROR: function getInteger()\n");
}
}
else
{
printf_s("ERROR: Module not imported\n");
}
return 0;
}

The problem is that 100*50/30 is not an integer, it is a float.
to get an integer use integer division: 100*50//30
If you are not sure about the returned type, you can use the Py_TYPE macro on pValue or just simply check for the type you are looking for with: PyLong_Check or PyLong_CheckExact
1: if PyImport_Import does not return null then the import was successful and the module was already executed by the time the function returned.
2: The PyObject_GetAttrString or the PyObject_GetAttr is the right way to get the imported module's objects.
3: Use these flags to ensure Python is embedded. Use Py_SetPath before Py_Initialize to add your module's path to sys.path.

Is it possible in pybind11 to use py::cast to access an abstract base class?

I have include a minimal working example below - it can be compiled using the typical pybind11 instructions (I use cmake).
I have an abstract base class, Abstract, which is pure virtual. I can easily wrap this in pybind11 using a "trampoline class" (this is well documented by pybind11).
Further, I have a concrete implementation of Abstract, ToBeWrapped, that is also wrapped using pybind11.
My issue is that I have some client code which accepts an arbitrary PyObject* (or, in the case of this example, pybind11's wrapper py::object) and expects to cast this to Abstract*.
However, as illustrated in my example, I am unable to cast the py::object to Abstract*.
I have no problem casting to ToBeWrapped* and then storing that as an Abstract*', however this would require my client code to know ahead of time what kind ofAbstract*` the python interpreter is sending, which defeats the purpose of the abstract base class.
TL;DR
Is it possible to modify this code such that the client accessMethod is able to arbitrarily handle an Abstract* passed from the python interpreter?
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;
// abstract base class - cannot be instantiated on its own
class Abstract
{
public:
virtual ~Abstract() = 0;
virtual std::string print() const = 0;
};
Abstract::~Abstract(){}
// concrete implementation of Abstract
class ToBeWrapped : public Abstract
{
public:
ToBeWrapped(const std::string& msg = "heh?")
: myMessage(msg){};
std::string print() const override
{
return myMessage;
}
private:
const std::string myMessage;
};
// We need a trampoline class in order to wrap this with pybind11
class AbstractPy : public Abstract
{
public:
using Abstract::Abstract;
std::string print() const override
{
PYBIND11_OVERLOAD_PURE(
std::string, // return type
Abstract, // parent class
print, // name of the function
// arguments (if any)
);
}
};
// I have client code that accepts a raw PyObject* - this client code base implements its
// own python interpreter, and calls this "accessMethod" expecting to convert the python
// object to its c++ type.
//
// Rather than mocking up the raw PyObject* method (which would be trivial) I elected to
// keep this minimal example 100% pybind11
void accessMethod(py::object obj)
{
// runtime error: py::cast_error
//Abstract* casted = obj.cast<Abstract*>();
// this works
Abstract* casted = obj.cast<ToBeWrapped*>();
}
PYBIND11_MODULE(PyMod, m)
{
m.doc() = R"pbdoc(
This is a python module
)pbdoc";
py::class_<Abstract, AbstractPy>(m, "Abstract")
.def("print", &Abstract::print)
;
py::class_<ToBeWrapped>(m, "WrappedClass")
.def(py::init<const std::string&>())
;
m.def("access", &accessMethod, "This method will attempt to access the wrapped type");
}

You need to declare the hierarchy relationship, so this:
py::class_<ToBeWrapped>(m, "WrappedClass")
should be:
py::class_<ToBeWrapped, Abstract>(m, "WrappedClass")

Boost python typedef

I am trying to expose the C++ class with name aliasing to python using boost python.
struct Foo
{
void hi() const { std::cout << "hi" << std::endl; }
};
BOOST_PYTHON_MODULE(Example)
{
typedef Foo Bar;
class_<Foo>("Foo")
.def("hi", &Foo::hi)
;
class_<Bar>("Bar")
.def("hi", &Bar::hi)
;
}
The code works as expected except the annoying RuntimeWarning.
RuntimeWarning: to-Python converter for Foo already registered; second conversion method ignore
Adding Bar = Foo in python also works. But I need to keep the definitions in the same module. Is there a better way to achieve this?

I'd go with the "C++ equivalent to the Python Bar = Foo" approach that Ulrich mentions.
You can use boost::python::scope to get access to the current module and its attributes.
#include <boost/python.hpp>
#include <iostream>
namespace bp = boost::python;
struct Foo
{
void hi() const { std::cout << "hi" << std::endl; }
};
BOOST_PYTHON_MODULE(Example)
{
bp::class_<Foo>("Foo")
.def("hi", &Foo::hi)
;
bp::scope().attr("Bar") = bp::scope().attr("Foo");
}

Since typedef only introduces an alias, your code just registers the same class under different names.
Suggestions:
Why would you want that anyway? Just register it once under its real name. As you mentioned, creating an alias in Python (again, why?) is easy.
If you just declared a baseclass and derived both Foo and Bar from it, you would have different types and the warning would vanish, too.
You could probably also write a C++ equivalent to the Python Bar = Foo, i.e. a simple assignment of an object to a name in the module namespace.
Given the feedback below that it's required to support legacy code, here's what I would do:
// same as above
struct Foo { ... };
// For legacy reasons, it is mandatory that Foo is exported
// under two names. In order to introduce new C++ types, we
// just derive from the original Foo. The approach using a
// typedef doesn't work because it only creates an alias but
// not an existing type.
struct FooType: Foo {};
struct BarType: Foo {};
BOOST_PYTHON_MODULE(Example)
{
class_<FooType>("Foo")
.def("hi", &FooType::hi)
;
class_<BarType>("Bar")
.def("hi", &BarType::hi)
;
}

Boost python getter/setter with the same name

I am wrapping C++ classes with boost-python and I am wondering is there is a better way to do it than what I am doing now.
The problem is that the classes have getters/setters that have the same name and there doesn't seem to be a painless way to wrap this with boost-python.
Here is a simplified version of the problem. Given this class:
#include <boost/python.hpp>
using namespace boost::python;
class Foo {
public:
double
x() const
{
return _x;
}
void
x(const double new_x)
{
_x = new_x;
}
private:
double _x;
};
I would like to do something like:
BOOST_PYTHON_MODULE(foo)
{
class_<Foo>("Foo", init<>())
.add_property("x", &Foo::x, &Foo::x)
;
}
This doesn't work because boost-python can't figure out which version of the function to use.
In fact, you can't even do
.def("x", &Foo::x)
for the same reason.
I was re-reading the tutorial at boost.org and the section on overloading seemed super promising. Unfortunately it doesn't seem to be what I'm looking for.
In the overloading section, it mentions a BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro that works like this:
if there were another member function in Foo that took defaulted arguments:
void z(int i=42)
{
std::cout << i << "\n";
}
you can then use the macro:
BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(z_member_overloads, z, 0, 1)
and then in the BOOST_PYTHON_MODULE:
.def("z", &Foo::z, z_member_overloads())
z_member_overloads lets you call def once and it will expose methods to python for both 0 arguments and 1 argument.
I was hoping that this would work for my x() and x(double val) getter/setter, but it doesn't work.
doing:
BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(x_member_overloads, x, 0, 1)
...
.def("x", &Foo::x, x_member_overloads())
doesn't compile:
error: no matching member function for call to 'def'
.def("x", &Foo::x, x_member_overloads())
~^~~
Question:
So, is there another macro or something that can make this work?
For completeness, this is how I'm currently handling cases like this:
.add_property(
"x",
make_function(
[](Foo& foo) {
return foo.x();
},
default_call_policies(),
boost::mpl::vector<double, Foo&>()
),
make_function(
[](Foo& foo, const double val) {
foo.x(val);
},
default_call_policies(),
boost::mpl::vector<void, Foo&, double>()
)
)

You can do this by casting to appropriate overload (untested):
class_<Foo>("Foo", init<>())
.add_property("x",
static_cast< double(Foo::*)() const >(&Foo::x), // getter
static_cast< void(Foo::*)(const double) >(&Foo::x)) // setter
;

Exposing virtual member functions from C++ to Python using boost::python

I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?

The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))