Develop Reference

Pybind11 and inconsistent types deduced for lambda

In cpp code that I can't modify, we have custom class and custom pointers.
This class reads an object and depending on kwargs might also write it.
I need to make pybind11 bindings for it.
Depending on the arguments passed, we get const or non-const pointer to the class.
Pybind complaints about inconsistent types for lambda return type.
It all goes well for making init() function that works only for const or non-const objects. Unfortunately, need to support both cases.
What would be the best way to make python binding to cpp code in this case?
Error
In lambda function:
error: inconsistent types 'std::shared_ptr<myClass>' and 'std::shared_ptr<const myClass>'
deduced for lambda return type
return *const_p;
How could the code below be modified to support both cases?
Using Cpp 11 compiler.
// those defined elsewhere
typedef std::shared_ptr< myClass > my_ptr;
typedef std::shared_ptr< const myClass > const_my_ptr;
//init function for pybind11
.def(py::init([](py::kwargs kwargs) {
bool object_writable = py::bool_(kwargs["rw"]);
int cache = py::bool_(kwargs["cache"]);
std::string path = py::str(kwargs["path"]);
if (object_writable){
//returns non const
my_ptr p = myClass::read_write(path)
return *p;
}
else{
//returns const
const_my_ptr const_p = myClass::read(path, cache)
return *const_p;
}
}))

Exposing virtual member functions from C++ to Python using boost::python

I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?

The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))

How do I pass a pointer to a C function with Python's CFFI?

How can I pass a pointer to a C function with Python's CFFI?
For example, if the library I'm wrapping has two functions:
void some_function(void (*callback)()) {
callback();
}
void some_callback() {
printf("callback!\n");
}
How can I call some_function passing in some_callback? For example, something like:
from mylib._ffi import lib
lib.some_function(lib.some_callback)
I know that I can use ffi.callback(…) to wrap a Python function in a callback, but I'm wondering if it's possible to avoid duplicating the type signature and whatnot of the C function.

With a recent version of cffi, in out-of-line mode, you can do:
lib.some_function(ffi.addressof(lib, "some_callback"))
Or, and this works with older cffi's too, you need to tweak the cdef to include "some_callback" as a constant function pointer instead of as a function:
ffi.cdef("void (*const some_callback)();")
lib.some_function(lib.some_callback)
If this seems too magical, then the more verbose but clearer solution would be:
ffi.cdef("""
typedef void (*my_callback_t)();
my_callback_t get_some_callback(void);
""")
ffi.set_source("example", # <- or, ffi.verify(
"""
// declare get_some_callback() explicitly here:
typedef void (*my_callback_t)();
static my_callback_t get_some_callback(void) {
return &some_callback;
}
""")
some_callback = lib.get_some_callback()
lib.some_function(some_callback)

finding out how many arguments a PyObject method needs

We can extract a PyObject pointing to a python method using
PyObject *method = PyDict_GetItemString(methodsDictionary,methodName.c_str());
I want to know how many arguments the method takes. So if the function is
def f(x,y):
return x+y
how do I find out it needs 2 arguments?

Followed through the link provided by Jon. Assuming you don't want to (or can't) use Boost in your application, the following should get you the number (easily adapted from How to find the number of parameters to a Python function from C?):
PyObject *key, *value;
int pos = 0;
while(PyDict_Next(methodsDictionary, &pos, &key, &value)) {
if(PyCallable_Check(value)) {
PyObject* fc = PyObject_GetAttrString(value, "func_code");
if(fc) {
PyObject* ac = PyObject_GetAttrString(fc, "co_argcount");
if(ac) {
const int count = PyInt_AsLong(ac);
// we now have the argument count, do something with this function
Py_DECREF(ac);
}
Py_DECREF(fc);
}
}
}
If you're in Python 2.x the above definitely works. In Python 3.0+, you seem to need to use "__code__" instead of "func_code" in the snippet above.
I appreciate the inability to use Boost (my company won't allow it for the projects I've been working on lately), but in general, I would try to knuckle down and use that if you can, as I've found that the Python C API can in general get a bit fiddly as you try to do complicated stuff like this.

SWIG: python callback with function template and shared pointer as return value

I'm creating wrappers using SWIG for python for my software library and I have the following function:
template<class PR>
boost::shared_ptr<JobT<PR> > Client::WaitForJob() {
boost::shared_ptr<JobT<PR> > job;
while (!job.get()) {
list<boost::shared_ptr<Job> > jobs = GetJobs(p_jobName, p_jobID, "", JobT<PR>::New);
while (jobs.size() > 0) {
job = boost::dynamic_pointer_cast<JobT<PR> >(jobs.front());
jobs.pop_front();
if (ClaimJob(job)) return job;
else job.reset();
}
}
return job;
}
In this case I'm able to create a class for the JobT and I also used the boos_pointer.
Currently I have the following in my interface file:
%template(jobme) JobT;
%include "boost_shared_ptr.i"
%shared_ptr(jobme)
%template(waitforme) Client::WaitForJob;
The waitforme function template return a pointer to a swig python object. This is correct but i want it to be the same as a jobme pointer.
Can someone point out if this is possible ??

The Swig object you get back is a proxy for the jobme. It should work just like a JobT wherever you need it.
Ah, I think I see the problem. You need to instantiate the JobT and Client::WaitForJob for each PR you have:
%include "boost_shared_ptr.i"
%shared_ptr(JobT<Foo>);
#include <JobT.hpp>
%template(Job##NAME) JobT<Foo>;
%template Client::WaitForJob<Foo>;
So %template instantiates a template, but it doesn't guess all the possible types you might use to instantiate it with.