I have a python extension module written in C++, which contains multiple functions. One of these generates an instance of a custom structure, which I then want to use with other functions of my module in Python as follows
import MyModule
var = MyModule.genFunc()
MyModule.readFunc(var)
To do this, I've tried using PyCapsule objects to pass a pointer to these objects between Python and C, but this produces errors when attempting to read them in the second C function ("PyCapsule_GetPointer called with invalid PyCapsule object"). Python, however, if asked to print the PyCapsule object (var) correctly identifies it as a "'capsule object "testcapsule"'. My C code appears as follows:
struct MyStruct {
int value;
};
static PyObject* genFunc(PyObject* self, PyObject *args) {
MyStruct var;
PyObject *capsuleTest;
var.value = 1;
capsuleTest = PyCapsule_New(&var, "testcapsule", NULL);
return capsuleTest;
}
static PyObject* readFunc(PyObject* self, PyObject *args) {
PyCapsule_GetPointer(args, "testcapsule");
return 0;
}
Thank you for your help.
Like stated in a comment to your question, you'll run into an issue when reading data from the local variable MyStruct var. For this you can use the third destructor to PyCapsule_New.
But that's not the reason for your problem just now. You're using PyCapsule_GetPointer(args, "testcapsule") on the args parameter. And since it's not a capsule, even though var is one, you might have defined the signature of the function as METH_VARARGS. Instead you need to unpack the tuple or use METH_O.
Related
I want to create a function in python, pass it's function pointer to c and execute it there.
So my python file:
import ctypes
import example
def tester_print():
print("Hello")
my_function_ptr = ctypes.CFUNCTYPE(None)(tester_print)
example.pass_func(my_function_ptr)
And here is what my function in c looks like:
typedef void (*MyFunctionType)(void);
PyObject* pass_func(PyObject *self, PyObject* args)
{
PyObject* callable_object;
if (!PyArg_ParseTuple(args, "O", &callable_object))
return NULL;
if (!PyCallable_Check(callable_object))
{
PyErr_SetString(PyExc_TypeError, "The object is not a callable function.");
return NULL;
}
PyObject* function_pointer = PyCapsule_New(callable_object, "my_function_capsule", NULL);
if (function_pointer == NULL) return NULL;
MyFunctionType my_function = (MyFunctionType) PyCapsule_GetPointer(function_pointer, "my_function_capsule");
if (my_function == NULL) return NULL;
my_function(); // Or (*my_function)() Both same result.
// PyCapsule_Free(function_pointer);
Py_RETURN_NONE;
}
Doing this causes a seg fault on my_function() call. How can I do this?
If you're just trying to pass a Python function to a C extension, pass it directly (don't use ctypes) and use PyObject_Call to call it:
example.pass_func(tester_print)
and
PyObject_CallNoArgs(callable_object);
If you need a real C function pointer for whatever reason, the usual approach is to write a C wrapper that takes the callable as an argument:
void callable_wrapper(PyObject *func) {
PyObject_CallNoArgs(func);
// plus whatever other code you need (e.g. reference counting, return value handling)
}
Most reasonable C APIs that take a callback function also provide a way to add an arbitrary argument to the callable ("user data"); for example, with pthreads:
result = pthread_create(&tid, &attr, callable_wrapper, callable_object);
Make sure to handle reference counting correctly: increment the reference on your callable object before passing it to the C API, and decrement the reference when it is no longer needed (e.g. if the callback is only called once, the callable_wrapper could DECREF before returning).
When using threads, you additionally need to ensure that you hold the GIL when calling any Python code; see https://docs.python.org/3/c-api/init.html#non-python-created-threads for more details and a code sample.
What your current code is doing is receiving a pointer to a ctypes CFUNCTYPE object as callable_object, placing that pointer in a capsule, taking it back out again, and calling it as if it was a C function pointer. This doesn't work, since it effectively attempts to call the CFUNCTYPE object as if it were a C function (the capsule stuff winds up being useless). When you're using the Python C API, there's almost never any need for ctypes in Python, because the C API can directly interact with Python objects.
I'm embedding Python into my C++ and creating PyObjects to represent my data/objects (ints, doubles, strings, etcetera).
I've put in several hours trying to find the answer to the above question, I expected there'd be a "name" property or "name()" method to set, and reference, the canonical object name used in Python script (as global/local objects), that there'd be a function:
PyObject *PyObject_ByName(PyObject *PyObjs, char* name)
Return value: New reference. Part of the Stable ABI.
Return a new PyObject reference from an array of PyObjs that match the 'name', or NULL on failure.
What am I missing? I see all the other pieces in place.
MSEXPORT PyObject* PyVariant(VarObj* data, tLvVarErr* error) {
PyObject* module_name, * module, * dict, * python_class, * object;
PyObject *pValue = NULL; // <- what I want to set name
switch (data->data.index())
{
case VarIdx::I32:
pValue = PyLong_FromLong((long) get<int32_t>(data->data));
break;
case VarIdx::DBL:
pValue = PyFloat_FromDouble ((double) get<double>(data->data));
break;
case VarIdx::Str :
pValue = PyUnicode_FromString((char*) get<string*>(data->data)->c_str());
break;
default:
return NULL;
}
return pValue;
}
What you need to know is that Python namespaces are implemented based on dictionaries (the exception is the local namespace inside the function). Global name lookup is just to obtain the global namespace dictionary and search through the string key. Therefore, what you need to do is:
Use PyEval_GetGlobals to get the global namespace.
Use PyUnicode_FromString to construct a string object through a C string.
Use PyDict_GetItem to get object from the global namespace.
As for the _PyDict_LoadGlobal I mentioned in the comment area, it is an API used internally by Python to quickly load objects (starting with an underscore of function names). You should avoid using it when writing C extensions.
I have an external class that is already wrapped (I mean, it is directly accessible by python with no further effort), and now I want it to be part of a larger cython module (in other words, embed it).
I could have explicitly python-import it. But the things is that the external class is already used in an extern function in the cython module (so that the class is eventually #included in source). Python import require the module to be compiled, then the two module may have two different copy of the same class...
How should I then use an external already wrapped class in cython?
(Possibly oversimplified) example:
Foo.cpp:
#include "Python.h"
#include "foo.hpp"
struct Foo_wrapper {
PyObject_HEAD
foo bar;
};
static int Foo_init(Foo_wrapper* self, PyObject* args, PyObject*) {
....
}
static PyTypeObject FooType {...};
Spam.pyx:
cdef extern from "some_functions.hpp":
cdef some_function1(some_type); // this returns a wrapped foo object
def spam(arg1, arg2, arg3):
// wrap arg1, arg2, arg3 to some_type
return some_function1(an_instance_of_some_type); // huh? but foo isn't available here!
And I want to use the foo class in spam.pyx.
This should be fine (almost) as it is. This cdef extern line isn't quite right:
cdef extern from "some_functions.hpp":
object some_function1(some_type); // this returns a wrapped foo object
Note the change to object - this tells Cython that the function returns a Python object. The C/C++ declaration would look like:
PyObject* some_function1(some_type);
// or
Foo_wrapper* some_function1(some_type);
Either will work.
The reason that the Cython code will be fine with it is that the PyObject_HEAD includes a pointer ob_type that points to the PyTypeObject FooType. This is set up when the object is created. The PyTypeObject contains all the details the Python interpreter needs to use the returned object and so everything should work fine.
The whole thing is basically the equivalent of the Python code:
# in "somemodule.py"
def a_function():
import something_else
return something_else.Class()
The Python interpreter can use the returned value, despite the Class not being known in the "global" namespace.
The one thing to be careful of is that you should ensure that the Foo module initialization function has been called at least once before creating a Foo_wrapper. The reason being is that this function usually does some things like calling PyType_Ready(&FooType) needed to make sure that FooType is properly set up. An easy way to do this would be to add the following lines to some_function1:
PyObject* m = PyImport_ImportModule("Foo");
if (m==NULL) return NULL; // an error
Py_CLEAR(m); // don't need to keep a reference to it
There are other ways of doing the same thing though.
How can I pass a pointer to a C function with Python's CFFI?
For example, if the library I'm wrapping has two functions:
void some_function(void (*callback)()) {
callback();
}
void some_callback() {
printf("callback!\n");
}
How can I call some_function passing in some_callback? For example, something like:
from mylib._ffi import lib
lib.some_function(lib.some_callback)
I know that I can use ffi.callback(…) to wrap a Python function in a callback, but I'm wondering if it's possible to avoid duplicating the type signature and whatnot of the C function.
With a recent version of cffi, in out-of-line mode, you can do:
lib.some_function(ffi.addressof(lib, "some_callback"))
Or, and this works with older cffi's too, you need to tweak the cdef to include "some_callback" as a constant function pointer instead of as a function:
ffi.cdef("void (*const some_callback)();")
lib.some_function(lib.some_callback)
If this seems too magical, then the more verbose but clearer solution would be:
ffi.cdef("""
typedef void (*my_callback_t)();
my_callback_t get_some_callback(void);
""")
ffi.set_source("example", # <- or, ffi.verify(
"""
// declare get_some_callback() explicitly here:
typedef void (*my_callback_t)();
static my_callback_t get_some_callback(void) {
return &some_callback;
}
""")
some_callback = lib.get_some_callback()
lib.some_function(some_callback)
I am trying to embed python within an exiting C++ application. However I am stuck at trying to call an instance member function from the python script.
I have an existing class in which I have a private member function that wraps the native function and meets the PyCFunction interface.
class MY_CLASS {
public:
void MY_CLASS();
void submit(TypeA a, int b);
private:
PyObject* PythonMethod_submit(PyObject*, PyObject*);
}
When I create the PyMethodDef I use the method name to identify the function I want however I get a compliation error indicating I have to the wrong type signature.
MY_CLASS::MY_CLASS() {
// Python C/API Intilization stuff
PyMethodDef ModuleMethods[] = {
{ "submit",
PythonMethod_submit,
METH_VARARGS,
"docstring" }
};
Py_InitModule("myclass", ModuleMethods);
// Further interaction with the embeded interpreter
}
If I use a static function as described in the Python C API this process works, when I use the class member I recieve an error asserting PyObject* (MY_CLASS::)(PyObject*, PyObject*) does not match PyObject* (*)(PyObject*, PyObject*).
Is it possible to pass a member function as function pointer as stated in the error?
No. A non-static member function has a MY_CLASS * implicitly passed in (as this), whereas a static member function, which behaves like a C function, does not. See this page for the different type signatures: http://www.parashift.com/c++-faq/fnptr-vs-memfnptr-types.html
What you can do as an alternative is have a member in your Python object that holds a pointer (MY_CLASS *) to your C++ object.