while trying to control an external device through USB COM with python, I encountered followings problem: The actual function, written in an DLL in Delphi and loaded into python using ctypes module look like this:
function DmxProOpen (cSerNo: pchar; var dHandle: DWORD): integer;
stdcall; external 'DmxPro.dll';
This require as argument a character pointer and a 32 bit int pointer. So in Python i wrote this:
SerNo = 'EN096445' #actual serial number
SerNo = ctypes.c_char_p
dLocalHandle = POINTER(c_uint32) #Local handle to pass onto the function
DLLIMPORT.DmxProOpen.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_uint32)]
DLLIMPORT.DmxProOpen(SerNo, dLocalHandle)
which generates following error:
argument 1: <class 'TypeError'>: wrong type
Even though i see the type match that of the requirements. So the question is: how can I pass the Serial number which was given as 'EN096445' onto the function when it requires a 8 bit pointer? I have tried to explicitly make the SerNo variable a real pointer by using pointer() with a variable that stores the characters but that also generate the error:
SERNO = ctypes.pointer(SerNo)
TypeError: _type_ must have storage info
it also did not work with the method byref(). I also want to ask if there is a way to declare that the argument of a from DLL imported functions is by reference with the method argtypes from ctypes. I really appreciate any help. Thanks in advance and have a really nice day!
To avoid a memory leak in the sutuation described below, I would like to call Py_DecRef directly from Python. Is there a way to do this? Is there a better solution for this problem?
I am using ctypes to interface my Python code to a C library for which I do not have the code. The C library was not written for Python, so it doesn't know anything about Python objects.
The C library uses two callbacks: the first creates an object and returns a void* pointer to it, and the second gets the pointer as parameter and is supposed to destroy it. In the C header files, the types of these callback functions are defined as follows:
typedef void* (*CreateCallback)();
typedef void (*DestroyCallback)(void*);
These callbacks could be defined in Python as shown below (simplified code). The current code has a memory leak as explained in the comments.
import ctypes
CreateCallback = ctypes.CFUNCTYPE(ctypes.py_object)
DestroyCallback = ctypes.CFUNCTYPE(None, ctypes.py_object)
class Object:
pass # In the real application, this contains more code
#CreateCallback
def create():
return Object()
# Ctypes correctly increments the reference count of the
# object, to make sure it does not get garbage collected
# while the C code holds a reference to it.
#DestroyCallback
def destroy(object):
pass
# Above, the reference count of the object should be
# decremented, because the C code no longer holds a
# reference to it. However, Ctypes does not know this so
# cannot do it automatically. How can I do this from
# Python? Is it possible to call Py_DecRef or similar
# directly from Python?
One option would be to create a C function that call Py_DecRef, compile that C function into a dll (or so for Linux), and call that from the destroy function above. That solution has at least two disadvantages:
It seems overly complex to create a dll just for one function
The C code would have to be compiled against a specific version of Python, instead of using whatever version of Python is running my Python code. Note that I need this to work on Windows, where a dll cannot contain undefined globals.
I am trying to wrap a function written in C such that I can access it in Python using SWIG
The problem I have is that it expects a pointer to a pointer to a struct, not a pointer to a struct, e.g.
int update_tracks(track_t **hash_tracks);
if it were:
int update_tracks(track_t *hash_tracks);
I would have no problem as I can create the argument and call the function from python as follows:
hash_tracks = track_t()
n = update_tracks(hash_tracks)
track_t is a simple C struct containing some ints, floats, arrays etc.
but I can't figure out how to get the pointer to hash_tracks that I need as the argument to the first function (i.e. the one I am actually trying to wrap)
The reason I need a track_t** argument (not just a track_t* argument) is that hash_tracks is a hash-table (using the uthash library) and as such the pointer to the track table can change as the function adds and removes track_t structs within its implementation.
I am stumped how to call such a function from python. Maybe I need to implement some 'helper' functions in C or use some SWIG typemaps to make it possible?
I think you want to use the INOUT typemap, documented here:
http://www.swig.org/Doc2.0/Arguments.html#Arguments_nn6
I am interfacing with a library that returns opaque pointers. Is it acceptable to subclass c_void_p to represent this in ctypes and provide for type checking for this particular flavor of c_void_p?
An easy way to do this type checking might be to create some arbitrary ctypes.Structure
class _Opaque(ctypes.Structure):
pass
Declare the return type of the relevant functions to be a pointer to this structure
lib.f.restype = ctypes.POINTER(_Opaque)
and either the argument type of a function which accepts this kind of pointer again:
lib.g.argtypes = [ctypes.POINTER(_Opaque)]
Now, ctypes ensures that the parameter to g is a pointer that was returned by f before. (Note that I used a leading _ to mark _Opaque for uses in this module only.)
I have a C++ class that requires a function pointer in it's constructor (float(*myfunction)(vector<float>*))
I've already exposed some function pointers to Python.
The ideal way to use this class is something like this:
import mymodule
mymodule.some_class(mymodule.some_function)
So I tell Boost about this class like so:
class_<SomeClass>("some_class", init<float(*)(vector<float>*)>);
But I get:
error: no matching function for call to 'register_shared_ptr1(Sample (*)(std::vector<double, std::allocator<double> >*))'
when I try to compile it.
So, does anyone have any ideas on how I can fix the error without losing the flexibility gained from function pointers (ie no falling back to strings that indicate which function to call)?
Also, the main point of writing this code in C++ is for speed. So it would be nice if I was still able to keep that benefit (the function pointer gets assigned to a member variable during initialization and will get called over a million times later on).
OK, so this is a fairly difficult question to answer in general. The root cause of your problem is that there really is no python type which is exactly equivalent to a C function pointer. Python functions are sort-of close, but their interface doesn't match for a few reasons.
Firstly, I want to mention the technique for wrapping a constructor from here:
http://wiki.python.org/moin/boost.python/HowTo#namedconstructors.2BAC8factories.28asPythoninitializers.29. This lets you write an __init__ function for your object that doesn't directly correspond to an actual C++ constructor. Note also, that you might have to specify boost::python::no_init in the boost::python::class_ construction, and then def a real __init__ function later, if your object isn't default-constructible.
Back to the question:
Is there only a small set of functions that you'll usually want to pass in? In that case, you could just declare a special enum (or specialized class), make an overload of your constructor that accepts the enum, and use that to look up the real function pointer. You can't directly call the functions yourself from python using this approach, but it's not that bad, and the performance will be the same as using real function pointers.
If you want to provide a general approach that will work for any python callable, things get more complex. You'll have to add a constructor to your C++ object that accepts a general functor, e.g. using boost::function or std::tr1::function. You could replace the existing constructor if you wanted, because function pointers will convert to this type correctly.
So, assuming you've added a boost::function constructor to SomeClass, you should add these functions to your python wrapping code:
struct WrapPythonCallable
{
typedef float * result_type;
explicit WrapPythonCallable(const boost::python::object & wrapped)
: wrapped_(wrapped)
{ }
float * operator()(vector<float>* arg) const
{
//Do whatever you need to do to convert into a
//boost::python::object here
boost::python::object arg_as_python_object = /* ... */;
//Call out to python with the object - note that wrapped_
//is callable using an operator() overload, and returns
//a boost::python::object.
//Also, the call can throw boost::python::error_already_set -
//you might want to handle that here.
boost::python::object result_object = wrapped_(arg_as_python_object);
//Do whatever you need to do to extract a float * from result_object,
//maybe using boost::python::extract
float * result = /* ... */;
return result;
}
boost::python::object wrapped_;
};
//This function is the "constructor wrapper" that you'll add to SomeClass.
//Change the return type to match the holder type for SomeClass, like if it's
//held using a shared_ptr.
std::auto_ptr<SomeClass> CreateSomeClassFromPython(
const boost::python::object & callable)
{
return std::auto_ptr<SomeClass>(
new SomeClass(WrapPythonCallable(callable)));
}
//Later, when telling Boost.Python about SomeClass:
class_<SomeClass>("some_class", no_init)
.def("__init__", make_constructor(&CreateSomeClassFromPython));
I've left out details on how to convert pointers to and from python - that's obviously something that you'll have to work out, because there are object lifetime issues there.
If you need to call the function pointers that you'll pass in to this function from Python, then you'll need to def these functions using Boost.Python at some point. This second approach will work fine with these def'd functions, but calling them will be slow, because objects will be unnecessarily converted to and from Python every time they're called.
To fix this, you can modify CreateSomeClassFromPython to recognize known or common function objects, and replace them with their real function pointers. You can compare python objects' identity in C++ using object1.ptr() == object2.ptr(), equivalent to id(object1) == id(object2) in python.
Finally, you can of course combine the general approach with the enum approach. Be aware when doing this, that boost::python's overloading rules are different from C++'s, and this can bite you when dealing with functions like CreateSomeClassFromPython. Boost.Python tests functions in the order that they are def'd to see if the runtime arguments can be converted to the C++ argument types. So, CreateSomeClassFromPython will prevent single-argument constructors def'd later than it from being used, because its argument matches any python object. Be sure to put it after other single-argument __init__ functions.
If you find yourself doing this sort of thing a lot, then you might want to look at the general boost::function wrapping technique (mentioned on the same page with the named constructor technique): http://wiki.python.org/moin/boost.python/HowTo?action=AttachFile&do=view&target=py_boost_function.hpp.