I have a couple of header files that are already defined in C (C++ if we're being technical, but the header is C compatible) that I use to define a bunch of data types (structs). I would like to make use of these in a python script that I am going use to test the corresponding C++ application. This is mostly to avoid having to redefine them in python as some of the structs are unwieldy, but also it would be nice to have them defined in one place so if changes happen down the road it will be easier to adapt.
When I started looking into this I thought that this was certainly doable but none of the examples I have come across get me quite there. The closest I got was using cffi. I got a simple example working how I want it to:
Data types header:
// mylib.h
struct Point2D
{
float x;
float y;
};
struct Point3D
{
float x;
float y;
float z;
};
Python code:
from cffi import FFI
with open("./mylib.h", "r") as fo:
header_text = fo.read()
ffi = FFI()
ffi.cdef(header_text)
point = ffi.new("struct Point2D*")
But this fails if I have #includes or #ifdefs in the header file, per the cffi documentation:
The declarations can contain types, functions, constants and global
variables. What you pass to the cdef() must not contain more than
that; in particular, #ifdef or #include directives are not supported.
Are there any tricks I can do to make this work?
You cannot directly access C structs in Python. You will need to 'bind' C functions to Python functions. This only allows you to access C functions from Python - not a C struct.
Testing C++ is generally done using Google Test. If you require using Python to test C++ functionality then you will need to create bindings in Python to access the C++ functions (as C functions using extern "C").
You can only bind to a C/C++ library. Google "Call C functions in Python" for more.
https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntqueryinformationfile?redirectedfrom=MSDN
How can I call the above kernel method in python? I found an example on another stackoverflow post: Winapi: Get the process which has specific handle of a file
The answer on this other post is essentially what I want to do, but in python. The goal is to be able to get a list of processes which currently are accessing/locking a file. This NtQueryInformationFile method seems to be exactly what I want. I know this can be done with ctypes, but I am not familiar or comfortable enough with ctypes to do this myself. How can I do this?
If there's no available wrapper for the function, you'll need to call the function yourself using ctypes.
The dlls windows uses are exposed through ctypes.windll, with cytpes.windll.ntdll being the one that exposes the function you need.
To help python convert arguments, it's usually a good idea to specify the function's argument and return types, which can be done through the argtypes and restype attributes on the function object, like so:
function = cytpes.windll.ntdll.NtQueryInformationFile
function.argtypes = [ctypes.wintypes.HANDLE, ...]
function.restype = ctypes.c_long
ctypes exposes the common window types in the ctypes.wintypes module, though for most structures like the PIO_STATUS_BLOCK in your function you'll need to define the struct yourself and add it to the argument list to use it properly. In case it's optional a void pointer and passing it None will suffice.
Also, do mind that windows handles are not the file descriptors that python exposes, to convert to/from them you can use the ..._osfhandle functions from the msvcrt module
I'm using python for my c++ program's sub script language.
I bind my c++ class to python to use it in python side.
This system must can share(pass) c++ instance to python.
And even Python (script side) can create new instance also it must be share by c++ side too.
But python(or..java, c#)'s class type is reference type.
#MyClass class has class type member test
data1 = MyClass()
data2 = MyClass()
#so data1's test will point data2.test object
data1.test = data2.test
So it will be work like a c++'s pointer.
But in c++, class is not reference type.
So assignment operation will call = operator.
if =operator will not overloaded then it will work value copy.
ClassInstance1 = ClassInstance2
Well If you create some objects in python, and share (pass) to c++ side.
like...
Character* pChara = PythonSideObjectToCpp("data1");
The data work rule is different between c++ and python.
So i think c++ can not 1:1 match with pythonlike above code.
It will need more interface about get, set or control python's reference type member.
How do you think about this?
I don't think that python instance can extract to c++ purely.
Well... but it can extract python instance's clone object.
I am attempting to write a .so library wrapper for an existing C source code project, and then call the functions in the .so library from Python. I have been able to call functions with primitive arguments and return types with no problem, so I am now working on interfacing with more complex functions that have arguments that are pointers to structures.
My problem is in creating the structures on the Python side so that I can call the C-library functions. Some of the structs in the .so library have hundreds of fields, so I was hoping there was an easier alternative to spelling out all the fields and types in a Python ctypes Structure object.
I would like to be able to write something like this is Python:
from ctypes import *
lib = cdll.LoadLibrary("./libexample.so")
class Input(Structure):
_fields_ = lib.example_struct._fields ## where `example_struct` is defined in the .so library
## I have no idea if you can actually get the fields of the struct!!
my_input = Input(a,b,c,...) ## pseudo-code
my_ptr = pointer(my_input) ## wrap the input with a pointer
result = lib.my_lib_func(my_ptr) ## call .so function with struct
This would allow me to easily replicate at least the structure definitions of the large C structs without having to create and maintain lengthy Python versions of the struct definitions. Is this possible? Or is there another way to achieve the same effect?
EDIT: The C source code is third party, so for now, I am looking for an approach where I don't have to modify the C source.
The Cython approach is to read and interpret the .h header file.
But I do not say it would be easy.
I'm just getting started with ctypes and would like to use a C++ class that I have exported in a dll file from within python using ctypes.
So lets say my C++ code looks something like this:
class MyClass {
public:
int test();
...
I would know create a .dll file that contains this class and then load the .dll file in python using ctypes.
Now how would I create an Object of type MyClass and call its test function? Is that even possible with ctypes? Alternatively I would consider using SWIG or Boost.Python but ctypes seems like the easiest option for small projects.
Besides Boost.Python(which is probably a more friendly solution for larger projects that require one-to-one mapping of C++ classes to python classes), you could provide on the C++ side a C interface. It's one solution of many so it has its own trade offs, but I will present it for the benefit of those who aren't familiar with the technique. For full disclosure, with this approach one wouldn't be interfacing C++ to python, but C++ to C to Python. Below I included an example that meets your requirements to show you the general idea of the extern "c" facility of C++ compilers.
//YourFile.cpp (compiled into a .dll or .so file)
#include <new> //For std::nothrow
//Either include a header defining your class, or define it here.
extern "C" //Tells the compile to use C-linkage for the next scope.
{
//Note: The interface this linkage region needs to use C only.
void * CreateInstanceOfClass( void )
{
// Note: Inside the function body, I can use C++.
return new(std::nothrow) MyClass;
}
//Thanks Chris.
void DeleteInstanceOfClass (void *ptr)
{
delete(std::nothrow) ptr;
}
int CallMemberTest(void *ptr)
{
// Note: A downside here is the lack of type safety.
// You could always internally(in the C++ library) save a reference to all
// pointers created of type MyClass and verify it is an element in that
//structure.
//
// Per comments with Andre, we should avoid throwing exceptions.
try
{
MyClass * ref = reinterpret_cast<MyClass *>(ptr);
return ref->Test();
}
catch(...)
{
return -1; //assuming -1 is an error condition.
}
}
} //End C linkage scope.
You can compile this code with
gcc -shared -o test.so test.cpp
#creates test.so in your current working directory.
In your python code you could do something like this (interactive prompt from 2.7 shown):
>>> from ctypes import cdll
>>> stdc=cdll.LoadLibrary("libc.so.6") # or similar to load c library
>>> stdcpp=cdll.LoadLibrary("libstdc++.so.6") # or similar to load c++ library
>>> myLib=cdll.LoadLibrary("/path/to/test.so")
>>> spam = myLib.CreateInstanceOfClass()
>>> spam
[outputs the pointer address of the element]
>>> value=CallMemberTest(spam)
[does whatever Test does to the spam reference of the object]
I'm sure Boost.Python does something similar under the hood, but perhaps understanding the lower levels concepts is helpful. I would be more excited about this method if you were attempting to access functionality of a C++ library and a one-to-one mapping was not required.
For more information on C/C++ interaction check out this page from Sun: http://dsc.sun.com/solaris/articles/mixing.html#cpp_from_c
The short story is that there is no standard binary interface for C++ in the way that there is for C. Different compilers output different binaries for the same C++ dynamic libraries, due to name mangling and different ways to handle the stack between library function calls.
So, unfortunately, there really isn't a portable way to access C++ libraries in general. But, for one compiler at a time, it's no problem.
This blog post also has a short overview of why this currently won't work. Maybe after C++0x comes out, we'll have a standard ABI for C++? Until then, you're probably not going to have any way to access C++ classes through Python's ctypes.
The answer by AudaAero is very good but not complete (at least for me).
On my system (Debian Stretch x64 with GCC and G++ 6.3.0, Python 3.5.3) I have segfaults as soon has I call a member function that access a member value of the class.
I diagnosticated by printing pointer values to stdout that the void* pointer coded on 64 bits in wrappers is being represented on 32 bits in Python. Thus big problems occurs when it is passed back to a member function wrapper.
The solution I found is to change:
spam = myLib.CreateInstanceOfClass()
Into
Class_ctor_wrapper = myLib.CreateInstanceOfClass
Class_ctor_wrapper.restype = c_void_p
spam = c_void_p(Class_ctor_wrapper())
So two things were missing: setting the return type to c_void_p (the default is int) and then creating a c_void_p object (not just an integer).
I wish I could have written a comment but I still lack 27 rep points.
Extending AudaAero's and Gabriel Devillers answer I would complete the class object instance creation by:
stdc=c_void_p(cdll.LoadLibrary("libc.so.6"))
using ctypes c_void_p data type ensures the proper representation of the class object pointer within python.
Also make sure that the dll's memory management be handled by the dll (allocated memory in the dll should be deallocated also in the dll, and not in python)!
I ran into the same problem. From trial and error and some internet research (not necessarily from knowing the g++ compiler or C++ very well), I came across this particular solution that seems to be working quite well for me.
//model.hpp
class Model{
public:
static Model* CreateModel(char* model_name) asm("CreateModel"); // static method, creates an instance of the class
double GetValue(uint32_t index) asm("GetValue"); // object method
}
#model.py
from ctypes import ...
if __name__ == '__main__':
# load dll as model_dll
# Static Method Signature
fCreateModel = getattr(model_dll, 'CreateModel') # or model_dll.CreateModel
fCreateModel.argtypes = [c_char_p]
fCreateModel.restype = c_void_p
# Object Method Signature
fGetValue = getattr(model_dll, 'GetValue') # or model_dll.GetValue
fGetValue.argtypes = [c_void_p, c_uint32] # Notice two Params
fGetValue.restype = c_double
# Calling the Methods
obj_ptr = fCreateModel(c_char_p(b"new_model"))
val = fGetValue(obj_ptr, c_int32(0)) # pass in obj_ptr as first param of obj method
>>> nm -Dg libmodel.so
U cbrt#GLIBC_2.2.5
U close#GLIBC_2.2.5
00000000000033a0 T CreateModel # <----- Static Method
U __cxa_atexit#GLIBC_2.2.5
w __cxa_finalize#GLIBC_2.2.5
U fprintf#GLIBC_2.2.5
0000000000002b40 T GetValue # <----- Object Method
w __gmon_start__
...
...
... # Mangled Symbol Names Below
0000000000002430 T _ZN12SHMEMWrapper4HashEPKc
0000000000006120 B _ZN12SHMEMWrapper8info_mapE
00000000000033f0 T _ZN5Model12DestroyModelEPKc
0000000000002b20 T _ZN5Model14GetLinearIndexElll
First, I was able to avoid the extern "C" directive completely by instead using the asm keyword which, to my knowledge, asks the compiler to use a given name instead of the generated one when exporting the function to the shared object lib's symbol table. This allowed me to avoid the weird symbol names that the C++ compiler generates automatically. They look something like the _ZN1... pattern you see above. Then in a program using Python ctypes, I was able to access the class functions directly using the custom name I gave them. The program looks like fhandle = mydll.myfunc or fhandler = getattr(mydll, 'myfunc') instead of fhandle = getattr(mydll, '_ZN12...myfunc...'). Of course, you could just use the long name; it would make no difference, but I figure the shorter name is a little cleaner and doesn't require using nm to read the symbol table and extract the names in the first place.
Second, in the spirit of Python's style of object oriented programming, I decided to try passing in my class' object pointer as the first argument of the class object method, just like when we pass self in as the first method in Python object methods. To my surprise, it worked! See the Python section above. Apparently, if you set the first argument in the fhandle.argtypes argument to c_void_ptr and pass in the ptr you get from your class' static factory method, the program should execute cleanly. Class static methods seem to work as one would expect like in Python; just use the original function signature.
I'm using g++ 12.1.1, python 3.10.5 on Arch Linux. I hope this helps someone.