Python support for Qt dll - python

I have my own C++ library project(with source) written in Qt and it uses QTcpsocket, QUdpSocket, QSerialPort signals and slots.
I would like to support this library in Python as well.
What is the preferred way to do this?
Writing a wrapper in Python, if so does it have obstacles?
Dont know if PyQt is just for this purpose?
Or do you thnink is it better to rewrite the lib in Python by just implementing the logic used in C++ library project.
As this is library is part of a SDK, same applies for supporting QT dll with .NET as well in fact, as a second step after supporting Python.
Example API of Qt.
quint16 SendCommandAsync(CmdBaseSpv1* pcommand,
ConnectionArg connectionArg,
emitLogOptions::emitLogOption logOption,
LogInfo &txLogInfo,
LogInfo &rxLogInfo);
I want to call this function from Python.
Function parameters CmdBaseSpv1, ConnectionArg, emitLogOption, LogInfo are all Qt classes.
Some of these arguments are using the QOBJECT base class.
As you see from the function name; it is an Asynchronous function call. Result will emit a signal so I need to get async result as well.

I'll write down what I know about wrapping C++ libraries and will try to source it, but as a huge disclaimer, I have only used this for something very, very simple myself.
You ask about rewriting the library in Python. I would say this depends. If the code is trivial, then I don't see why not. If it is larger and has to be kept up-to-date with other code (as you imply with .Net), I wouldn't. It makes sense to reuse the same code for both.
 My suggestion
From what I see of your code I would try to wrap it using boost::python or SWIG.
How to wrap
The main trouble is going to be to create CmdBaseSpv1, ConnectionArg, etc. in Python.
If you don't need any Qt-classes to instantiate your classes, this should be straightforward. However, in case you need the Qt types inside of Python (e.g. because the constructor of CmdBaseSpv1 requires a QString), your task is a lot more complicated because you need a way to convert a Python-string into a QString. If you can, you should only use stl-types.
Everything in Python
The simplest way to wrap a small C library is to use the cffi module (or ctypes). You can write the full binding in Python. However, this is a lot of manual work if your API is large and can get difficult.
There is another problem: ctypes is only compatible with C, not C++. So you'd need to change your interface to be compatible with C, internally you could still use C++ and Qt.
Wrap by hand
An alternative is to wrap the library calls yourself. You can either do this by using the Python API. There are also a few libraries that help you create the bindings. Boost::python seems especially promising and works with C++.
Binding generators
If your API is very large, you should use a binding generator which parses the C++ code and generates the bindings itself. For example sip is one of them. It is used to create the bindings for the whole Qt library. There are a few binding generators out there, one mentioned in the Python docs is SWIG. PySide uses Shiboken and also has a nice description of it on their website.
SWIG has the additional advantage, that you can create bindings for multiple languages, including C#.
PyQt
PyQt is a binding generated from Qt using sip. You'll probably not need it, unless you need to access the full power of Qt from inside Python. If this is the case, consider using sipfor generating the bindings, so things like the signal-slot mechanism are compatible between your library and PyQt.
Challenges with bindings
Bindings come with a few challenges because Python and C++ are different in some key areas.
Memory-management
Memory management in Python is almost automatic, in C++ you're required to do it manually. For example
def myfunc():
mywidget = QWidget()
at the end of myfunc() mywidget gets garbage collected. In C++ however
void myfunc() {
auto mywidget = new QWidget();
}
mywidget is still around. This means that even when inside Python, you need to take care of the C++ memory management. The problems I've seen are memory leaks and dangling pointers. Watch out for this when using callbacks, you don't want Python to garbage collect the callback while C++ thinks it's still alive.
Exceptions
Not all programming languages have exceptions or deal with them the same way. For example, it would be nice if an exception inside C++ can be caught inside Python.
Links to related question
How to wrap a c++ library for python? (example of boost::python)
Exposing a C++ API to Python (discussion about boost::python, SWIG and more)
https://stackoverflow.com/a/5686873 (discusses Cython, another choice)

Related

Difference between pybind and process.call

I have a python code and I would like to call to a function from a Library. I see two ways to achieve it
pybind
This means writing a wrapper around the library, making it a module, install it and calling it directly from python.
This is the more native approach, but adding interfaces is a pain.
process.call
Assuming I can send/receive data through ros, the Library can ba called through process.call and the results received through ros.
This is easier when I would like to add interface or make changes.
For a process which is not realtime (well, this is python...), Is there's any major disadvantage to use process.call?

Creating a C++ Qt Gui for a Python logic

I was presented with a Python logic for which I need to create a GUI. I want to use Qt for that purpose and ideally I would like to program it in C++, without using the Qt Creator.
What are recommended ways of combining a C++ Qt GUI and a Python logic, perhaps with a controlling layer which is either in Python or C++?
Due to prior research, I'm aware of certain tools like PySide, PythonQt, SWIG, Shiboken and others (although I haven't used them so far) and that it is very simple to import *.ui files into Python, but the number of possible ways is a bit overwhelming and I could not come up with a "good solution" so far.
Ideally, I would not use Qt Designer, but create the GUI (windows, custom widgets, helper classes etc) by hand, combine them into one or few classes which I expose to a Python/C++ controlling layer (by wrappers? DLL?) which manages the communication between GUI and logic.
How can I achieve this?
A short version of this question might just be: How can I use a Qt C++ GUI with a Python logic?
You can create your GUI application using qt c++ and wrap controler logic written in a python api embedding the python api.
https://docs.python.org/2/extending/embedding.html
But if you have many methods that will be boring to do and you may have some memory leaks if not well written in th c++ side.
Another solution may be using rpc calls to a python api using a webservice (rest api/json rpc ....Etc...). Qt application is client of a python rest api.
like this you split the c code from the python one. You can do that using json rpc calls , or other apis,like jcon (https://github.com/joncol/jcon-cpp).
You will too be able to find python and QT xml rpc api able to talk together.
swig (http://swig.org) may have some capability , but it is better used in the other way : calling c func from python

Run c++ class from python

Is there a way to run a c++ class from python without using any external libraries like Boost.Python, SWING ect? I don't want to pass any arguments to this class or call a specific method and in my c++ class I have only a void main method, I just want to run the main and that is all.
Or if this is not possible a saw this tutorial http://intermediate-and-advanced-software-carpentry.readthedocs.io/en/latest/c++-wrapping.html#manual-wrapping. But I didn't understand if I should put the hello_wrapper function in the same c++ class where I have the original hello function. And also how can I create a modulo in Python(second part in the tutorial) and where should I put this code
DL_EXPORT(void) inithello(void)
{
Py_InitModule("hello", HelloMethods);
}
Thanks
is there a way to run a c++ class
you don't run C++ classes. They are data types!
Boost.Python, SWING
It's called SWIG, not SWING :)
You can add your own C wrapper code that initializes a PyObject. I'd recommend reading the CPython docs and the examples in the tutorial on extending python. Since you didn't specify a version, I can't give you a discrete link.
Note that python is C, and C++ isn't; which means that you'll have to export several things with a C ABI, i.e. by using external "C" in your code. That might not be something for the uninitiated, and you should certainly evaluate whether not using external wrapper generators is really worth the trouble – especially since using e.g. SWIG properly (which is really a pain) you can get Python objects that really behave like python objects, e.g. you can extend them with python etc.

Wrapping C++ class to use in Python

I have a device which can be controlled through a C++ class (https://github.com/stanleyseow/RF24/tree/master/RPi/RF24).
I'd like to be able to use this class in Python, and thought I could wrap it.
I found many ways to do it, but not much detailed documentation with examples. In particular, I found Boost, Cython, SWIG and the native C Python API.
Which one is the best method in which case ? And do you have some links to detailed documentations / examples about this ?
Thanks !
There is no "best"; it depends entirely on your circumstances.
For a single class, the native C Python API isn't too difficult,
but you do have to create an entire module, then the class. It
would be simpler if you exposed a procedural interface, rather
than a class. If you only have one instance of the device, this
would be an appropriate solution.
SWIG is very good for taking C++ class definitions and
generating a Python module which contains them. The resulting
code is relatively complex, since SWIG tries to cover all
possible versions of Python; for anything 2.7 or later (and
perhaps a little earlier), you can do everything directly in
C++, without any intermediate Python.
Boost makes extensive use of templates. This isn't really an
appropriate solution for the problem; it adds a lot of
complexity for something that is relatively simple if done with
external tools, rather than metaprogramming. Still, if the
underlying complexity doesn't scare you, it might not be that
hard to use.
I'm not familiar with Cython.
Globally, if all you have is one instance of one simple class,
using the native C API is probably no more difficult than the
other solutions, and introduces a minimum of added internal
complexity.

Python: SWIG vs ctypes

In python, under what circumstances is SWIG a better choice than ctypes for calling entry points in shared libraries? Let's assume you don't already have the SWIG interface file(s). What are the performance metrics of the two?
I have a rich experience of using swig. SWIG claims that it is a rapid solution for wrapping things. But in real life...
Cons:
SWIG is developed to be general, for everyone and for 20+ languages. Generally, it leads to drawbacks:
- needs configuration (SWIG .i templates), sometimes it is tricky,
- lack of treatment of some special cases (see python properties further),
- lack of performance for some languages.
Python cons:
1) Code style inconsistency. C++ and python have very different code styles (that is obvious, certainly), the possibilities of a swig of making target code more Pythonish is very limited. As an example, it is butt-heart to create properties from getters and setters. See this q&a
2) Lack of broad community. SWIG has some good documentation. But if one caught something that is not in the documentation, there is no information at all. No blogs nor googling helps. So one has to heavily dig SWIG generated code in such cases... That is terrible, I could say...
Pros:
In simple cases, it is really rapid, easy and straight forward
If you produced swig interface files once, you can wrap this C++ code to ANY of other 20+ languages (!!!).
One big concern about SWIG is a performance. Since version 2.04 SWIG includes '-builtin' flag which makes SWIG even faster than other automated ways of wrapping. At least some benchmarks shows this.
When to USE SWIG?
So I concluded for myself two cases when the swig is good to use:
2) If one needs to wrap C++ code for several languages. Or if potentially there could be a time when one needs to distribute the code for several languages. Using SWIG is reliable in this case.
1) If one needs to rapidly wrap just several functions from some C++ library for end use.
Live experience
Update :
It is a year and a half passed as we did a conversion of our library by using SWIG.
First, we made a python version. There were several moments when we experienced troubles with SWIG - it is true. But right now we expanded our library to Java and .NET. So we have 3 languages with 1 SWIG. And I could say that SWIG rocks in terms of saving a LOT of time.
Update 2:
It is two years as we use SWIG for this library. SWIG is integrated into our build system. Recently we had major API change of C++ library. SWIG worked perfectly. The only thing we needed to do is to add several %rename to .i files so our CppCamelStyleFunctions() now looks_more_pythonish in python. First I was concerned about some problems that could arise, but nothing went wrong. It was amazing. Just several edits and everything distributed in 3 languages. Now I am confident that it was a good solution to use SWIG in our case.
Update 3:
It is 3+ years we use SWIG for our library. Major change: python part was totally rewritten in pure python. The reason is that Python is used for the majority of applications of our library now. Even if the pure python version works slower than C++ wrapping, it is more convenient for users to work with pure python, not struggling with native libraries.
SWIG is still used for .NET and Java versions.
The Main question here "Would we use SWIG for python if we started the project from the beginning?". We would! SWIG allowed us to rapidly distribute our product to many languages. It worked for a period of time which gave us the opportunity for better understanding our users requirements.
SWIG generates (rather ugly) C or C++ code. It is straightforward to use for simple functions (things that can be translated directly) and reasonably easy to use for more complex functions (such as functions with output parameters that need an extra translation step to represent in Python.) For more powerful interfacing you often need to write bits of C as part of the interface file. For anything but simple use you will need to know about CPython and how it represents objects -- not hard, but something to keep in mind.
ctypes allows you to directly access C functions, structures and other data, and load arbitrary shared libraries. You do not need to write any C for this, but you do need to understand how C works. It is, you could argue, the flip side of SWIG: it doesn't generate code and it doesn't require a compiler at runtime, but for anything but simple use it does require that you understand how things like C datatypes, casting, memory management and alignment work. You also need to manually or automatically translate C structs, unions and arrays into the equivalent ctypes datastructure, including the right memory layout.
It is likely that in pure execution, SWIG is faster than ctypes -- because the management around the actual work is done in C at compiletime rather than in Python at runtime. However, unless you interface a lot of different C functions but each only a few times, it's unlikely the overhead will be really noticeable.
In development time, ctypes has a much lower startup cost: you don't have to learn about interface files, you don't have to generate .c files and compile them, you don't have to check out and silence warnings. You can just jump in and start using a single C function with minimal effort, then expand it to more. And you get to test and try things out directly in the Python interpreter. Wrapping lots of code is somewhat tedious, although there are attempts to make that simpler (like ctypes-configure.)
SWIG, on the other hand, can be used to generate wrappers for multiple languages (barring language-specific details that need filling in, like the custom C code I mentioned above.) When wrapping lots and lots of code that SWIG can handle with little help, the code generation can also be a lot simpler to set up than the ctypes equivalents.
CTypes is very cool and much easier than SWIG, but it has the drawback that poorly or malevolently-written python code can actually crash the python process. You should also consider boost python. IMHO it's actually easier than swig while giving you more control over the final python interface. If you are using C++ anyway, you also don't add any other languages to your mix.
In my experience, ctypes does have a big disadvantage: when something goes wrong (and it invariably will for any complex interfaces), it's a hell to debug.
The problem is that a big part of your stack is obscured by ctypes/ffi magic and there is no easy way to determine how did you get to a particular point and why parameter values are what they are..
You can also use Pyrex, which can act as glue between high-level Python code and low-level C code. lxml is written in Pyrex, for instance.
ctypes is great, but does not handle C++ classes. I've also found ctypes is about 10% slower than a direct C binding, but that will highly depend on what you are calling.
If you are going to go with ctypes, definitely check out the Pyglet and Pyopengl projects, that have massive examples of ctype bindings.
I'm going to be contrarian and suggest that, if you can, you should write your extension library using the standard Python API. It's really well-integrated from both a C and Python perspective... if you have any experience with the Perl API, you will find it a very pleasant surprise.
Ctypes is nice too, but as others have said, it doesn't do C++.
How big is the library you're trying to wrap? How quickly does the codebase change? Any other maintenance issues? These will all probably affect the choice of the best way to write the Python bindings.
Just wanted to add a few more considerations that I didn't see mentioned yet.
[EDIT: Ooops, didn't see Mike Steder's answer]
If you want to try using a non Cpython implementation (like PyPy, IronPython or Jython), then ctypes is about the only way to go. PyPy doesn't allow writing C-extensions, so that rules out pyrex/cython and Boost.python. For the same reason, ctypes is the only mechanism that will work for IronPython and (eventually, once they get it all working) jython.
As someone else mentioned, no compilation is required. This means that if a new version of the .dll or .so comes out, you can just drop it in, and load that new version. As long as the none of the interfaces changed, it's a drop in replacement.
Something to keep in mind is that SWIG targets only the CPython implementation. Since ctypes is also supported by the PyPy and IronPython implementations it may be worth writing your modules with ctypes for compatibility with the wider Python ecosystem.
I have found SWIG to be be a little bloated in its approach (in general, not just Python) and difficult to implement without having to cross the sore point of writing Python code with an explicit mindset to be SWIG friendly, rather than writing clean well-written Python code. It is, IMHO, a much more straightforward process to write C bindings to C++ (if using C++) and then use ctypes to interface to any C layer.
If the library you are interfacing to has a C interface as part of the library, another advantage of ctypes is that you don't have to compile a separate python-binding library to access third-party libraries. This is particularly nice in formulating a pure-python solution that avoids cross-platform compilation issues (for those third-party libs offered on disparate platforms). Having to embed compiled code into a package you wish to deploy on something like PyPi in a cross-platform friendly way is a pain; one of my most irritating points about Python packages using SWIG or underlying explicit C code is their general inavailability cross-platform. So consider this if you are working with cross-platform available third party libraries and developing a python solution around them.
As a real-world example, consider PyGTK. This (I believe) uses SWIG to generate C code to interface to the GTK C calls. I used this for the briefest time only to find it a real pain to set up and use, with quirky odd errors if you didn't do things in the correct order on setup and just in general. It was such a frustrating experience, and when I looked at the interace definitions provided by GTK on the web I realized what a simple excercise it would be to write a translator of those interface to python ctypes interface. A project called PyGGI was born, and in ONE day I was able to rewrite PyGTK to be a much more functiona and useful product that matches cleanly to the GTK C-object-oriented interfaces. And it required no compilation of C-code making it cross-platform friendly. (I was actually after interfacing to webkitgtk, which isn't so cross-platform). I can also easily deploy PyGGI to any platform supporting GTK.

Categories

Resources