Related
You know, we can bind every C library on Python or Perl programming languages. A good instance is PyQt; PyQt is bound from Qt.
My question is: Can I do the reverse of the above? I mean: suppose I have a library in Python or Perl, and I want to convert it to a C library...can it be done?
However, you can think to convert a web program to shared library or a set of functions.
My Goal: I want to improve a set of security features.
Yes, you can. The term of art is "embedding," as in "embedded Python" or "embed a Python interpreter." Python has a document about it here: https://docs.python.org/2/extending/embedding.html - the general idea is you must use (at least a small part of) the Python C API to launch Python within a C or C++ application.
Once you realize you can embed a scripting language in C, and that scripting languages can invoke C, you then realize you can also embed one scripting language in another, using C as the bridge between them. For example, RubyLuaBridge: https://bitbucket.org/neomantra/rubyluabridge
A lot of commercial applications embed a scripting language interpreter within a C or C++ host program. A good, well-documented example is Adobe Lightroom, which is roughly half C++ and half Lua. You can read about that from the horse's mouth starting on page xi here: http://www.lua.org/gems/front.pdf
Yes, for python at least: Convert Python program to C/C++ code?.
And for Perl: http://perldoc.perl.org/5.8.9/perlcompile.html.
[EDIT] Per the comment, I'll expand. First, the question "Can I translate Python to C?" has already been answered on SO. See the link.
Second, Perl is actually an interpreted language (just like Python), and has the capability to take that intermediate code and translate it into full blown C for native executables. This is done using the 'B' module and it's other companion modules, such as B::C. There's also a standalone program, 'perlcc' for doing just this. [/EDIT]
I have a C++ library that I need to be able to interface with python. I read this question to understand the choice I need to adapt.
I saw SWIG and Cython and wanted to go with SWIG, mainly because my python programming experience is very minimal. However, I realise that with Swig I have to write an interface (.i extensions) for every class. Now, my C++ project is huge and I feel it will take me a lot of time to get the wrappers in place (or maybe I am wrong).
So right now since my application is large I need to make a choice. In the quoted thread I came across Boost Python. Now I can no longer decide and want input from people who can tell me the pros and cons of one over the other. Note my preference is on easy of use and how quickly can it be done. I am willing to compromise system performance for this. I would appreciate immensely if someone could provide me a SWIG implemented project or Boost Python implemented project link (a complete module instead of a sample tutorial would be much better !)
Boost::python provides a nearly wrapper-less interface between C++ and Python. It also allows you to write custom converters and other neat things that make the Python interfaces much nicer. The interfaces are pure C++, but they rely on templates and clever design patterns to make it look all nice and declarative. You also get the benefit of your connector code being checked by the compiler directly.
With Swig, you write interface declarations in Swig's own DSL, which takes a few days to get a hang of. In addition, it always inserts a wrapper layer, so it could be a bit slower. However, it does have the nice feature of automatically converting many things for ya without having to declare anything extra. The wrappers it generates are quite hard to debug though.
IMHO boost::python is the better choice, because you're working pretty directly with CPython's native C interfaces. I use Swig for Java and C++ interaction, because JNI is a bear, Python's C interface is actually quite usable all by itself.
If you already have a bunch of Swig wrappers, I would keep those because you'd have to redo all that work. However, starting a new project, or if you require maximum performance, boost::python all the way!
I do understand that this topic has been covered in some way at StackOverflow but I'm still not able to figure out the exact answer: can I treat IronPython as a Pythonic replacement to C#?
I use CPython every day, I love the Zen :) but my current task is a Windows-only application with a complex GUI and some other features which I would like to implement using .NET.
IronPython is NOT equivalent to "other languages that run on .NET", as the language has support for substantially fewer CLR runtime features.
IronPython classes are not "real" .NET classes, and DLR APIs need to be used when calling IronPython code from traditional CLR-based languages; this means that if you want genuinely easy interoperability, you're stuck writing glue to "hide" the DLR.
Boo is a much more complete Pythonically-inspired language targeting the CLR. Its (dynamically inferrable) static typing (which can be replaced with duck typing on a variable-by-variable basis) also allows libraries written in Boo to be natively used from C# and other CLR-based languages, without needing to make any allowances for the language in use.
That depends on what it is about C# that you need, and which needs replacing.
If the reason you use C# is that you need a reasonably high performance statically typed language then no, IronPython is likely not going to be a replacement.
If the reason you use it is simply "I need something that runs on .NET and can access .NET libraries", then yes, any language that runs on .NET can be used to replace it.
If you use C# because you're working with a team of programmers who only know C-like languages, C# might also be difficult to replace with IronPython.
It depends on what characteristics about C# it is that you care about, and need to find replacements for.
One thing to consider - and I have no idea how IronPython behaves in that respect - is Common Language Subsystem (CLS) compliance for assemblies. CLS compliance guarantees that any .Net language can access a compiled DLL of your code. That means e.g. in C# you cannot have any public or protected method or parameter that is a unsigned integer. I have no idea how easy it is to achieve CLS compliant code in IronPython, an interesting blog entry that I found dates from 2008.
The question "Can I treat IronPython as a Pythonic replacement for C#?" has been answered pretty well by jalf. If the question were "Is IronPython a Pythonic .NET Language?" though, then the answer would absolutely be yes. The principles of Zen - esp. least surprise - absolutely apply to IronPython's integration with the CLR as well.
I think if you were to do that,
it would be easier in .NET 4.0.
I think you can use the newly released IronPython 2.6.1 for .NET 4.0,
It is already easy to use C# in IronPython.
As you can see here, you can easily do it the other way around (using IronPython from C#) with .NET 4.0.
I think its possible, but I agree Boo is a safer way to go.
"Complex" UI usually entails not "writing" it but building it within Visual Studio with point and click. All the callbacks and eventing code is inserted by itself. There is almost nothing like that on python side. I'd say go for C# straight out.
There is one nagging thing though. If you are true Pythonista, the static typing will get to you very very quickly and you will want to start throwing heavy objects at random people.
If that point comes think about building out the UI with C# and embedding IronPython as a scripting engine for implementing your business logic. That could be a tolerable middle ground.
So I'm aware of the big ammount of general-purpose scripting languages like Ruby, Python, Perl, maybe even PHP, etc. that actually claim being usable for creating desktop applications too.
I think my question can be answered clearly
Are there actually companies using a special scripting language only to create their applications?
Are there any real advantages on creating a product in a language like Python only?
I'm not talking about the viability of those languages for web-development!
Should I stick with C(++) for desktop apps?
best regards,
lamas
The company I work for uses Perl and Tk with PerlApp to build executable packages to produce or major software application.
Perl beats C and C++ for simplicity of code. You can do things in one line of Perl that take 20 lines of C.
We've used WxPerl for a few smaller projects. We'd like to move fully to WxPerl, but existing code works, so the move has a low priority until Wx can give us something we need that Tk can't.
Python is popular for building GUI apps, too. You may have heard about Chandler. That's a big Python app. There are many others as well.
Ruby is also a suitable choice.
PHP is breaking into the world of command line apps. I am not sure about the power or flexibility of its GUI toolkits.
The languages you list aren't really scripting languages, as that tends to describe languages designed to work inside of a larger framework (like javascript) that provides its interface to the world. While you can certainly write scripts in those languages, each is a proper programming language (referred to as a dynamic or interpreted language, in contrast to compiled languages like C or C++).
There are many mature gui toolkits for creating desktop apps with interpreted languages. A search for any of those languages with "gui" on SO will yield many results.
The advantage of the languages you list are rapid development and concise code.
The advantage of compiled languages is mainly speed, and deeper ties with the internals of the operating system. But for most desktop apps, the ease of development in an interpreted language outweighs any small performance gains (unless you are writing a cpu intensive app, in which case, write the cpu heavy bits in C, and then call them from the interpreted language, which can handle the gui)
Many interpreted languages offer easy escapes to C or other languages (often with a nice inline syntax).
I would encourage you to take a look at some examples on http://rosettacode.org to see the fundamental differences between how programs come together with the languages you are interested in.
Python (combined with PyQt) is a very solid combination for GUI desktop applications (note that while QT is LGPL, PyQt (the Python bindings) is dual licensed: GPL or commercial).
It offers the same (GUI library-wise) as Qt on C++ but with Python's specific strenghts. I'll list some of the more obvious ones:
rapid prototyping
extremely readable (hence maintainable) code
Should I stick with C(++) for desktop apps?
In general: no, unless you want to / need to (for a specific reason).
I would recommend you not try to look for a language that is best for GUI apps but instead look for the language that you like the most and then use that to write your app.
Ruby, Python, Perl all have GUI tool kits available to them. Most of them have access to the same often used tool kits like TK, GTK, and Wx. The look and feel of a an app will be dependent more on the GUI tool kit than on the language, and performance wise your likely to see more impact for how you write your app than language choice.
If your comfortable with C++ then you should also look at C# or Java as options. While not scripting languages they have many of the same benefits like memory management and more sane string implementations.
I have used a number of programs that were developed using scripted languages. Several embedded device vendors ship my group Windows-based configuration and debugging utilities written in TCL. Google's drawing program SketchUp has a lot of Ruby inside it (and users can create add-ons using Ruby). I have seen many Linux applications written in Python. There are many more examples out there, but often times finished applications are bundled up to the point where you can't really tell what's powering it on the inside.
Yes, there can be advantages to working with scripted languages. Some scripted languages make it easier to do specific tasks; for example, text processing is much easier (IMO) in a language like Ruby that has regular expression support and a robust String class than it is in plain old C. Generating a UI using a scripted language may make it easier to support multiple platforms, as all the platform-specific code is taken care of inside the language interpreter or pre-compiled libraries. For example, our suppliers who build TCL-based apps claim they can build the UI for an app using TCL in a fraction of the time it would take them to build it in C++ or VB, and then they can port it to Linux almost effortlessly.
On the other hand there are a few things that scripted languages typically aren't suited for, such as writing drivers or doing anything that requires low-level hardware access.
Most importantly, however, is this: modern languages have become quite powerful to the point where choice of language doesn't make as big of a difference as it used to be. Use the language you are most comfortable with. The learning curve associated with learning a new language will usually have a much larger impact on your project.
http://www.pygtk.org/applications.html
Seems like a really long list of GUI applications in Python using just one of the frameworks.
Some part of MPICH2 is written in Python. I didn't check everything, but many parts of it used for running mpi applications are written it Python. Maybe MPICH2 is not used by everyone, but for sure it is good piece of software.
In python, under what circumstances is SWIG a better choice than ctypes for calling entry points in shared libraries? Let's assume you don't already have the SWIG interface file(s). What are the performance metrics of the two?
I have a rich experience of using swig. SWIG claims that it is a rapid solution for wrapping things. But in real life...
Cons:
SWIG is developed to be general, for everyone and for 20+ languages. Generally, it leads to drawbacks:
- needs configuration (SWIG .i templates), sometimes it is tricky,
- lack of treatment of some special cases (see python properties further),
- lack of performance for some languages.
Python cons:
1) Code style inconsistency. C++ and python have very different code styles (that is obvious, certainly), the possibilities of a swig of making target code more Pythonish is very limited. As an example, it is butt-heart to create properties from getters and setters. See this q&a
2) Lack of broad community. SWIG has some good documentation. But if one caught something that is not in the documentation, there is no information at all. No blogs nor googling helps. So one has to heavily dig SWIG generated code in such cases... That is terrible, I could say...
Pros:
In simple cases, it is really rapid, easy and straight forward
If you produced swig interface files once, you can wrap this C++ code to ANY of other 20+ languages (!!!).
One big concern about SWIG is a performance. Since version 2.04 SWIG includes '-builtin' flag which makes SWIG even faster than other automated ways of wrapping. At least some benchmarks shows this.
When to USE SWIG?
So I concluded for myself two cases when the swig is good to use:
2) If one needs to wrap C++ code for several languages. Or if potentially there could be a time when one needs to distribute the code for several languages. Using SWIG is reliable in this case.
1) If one needs to rapidly wrap just several functions from some C++ library for end use.
Live experience
Update :
It is a year and a half passed as we did a conversion of our library by using SWIG.
First, we made a python version. There were several moments when we experienced troubles with SWIG - it is true. But right now we expanded our library to Java and .NET. So we have 3 languages with 1 SWIG. And I could say that SWIG rocks in terms of saving a LOT of time.
Update 2:
It is two years as we use SWIG for this library. SWIG is integrated into our build system. Recently we had major API change of C++ library. SWIG worked perfectly. The only thing we needed to do is to add several %rename to .i files so our CppCamelStyleFunctions() now looks_more_pythonish in python. First I was concerned about some problems that could arise, but nothing went wrong. It was amazing. Just several edits and everything distributed in 3 languages. Now I am confident that it was a good solution to use SWIG in our case.
Update 3:
It is 3+ years we use SWIG for our library. Major change: python part was totally rewritten in pure python. The reason is that Python is used for the majority of applications of our library now. Even if the pure python version works slower than C++ wrapping, it is more convenient for users to work with pure python, not struggling with native libraries.
SWIG is still used for .NET and Java versions.
The Main question here "Would we use SWIG for python if we started the project from the beginning?". We would! SWIG allowed us to rapidly distribute our product to many languages. It worked for a period of time which gave us the opportunity for better understanding our users requirements.
SWIG generates (rather ugly) C or C++ code. It is straightforward to use for simple functions (things that can be translated directly) and reasonably easy to use for more complex functions (such as functions with output parameters that need an extra translation step to represent in Python.) For more powerful interfacing you often need to write bits of C as part of the interface file. For anything but simple use you will need to know about CPython and how it represents objects -- not hard, but something to keep in mind.
ctypes allows you to directly access C functions, structures and other data, and load arbitrary shared libraries. You do not need to write any C for this, but you do need to understand how C works. It is, you could argue, the flip side of SWIG: it doesn't generate code and it doesn't require a compiler at runtime, but for anything but simple use it does require that you understand how things like C datatypes, casting, memory management and alignment work. You also need to manually or automatically translate C structs, unions and arrays into the equivalent ctypes datastructure, including the right memory layout.
It is likely that in pure execution, SWIG is faster than ctypes -- because the management around the actual work is done in C at compiletime rather than in Python at runtime. However, unless you interface a lot of different C functions but each only a few times, it's unlikely the overhead will be really noticeable.
In development time, ctypes has a much lower startup cost: you don't have to learn about interface files, you don't have to generate .c files and compile them, you don't have to check out and silence warnings. You can just jump in and start using a single C function with minimal effort, then expand it to more. And you get to test and try things out directly in the Python interpreter. Wrapping lots of code is somewhat tedious, although there are attempts to make that simpler (like ctypes-configure.)
SWIG, on the other hand, can be used to generate wrappers for multiple languages (barring language-specific details that need filling in, like the custom C code I mentioned above.) When wrapping lots and lots of code that SWIG can handle with little help, the code generation can also be a lot simpler to set up than the ctypes equivalents.
CTypes is very cool and much easier than SWIG, but it has the drawback that poorly or malevolently-written python code can actually crash the python process. You should also consider boost python. IMHO it's actually easier than swig while giving you more control over the final python interface. If you are using C++ anyway, you also don't add any other languages to your mix.
In my experience, ctypes does have a big disadvantage: when something goes wrong (and it invariably will for any complex interfaces), it's a hell to debug.
The problem is that a big part of your stack is obscured by ctypes/ffi magic and there is no easy way to determine how did you get to a particular point and why parameter values are what they are..
You can also use Pyrex, which can act as glue between high-level Python code and low-level C code. lxml is written in Pyrex, for instance.
ctypes is great, but does not handle C++ classes. I've also found ctypes is about 10% slower than a direct C binding, but that will highly depend on what you are calling.
If you are going to go with ctypes, definitely check out the Pyglet and Pyopengl projects, that have massive examples of ctype bindings.
I'm going to be contrarian and suggest that, if you can, you should write your extension library using the standard Python API. It's really well-integrated from both a C and Python perspective... if you have any experience with the Perl API, you will find it a very pleasant surprise.
Ctypes is nice too, but as others have said, it doesn't do C++.
How big is the library you're trying to wrap? How quickly does the codebase change? Any other maintenance issues? These will all probably affect the choice of the best way to write the Python bindings.
Just wanted to add a few more considerations that I didn't see mentioned yet.
[EDIT: Ooops, didn't see Mike Steder's answer]
If you want to try using a non Cpython implementation (like PyPy, IronPython or Jython), then ctypes is about the only way to go. PyPy doesn't allow writing C-extensions, so that rules out pyrex/cython and Boost.python. For the same reason, ctypes is the only mechanism that will work for IronPython and (eventually, once they get it all working) jython.
As someone else mentioned, no compilation is required. This means that if a new version of the .dll or .so comes out, you can just drop it in, and load that new version. As long as the none of the interfaces changed, it's a drop in replacement.
Something to keep in mind is that SWIG targets only the CPython implementation. Since ctypes is also supported by the PyPy and IronPython implementations it may be worth writing your modules with ctypes for compatibility with the wider Python ecosystem.
I have found SWIG to be be a little bloated in its approach (in general, not just Python) and difficult to implement without having to cross the sore point of writing Python code with an explicit mindset to be SWIG friendly, rather than writing clean well-written Python code. It is, IMHO, a much more straightforward process to write C bindings to C++ (if using C++) and then use ctypes to interface to any C layer.
If the library you are interfacing to has a C interface as part of the library, another advantage of ctypes is that you don't have to compile a separate python-binding library to access third-party libraries. This is particularly nice in formulating a pure-python solution that avoids cross-platform compilation issues (for those third-party libs offered on disparate platforms). Having to embed compiled code into a package you wish to deploy on something like PyPi in a cross-platform friendly way is a pain; one of my most irritating points about Python packages using SWIG or underlying explicit C code is their general inavailability cross-platform. So consider this if you are working with cross-platform available third party libraries and developing a python solution around them.
As a real-world example, consider PyGTK. This (I believe) uses SWIG to generate C code to interface to the GTK C calls. I used this for the briefest time only to find it a real pain to set up and use, with quirky odd errors if you didn't do things in the correct order on setup and just in general. It was such a frustrating experience, and when I looked at the interace definitions provided by GTK on the web I realized what a simple excercise it would be to write a translator of those interface to python ctypes interface. A project called PyGGI was born, and in ONE day I was able to rewrite PyGTK to be a much more functiona and useful product that matches cleanly to the GTK C-object-oriented interfaces. And it required no compilation of C-code making it cross-platform friendly. (I was actually after interfacing to webkitgtk, which isn't so cross-platform). I can also easily deploy PyGGI to any platform supporting GTK.