Develop Reference

Which language to use for writing an admin console à la webmin?

We have an in house developed web-based admin console that uses a combination of C CGI and Perl scripts to administer our mail server stack. Of late we have been thinking of cleaning up the code (well, replacing most of it), making the implementation more secure, and improving the overall behavior.
I don't have much programming knowledge, but I use Ruby on and off (mainly for writing erb templates), and hence was thinking of using ruby/rails for developing such an app (off-duty for now, I also need to learn stuff !).
Before blindly picking up a language though, what would you folks suggest ? Please let me know if this is too vague a question, I'll try to supply more information, if needed.

Have you considered writing your applications as Webmin modules?
You get a lot of stuff for free when you do so (users and groups, tons of security features, a pretty big variety of helper functions related to config files, and tons of existing code for most aspects of a UNIX/Linux system). You also get a lot of stuff for nearly free, like action logging, packages and updates via wbm or apt or yum, an online help system, etc.
There are some cons, as well. It's an old codebase, so it has some clunky bits in the API among other places. A lot of the old modules can be a bit hard to grok if you're not an old-school Perl programmer. But, it's a well-maintained codebase, and it's been banged on by millions of users for over a dozen years. It's pretty robust. The UI isn't beautiful, but it is relatively theme-able, and if you're distributing a minimized version it becomes easier to customize the UI.
I suspect you can be up and running a lot faster than starting from scratch or using most existing frameworks that aren't targeted specifically to building systems management interfaces the way Webmin is.
Also, it's BSD licensed, so you can do whatever you want with it, including building a custom commercial app with it (hundreds of companies have done so over the years).

If you already know a bit of ruby, then there's no reason not to use that.
If you're interested specifically in learning another language, then what you're trying to do could be done in pretty much any language/framework, it's just a matter of which one you want to learn.

Without knowing much about your existing application I'd say that this effectively boils down to "which language do you like to work with?".
Python and Ruby are both mature languages with ample library infrastructure. They also boast popular, similar web application frameworks namely Django and Ruby-on-Rails respectively.
Since you are porting an existing Perl app(lets) it may be worthwhile to note that Ruby is relatively more similar to Perl. Not surprising given that Ruby was influenced "primarily by Perl, Smalltalk, Eiffel and Lisp".

django has a nice admin interface

Which cross platform scripting language should we adopt for a group of DBAs?

I wanted to get the community's feedback on a language choice our team is looking to make in the near future. We are a software developer, and I work in a team of Oracle and SQL Server DBAs supporting a cross platform Java application which runs on Oracle Application Server. We have SQL Server and Oracle code bases, and support customers on Windows, Solaris and Linux servers.
Many of the tasks we do on a frequent basis are insufficiently automated, and where they are, tend to be much more automated via shell scripts, with little equivalent functionality on Windows. Unfortunately, we now have this problem of redeveloping scripts and so on, on two platforms. So, I wish for us to choose a cross platform language to script in, instead of using Bash and awkwardly translating to Cygwin or Batch files where necessary.
It would need to be:
Dynamic (so don't suggest Java or C!)
Easily available on each platform (Windows, Solaris, Linux, perhaps AIX)
Require very little in the way of setup (root access not always available!)
Be easy for shell scripters, i.e. DBAs, to adopt, who are not hardcore developers.
Be easy to understand other people's code
Friendly with SQL Server and Oracle, without messing around.
A few nice XML features wouldn't go amiss.
It would be preferable if it would run on the JVM, since this will almost always be installed on every server (certainly on all application servers) and we have many Java developers in our company, so sticking to the JVM makes sense. This isn't exclusive though, since I know Python is a very viable language here.
I have created a list of options, but there may be more: Groovy, Scala, Jython, Python, Ruby, Perl.
No one has much experience of any, except I have quite a lot of Java and Groovy experience myself. We are looking for something dynamic, easy to pick up, will work with both SQL server and Oracle effortlessly, has some XML simplifying features, and that won't be a turnoff for DBAs. Many of us are very Bash orientated - what could move us away from this addiction?
What are people's opinions on this?
thanks!
Chris

You can opt for Python. Its dynamic(interpreted) , is available on Windows/Linux/Solaris, has easy to read syntax so that your code maintenance is easy. There modules/libraries for Oracle interaction and various other database servers as well. there are also library support for XML. All 7 points are covered.

I think your best three options are Groovy, Python, and Scala. All three let you write code at a high level (compared to C/Java). Python has its own perfectly adequate DB bindings, and Groovy and Scala can use ones made for Java.
The advantages of Python are that it is widely used already, so there are tons of tools, libraries, expertise, etc. available around it. It has a particularly clean syntax, which makes working with it aesthetically pleasing. The disadvantages are that it is slow (which may not be an issue for you), untyped (so you have runtime errors instead of compile-time errors), and you can't really switch back and forth between Jython and Python, so you have to pick whether you want the large amount of Python stuff, or the huge amount of Java stuff, minus a lot of the nice Python stuff.
The advantages of Groovy are that you know it already and it interoperates well with Java libraries. Its disadvantages are also slowness and lack of static typing. (So in contrast to Python, the choice is: do you value Python's clean syntax and wide adoption more, or do you value the vast set of Java libraries more in a language made to work well in that environment?)
The advantages of Scala are that it is statically typed (i.e. if the code gets past the compiler, it has a greater chance of working), is fast (as fast as Java if you care to work hard enough), and interoperates well with Java libraries. The disadvantages are that it imposes a bit more work on you to make the static typing work (though far, far less than Java while simultaneously being more safe), and that the canonical style for Scala is a hybrid object/functional blend that feels more different than the other two (and thus requires more training to use at full effectiveness IMO). In contrast to Groovy, the question would be whether familiarity and ease of getting started is more important than speed and correctness.
Personally, I now do almost all of my work in Scala because my work requires speed and because the compiler catches those sort of errors in coding that I commonly make (so it is the only language I've used where I am not surprised when large blocks of code run correctly once I get them to compile). But I've had good experiences with Python in other contexts--interfacing with large databases seems like a good use-case.
(I'd rule out Perl as being harder to maintain with no significant benefits over e.g. Python, and I'd rule out Ruby as being not enough more powerful than Python to warrant the less-intuitive syntax and lower rate of adoption/tool availability.)

The XML thing almost calls for Scala. Now, I love Scala, but I suggest Python here.

If you want a dynamic language and there already a lot of Java developers in your company, then Groovy seems an obvious choice, as it's very easy for Java developers to pick up (also, you said you have some Groovy experience yourself).
Groovy runs on the JVM and has excellent support for working with XML. It also has provides a very straightforward syntax for working with relational databases.
It comes with a console and a shell (though I never use the shell) which make it really easy to test/run scripts or snippets of Groovy code.

Although I prefer working on the JVM, one thing that turns me off is having to spin up a JVM to run a script. If you can work in a REPL this is not such a big deal, but it really slows you down when doing edit-run-debug scripting.
Now of course Oracle has a lot of Java stuff where interaction moght be needed, but that is something only you can estimate how important it is. For plain Oracle DB work I have seen very little Java and lots fo PLSQL/SQL.
If your dba now do their work in bash, then they will very likely pickup perl in a short time as there is a nice, logical progression path.
Since ruby was designed to be an improved version of perl, it might fit in that category too. Actually python also.
Scala is statically typed like Java, albeit with much better type inference.
My recommendation would be to go the Perl route. The CPAN is its ace in the hole, you do not have to deal with the OO stuff which might turn off some DBA's (although it is there for the power users).

I've been in a similar situation, though on a small scale. The previous situation was that any automation on the SQL Server DBs was done with VBScript, which I did start out using. As I wanted something cross-platform (and less annoying than VBScript) I went with Python.
What I learnt is:
Obviously you want a language that comes with libraries to access your databases comfortably. I wasn't too concerned with abstracting the differences away (ie, I still wrote SQL queries in the relevant dialect, with parameters). However, I'd be a bit less happy with PHP, for example, which has only very vendor-specific libraries and functions for certain databases. I see it's not on your list.
THE major obstacle was authentication. If your SQL Server uses Windows domain authentication, you'll have to work to get in. Another system also had specific needs as it required RSA tokens to be supported.
For the second point, Python is quite versatile enough to work around the difficulties, but it was getting into "badly supported" territory, especially on Windows. It was easy to work around the first problem from a Windows host, and for a Unix host it is possible though not easy. If you're using SQL Server authentication, it becomes a lot easier.
From your other choices, I'd expect various ways of authenticating and DB drivers to exist for Perl, which philosophically would be easier for DBAs used to shell scripting. Ruby - no experience, but it tends to have spotty support for some of the odder authentication methods and connectors. Scala I'd expect to be a bit too much of a "programmer's programming language" -- OOO and FP? It's a very interesting language, but maybe not the one I'd chose at first. As for the rest of the Java-based options, I don't have an opinion, but do check that all the connection types you want to make are solidly supported.

"Real" and non-embedded use of Ruby, Python and their friends

So I'm aware of the big ammount of general-purpose scripting languages like Ruby, Python, Perl, maybe even PHP, etc. that actually claim being usable for creating desktop applications too.
I think my question can be answered clearly
Are there actually companies using a special scripting language only to create their applications?
Are there any real advantages on creating a product in a language like Python only?
I'm not talking about the viability of those languages for web-development!
Should I stick with C(++) for desktop apps?
best regards,
lamas

The company I work for uses Perl and Tk with PerlApp to build executable packages to produce or major software application.
Perl beats C and C++ for simplicity of code. You can do things in one line of Perl that take 20 lines of C.
We've used WxPerl for a few smaller projects. We'd like to move fully to WxPerl, but existing code works, so the move has a low priority until Wx can give us something we need that Tk can't.
Python is popular for building GUI apps, too. You may have heard about Chandler. That's a big Python app. There are many others as well.
Ruby is also a suitable choice.
PHP is breaking into the world of command line apps. I am not sure about the power or flexibility of its GUI toolkits.

The languages you list aren't really scripting languages, as that tends to describe languages designed to work inside of a larger framework (like javascript) that provides its interface to the world. While you can certainly write scripts in those languages, each is a proper programming language (referred to as a dynamic or interpreted language, in contrast to compiled languages like C or C++).
There are many mature gui toolkits for creating desktop apps with interpreted languages. A search for any of those languages with "gui" on SO will yield many results.
The advantage of the languages you list are rapid development and concise code.
The advantage of compiled languages is mainly speed, and deeper ties with the internals of the operating system. But for most desktop apps, the ease of development in an interpreted language outweighs any small performance gains (unless you are writing a cpu intensive app, in which case, write the cpu heavy bits in C, and then call them from the interpreted language, which can handle the gui)
Many interpreted languages offer easy escapes to C or other languages (often with a nice inline syntax).
I would encourage you to take a look at some examples on http://rosettacode.org to see the fundamental differences between how programs come together with the languages you are interested in.

Python (combined with PyQt) is a very solid combination for GUI desktop applications (note that while QT is LGPL, PyQt (the Python bindings) is dual licensed: GPL or commercial).
It offers the same (GUI library-wise) as Qt on C++ but with Python's specific strenghts. I'll list some of the more obvious ones:
rapid prototyping
extremely readable (hence maintainable) code
Should I stick with C(++) for desktop apps?
In general: no, unless you want to / need to (for a specific reason).

I would recommend you not try to look for a language that is best for GUI apps but instead look for the language that you like the most and then use that to write your app.
Ruby, Python, Perl all have GUI tool kits available to them. Most of them have access to the same often used tool kits like TK, GTK, and Wx. The look and feel of a an app will be dependent more on the GUI tool kit than on the language, and performance wise your likely to see more impact for how you write your app than language choice.
If your comfortable with C++ then you should also look at C# or Java as options. While not scripting languages they have many of the same benefits like memory management and more sane string implementations.

I have used a number of programs that were developed using scripted languages. Several embedded device vendors ship my group Windows-based configuration and debugging utilities written in TCL. Google's drawing program SketchUp has a lot of Ruby inside it (and users can create add-ons using Ruby). I have seen many Linux applications written in Python. There are many more examples out there, but often times finished applications are bundled up to the point where you can't really tell what's powering it on the inside.
Yes, there can be advantages to working with scripted languages. Some scripted languages make it easier to do specific tasks; for example, text processing is much easier (IMO) in a language like Ruby that has regular expression support and a robust String class than it is in plain old C. Generating a UI using a scripted language may make it easier to support multiple platforms, as all the platform-specific code is taken care of inside the language interpreter or pre-compiled libraries. For example, our suppliers who build TCL-based apps claim they can build the UI for an app using TCL in a fraction of the time it would take them to build it in C++ or VB, and then they can port it to Linux almost effortlessly.
On the other hand there are a few things that scripted languages typically aren't suited for, such as writing drivers or doing anything that requires low-level hardware access.
Most importantly, however, is this: modern languages have become quite powerful to the point where choice of language doesn't make as big of a difference as it used to be. Use the language you are most comfortable with. The learning curve associated with learning a new language will usually have a much larger impact on your project.

http://www.pygtk.org/applications.html
Seems like a really long list of GUI applications in Python using just one of the frameworks.

Some part of MPICH2 is written in Python. I didn't check everything, but many parts of it used for running mpi applications are written it Python. Maybe MPICH2 is not used by everyone, but for sure it is good piece of software.

Python Applications: Can You Secure Your Code Somehow?

If there is truly a 'best' way, what is the best way to ship a python app and ensure people can't (easily) reverse engineer your algorithms/security/work in general?
If there isn't a 'best' way, what are the different options available?
Background:
I love coding in Python and would love to release more apps with it. One thing that I wonder about is the possibility of people circumventing any licensing code I put in, or being able to just rip off my entire source base. I've heard of Py2Exe and similar applications, but I'm curious if there are 'preferred' ways of doing it, or if this problem is just a fact of life.

Security through obscurity never works. If you must use a proprietary license, enforce it through the law, not half-baked obfuscation attempts.
If you're worried about them learning your security (e.g. cryptography) algorithm, the same applies. Real, useful, security algorithms (like AES) are secure even though the algorithm is fully known.

Even if you use a compiled language like C# or Java, people can perform reverse engineering if they are motivated and technically competent. Obfuscation is not a reliable protection against this.
You can add prohibition against reverse-engineering to your end-user license agreement for your software. Most proprietary companies do this. But that doesn't prevent violation, it only gives you legal recourse.
The best solution is to offer products and services in which the user's access to read your code does not harm your ability to sell your product or service. Base your business on service provided, or subscription to periodic updates to data, rather than the code itself.
Example: Slashdot actually makes their code for their website available. Does this harm their ability to run their website? No.
Another remedy is to set your price point such that the effort to pirate your code is more costly than simply buying legitimate licenses to use your product. Joel Spolsky has made a recommendation to this effects in his articles and podcasts.

Shipping a commercial mac desktop app in Python, we do exactly as described in the other answers; protect yourself by law with a decent EULA, not by obfuscating.
We have never had any troubles with people reverse engineering our code. And if we do, I feel confident we can take legal action. So yes, it's a fact of life. But one that is not too hard to live with. Just get a decent lawyer that writes a decent EULA.

The word you're looking for is obfuscate. A quick google reveals:
http://www.lysator.liu.se/~astrand/projects/pyobfuscate/
but:
a) If copyright infringement becomes a problem, then the law is on your side (as long as you include the appropriate copyright notices in all files).
b) It's also possible to make a profit on open source applications if you're clever about it.
c) If you want your Intellectual Property to be truly secure, then the only answer is to not let anyone have it in the first place: Write your application as a web app, (I recommend using django) and only your web hosting provider has access to your code.

py2exe
On windows py2exe is one way of shipping code to end-users, py2exe bundles the python interpreter, the necessary dlls and your code compiled to python bytecode.
Here are the python bytecode instructions to get some clue what it looks like:
http://www.python.org/doc/2.5.2/lib/bytecodes.html
Or you can use dis to disassemble some pyc/pyo files.
So, using py2exe is similar to distributing compiled python (pyc/pyo) files.
Shedskin C++ compiler
The Shedskin compiler compiles a subset of python to C++ which you can compile to native code using any compiler.
pypy
I don't know about PyPy too much. According to their docs Pypy is able to generate C code.

Anyone using Python for embedded projects? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
My company is using Python for a relatively simple embedded project. Is anyone else out there using Python on embedded platforms? Overall it's working well for us, quick to develop apps, quick to debug. I like the overall "conciseness" of the language.
The only real problem I have in day to day work is that the lack of static checking vs a regular compiler can cause problems to be thrown at run-time, e.g. a simple accidental cat of a string and an int in a print statement can bring the whole application down.

We use python in quite a lot of embedded boards with ARM processors and 16 MB of RAM (running linux).
It works really well and is really easy to make custom code quickly - one of the strong points of python.
As for reliability of the code - we try to have 100% test coverage. Writing tests with python is very quick and it gives you a wonderful feeling of confidence. We use twisted trial to run the tests and report on coverage, but there are many other tools available.
In my experience python + tests is more reliable and much quicker to write than any other alternatives.
The only downsides for embedded work is that sometimes python can be slow and sometimes it uses a lot of memory (relatively speaking). This hasn't causes us a show stopping problem yet, and python is quite easy to profile for both speed and memory if it becomes a problem.
pychecker is a very useful too also which will catch quite a lot of common errors.

BTW, see this blog post: "Type inference for Python" for an interesting discussion of type inference and static typing, including links to some Guido van Rossum blog posts describing adding optional static typing to Python.
I agree with Bruce Eckel that one is better off practicing "strong testing" than relying on strong typing. I think that applies equally well to embedded development.

Personally, I've worked on some of the software that runs in the device used by BusRadio. It's an example of an embedded project built on Twisted and Python. The device is an embedded XScale processor running a debian-derived distribution, so it might not meet certain definitions of "embedded", but it is pretty dang small: it fits into the dashboard of a school bus.
There were some interesting issues with using Python with large libraries - the interpreter can take quite a while to start up and load all the code for Twisted on a really slow chip, and some things needed special-case optimizations. However, at no point was the dynamic nature of Python a problem. The software in question certainly wasn't perfect, but at least when using Twisted, a simple programming error will not "bring the whole application down". A traceback will get logged, and processing continues.
So, if you're in an embedded environment sufficiently unconstrained that you can use Python in the first place, it's no different than developing "regular" programs (games, desktop applications, web apps). You don't need static typing there, and you don't need it here either.

At my previous employer I had wanted to spend some time playing with building embedded systems in tinypy, which is a "minimalist implementation of Python in 64k of code". (But I never got to it and I no longer have time.)

Telit makes GSM/GPRS modem modules that include an embedded Python interpreter.
I haven't tried them myself, so I don't know how the Python interpreter compares or differs from a PC implementation, such as which included modules, RAM and ROM memory limits, execution speed, etc.
However, as user foresightyj pointed out in a comment, it appears that they use Python 1.5.x, which is a truly ancient version, and so I would have trouble taking them seriously. Python developers would not enjoy downgrading to such an ancient version without so many modern Python features. I would be concerned about security issues with such an old version.

I've been working on microwave telecommunication equipments based on old and slow powerpc and 16Mb of RAM.
I've been able to port the Python 2.6.1 interpreter on VxWorks, in order to have the command line interpreter available directly from the target shell, or to execute python scripts uploaded to the target flash.
We used those scripts to perform autotest on the target or execute diagnostic procedures.
Here some details on the whole procedure: HOW TO: Port Python to VxWorks

The only real problem I have in day to
day work is that the last of static
checking vs a regular compiler can
cause problems to be thrown at
run-time, e.g. a simple accidental cat
of a string and an int in a print
statement can bring the whole
application down.
Unit tests are your only safety against these things.

Indeed, Python is often used as a 'support language' while you need to write some kind of tests - i.e. I was involved in a project, which (Python based) test framework code base was (is?) almost as big as that of the main product.
Python 'agents' works on QNX, VxWorks - and most problems we have, was to port properly threading and network related parts of our code.
It might be worth to take a look OpenMoko project a lot of embedded development in Python is done there.
Things to watch-out:
- support for Python/C extension module might behave quite strangely depending on platform/OS
- most of embedded platforms offers quite out-dated versions of Python
- finally you will find out that there is a difference between 'proper' embedded software in which every bit counts, and 'modern' embedded software that is performed on >412Mhz XScale CPUs with more thatn 128MB, and then Python just don't match the hardware that you would like to target :(

We use Python here at the university for embedded applications based on the Gumstix hardware platform. Although more capable than traditional embedded systems, we find the mix of small formfactor, low (ish) power consumption and the ease in transferring code between development on desktop machines and the target hardware invaluable.
Python is also a great language to teach the students, and with the Gumstix its great they can get code working on a low power system, rather than the headache and heartbreak that comes with using dedicated languages such as NesC.

My team wrote an embedded software made out of C++ and Python. We decided to write basic classes and heavy computational routines in C++. We wrote logic in Python. Boost libraries as glue. Using boost is never easy, but the results is excellent. Fast and easy to modify. Using python to represent the custom needings, we are able to satisfy customers' needings realtime, changing the code using injection technics. Something really exciting! (ok, I'm a geek ;)
We started prototyping in python but we suddenly realized that it was clearly too slow. So we decided to structure the program in different computational layers, in order to reach the speed requirements. C++ was the best solution.
In order to use python and c++ together we had to keep a strict control on typing.

I worked for a company which used Python on an embedded product based around an Atmel AVR32 and running embedded Linux. The firmware was initially developed on a PC (due to lack of a working hardware prototype), then later moved to the embedded hardware running on the cross-compiled Python interpreter.
The ability to debug and modify source code "live" on the device was a big plus during development, and saved a lot of time. The big disadvantages were speed and memory usage of the Python interpreter.
Following the first release of production firmware we ported critical sections of code over to C/C++. The porting effort was quite straightforward and resulted in an improvement of several orders of magnitude on speed-critical code (as you would expect).
Incidently most of the design and production testing code was written in Python, mainly running inside a test harness on a PC.

In my experience, Python has been traditionally used in desktop environments more than in the embedded field. There are two reasons, related to the fact that Python is interpreted:
C/C++ languages have higher performance than Python (and this is important in embedded systems with a slow microcontroller)
C/C++ languages have more deterministic response times (and this is important in real-time embedded systems controlling something).
Of course, as embedded systems will become faster and time-to-market shorter, Python will be more adopted in the embedded sector.

I have a Python server (using Twisted) and some helper scripts running under XP Embedded, and it's been working great.

Recent developments
MicroPython is a lean and fast implementation of the Python 3 programming language that is optimised to run on a microcontroller.
The European Space Agency (ESA) is funding further development of MicroPython. It is doing so to assess the suitability of the language for space-based applications, in particular for payloads.
WiPy 1.0 & 2.0, LoPy & SiPy are wireless MicroPython platforms sold by Pycom.

Isn't the EVE Online client a showpiece of real-time, high-performance Python?

I'm using a Gatetel GT-HE910 series module which embeds the Telit modem including 3G, GPS, AD, IO and Python 2.7. This is used for a remote data aquisition application. Python is fairy slow on these modules but we only need an update every 15 minutes or in an alarm condition so they work well.
http://www.gatetel.com/#!gt-series/cscb

Blockquote
The only real problem I have in day to day work is that the last of static checking vs a regular compiler can cause problems to be thrown at run-time, e.g. a simple accidental cat of a string and an int in a print statement can bring the whole application down
To me it is a huge deal. Problems you could find at compile time and fix the problem now have to rely at run time. Not knowing the data type and having to write additional function just to check the datatype is hassle. There is no need to do that in C. How would you declare 'volatile' in python?
Blockquote
The only downsides for embedded work is that sometimes python can be slow and sometimes it uses a lot of memory (relatively speaking). This hasn't causes us a show stopping problem yet, and python is quite easy to profile for both speed and memory if it becomes a problem.
This is also huge. For Embedded sytems or RTOS time constraint is very important.
Python is not necessary quick to code. It really depends what language you are comfortable with. Honestly it takes me 1 day to write function and unnecessary object orientation stuff which I can do in 2 hours in C.
Testing is so inconvenient I have to write the code, py_compile, copy pyc in the target then run the program, then python quits complaining variable not defined or type cast error or some petty thing like that.
My suggestion is C toolchain is available for any target. C is fast, hardware oriented,challenging and fun. Stick with C for Embedded systems. No need to install configure silly python packages just to run it.