I'm looking to call Python code from Ruby. There are a few existing tools to do this and a few questions on this site recommending http://rubypython.rubyforge.org/, which works by embedding the Python interpreter in Ruby. I'm working on an app that uses libraries unique to Python (namely graph-tool, which I have reasons for using over, say RGL), but the final project is in Rails so having Ruby code do the controlling work would be ideal. I want it to be speedy so I'm using PyPy. Is there a way to get the PyPy interpreter embedded in Ruby code, or to make the Python interpreter in rubypython run PyPy?
No. Well, not without a lot of work.
First, RubyPython doesn't really include an embedded Python interpreter; it just wraps the interpreter at runtime. As shown in the docs, you can run it with any Python you want, e.g.:
>> RubyPython.start(:python_exe => "python2.6")
So, what happens when you try?
>> RubyPython.start(:python_exe => "/usr/local/bin/pypy")
RubyPython::InvalidInterpreter: An invalid interpreter was specified.
from /Library/Ruby/Gems/1.8/gems/rubypython-0.6.3/lib/rubypython.rb:67:in `start'
from /Library/Ruby/Gems/1.8/gems/rubypython-0.6.3/lib/rubypython/python.rb:10:in `synchronize'
from /Library/Ruby/Gems/1.8/gems/rubypython-0.6.3/lib/rubypython/python.rb:10:in `synchronize'
from /Library/Ruby/Gems/1.8/gems/rubypython-0.6.3/lib/rubypython.rb:54:in `start'
from (irb):4
Unfortunately, it requires CPython 2.4-2.7. It doesn't work with CPython 3.x, PyPy, Jython, etc. Again, from the docs:
RubyPython has been tested with the C-based Python interpreter (cpython), versions 2.4 through 2.7. Work is planned to enable Python 3 support, but has not yet been started. If you’re interested in helping us enable Python 3 support, please let us know.
Without looking at the code, I'm guessing rubypython is using rubyffi to either:
* Wrap the CPython embedding APIs, or
* Directly call CPython VM internals via its dll/so/dylib exports.
If it's the former, the project might be doable, but still a lot of work. PyPy doesn't support CPython's embedding APIs. If it had its own embedded APIs, you could potentially rewrite rubypython's lower level to wrap those instead, and leave the higher-level code alone. But embedding PyPy at all is still a work in progress, (See http://mail.python.org/pipermail/pypy-dev/2012-March/009661.html for the state of affairs 6 months ago.) So, you'd need to first help get PyPy embedding ready for prime time and stable, and then port the lower level of rubypython to use the different APIs.
If it's the latter, you're pretty much SOL. PyPy will never support the CPython internals, and much of what's internal for CPython is actually written in RPython or Python and then compiled for PyPy, so it's not even possible in principle. You'd have to drastically rewrite all of rubypython to find some way to make it work, instead of just porting the lower level.
One alternative is to port Ruby to RPython and use PyPy to build a Ruby interpreter and a Python interpreter that can talk to each other at a higher level; then, writing something like rubypython for PyRuby and PyPy would be trivial. But that first step is a doozy.
Related
This is an obvious question, that I haven't been able to find a concrete answer to.
Is the Python Byte-Code and Python Code itself interpreter independent,
Meaning by this, that If I take a CPython, PyPy, Jython, IronPython, Skulpt, etc, Interpreter, and I attempt to run, the same piece of code in python or bytecode, will it run correctly? (provided that they implement the same language version, and use modules strictly written in Python or standard modules)
If so, is there is a benchmark, or place where I can compare performance comparison from many interpreters?
I've been playing for a while with CPython, and now I want to explore new interpreters.
And also a side question, What are the uses for the others implementations of python?
Skulpt I get it, browsers, but the rest? Is there a specific industry or application that requires a different interpreter (which)?
From https://docs.python.org/3/library/dis.html#module-dis
Bytecode is an implementation detail of the CPython interpreter. No
guarantees are made that bytecode will not be added, removed, or
changed between versions of Python. Use of this module should not be
considered to work across Python VMs or Python releases.
On the other hand, Jython "consists of a compiler to compile Python source code down to Java bytecodes which can run directly on a JVM" and IronPython compiles to CIL to run on the .NET VM.
The purpose is to better integrate into your programming environment. CPython allows you to write C extensions, but this is not necessarily true of other implementations. Jython allows you to interact with Java code. I'm sure similar is true of IronPython.
If so, is there is a benchmark, or place where I can compare
performance comparison from many interpreters?
speed.pypy.org compares pypy to cpython
Is there a difference in python programming while using just python and while using pypy compiler? I wanted to try using pypy so that my program execution time becomes faster. Does all the syntax that work in python works in pypy too? If there is no difference, can you tell me how can i install pypy on debian lunux and some usage examples on pypy? Google does not contain much info on pypy other than its description.
From the pypy features page:
PyPy 1.9 implements Python 2.7.2 and runs on Intel x86 (IA-32) and
x86_64 platforms, with ARM and PPC being underway. It supports all of
the core language, passing the Python test suite.
This means that pretty much any code that you've written in Python 2.7 will work. The only exceptions worth mentioning are some python extensions written in C, such as numpy.
Installation should be fairly easy, you can download a linux binary from here. Then simply extract the interpreter. From this point, you can run your python programs similar to how you would run them with the normal python interpreter.
At the command line, instead of:
python my_program.py
Use:
path/to/where/you/installed/pypy my_program.py
For examples of how/why you might want to use pypy, check out this video from PyCon 2012.
pypy is a compliant alternative implementation of the python language. This means that there are few (intentional) differences. One of the few differences is pypy does not use reference counting. This means, for instance, you have to manually close your files, they will not be automatically closed when your file variable goes out of scope as in CPython.
I am currently looking for a possible way to integrate my C++/C# application with some of my Python scripts. At this point, IronPython seems like the way to go.
However, before proceeding, I would like to ask the following:
How stable is IronPython right now? Is it ready for production use? Are there any known major quirks/bugs?
What is the future of IronPython? Will it be maintained for bug fixes? Will there be new versions?
I am particularly interested in using IronPython to run a Python web framework such as Django or Web2py. I am very well aware that current Python web frameworks don't play very well with it. Therefore any insights on the future of IronPython's web framework support would be much appreciated as well.
To answer your second question, yes, IronPython will be developed in the future. Right now, there is a "language change moratorium" on CPython, the main branch of Python (see PEP 3003. The Python folks want CPython, Jython, and other branches of Python development to catch up with CPython, and they've been doing just that. If all goes as planned, by the time the moratorium is over, IronPython and others will be up to speed and will have implementations that follow the syntax and features of Python 3.x. Also, since IronPython is backed by Microsoft and is a key part of their Dynamic Language Runtime (whatever that is), it's unlikely to get cancelled.
Right now, IronPython is making good progress. According to their svn, code is being changed fairly regularly (1 check-in every other day or so). A Python 2.7 compatible implementation is in the works, and the alpha was released July 16 (so IronPython 2.7 can be expected in the near future).
As for the stability of the interpreter, it seems rather stable. I haven't used IronPython extensively, but the 2.6.1 release behaves almost exactly like the CPython 2.6 interpreter, disregarding standard library.
A lot of the extensions for CPython don't work with IronPython. So, if you want to use Django or something like that, it's probably not smart to use IronPython because it isn't really cross-platform, doesn't work with some frameworks, and it performs worse than CPython. The real advantage to IronPython is access to everything that .NET has to offer, including ASP.NET (a web framework) and Silverlight.
If you want to use .NET, IronPython is the best route as far as scripting goes.
I'm starting to study Python, and for now I like it very much. But, if you could just answer a few questions for me, which have been troubling me, and I can't find any definite answers to them:
What is the relationship between Python's C implementation (main version from python.org) and IronPython, in terms of language compatibility ? Is it the same language, and do I by learning one, will be able to smoothly cross to another, or is it Java to JavaScript ?
What is the current status to IronPython's libraries ? How much does it lags behind CPython libraries ? I'm mostly interested in numpy/scipy and f2py. Are they available to IronPython ?
What would be the best way to access VB from Python and the other way back (connecting some python libraries to Excel's VBA, to be exact) ?
1) IronPython and CPython share nearly identical language syntax. There is very little difference between them. Transitioning should be trivial.
2) The libraries in IronPython are very different than CPython. The Python libraries are a fair bit behind - quite a few of the CPython-accessible libraries will not work (currently) under IronPython. However, IronPython has clean, direct access to the entire .NET Framework, which means that it has one of the most extensive libraries natively accessible to it, so in many ways, it's far ahead of CPython. Some of the numpy/scipy libraries do not work in IronPython, but due to the .NET implementation, some of the functionality is not necessary, since the perf. characteristics are different.
3) Accessing Excel VBA is going to be easier using IronPython, if you're doing it from VBA. If you're trying to automate excel, IronPython is still easier, since you have access to the Execl Primary Interop Assemblies, and can directly automate it using the same libraries as C# and VB.NET.
What is the relationship between
Python's C implementation (main
version from python.org) and
IronPython, in terms of language
compatibility ? Is it the same
language, and do I by learning one,
will be able to smoothly cross to
another, or is it Java to JavaScript ?
Same language (at 2.5 level for now -- IronPython's not 2.6 yet AFAIK).
What is the current status to
IronPython's libraries ? How much does
it lags behind CPython libraries ? I'm
mostly interested in numpy/scipy and
f2py. Are they available to IronPython
?
Standard libraries are in a great state in today's IronPython, huge third-party extensions like the ones you mention far from it. numpy's starting to get feasible thanks to ironclad, but not production-level as numpy is from IronPython (as witnessed by the 0.5 version number for ironclad, &c). scipy is huge and sprawling and chock full of C-coded and Fortran-coded extensions: I have no first-hand experience but I suspect less than half will even run, much less run flawlessly, under any implementation except CPython.
What would be the best way to access
VB from Python and the other way back
(connecting some python libraries to
Excel's VBA, to be exact) ?
IronPython should make it easier via .NET approaches, but CPython is not that far via COM implementation in win32all.
Last but not least, by all means check out the book IronPython in Action -- as I say every time I recommend it, I'm biased (by having been a tech reviewer for it AND by friendship with one author) but I think it's objectively the best intro to Python for .NET developers AND at the same time the best intro to .NET for Pythonistas.
If you need all of scipy (WOW, but that's some "Renaissance Man" computational scientist!-), CPython is really the only real option today. I'm sure other large extensions (PyQt, say, or Mayavi) are in a similar state. For deep integration to today's Windows, however, I think IronPython may have an edge. For general-purpose uses, CPython may be better (esp. thanks to the many new features in 2.6), unless you're really keen to use many cores to the hilt within a single process, in which case IronPython (which is GIL-less) may prove advantageous again.
One way or another (or even on the JVM via Jython, or in peculiar environments via PyPy) Python is surely an awesome language, whatever implementation(s) you pick for a given application!-) Note that you don't need to stick with ONE implementation (though you should probably pick one VERSION -- 2.5 for maximal compatibility with IronPython, Jython, Google App Engine, etc; 2.6 if you don't care about any deployment options except "CPython on a machine under my own sole or virtual control";-).
IronPython version 2.0.2, the current release, supports Python 2.5 syntax. With the next release, 2.6, which is expected sometime over the next month or so (though I'm not sure the team have set a hard release date; here's the beta), they will support Python 2.6 syntax. So, less Java to JavaScript and more Java to J# :-)
All of the libraries that are themselves written in Python work pretty much perfectly. The ones that are written in C are more problematic; there is an open source project called Ironclad (full disclosure: developed and supported by my company), currently at version 0.8.5, which supports numpy pretty well but doesn't cover all of scipy. I don't think we've tested it with f2py. (The version of Ironclad mentioned below by Alex Martelli is over a year old, please avoid that one!)
Calling regular VB.NET from IronPython is pretty easy -- you can instantiate .NET classes and call methods (static or instance) really easily. Calling existing VBA code might be trickier; there are open source projects called Pyinex and XLW that you might want to take a look at. Alternatively, if you want a spreadsheet you can code in Python, then there's always Resolver One (another one from my company... :-)
1) The language implemented by CPython and IronPython are the same, or at most a version or two apart. This is nothing like the situation with Java and Javascript, which are two completely different languages given similar names by some bone-headed marketing decision.
2) 3rd-party libraries implemented in C (such as numpy) will have to be evaluated carefully. IronPython has a facility to execute C extensions (I forget the name), but there are many pitfalls, so you need to check with each library's maintainer
3) I have no idea.
CPython is implemented by C for corresponding platform, such as Windows, Linux or Unix; IronPython is implemented by C# and Windows .Net Framework, so it can only run on Windows Platform with .Net Framework.
For gramma, both are same. But we cannot say they are one same language. IronPython can use .Net Framework essily when you develop on windows platform.
By far, July 21, 2009 - IronPython 2.0.2, our latest stable release of IronPython, was released. you can refer to http://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython.
You can access VB with .Net Framework function by IronPython. So, if you want to master IronPython, you'd better learn more .Net Framework.
How does IronPython stack up to the default Windows implementation of Python from python.org? If I am learning Python, will I be learning a subtley different language with IronPython, and what libraries would I be doing without?
Are there, alternatively, any pros to IronPython (not including .NET IL compiled classes) that would make it more attractive an option?
There are a number of important differences:
Interoperability with other .NET languages. You can use other .NET libraries from an IronPython application, or use IronPython from a C# application, for example. This interoperability is increasing, with a movement toward greater support for dynamic types in .NET 4.0. For a lot of detail on this, see these two presentations at PDC 2008.
Better concurrency/multi-core support, due to lack of a GIL. (Note that the GIL doesn't inhibit threading on a single-core machine---it only limits performance on multi-core machines.)
Limited ability to consume Python C extensions. The Ironclad project is making significant strides toward improving this---they've nearly gotten Numpy working!
Less cross-platform support; basically, you've got the CLR and Mono. Mono is impressive, though, and runs on many platforms---and they've got an implementation of Silverlight, called Moonlight.
Reports of improved performance, although I have not looked into this carefully.
Feature lag: since CPython is the reference Python implementation, it has the "latest and greatest" Python features, whereas IronPython necessarily lags behind. Many people do not find this to be a problem.
There are some subtle differences in how you write your code, but the biggest difference is in the libraries you have available.
With IronPython, you have all the .Net libraries available, but at the expense of some of the "normal" python libraries that haven't been ported to the .Net VM I think.
Basically, you should expect the syntax and the idioms to be the same, but a script written for IronPython wont run if you try giving it to the "regular" Python interpreter. The other way around is probably more likely, but there too you will find differences I think.
Well, it's generally faster.
Can't use modules, and only has a subset of the library.
Here's a list of differences.
See the blog post IronPython is a one-way gate. It summarizes some things I've learned about IronPython from asking questions on StackOverflow.
Python is Python, the only difference is that IronPython was designed to run on the CLR (.NET Framework), and as such, can inter-operate and consume .NET assemblies written in other .NET languages. So if your platform is Windows and you also use .NET or your company does then should consider IronPython.
One of the pros of IronPython is that, unlike CPython, IronPython doesn't use the Global Interpreter Lock, thus making threading more effective.
In the standard Python implementation, threads grab the GIL on each object access. This limits parallel execution, which matters especially if you expect to fully utilize multiple CPUs.
Pro: You can run IronPython in a browser if SilverLight is installed.
It also depends on whether you want your code to work on Linux. Dunno if IronPython will work on anything beside windows platforms.