A person for which I am working under constantly has reserves about me updating programs that were written in python 2.5 and 2.7 to python 3.3. I work in bioninformatics and a lot of the python code I am trying to work with is pre 3.0 and while I have a linux that runs 2.7 it on a virtual machine, on my main machine I am already at python 3.3 and develop my programs with it. I understand that if the program has heavy reliance on the libraries then there could be some compatibility issues but besides that I don't see why I can't just spend a little time upgrading it. I feel I should clarify that most of this programs are not much more then the a few hundred lines of code.
What I really want to know is;
Are there some real differences between the two version that might cause a program to run different?
Is it possible to just simply update to 3.3 and clean it up by just changing the over things like print to print() or urlib2 to the update urlib?
It is quite easy to write new code that works both python 2.6+ and 3.3+. Use the following at the beginning of each file
from __future__ import division, absolute_import \
unicode_literals, print_function
And then know the differences; you can use the six to ease the porting. However be warned that many needed modules might be written for 2.x only (I suspect more so in the field of bioinformatics than in general programming) and it might not only be your code that needs porting. I'd expect you to still need the 2.x for some time. I'd advice against using the 2to3, to me the right way to proceed is to write code that works on both python 2.x and 3.x out of box - it eases the development and testing.
As for the old code, be warned that the str/unicode confusion will hit you hard many times - some python 2 str should be bytes, and some should be python 3 str.
I don't know if SO is the best place to ask this question, but this link here lists every difference between Python 2.x and 3.x: http://docs.python.org/3.0/whatsnew/3.0.html
After reading it, you can easily tell what needs to be done to your 2.x programs to bring them into 3.x.
As an aside: are you familiar with 2to3?
The list of things that can 'go wrong' is basically too large for this Q+A format. Here is what the docs have to say about porting code. The most vital point is to have well-written tests, but in academia I realize that's not always reality. In lieu of that, see the 'gotchas' section for the most common ones that automation will not/can not pick up.
The two that I see most in academic code are reliance on integer division and opening files in non-binary mode. Without knowing specifics, I'd say to watch out for those.
Related
I have written a Python script that I want to be able to run separately from the OS. As a result, I want it to be able to run as pretty much its own OS. Everything is shell based and there are no GUI components. How would I go about making a bootable version of this Python script, and is it even possible? Or would i have to put a bare-bones OS with the Python script installed as an addition?
For all practical purposes, the answer is no, it's not possible. The bare-bones os with a python install is your best option.
It is theoretically possible to implement a stand-alone Python interpreter, which runs as the OS of a system. That has been done with several interpreter languages. First of all, BASIC, then APL, Forth, Smalltalk, Java, and most likely some I'm not aware of. There is no reason why this couldn't be done with Python, it merely needs a bit of implementation work.
To give a rough estimation of the amount of work: a caffeine-driven expert coder takes about two days to implement a stand alone Forth interpreter, doubling as operating system, from scratch, on a platform (s)he's familiar with. By then the system will be far from finished or complete, but it will compile source, and allow extending itself with the results of that compilation. Other languages would differ. In case of Java, only porting an assembly implementation from one to an entirely different CPU took a team of 4 people about 2 weeks, while the back end of Java, the JVM, has a considerable resemblance to Forth. I'm not knowledgeable enough about Python inner workings to be able to give a meaningful estimate of the effort to implement a stand alone Python interpreter, but it could be roughly comparable to the effort of implementing a Java VM.
I know that when it launched, a lot of people didn't think it would be picked up very easily. I was wondering if thats changed at all. Do many of the major modules and libraries support Python 3 yet? Is it gaining acceptance or are people mostly sticking with 2.x?
If you go to python.org and click on Download, there's this message:
If you don't know which version to use, start with Python 2.7; more existing third party software is compatible with Python 2 than Python 3 right now.
I think that's the case. Python 2 is still more useful.
Example: you don't have WSGI working on python3 yet.
You could always write code in python2, and then port it to python3 later if all libraries are already ported, by running 2to3. The opposite is not true: If you write python3 code and later want to use a library only on python 2, you're out of luck :( I think that's why python.org recommends python2 even for new projects, by suggesting you start with python 2.7.
Using python3 also doesn't buy you anything right now - there are some small niceties, but losing full 3rd party support far outweights the tiny gains of the small syntax changes and module relocations.
Don't get me wrong -- I love python3 and all the new nice things it can do, it's beautiful and much closer to the way I think python should be.
The thing is that using it just doesn't pay back, yet. You need either more benefits or less downfalls to make it worth the trouble.
The answer to all your questions are "Yes". :)
Do many of the major modules and libraries support Python 3 yet?
Yes.
Is it gaining acceptance
Yes.
or are people mostly sticking with 2.x?
Yes.
Your question is a bit fuzzy, namely. If you are wondering if you should use Python 2 or Python 3, then the answer is "Use Python 2". There are many who uses Python 3, but they wouldn't ask the question. :-)
I'd expect that to change in 1-2 years, when the answer will be "well, it depends". And in around 3 years the answer will be "Are you using a library/framework that hasn't been ported?"
G'day,
I'm wanting to go back to Python after not using it for a while and I saw this question "Python Version for a Newbie" while wondering about getting back into Python 2.6 or Python 3.
Almost all of the questions' answers were along the lines that most of the code out there, libraries, legacy systems, etc., is 2.5 or 2.6 rather than 3 so start with 2.x now and then head towards 3 later on.
Given that the question and all answers date from early December 2008 I was wondering is this still the case?
Should someone who wants to get back into Python maybe start off with 2.6 and then head towards 3 later on?
Yes. Virtually all live production systems will use 2.5/2.6 for a long time yet. There's no point learning 3.0, only to have to downgrade it because your host doesn't support it.
95% of what you will learn in 2.5/2.6 is applicable to 3 anyway.
Depends on the amount of libraries you're going to use.
Raw Python, or all libs are available for Py3k - go for it without any doubts.
Python code distributed as standalone app (using PyInstaller), relying on some GUI lib, XML-lib, win32api etc - double check if all libs are available at least as betas for Py3k. Chances are still quite high that some older lib is not available for Python 3.x, and either you port it by yourself to new Python version, or you switch to some other lib or - stick to Python 2.6 for a while.
If you want to use only standard library then try Python 3.1. If you want to use others libraries/frameworks then they dictate the version to use. For example web2py framework will work best on 2.5.
I would say that Python 2.4 is the safest to learn, but the changes from 2.4->2.5->2.6 make some small progress towards Python 3.x, even if they may never make it (if I recall there will be some more steps?).
Python 3.1 can be used if you own a dedicated server and intend to build your own applications from the ground up. WSGI does support this, but I wouldn't recommend it.
As has already been said, I would learn the Python 2.5 or Python 2.6 style, but I would make a few changes.
Look at the Python 3 style regarding brackets.
e.g. The print function in 2.x has always been just
print "Hello World"
Where as in 3.x you need to enclose it
print("Hello World")
This is probably a good practice to pick up on, but things like Exceptions will cause issues if you use 3.x in 2.x. I know it's probably a bit confusing, but if you make sure you wrap your functions (additional brackets shouldn't really hurt most things) so that nothing is bare (bare like the first code snippet above), then it'll help with the transition.
The problem is, if you started with 2.4 or more it is better if you start from there, so you'll get on track faster, after some time when you feel comfortable with you code you can try 3.0 and find out what did they change and learn the new style.
I for once still code in 2.6 style and follow those guidelines, still haven't seen the changes in 3.0
This question already has answers here:
How are you planning on handling the migration to Python 3?
(7 answers)
Closed 9 years ago.
So with the final releases of Python 3.0 (and now 3.1), a lot of people are facing the worry of how to upgrade without losing half their codebase due to backwards incompatibility.
What are people's best tips for avoiding the many pitfalls that will almost-inevitably result from switching to the next-generation of python?
Probably a good place to start is "use 2to3 to convert your python 2.x code to 3.x" :-)
First, this question is very similar to How are you planning on handling the migration to Python 3?. Check the answers there.
There is also a section in the Python Wiki about porting applications to Python 3.x
The Release Notes for python 3.0 contains a section about porting. I'm quoting the tips there:
(Prerequisite:) Start with excellent test coverage.
Port to Python 2.6. This should be no more work than the average por
from Python 2.x to Python 2.(x+1).
Make sure all your tests pass.
(Still using 2.6:) Turn on the -3 command line switch. This enables warnings about features that will be
removed (or change) in 3.0. Run your
test suite again, and fix code that
you get warnings about until there are
no warnings left, and all your tests
still pass.
Run the 2to3 source-to-source translator over your source code tree.
(See 2to3 - Automated Python 2 to 3
code translation for more on this
tool.) Run the result of the
translation under Python 3.0. Manually
fix up any remaining issues, fixing
problems until all tests pass again.
It is not recommended to try to write
source code that runs unchanged under
both Python 2.6 and 3.0; you’d have to
use a very contorted coding style,
e.g. avoiding print statements,
metaclasses, and much more. If you are
maintaining a library that needs to
support both Python 2.6 and Python
3.0, the best approach is to modify step 3 above by editing the 2.6
version of the source code and running
the 2to3 translator again, rather than
editing the 3.0 version of the source
code.
I write a free book about this. You can read it here:
http://python3porting.com/
In short:
Make sure all your third party libraries are available for Python 3.
Prepare your code by removing common ambiguities:
Use // if you really want integer division.
Make sure you flag binary files with the 'b' flag when you open them, to clearly
indicate if you mean the data to be binary or not.
The higher your test coverage is, the better.
Make sure it runs without warnings under "Python 2.7 -3".
Now run 2to3.
Fix any bugs.
That's it, more or less.
Without a really compelling reason to upgrade, I would stick with what works. I looked at upgrading the scripts I use daily and it was too much work for no benefit that I could see.
"If it ain't broke, don't fix it!"
Suppose I've developed a general-purpose end user utility written in Python. Previously, I had just one version available which was suitable for Python later than version 2.3 or so. It was sufficient to say, "download Python if you need to, then run this script". There was just one version of the script in source control (I'm using Git) to keep track of.
With Python 3, this is no longer necessarily true. For the foreseeable future, I will need to simultaneously develop two different versions, one suitable for Python 2.x and one suitable for Python 3.x. From a development perspective, I can think of a few options:
Maintain two different scripts in the same branch, making improvements to both simultaneously.
Maintain two separate branches, and merge common changes back and forth as development proceeds.
Maintain just one version of the script, plus check in a patch file that converts the script from one version to the other. When enough changes have been made that the patch no longer applies cleanly, resolve the conflicts and create a new patch.
I am currently leaning toward option 3, as the first two would involve a lot of error-prone tedium. But option 3 seems messy and my source control system is supposed to be managing patches for me.
For distribution packaging, there are more options to choose from:
Offer two different download packages, one suitable for Python 2 and one suitable for Python 3 (the user will have to know to download the correct one for whatever version of Python they have).
Offer one download package, with two different scripts inside (and then the user has to know to run the correct one).
One download package with two version-specific scripts, and a small stub loader that can run in both Python versions, that runs the correct script for the Python version installed.
Again I am currently leaning toward option 3 here, although I haven't tried to develop such a stub loader yet.
Any other ideas?
Edit: my original answer was based on the state of 2009, with Python 2.6 and 3.0 as the current versions. Now, with Python 2.7 and 3.3, there are other options. In particular, it is now quite feasible to use a single code base for Python 2 and Python 3.
See Porting Python 2 Code to Python 3
Original answer:
The official recommendation says:
For porting existing Python 2.5 or 2.6
source code to Python 3.0, the best
strategy is the following:
(Prerequisite:) Start with excellent test coverage.
Port to Python 2.6. This should be no more work than the average port
from Python 2.x to Python 2.(x+1).
Make sure all your tests pass.
(Still using 2.6:) Turn on the -3 command line switch. This enables
warnings about features that will be
removed (or change) in 3.0. Run your
test suite again, and fix code that
you get warnings about until there are
no warnings left, and all your tests
still pass.
Run the 2to3 source-to-source translator over your source code tree.
(See 2to3 - Automated Python 2 to 3
code translation for more on this
tool.) Run the result of the
translation under Python 3.0. Manually
fix up any remaining issues, fixing
problems until all tests pass again.
It is not recommended to try to write
source code that runs unchanged under
both Python 2.6 and 3.0; you’d have to
use a very contorted coding style,
e.g. avoiding print statements,
metaclasses, and much more. If you are
maintaining a library that needs to
support both Python 2.6 and Python
3.0, the best approach is to modify step 3 above by editing the 2.6
version of the source code and running
the 2to3 translator again, rather than
editing the 3.0 version of the source
code.
Ideally, you would end up with a single version, that is 2.6 compatible and can be translated to 3.0 using 2to3. In practice, you might not be able to achieve this goal completely. So you might need some manual modifications to get it to work under 3.0.
I would maintain these modifications in a branch, like your option 2. However, rather than maintaining the final 3.0-compatible version in this branch, I would consider to apply the manual modifications before the 2to3 translations, and put this modified 2.6 code into your branch. The advantage of this method would be that the difference between this branch and the 2.6 trunk would be rather small, and would only consist of manual changes, not the changes made by 2to3. This way, the separate branches should be easier to maintain and merge, and you should be able to benefit from future improvements in 2to3.
Alternatively, take a bit of a "wait and see" approach. Proceed with your porting only so far as you can go with a single 2.6 version plus 2to3 translation, and postpone the remaining manual modification until you really need a 3.0 version. Maybe by this time, you don't need any manual tweaks anymore...
For developement, option 3 is too cumbersome. Maintaining two branches is the easiest way although the way to do that will vary between VCSes. Many DVCS will be happier with separate repos (with a common ancestry to help merging) and centralized VCS will probably easier to work with with two branches. Option 1 is possible but you may miss something to merge and a bit more error-prone IMO.
For distribution, I'd use option 3 as well if possible. All 3 options are valid anyway and I have seen variations on these models from times to times.
I don't think I'd take this path at all. It's painful whichever way you look at it. Really, unless there's strong commercial interest in keeping both versions simultaneously, this is more headache than gain.
I think it makes more sense to just keep developing for 2.x for now, at least for a few months, up to a year. At some point in time it will be just time to declare on a final, stable version for 2.x and develop the next ones for 3.x+
For example, I won't switch to 3.x until some of the major frameworks go that way: PyQt, matplotlib, numpy, and some others. And I don't really mind if at some point they stop 2.x support and just start developing for 3.x, because I'll know that in a short time I'll be able to switch to 3.x too.
I would start by migrating to 2.6, which is very close to python 3.0. You might even want to wait for 2.7, which will be even closer to python 3.0.
And then, once you have migrated to 2.6 (or 2.7), I suggest you simply keep just one version of the script, with things like "if PY3K:... else:..." in the rare places where it will be mandatory. Of course it's not the kind of code we developers like to write, but then you don't have to worry about managing multiple scripts or branches or patches or distributions, which will be a nightmare.
Whatever you choose, make sure you have thorough tests with 100% code coverage.
Good luck!
Whichever option for development is chosen, most potential issues could be alleviated with thorough unit testing to ensure that the two versions produce matching output. That said, option 2 seems most natural to me: applying changes from one source tree to another source tree is a task (most) version control systems were designed for--why not take advantages of the tools they provide to ease this.
For development, it is difficult to say without 'knowing your audience'. Power Python users would probably appreciate not having to download two copies of your software yet for a more general user-base it should probably 'just work'.