Is there any automated way to test that code is compatible with both Python 2 and 3? I've seen plenty of documentation on how to write code that is compatible with both, but nothing on automatically checking. Basically a kind of linting for compatibility between versions rather than syntax/style.
I have thought of either running tests with both interpreters or running a tool like six or 2to3 and checking that nothing is output; unfortunately, the former requires that you have 100% coverage with your tests and I would assume the latter requires that you have valid Python 2 code and would only pick up issues in compatibility with Python 3.
Is there anything out there that will accomplish this task?
There is no "fool-proof" way of doing this other than running the code on both versions and finding inconsistencies. With that said, CPython2.7 has a -3 flag which (according to the man page) says:
Warn about Python 3.x incompatibilities that 2to3 cannot trivially fix.
As for the case where you have valid python3 code and you want to backport it to python2.x -- You likely don't actually want to do this. python3.x is the future. This is likely to be a very painful problem in the general case. A lot of the reason to start using python3.x is because then you gain access to all sorts of cool new features. Trying to re-write code that is already relying on cool new features is frequently going to be very difficult. Your much better off trying to upgrade python2.x packages to work on python3.x than doing things the other way around.
Could you not look at the output from 2to3 to see if any code changes may be necessary ?
Related
I've read in few places that generally, Python doesn't provide backward compatibility, which means that any newer version of Python may break code that worked fine for earlier versions. If so, what is my way as a developer to know what versions of Python can execute my code successfully? Is there any set of rules/guarantees regarding this? Or should I just tell my users: Just run this with Python 3.8 (for example) - no more no less...?
99% of the time, if it works on Python 3.x, it'll work on 3.y where y >= x. Enabling warnings when running your code on the older version should pop DeprecationWarnings when you use a feature that's deprecated (and therefore likely to change/be removed in later Python versions). Aside from that, you can read the What's New docs for each version between the known good version and the later versions, in particular the Deprecated and Removed sections of each.
Beyond that, the only solution is good unit and component tests (you are using those, right? 😉) that you rerun on newer releases to verify stuff still works & behavior doesn't change.
According to PEP387, section "Making Incompatible Changes", before incompatible changes are made, a deprecation warning should appear in at least two minor Python versions of the same major version, or one minor version in an older major version. After that, it's a free game, in principle. This made me cringe with regards to safety. Who knows if people run airplanes on Python and if they don't always read the python-dev list. So if you have something that passes 100% coverage unit tests without deprecation warnings, your code should be safe for the next two minor releases.
You can avoid this issue and many others by containerizing your deployments.
tox is great for running unit tests against multiple Python versions. That’s useful for at least 2 major cases:
You want to ensure compatibility for a certain set of Python versions, say 3.7+, and to be told if you make any breaking changes.
You don’t really know what versions your code supports, but want to establish a baseline of supported versions for future work.
I don’t use it for internal projects where I can control over the environment where my code will be running. It’s lovely for people publishing apps or libraries to PyPI, though.
I am trying to incorporate an autocomplete feature in Vim. I have used the omnicompletion of Vim as well as Pydiction. But in both the cases this message pops up:
Omni completion (^O^N^P) Pattern not found
I have tried Vim's Omnicompletion with Python just doesn't work. But this is not helping. Is there a workaround?
Try Jedi! There's a Vim plugin at https://github.com/davidhalter/jedi-vim.
It works just much better than anything else for Python in Vim. It even has support for renaming, goto, etc. The best part is probably that it really tries to understand your code (decorators, generators, etc. Just look at the feature list).
As pointed out by in the comments, this answers is outdated. youcompleteme now supports python3 and jedi-vim no longer breaks the undo history.
Original answer below.
AFAIK there are three options, each with its disadvantages:
youcompleteme: unfriendly to install, but works nice if you manage to get it working. However python3 is not supported.
jedi-vim: coolest name, but breaks your undo history.
python-mode does a lot more the autocomplete: folding, syntax checking, highlighting. Personally I prefer scripts that do 1 thing well, as they are easier to manage (and replace). Differently from the two other options, it uses rope instead of jedi for autocompletion.
Python 3 and undo history (gundo!) are a must for me, so options 1 and 2 are out.
This can be a good option if you want python completion as well as other languages. https://github.com/Valloric/YouCompleteMe
The python completion is jedi based same as jedi-vim.
I tried pydiction (didn't work for me) and the normal omnicompletion (too limited). I looked into Jedi as suggested but found it too complex to set up. I found python-mode, which in the end satisfied my needs. Thanks #klen.
I found a good choice to be coc.nvim with the python language server.
It takes a bit of effort to set up. I got frustrated with jedi-vim, because it would always freeze vim for a bit when completing. coc.nvim doesn't do it because it's asyncronous, meaning that . It also gives you linting for your code. It supports many other languages and is highly configurable.
The python language server uses jedi so you get the same completion as you would get from jedi.
Use install coc.nvim package and CocInstall coc-pyright
if you are using vim-plug:
Plug 'neoclide/coc.nvim'
:PlugInstall
CocInstall coc-pyright
I ran into this on my Mac using the MacPorts vim with +python. Problem was that the MacPorts vim will only bind to python 2.5 with +python, while my extensions were installed under python 2.7. Installing the extensions using pip-2.5 solved it.
G'day,
I'm wanting to go back to Python after not using it for a while and I saw this question "Python Version for a Newbie" while wondering about getting back into Python 2.6 or Python 3.
Almost all of the questions' answers were along the lines that most of the code out there, libraries, legacy systems, etc., is 2.5 or 2.6 rather than 3 so start with 2.x now and then head towards 3 later on.
Given that the question and all answers date from early December 2008 I was wondering is this still the case?
Should someone who wants to get back into Python maybe start off with 2.6 and then head towards 3 later on?
Yes. Virtually all live production systems will use 2.5/2.6 for a long time yet. There's no point learning 3.0, only to have to downgrade it because your host doesn't support it.
95% of what you will learn in 2.5/2.6 is applicable to 3 anyway.
Depends on the amount of libraries you're going to use.
Raw Python, or all libs are available for Py3k - go for it without any doubts.
Python code distributed as standalone app (using PyInstaller), relying on some GUI lib, XML-lib, win32api etc - double check if all libs are available at least as betas for Py3k. Chances are still quite high that some older lib is not available for Python 3.x, and either you port it by yourself to new Python version, or you switch to some other lib or - stick to Python 2.6 for a while.
If you want to use only standard library then try Python 3.1. If you want to use others libraries/frameworks then they dictate the version to use. For example web2py framework will work best on 2.5.
I would say that Python 2.4 is the safest to learn, but the changes from 2.4->2.5->2.6 make some small progress towards Python 3.x, even if they may never make it (if I recall there will be some more steps?).
Python 3.1 can be used if you own a dedicated server and intend to build your own applications from the ground up. WSGI does support this, but I wouldn't recommend it.
As has already been said, I would learn the Python 2.5 or Python 2.6 style, but I would make a few changes.
Look at the Python 3 style regarding brackets.
e.g. The print function in 2.x has always been just
print "Hello World"
Where as in 3.x you need to enclose it
print("Hello World")
This is probably a good practice to pick up on, but things like Exceptions will cause issues if you use 3.x in 2.x. I know it's probably a bit confusing, but if you make sure you wrap your functions (additional brackets shouldn't really hurt most things) so that nothing is bare (bare like the first code snippet above), then it'll help with the transition.
The problem is, if you started with 2.4 or more it is better if you start from there, so you'll get on track faster, after some time when you feel comfortable with you code you can try 3.0 and find out what did they change and learn the new style.
I for once still code in 2.6 style and follow those guidelines, still haven't seen the changes in 3.0
This question already has answers here:
How are you planning on handling the migration to Python 3?
(7 answers)
Closed 9 years ago.
So with the final releases of Python 3.0 (and now 3.1), a lot of people are facing the worry of how to upgrade without losing half their codebase due to backwards incompatibility.
What are people's best tips for avoiding the many pitfalls that will almost-inevitably result from switching to the next-generation of python?
Probably a good place to start is "use 2to3 to convert your python 2.x code to 3.x" :-)
First, this question is very similar to How are you planning on handling the migration to Python 3?. Check the answers there.
There is also a section in the Python Wiki about porting applications to Python 3.x
The Release Notes for python 3.0 contains a section about porting. I'm quoting the tips there:
(Prerequisite:) Start with excellent test coverage.
Port to Python 2.6. This should be no more work than the average por
from Python 2.x to Python 2.(x+1).
Make sure all your tests pass.
(Still using 2.6:) Turn on the -3 command line switch. This enables warnings about features that will be
removed (or change) in 3.0. Run your
test suite again, and fix code that
you get warnings about until there are
no warnings left, and all your tests
still pass.
Run the 2to3 source-to-source translator over your source code tree.
(See 2to3 - Automated Python 2 to 3
code translation for more on this
tool.) Run the result of the
translation under Python 3.0. Manually
fix up any remaining issues, fixing
problems until all tests pass again.
It is not recommended to try to write
source code that runs unchanged under
both Python 2.6 and 3.0; you’d have to
use a very contorted coding style,
e.g. avoiding print statements,
metaclasses, and much more. If you are
maintaining a library that needs to
support both Python 2.6 and Python
3.0, the best approach is to modify step 3 above by editing the 2.6
version of the source code and running
the 2to3 translator again, rather than
editing the 3.0 version of the source
code.
I write a free book about this. You can read it here:
http://python3porting.com/
In short:
Make sure all your third party libraries are available for Python 3.
Prepare your code by removing common ambiguities:
Use // if you really want integer division.
Make sure you flag binary files with the 'b' flag when you open them, to clearly
indicate if you mean the data to be binary or not.
The higher your test coverage is, the better.
Make sure it runs without warnings under "Python 2.7 -3".
Now run 2to3.
Fix any bugs.
That's it, more or less.
Without a really compelling reason to upgrade, I would stick with what works. I looked at upgrading the scripts I use daily and it was too much work for no benefit that I could see.
"If it ain't broke, don't fix it!"
Suppose I've developed a general-purpose end user utility written in Python. Previously, I had just one version available which was suitable for Python later than version 2.3 or so. It was sufficient to say, "download Python if you need to, then run this script". There was just one version of the script in source control (I'm using Git) to keep track of.
With Python 3, this is no longer necessarily true. For the foreseeable future, I will need to simultaneously develop two different versions, one suitable for Python 2.x and one suitable for Python 3.x. From a development perspective, I can think of a few options:
Maintain two different scripts in the same branch, making improvements to both simultaneously.
Maintain two separate branches, and merge common changes back and forth as development proceeds.
Maintain just one version of the script, plus check in a patch file that converts the script from one version to the other. When enough changes have been made that the patch no longer applies cleanly, resolve the conflicts and create a new patch.
I am currently leaning toward option 3, as the first two would involve a lot of error-prone tedium. But option 3 seems messy and my source control system is supposed to be managing patches for me.
For distribution packaging, there are more options to choose from:
Offer two different download packages, one suitable for Python 2 and one suitable for Python 3 (the user will have to know to download the correct one for whatever version of Python they have).
Offer one download package, with two different scripts inside (and then the user has to know to run the correct one).
One download package with two version-specific scripts, and a small stub loader that can run in both Python versions, that runs the correct script for the Python version installed.
Again I am currently leaning toward option 3 here, although I haven't tried to develop such a stub loader yet.
Any other ideas?
Edit: my original answer was based on the state of 2009, with Python 2.6 and 3.0 as the current versions. Now, with Python 2.7 and 3.3, there are other options. In particular, it is now quite feasible to use a single code base for Python 2 and Python 3.
See Porting Python 2 Code to Python 3
Original answer:
The official recommendation says:
For porting existing Python 2.5 or 2.6
source code to Python 3.0, the best
strategy is the following:
(Prerequisite:) Start with excellent test coverage.
Port to Python 2.6. This should be no more work than the average port
from Python 2.x to Python 2.(x+1).
Make sure all your tests pass.
(Still using 2.6:) Turn on the -3 command line switch. This enables
warnings about features that will be
removed (or change) in 3.0. Run your
test suite again, and fix code that
you get warnings about until there are
no warnings left, and all your tests
still pass.
Run the 2to3 source-to-source translator over your source code tree.
(See 2to3 - Automated Python 2 to 3
code translation for more on this
tool.) Run the result of the
translation under Python 3.0. Manually
fix up any remaining issues, fixing
problems until all tests pass again.
It is not recommended to try to write
source code that runs unchanged under
both Python 2.6 and 3.0; you’d have to
use a very contorted coding style,
e.g. avoiding print statements,
metaclasses, and much more. If you are
maintaining a library that needs to
support both Python 2.6 and Python
3.0, the best approach is to modify step 3 above by editing the 2.6
version of the source code and running
the 2to3 translator again, rather than
editing the 3.0 version of the source
code.
Ideally, you would end up with a single version, that is 2.6 compatible and can be translated to 3.0 using 2to3. In practice, you might not be able to achieve this goal completely. So you might need some manual modifications to get it to work under 3.0.
I would maintain these modifications in a branch, like your option 2. However, rather than maintaining the final 3.0-compatible version in this branch, I would consider to apply the manual modifications before the 2to3 translations, and put this modified 2.6 code into your branch. The advantage of this method would be that the difference between this branch and the 2.6 trunk would be rather small, and would only consist of manual changes, not the changes made by 2to3. This way, the separate branches should be easier to maintain and merge, and you should be able to benefit from future improvements in 2to3.
Alternatively, take a bit of a "wait and see" approach. Proceed with your porting only so far as you can go with a single 2.6 version plus 2to3 translation, and postpone the remaining manual modification until you really need a 3.0 version. Maybe by this time, you don't need any manual tweaks anymore...
For developement, option 3 is too cumbersome. Maintaining two branches is the easiest way although the way to do that will vary between VCSes. Many DVCS will be happier with separate repos (with a common ancestry to help merging) and centralized VCS will probably easier to work with with two branches. Option 1 is possible but you may miss something to merge and a bit more error-prone IMO.
For distribution, I'd use option 3 as well if possible. All 3 options are valid anyway and I have seen variations on these models from times to times.
I don't think I'd take this path at all. It's painful whichever way you look at it. Really, unless there's strong commercial interest in keeping both versions simultaneously, this is more headache than gain.
I think it makes more sense to just keep developing for 2.x for now, at least for a few months, up to a year. At some point in time it will be just time to declare on a final, stable version for 2.x and develop the next ones for 3.x+
For example, I won't switch to 3.x until some of the major frameworks go that way: PyQt, matplotlib, numpy, and some others. And I don't really mind if at some point they stop 2.x support and just start developing for 3.x, because I'll know that in a short time I'll be able to switch to 3.x too.
I would start by migrating to 2.6, which is very close to python 3.0. You might even want to wait for 2.7, which will be even closer to python 3.0.
And then, once you have migrated to 2.6 (or 2.7), I suggest you simply keep just one version of the script, with things like "if PY3K:... else:..." in the rare places where it will be mandatory. Of course it's not the kind of code we developers like to write, but then you don't have to worry about managing multiple scripts or branches or patches or distributions, which will be a nightmare.
Whatever you choose, make sure you have thorough tests with 100% code coverage.
Good luck!
Whichever option for development is chosen, most potential issues could be alleviated with thorough unit testing to ensure that the two versions produce matching output. That said, option 2 seems most natural to me: applying changes from one source tree to another source tree is a task (most) version control systems were designed for--why not take advantages of the tools they provide to ease this.
For development, it is difficult to say without 'knowing your audience'. Power Python users would probably appreciate not having to download two copies of your software yet for a more general user-base it should probably 'just work'.