Develop Reference

How to install multiple versions of Python in Windows?

up until recently I have only worked with one version of Python and used virtual environments every now and then. Now, I am working with some libraries that require older version of Python. So, I am very confused. Could anyone please clear up some of my confusion?
How do I install multiple Python versions?
I initially had Python version 3.8.x but upgraded to 3.10.x last month. There is currently only that one version on my PC now.
I wanted to install one of the Python 3.8.x version and went to https://www.python.org/downloads/. It lists a lot of versions and subversions like 3.6, 3.7, 3.8 etc. etc. with 3.8.1, 3.8.2 till 3.8.13. Which one should I pick?
I actually went ahead with 3.8.12 and downloaded the Tarball on the page: https://www.python.org/downloads/release/python-3812/
I extracted the tarball (23.6MB) and it created a folder with a setup.py file.
Is Python 3.8.12 now installed? Clicking on the setup.py file simply flashes the terminal for a second.
I have a few more questions. Hopefully, they won't get me downvoted. I am just confused and couldn't find proper answers for them.
Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
For example, this question
How can I run Mozilla TTS/Coqui TTS training with CUDA on a Windows system?. This seems very beginner unfriendly. Slightly mismatched package
version can prevent any program from running.
Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.

Your questions depend a bit on "all the other software". For example, as #leiyang indicated, the answer will be different if you use conda vs just pip on vanilla CPython (the standard Windows Python).
I'm also going to assume you're actually on Windows, because on Linux I would recommend looking at pyenv. There is a pyenv-win, which may be worth looking into, but I don't use it myself because it doesn't play as nice if you also want (mini)conda environments.
1. (a) How do I install multiple Python versions?
Simply download the various installers and install them in sensible locations. E.g. "C:\Program Files\Python39" for Python 3.9, or some other location where you're allowed to install software.
Don't have Python add itself to the PATH though, since that'll only find the last version to do so and can really confuse things.
Also, you probably want to use virtual environments consistently, as this ties a specific project very clearly to a specific Python version, avoiding future confusion or problems.
1. (b) "3.8.1, 3.8.2 till 3.8.13" which should I pick?
Always pick the latest 3.x.y, so if there's a 3.8.13 for Windows, but no 3.8.14, pick that. Check if the version is actually available for your operating system, sometimes there are later versions for one OS, but not for another.
The reason is that between a verion like 3.6 and 3.7, there may be major changes that change how Python works. Generally, there will be backwards compatibility, but some changes may break how some of your packages work. However, when going up a minor version, there won't be any such breaking changes, just fixes and additions that don't get in the way of what was already there. A change from 2.x to 3.x only happens if the language itself goes through a major change, and rarely happens (and perhaps never will again, depending on who you ask).
An exception to the "no minor version change problems" is of course if you run some script that very specifically relies on something that was broken in 3.8.6, but no fixed in 3.8.7+ (as an example). However, that's very bad coding, to rely on what's broken and not fixing it later, so only go along with that if you have no other recourse. Otherwise, just the latest minor version of any version you're after.
Also: make sure you pick the correct architecture. If there's no specific requirement, just pick 64-bit, but if your script needs to interact with other installed software at the binary level, it may require you to install 32-bit Python (and 32-bit packages as well). If you have no such requirement, 64-bit allows more memory access and has some other benefits on modern computers.
2. Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
It's not just Python, this is true for many languages. It's just more visible to the end user for Python, because you run it as an interpreted language. It's only compiled at the very last moment, on the computer it's running on.
This has the advantage that the code can run on a variety of computers and operating systems, but the downside that you need the right environment where you're running it. For people who code in languages like C++, they have to deal with this problem when they're coding, but target a much smaller number of environments (although there's still runtimes to contend with, and DirectX versions, etc.). Other languages just roll everything up into the program that's being distributed, while a Python script by itself can be tiny. It's a design choice.
There are a lot of tools to help you automate the process though and well-written packages will make the process quite painless. If you feel Python is very shakey when it comes to this, that's probable to blame on the packages or scripts you're using, not really the language. The only fault of the language is that it makes it very easy for developers to make such a mess for you and make your life hard with getting specific requirements.
Look for alternatives, but if you can't avoid using a specific script or package, once you figure out how to install or use it, document it or better yet, automate it so you don't have to think about it again.
3. Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.
Not all of them, but quite a few of them. However, you still need the original installation to be present on the system. Also, you can't pick up a virtual environment and put it somewhere else, not even on the same PC without some careful changes (often better to just recreate it).
You're right that this is a bit wasteful - but this is a difficult choice.
Either Python would be even more complicated, having to manage many different version of packages in a single environment (Java developers will be able to tell you war stories about this, with their dependency management - or wax lyrically about it, once they get it themselves).
Or you get what we have: a bit wasteful, but in the end diskspace is a lot cheaper than your time. And unlike your time, diskspace is almost infinitely expandable.
You can share virtual environments between very similar projects though, but especially if you get your code from someone else, it's best to not have to worry and just give up a few dozen MB for the project. On the upside: you can just delete a virtual environment directory and that pretty much gets rid of the whole things. Some applications like PyCharm may remember that it was once there, but other than that, that's the virtual environment gone.

Just install them. You can have any number of Python installations side by side. Unless you need to have 2 different minor versions, for example 3.10.1 and 3.10.2, there is no need to do anything special. (And if you do need that then you don't need any advice.) Just set up separate shortcuts for each one.
Remember you have to install any 3rd-party libraries you need in each version. To do this, navigate to the Scripts folder in the version you want to do the install in, and run pip from that folder.
Python's 3rd-party libraries are open-source and come from projects that have release schedules that don't necessarily coincide with Python's. So they will not always have a version available that coincides with the latest version of Python.
Often you can get around this by downloading unofficial binaries from Christoph Gohlke's site. Google Python Gohlke.

Install Python using the windows executable installers from python.org. If the version is 3.x.y, use the highest y that has a windows executable installer. Unless your machine is very old, use the 64-bit versions. Do not have them add python to your PATH environment variable, but in only one of the installs have it install the python launcher py. That will help you in using multiple versions. See e.g. here.
Python itself does not. But some modules/libraries do. Especially those that are not purely written in Python but contain extensions written in C(++). The reason for this is that compiling programs on ms-windows can be a real PITA. Unlike UNIX-like operating systems with Linux, ms-windows doesn't come with development tools as standard. Nor does it have decent package management. Since the official Python installers are built with microsoft tools, you need to use those with C(++) extensions as well. Before 2015, you even had to use exactly the same version of the compiler that Python was built with. That is still a good idea, but no longer strictly necessary. So it is a signigicant amount of work for developers to release binary packages for each supported Python version. It is much easier for them to say "requires Python 3.x".

PYTHONPATH on Linux [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I'm novice in this, and I have started learning Python, but I have some questions that I'm not be able to understand,
What exactly is the PYTHONPATH (on Ubuntu)? Is it a folder?
Is Python provided by default on Ubuntu, or does it have to be installed explicitly?
Where is the folder in which all modules are (I have a lot folders called python_)?
If I wish a new module to work when I'm programming (such as pyopengl) where should I go to introduce all the folders I've got in the folder downloaded?
Coming back from the PYTHONPATH issue, how do I configure the PYTHONPATH in order to start working on my new module?

PYTHONPATH is an environment variable which you can set to add additional directories where python will look for modules and packages. e.g.:
# make python look in the foo subdirectory of your home directory for
# modules and packages
export PYTHONPATH=${PYTHONPATH}:${HOME}/foo
Here I use the sh syntax. For other shells (e.g. csh,tcsh), the syntax would be slightly different. To make it permanent, set the variable in your shell's init file (usually ~/.bashrc).
Ubuntu comes with python already installed. There may be reasons for installing other (independent) python versions, but I've found that to be rarely necessary.
The folder where your modules live is dependent on PYTHONPATH and where the directories were set up when python was installed. For the most part, the installed stuff you shouldn't care about where it lives -- Python knows where it is and it can find the modules. Sort of like issuing the command ls -- where does ls live? /usr/bin? /bin? 99% of the time, you don't need to care -- Just use ls and be happy that it lives somewhere on your PATH so the shell can find it.
I'm not sure I understand the question. 3rd party modules usually come with install instructions. If you follow the instructions, python should be able to find the module and you shouldn't have to care about where it got installed.
Configure PYTHONPATH to include the directory where your module resides and python will be able to find your module.

PYTHONPATH is an environment variable
Yes (see https://unix.stackexchange.com/questions/24802/on-which-unix-distributions-is-python-installed-as-part-of-the-default-install)
/usr/lib/python2.7 on Ubuntu
you shouldn't install packages manually. Instead, use pip. When a package isn't in pip, it usually has a setuptools setup script which will install the package into the proper location (see point 3).
if you use pip or setuptools, then you don't need to set PYTHONPATH explicitly
If you look at the instructions for pyopengl, you'll see that they are consistent with points 4 and 5.

PYTHONPATH is an environment variable those content is added to the sys.path where Python looks for modules. You can set it to whatever you like.
However, do not mess with PYTHONPATH. More often than not, you are doing it wrong and it will only bring you trouble in the long run. For example, virtual environments could do strange things…
I would suggest you learned how to package a Python module properly, maybe using this easy setup. If you are especially lazy, you could use cookiecutter to do all the hard work for you.

How to easily distribute Python software that has Python module dependencies? Frustrations in Python package installation on Unix

My goal is to distribute a Python package that has several other widely used Python packages as dependencies. My package depends on well written, Pypi-indexed packages like pandas, scipy and numpy, and specifies in the setup.py that certain versions or higher of these are needed, e.g. "numpy >= 1.5".
I found that it's immensely frustrating and nearly impossible for Unix savvy users who are not experts in Python packaging (even if they know how to write Python) to install a package like mine, even when using what are supposed to be easy to use package managers. I am wondering if there is an alternative to this painful process that someone can offer, or if my experience just reflects the very difficult current state of Python packaging and distribution.
Suppose users download your package onto their system. Most will try to install it "naively", using something like:
$ python setup.py install
Since if you google instructions on installing Python packages, this is usually what comes up. This will fail for the vast majority of users, since most do not have root access on their Unix/Linux servers. With more searching, they will discover the "--prefix" option and try:
$ python setup.py install --prefix=/some/local/dir
Since the users are not aware of the intricacies of Python packaging, they will pick an arbitrary directory as an argument to --prefix, e.g. "~/software/mypackage/". It will not be a cleanly curated directory where all other Python packages reside, because again, most users are not aware of these details. If they install another package "myotherpackage", they might pass it "~/software/myotherpackage", and you can imagine how down the road this will lead to frustrating hacking of PYTHONPATH and other complications.
Continuing with the installation process, the call to "setup.py install" with "--prefix" will also fail once users try to use the package, even though it appeared to have been installed correctly, since one of the dependencies might be missing (e.g. pandas, scipy or numpy) and a package manager is not used. They will try to install these packages individually. Even if successful, the packages will inevitably not be in the PYTHONPATH due to the non-standard directories given to "--prefix" and patient users will dabble with modifications of their PYTHONPATH to get the dependencies to be visible.
At this stage, users might be told by a Python savvy friend that they should use a package manager like "easy_install", the mainstream manager, to install the software and have dependencies taken care of. After installing "easy_install", which might be difficult, they will try:
$ easy_install setup.py
This too will fail, since users again do not typically have permission to install software globally on production Unix servers. With more reading, they will learn about the "--user" option, and try:
$ easy_install setup.py --user
They will get the error:
usage: easy_install [options] requirement_or_url ...
or: easy_install --help
error: option --user not recognized
They will be extremely puzzled why their easy_install does not have the --user option where there are clearly pages online describing the option. They might try to upgrade their easy_install to the latest version and find that it still fails.
If they continue and consult a Python packaging expert, they will discover that there are two versions of easy_install, both named "easy_install" so as to maximize confusion, but one part of "distribute" and the other part of "setuptools". It happens to be that only the "easy_install" of "distribute" supports "--user" and the vast majority of servers/sys admins install "setuptools"'s easy_install and so local installation will not be possible. Keep in mind that these distinctions between "distribute" and "setuptools" are meaningless and hard to understand for people who are not experts in Python package management.
At this point, I would have lost 90% of even the most determined, savvy and patient users who try to install my software package -- and rightfully so! They wanted to install a piece of software that happened to be written in Python, not to become experts in state of the art Python package distribution, and this is far too confusing and complex. They will give up and be frustrated at the time wasted.
The tiny minority of users who continue on and ask more Python experts will be told that they ought to use pip/virtualenv instead of easy_install. Installing pip and virtualenv and figuring out how these tools work and how they are different from the conventional "python setup.py" or "easy_install" calls is in itself time consuming and difficult, and again too much to ask from users who just wanted to install a simple piece of Python software and use it. Even those who pursue this path will be confused as to whether whatever dependencies they installed with easy_install or setup.py install --prefix are still usable with pip/virtualenv or if everything needs to be reinstalled from scratch.
This problem is exacerbated if one or more of the packages in question depends on installing a different version of Python than the one that is the default. The difficulty of ensuring that your Python package manger is using the Python version you want it to, and that the required dependencies are installed in the relevant Python 2.x directory and not Python 2.y, will be so endlessly frustrating to users that they will certainly give up at that stage.
Is there a simpler way to install Python software that doesn't require users to delve into all of these technical details of Python packages, paths and locations? For example, I am not a big Java user, but I do use some Java tools occasionally, and don't recall ever having to worry about X and Y dependencies of the Java software I was installing, and I have no clue how Java package managing works (and I'm happy that I don't -- I just wanted to use a tool that happened to be written in Java.) My recollection is that if you download a Jar, you just get it and it tends to work.
Is there an equivalent for Python? A way to distribute software in a way that doesn't depend on users having to chase down all these dependencies and versions? A way to perhaps compile all the relevant packages into something self-contained that can just be downloaded and used as a binary?
I would like to emphasize that this frustration happens even with the narrow goal of distributing a package to savvy Unix users, which makes the problem simpler by not worrying about cross platform issues, etc. I assume that the users are Unix savvy, and might even know Python, but just aren't aware (and don't want to be made aware) about the ins and outs of Python packaging and the myriad of internal complications/rivalries of different package managers. A disturbing feature of this issue is that it happens even when all of your Python package dependencies are well-known, well-written and well-maintained Pypi-available packages like Pandas, Scipy and Numpy. It's not like I was relying on some obscure dependencies that are not properly formed packages: rather, I was using the most mainstream packages that many might rely on.
Any help or advice on this will be greatly appreciated. I think Python is a great language with great libraries, but I find it virtually impossible to distribute the software I write in it (once it has dependencies) in a way that is easy for people to install locally and just run. I would like to clarify that the software I'm writing is not a Python library for programmatic use, but software that has executable scripts that users run as individual programs. Thanks.

We also develop software projects that depend on numpy, scipy and other PyPI packages. Hands down, the best tool currently available out there for managing remote installations is zc.buildout. It is very easy to use. You download a bootstrapping script from their website and distribute that with your package. You write a "local deployment" file, called normally buildout.cfg, that explains how to install the package locally. You ship both the bootstrap.py file and buildout.cfg with your package - we use the MANIFEST.in file in our python packages to force the embedding of these two files with the zip or tar balls distributed by PyPI. When the user unpackages it, it should execute two commands:
$ python bootstrap.py # this will download zc.buildout and setuptools
$ ./bin/buildout # this will build and **locally** install your package + deps
The package is compiled and all dependencies are installed locally, which means that the user installing your package doesn't even need root privileges, which is an added feature. The scripts are (normally) placed under ./bin, so the user can just execute them after that. zc.buildout uses setuptools for interaction with PyPI so everything you expect works out of the box.
You can extend zc.buildout quite easily if all that power is not enough - you create the so-called "recipes" that can help the user to create extra configuration files, download other stuff from the net or instantiate custom programs. zc.buildout website contains a video tutorial that explains in details how to use buildout and how to extend it. Our project Bob makes extensive use of buildout for distributing packages for scientific usage. If you would like, please visit the following page that contains detailed instructions for our developers on how they can setup their python packages so other people can build and install them locally using zc.buildout.

We're currently working to make it easier for users to get started installing Python software in a platform independent manner (in particular see https://python-packaging-user-guide.readthedocs.org/en/latest/future.html and http://www.python.org/dev/peps/pep-0453/)
For right now, the problem with two competing versions of easy_install has been resolved, with the competing fork "distribute" being merged backing into the setuptools main line of development.
The best currently available advice on cross-platform distribution and installation of Python software is captured here: https://packaging.python.org/

Use NLTK without installing [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
Learning Python with the Natural Language Toolkit has been great fun, and they work fine on my local machine, though I had to install several packages in order to use it. Exactly how the NLTK resources are now integrated on my system remains a mystery to me, though it seems clear that the NLTK source code is not simply sitting someplace where the Python interpreter knows to find it.
I would like to use the Toolkit on my website, which is hosted by another company. Simply uploading the NLTK source code files to my server and telling scripts in the root directory to "import nltk" has not worked; I kind of doubted it would.
What, then, is the difference between whatever the NLTK install routine does and straightforward imports, and why should the toolkit be inaccessible to straightforward imports? Is there a way to use the NLTK source files without essentially altering my host's Python?
Many thanks for your thoughts and notes.
-G

Not only do you need NLTK on your PYTHONPATH (as #dhg points out), you need whatever dependencies it has; a quick local test indicates that this is really only PyYAML. You should just use pip to install packages. It's much less error-prone than trying to manually figure out all the dependencies and tweak the PYTHONPATH accordingly. If this is a shared host where you don't have the proper access to run a pip install, you should ask the host to do it for you.
To address the more general "Whatever the install script is doing" portion of your question: most Python packages are managed using setup.py, which is built on top of distutils (and sometimes setuputils). If this is something you're really interested in, check out The Hitchhiker’s Guide to Packaging.

You don't need system-install support, just the right modules where python can find them. I've set up the NLTK without system install rights with relatively little trouble--but I did have commandline access so I could see what I was doing.
To get this working, you should put together a local install on a computer you do control-- ideally one that never had NLTK installed, since you may have forgotten (or not know) what was configured for you. Once you figure out what you need, copy the bundle to the hosting computer. But at that point, check that you're using the module versions that are appropriate for the webserver's architecture. Numpy in particular has different 32/64 bit versions, IIRC.
It's also worth your while to figure out how to see the error messages from the hosting computer. If you can't see them by default, you could catch ImportError and display the message it contains, or you could redirect stderr... it depends on your configuration.

Let's assume that you have the NLTK source located in /some/dir/, so that
dhg /some/dir/$ ls nltk
...
app
book.py
ccg
chat
chunk
classify
...
You can either launch the python interpreter from the directory in which the nltk source directory is found:
dhg /some/dir/$ python
Python 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34)
>>> import nltk
Or you can add its location to the PYTHONPATH environment variable, which makes NLTK available from anywhere:
dhg /whatever/$ export PYTHONPATH="$PYTHONPATH:/some/dir/"
dhg /whatever/$ python
Python 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34)
>>> import nltk
Any other dependencies, including those that NLTK depends on, can also be added to the PYTHONPATH in the same way.

Multiple Versions of Python on my Windows machine: Which is the "right" one?

In continuing to research a solution for this question on ServerFault:
https://serverfault.com/questions/221203/mercurial-hook-fails-on-windows
I discovered an interesting and somewhat disturbing thing: I have seem three different versions of Python on my machine (four if you count the "official" version which doesn't appear to have a DLL with it....). Here's shot from my file search tool:
More Info:
I am running Windows 7 64-bit
Both the TortoiseHG and the Mercurial directories are on my path, with the Mercurial directory listed first.
I have Python 2.6 installed in c:\Python26
I have no entry for any type of PYTHON-based environmental variable. (Should I?)
I suspect that this is the source of the my problem from the question above, but I thought I'd ask here, as this is particular issue is a Python deal.
I tried to replace both DLLs with each other, but when I use the one that comes with Mercurial, then TortoiseHg stops working.
It seems to me that "there should only be one" Python on my machine. How do I achieve that?

For the problem that you mentioned earlier, the mercurial package got installed within python under mercurial home but you are executing scripts under C:\python26. So you need to install and execute your script under mercurial python
As seth mentioned earlier it is perfectly ok to multiple python homes in the same machine but you just to pay attention when installing python libraries to make sure that you are under the right home which means you set the path right before calling python.

Side note: The Python installation in "C:\Python26" installs its DLL to the Windows directory, in your case "C:\Windows\SysWOW64".
Answering your serverfault question: As you installed Mercurial as standalone version, you'll have to place any packages that are accessed by hooks into Mercurial's library folder (if it has one, could also be "library.zip").
I would recommend you to uninstall the Mercurial standalone version and instead install Mercurial with pip. This makes updates easier and you can use your normal "site-packages" directory for both normal Python libraries and hg hooks.

I would assume that tortoise/mercurial have just embedded their own versions of python to do whatever they need to do.
I wouldn't worry about it, the DLLs won't stomp on each other -- the PATH is the last placed that windows searches to find DLLs.
See: http://msdn.microsoft.com/en-us/library/7d83bc18(v=vs.80).aspx

Each DLL is for that application. There is only one in your search path so you don't need to worry about conflicts.
Is something not working that prompted you to worry about this??
Your assumption that there should only be one is wrong, each application has bundled a specific version with a fixed API, you can't just drop another in and hope it'll work.

The Python DLL naming structure only provides the major version and revision numbers. You are probably looking at the DLLs for versions 2.6.1, 2.6.4, 2.6.5, and 2.6.6.
All of this doesn't really matter as long as each application contains its own copy of the python26.dll. Windows will not explore the PATH environment variable if there is a local copy of the file.