I am not a regular Linux user so this might be completely trivial question. I am running 6.2 PUIAS version i386_64 on one of my GPU based "super" computers due to the unavailability of NVidia drivers for NetBSD. The installed version of Python is 2.6.6. I need 2.7.2 Python and newer version of scipy, numpy, matlibplot and friends. I have PUIAS and EPEL repositories enabled. However they do not have newer versions of Python. What is the "recommended" way to install newer version of Python without braking the system which depends on it. I am not interested in Python 3.2 due to the lack of libraries for scientific computing.
When the install-Python-from-source routine tells you to use make install, type make altinstall instead. This will leave the normal python executable untouched and instead create python2.7 for you to use. Install the other packages from source using this new executable. Don't forget to change the shebang line in your scripts accordingly.
I am going to answer my own question. For people who are using Python for scientific computing on RedHat clones (PUIAS for example) the easiest way to get all they need is to use rpm package manager and Enthought Python Distribution (EPD for short). EPD installs everything in a sandbox so system tools which are based on an obsolete version of Python are not massed up. However, paths have to be adjusted for system or even easier on the user base so that the using shell invokes non-system tools. One should never compile Python from source unless you are interesting in Python itself or in porting it to your favorite operating system rather than in your own research!
Related
up until recently I have only worked with one version of Python and used virtual environments every now and then. Now, I am working with some libraries that require older version of Python. So, I am very confused. Could anyone please clear up some of my confusion?
How do I install multiple Python versions?
I initially had Python version 3.8.x but upgraded to 3.10.x last month. There is currently only that one version on my PC now.
I wanted to install one of the Python 3.8.x version and went to https://www.python.org/downloads/. It lists a lot of versions and subversions like 3.6, 3.7, 3.8 etc. etc. with 3.8.1, 3.8.2 till 3.8.13. Which one should I pick?
I actually went ahead with 3.8.12 and downloaded the Tarball on the page: https://www.python.org/downloads/release/python-3812/
I extracted the tarball (23.6MB) and it created a folder with a setup.py file.
Is Python 3.8.12 now installed? Clicking on the setup.py file simply flashes the terminal for a second.
I have a few more questions. Hopefully, they won't get me downvoted. I am just confused and couldn't find proper answers for them.
Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
For example, this question
How can I run Mozilla TTS/Coqui TTS training with CUDA on a Windows system?. This seems very beginner unfriendly. Slightly mismatched package
version can prevent any program from running.
Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.
Your questions depend a bit on "all the other software". For example, as #leiyang indicated, the answer will be different if you use conda vs just pip on vanilla CPython (the standard Windows Python).
I'm also going to assume you're actually on Windows, because on Linux I would recommend looking at pyenv. There is a pyenv-win, which may be worth looking into, but I don't use it myself because it doesn't play as nice if you also want (mini)conda environments.
1. (a) How do I install multiple Python versions?
Simply download the various installers and install them in sensible locations. E.g. "C:\Program Files\Python39" for Python 3.9, or some other location where you're allowed to install software.
Don't have Python add itself to the PATH though, since that'll only find the last version to do so and can really confuse things.
Also, you probably want to use virtual environments consistently, as this ties a specific project very clearly to a specific Python version, avoiding future confusion or problems.
1. (b) "3.8.1, 3.8.2 till 3.8.13" which should I pick?
Always pick the latest 3.x.y, so if there's a 3.8.13 for Windows, but no 3.8.14, pick that. Check if the version is actually available for your operating system, sometimes there are later versions for one OS, but not for another.
The reason is that between a verion like 3.6 and 3.7, there may be major changes that change how Python works. Generally, there will be backwards compatibility, but some changes may break how some of your packages work. However, when going up a minor version, there won't be any such breaking changes, just fixes and additions that don't get in the way of what was already there. A change from 2.x to 3.x only happens if the language itself goes through a major change, and rarely happens (and perhaps never will again, depending on who you ask).
An exception to the "no minor version change problems" is of course if you run some script that very specifically relies on something that was broken in 3.8.6, but no fixed in 3.8.7+ (as an example). However, that's very bad coding, to rely on what's broken and not fixing it later, so only go along with that if you have no other recourse. Otherwise, just the latest minor version of any version you're after.
Also: make sure you pick the correct architecture. If there's no specific requirement, just pick 64-bit, but if your script needs to interact with other installed software at the binary level, it may require you to install 32-bit Python (and 32-bit packages as well). If you have no such requirement, 64-bit allows more memory access and has some other benefits on modern computers.
2. Why does Python have such heavy dependency on the exact versions of libraries and packages etc?
It's not just Python, this is true for many languages. It's just more visible to the end user for Python, because you run it as an interpreted language. It's only compiled at the very last moment, on the computer it's running on.
This has the advantage that the code can run on a variety of computers and operating systems, but the downside that you need the right environment where you're running it. For people who code in languages like C++, they have to deal with this problem when they're coding, but target a much smaller number of environments (although there's still runtimes to contend with, and DirectX versions, etc.). Other languages just roll everything up into the program that's being distributed, while a Python script by itself can be tiny. It's a design choice.
There are a lot of tools to help you automate the process though and well-written packages will make the process quite painless. If you feel Python is very shakey when it comes to this, that's probable to blame on the packages or scripts you're using, not really the language. The only fault of the language is that it makes it very easy for developers to make such a mess for you and make your life hard with getting specific requirements.
Look for alternatives, but if you can't avoid using a specific script or package, once you figure out how to install or use it, document it or better yet, automate it so you don't have to think about it again.
3. Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.
Not all of them, but quite a few of them. However, you still need the original installation to be present on the system. Also, you can't pick up a virtual environment and put it somewhere else, not even on the same PC without some careful changes (often better to just recreate it).
You're right that this is a bit wasteful - but this is a difficult choice.
Either Python would be even more complicated, having to manage many different version of packages in a single environment (Java developers will be able to tell you war stories about this, with their dependency management - or wax lyrically about it, once they get it themselves).
Or you get what we have: a bit wasteful, but in the end diskspace is a lot cheaper than your time. And unlike your time, diskspace is almost infinitely expandable.
You can share virtual environments between very similar projects though, but especially if you get your code from someone else, it's best to not have to worry and just give up a few dozen MB for the project. On the upside: you can just delete a virtual environment directory and that pretty much gets rid of the whole things. Some applications like PyCharm may remember that it was once there, but other than that, that's the virtual environment gone.
Just install them. You can have any number of Python installations side by side. Unless you need to have 2 different minor versions, for example 3.10.1 and 3.10.2, there is no need to do anything special. (And if you do need that then you don't need any advice.) Just set up separate shortcuts for each one.
Remember you have to install any 3rd-party libraries you need in each version. To do this, navigate to the Scripts folder in the version you want to do the install in, and run pip from that folder.
Python's 3rd-party libraries are open-source and come from projects that have release schedules that don't necessarily coincide with Python's. So they will not always have a version available that coincides with the latest version of Python.
Often you can get around this by downloading unofficial binaries from Christoph Gohlke's site. Google Python Gohlke.
Install Python using the windows executable installers from python.org. If the version is 3.x.y, use the highest y that has a windows executable installer. Unless your machine is very old, use the 64-bit versions. Do not have them add python to your PATH environment variable, but in only one of the installs have it install the python launcher py. That will help you in using multiple versions. See e.g. here.
Python itself does not. But some modules/libraries do. Especially those that are not purely written in Python but contain extensions written in C(++). The reason for this is that compiling programs on ms-windows can be a real PITA. Unlike UNIX-like operating systems with Linux, ms-windows doesn't come with development tools as standard. Nor does it have decent package management. Since the official Python installers are built with microsoft tools, you need to use those with C(++) extensions as well. Before 2015, you even had to use exactly the same version of the compiler that Python was built with. That is still a good idea, but no longer strictly necessary. So it is a signigicant amount of work for developers to release binary packages for each supported Python version. It is much easier for them to say "requires Python 3.x".
Short Description
Is it possible to install Python 2.7.1 64/32bit from python.org on top of an existing install (from python.org) of Python 2.7.1 32bit?
Background
I installed the 32bit version for wxPython(2.8) support which until now has given me zero issues. There are a few modules that I am now having difficultly installing (psycopg2 and mysql-python). The warning messages in homebrew constantly warn me about not having a 64bit version of Python on the path. These warnings only add to my list of wanting to use a 64 bit version.
At the time I selected the 32bit install, the GUI for a particular project was the main focus. Now the GUI has been come very simple and the database back-end support (using Django) is much more important. This being said, using the development version of wxPython 2.9 (which supports OS X and 64bit 2.7) has become acceptable.
For the non-mac users, to help explain why I had to use the 32bit version please see this Brief guide to using virtualenv in a wxpython project
System Information
Development System
OS: Mac OS X Snow Leopard (10.6.7)
Python: 2.7.1 with virutalenv / virutalenv-wrapper
Project Dependencies:
Note that MySQL could be PostgresSQL's psycopg2 if I can get the postgresql to install with homebrew
Django==1.2.5
MySQL-python==1.2.3
PIL==1.1.7
PyVISA==1.3
pyserial==2.5
virtualenv==1.5.1
virtualenvwrapper==2.6.3
wsgiref==0.1.2
wxPython==2.8.11.0
wxPython-common==2.8.11.0
Deployment System
OS: Windows XP / Windows 7
Python: Hopefully none (goal to use py2exe, or similar tool)
Current Thoughts
I fear that my goal cannot be accomplished based on the file paths alone. In Windows 7 the identifier (x86) is placed in the path showing that it is a 32bit program, but on OS X the path would be the same for 32bit or 64/32bit installs (/Library/Frameworks/Python.framework/Versions/2.7/).
Any thoughts or comments would be helpful!
Update 5-18-2011: 8:40 AM
I have confirmed that using the pre-compiled (.dmg) framework builds, installing the a 64 bit version does blow away the 32 bit install. This did negatively effect my virtual environments in that all of the things I have installed in the environments was based on the 32 bit install nearly every module threw an error of some sort.
I still have not achieved installing 32 bit and 64 bit on the same machine, however looking into homebrew with greater detail does look this could be possible. The trick would be to define your own formulas for each of the python builds (from source) and rename the install directory to something like 'Python27_32' and 'Python27_64'. I'll keep exploring this front as I have time.
EDIT 7-12-2011: 10:51 AM CST
Has anyone out there been able control how homebrew compile options? Specifically how to select to use a 64bit compiler or a 32 bit compiler? I will create a new SO question if this doesn't bring anything up.
Check out http://www.macports.org/ Which provides ports of various flavors of linux/unix tools that don't appear in the default MacInstallation and duplicates those that it needs otherwise. It installs everthing in /opt/local instead of stepping on the installed base. With some manipulation of PATH and LD_LIBRARY_PATH this might give you the options you need.
I would use something like virtualenv. This way you can have as many different installs of python as you wish.
EDIT: To use different versions of python use:
virtualenv -p python2.6
Reference
Another possible solution that was provided in an answer to one of my questions aimed at using a virtual environment in the Windows world was using Buildout. This approach looks like it would allow me create an environment with the 32bit installed and another one with the 64bit.
I have not tested this out yet, but wanted to pass it along to anyone else attempting this feat.
Cheers!
I'm a newbie programmer just installing Python 3.2, but I know I also have an older version of Python on my machine. in fact, I think Macbook comes with it installed. Do I have to worry about having different versions on my computer when I try to start learning Python?
For the most part, you don't have to worry about conflicts with system Python. In fact it is recommended to install a different Python version instead of working with system Python. Also consider using virtualenv and virtualenvwrapper to maintain any dependencies for each project easily without conflicts.
It really depends what OS you're talking about. I'm assuming you're talking about a Mac, since you mentioned Macbook.
Macs come with 2.5 and 2.6 installed as far as I'm aware. At least mine has both those versions, and I've only installed 2.7 manually.
You can check which version of python is the current 'system' python by doing the following in terminal:
// check the version of system python
python --version
// tells you where the system version of python is on your PATH
which python
On *nix type Operating Systems, like your Mac, applications aren't really 'installed', like they are in Windows (eliding details). Instead, application files are placed in various different parts of the file system. Python, for example, is placed into the following directory (by default) when installing 2.7:
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
Since this directory isn't on the system PATH, this version of python won't be used when simply calling python from the command line. The system will search all the folders in the PATH environment variable for an executable file called python. It will usually find it in /usr/bin/ or something similar.
To make a new version of Python the 'system' python, you have a couple of options:
Modify your .bash_profile, and prepend the path to your new python to the PATH environment variable.
symlink the new version of python to a directory already on your PATH like /usr/bin/
Be aware that Mac python installers can modify your .bash_profile (in your home directory), to force the new version to be the default system version. This is what my bash_profile shows:
# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
You can happily run multiple versions of python on the same system. A particular version is usually the default though, and that's whatever executable python file is found on the PATH first.
If you want to use a different version at any particular point in time, you can:
/path/to/python/2.4/python some_script.py
/path/to/python/2.7/python some_script.py
/path/to/python/3.2/python some_script.py
That will execute the script some_script.py under 3 different versions of python. Of course, you need to make sure that the /path/to/python is correct.
So yes, you need to be mindful about what version of python you are going to be using, hopefully this will guide you into understanding how applications are installed and which version of an application is launched by default when you don't provide a path.
Yes, 3.x Python syntax is not backward-compatible with 2.x. So if you learn Python 3.x you might not be able to port your knowledge to Python 2.x.
Moreover you should choose if you want to learn 3.x or 2.x. 2.x is far more widespread than 3.x, but 3.x is where Python is heading. No more innovation will happen in 2.x, and in mid-term most frameworks will be ported to 3.x (right now there are some notable exceptions)
Hope that helps!
In general, you should be fine. Since the Mac is BSD-based, it should maintain the "python" command as pointing to the version that your system requires, which is usually an older version like 2.5. You may have to use a command like python3 to run your Python 3 programs, but other than that it should be transparent to you.
As you learn and become more advanced, you can begin using the virtualenv system to maintain separate Python installations for multiple projects.
Python version with different major or minor version numbers can be installed in parallel. For example, you can have 2.4, 2.5, 2.6, 2.7 and 3.1 on the same machine. However, you can't have versions with the same major and minor number installed at the same time (at least, not without tricks), so you can't have 2.5.2 and 2.5.4 at the same time.
Note that you will have to install any third-party libraries once for every Python version.
It is very well possible to have multiple versions of python on your machine. Just make sure, that if you call python in your console it uses the python you want it to use. Same goes for your IDE.
Regarding the version: It is always nice to have the latest version on board (in python however there are compatibility issues to take into account) , since there might be features you want to use, that are only available with a certain version and upwards. Since this is sometimes tricky to find out, especially if you are new to the field, going with the latest version might be how you should proceed.
Be careful before installing new version of python.
Python has no backward compatibility.
Scripts written for python 2.7.* won't work on python 3
For example,
print "Hello" will work on python 2.7 but not on version3
I have Python2.6.5 and Python2.4.4 on my linux machine.
At the moment, all the modules I have (wx, ply, pyserial, twisted, to name a few) are installed for the 2.6 version of python. If I try to import wx on Python2.4, I get the expected no module error.
The problem here, is that I have a lot of devices (Let's say over a thousand) all running 2.4.4, which will soon need to be supported by this machine (For builds of code, releases etc). Until now, I've been using an EeePC (Same device as the ones I'm supporting) to do builds and releases, which has worked well. (I develop on the 2.6 machine, and build on the EeePC).
How would I go about getting these modules to work for Python2.4? I've tried reinstalling (With 2.4 as my primary), but that just caused errors. The blogs/answers I've found say to use easy_install, but that doesn't support the packages I need (Or at least, it just died when I tried).
In short: I'm currently using python 2,6, but I'd like it to change to 2.4 for all the modules as that's what I'm going to be using.
You can't share modules between different versions of Python. If you want to use wxPython from Python 2.4, you need to install it for Python 2.4.
You said you tried to install it with Python 2.4 as your "primary". I'm not sure what that means. You would install wxPython for Python 2.4 by running the installer with Python 2.4, like so:
$sudo /usr/bin/python2.4 setup.py install
Or similar.
You can use easy_install as well, but then you need to install Distribute for Python 2.4 first. Did you do that?
I recently wrote a full explanation on my blog about this: http://regebro.wordpress.com/2011/02/02/newbie-hint-on-installing-python-and-its-modules-and-packages/
Don't attempt to share them; this has some chance of success with pure Python modules, but C modules will fail to work. Instead, install them using the appropriate interpreter executable, e.g. python2.4 setup.py install.
I have a linux VPS that uses an older version of python (2.4.3). This version doesn't include the UUID module, but I need it for a project. My options are to upgrade to python2.6 or find a way to make uuid work with the older version. I am a complete linux newbie. I don't know how to upgrade python safely or how I could get the UUID modules working with the already installed version. What is a better option and how would I go about doing it?
The safest way to upgrading Python is to install it to a different location (away from the default system path).
To do this, download the source of python and do a
./configure --prefix=/opt
(Assuming you want to install it to /opt which is where most install non system dependant stuff to)
The reason why I say this is because some other system libraries may depend on the current version of python.
Another reason is that as you are doing your own custom development, it is much better to have control over what version of the libraries (or interpreters) you are using rather than have a operating system patch break something that was working before. A controlled upgrade is better than having the application break on you all of a sudden.
The UUID module exists as a separate package for Python 2.3 and up:
http://pypi.python.org/pypi/uuid/1.30
So you can either install that in your Python2.4, or install Python2.6. If your distro doesn't have it, then Python is quite simple to compile from source. Look through the requirements to make sure all the libraries you need/want are installed before compiling Python. That's it.
The best solution will be installing python2.6 in the choosen directory - It will you give you access to many great features and better memory handling (infamous python=2.4 memory leak problem).
I have got several pythons installed onto my two computers, I found that the best solution for are two directories:
$HOME/usr-32
$HOME/usr-64
respectively to using operating system (I share $HOME between 32 and 64 bit versions of Linux).
In each I have one directory for every application/program, for example:
ls ~/usr-64/python-2.6.2/
bin include lib share
It leads completetely to avoiding conflicts between version and gives great portability (you can use usb pendrives etc).
Python 2.6.2 in previously example has been installed with option:
./configure --prefix=$HOME/usr-64/python-2.6.2