I am trying to clean up a few things now that I have Mavericks. I used Homebrew to install python 2.7 and 3.3. I also have their respective pips. Now I want to have a sane virtualenv setup.
I see from virtualenv with python2 and python3 via Homebrew that I can specify which version of python a newly-created virtualenv will use. What is the best (i.e., easiest to use/remember) way to make sure that a virtualenv using Python 3.3 uses pip3?
ETA: I've not had to use 3.3 before, so this is my first experience trying to maintain different versions. Sorry if this has an obvious answer.
Credit goes to scythargon for this one (left in a comment to Kristof's answer in this thread) and, since it did exactly what I needed, I figured it deserved its own answer.
When creating a virtualenv with virtualenvwrapper use the -p flag to explicitly associate it with a specific version of Python. In the examples below replace {virtualenv_name} with your intended virtual environment name:
Python 3.3:
$ mkvirtualenv {virtualenv_name} -ppython3.3
Python 3.4:
$ mkvirtualenv {virtualenv_name} -ppython3.4
Python 2.7:
$ mkvirtualenv {virtualenv_name} -ppython2.7
Have a look at this article: Python Development Environment on Mac OS X Mavericks 10.9. It describes quite clearly how to set up both versions of Python on the same machine using Homebrew and contains some helpful pointers.
In the article, the author appends -py3 to a Python3 virtualenv, as #moliware already suggested.
Related
I've just installed Python version 3.11 (I also moved versions 3.8 and 3.9 to the trash from my Applications folder).
I can see it in the following:
$ myname#name-MBP miniconda3 % ls /usr/local/bin/py*
/usr/local/bin/pydoc3 /usr/local/bin/python3-intel64
/usr/local/bin/pydoc3.11 /usr/local/bin/python3.11
/usr/local/bin/python3 /usr/local/bin/python3.11-config
/usr/local/bin/python3-config /usr/local/bin/python3.11-intel64
(Any additional recommendations on whether I need to clean things up would be much appreciated.)
Checking python3 --version still displays Python 3.8.13.
First attempt to solve
Initially I tried installing it from the command line using homebrew and specifying the version:
brew install python#3.11
I also tried using conda, but neither of these worked.
Second attempt to solve
My initial thought was to check my PATH (I think this is how Python decides which version to use, but please correct me if I'm wrong).
This still only contained version 8 paths:
/Users/myname/miniconda3/lib/python38.zip
/Users/myname/miniconda3/lib/python3.8
/Users/myname/miniconda3/lib/python3.8/lib-dynload
/Users/myname/.local/lib/python3.8/site-packages
/Users/myname/miniconda3/lib/python3.8/site-packages
So I added the Python3.11 path to it using
PYTHONPATH="/usr/local/bin/pydoc3.11/:$PYTHONPATH"
export PYTHONPATH
Now it includes the v3.11 path when I print out sys.path:
/usr/local/bin/python3.11
/usr/local/bin/pydoc3.11
/Users/myname/miniconda3
/Users/myname/miniconda3/lib/python38.zip
/Users/myname/miniconda3/lib/python3.8
/Users/myname/miniconda3/lib/python3.8/lib-dynload
/Users/myname/.local/lib/python3.8/site-packages
/Users/myname/miniconda3/lib/python3.8/site-packages
But the python3 --version output is still unchanged.
Questions
I'm nervous to keep playing around with the contents of my path and entering random command line executions to try to solve this because I really have no clue what I'm doing.
What's happening here?
How can I get the output of python3 --version to be 3.11?
So first thing to understand is setting the variable PYTHONPATH will not affect which version of python is executed by the shell. The shell (bash/zsh) only knows to scan the paths in the PATH env var to figure out all the executables.
Now there are two ways to solve this.
1. Using the python#3.11 from homebrew.
There are several downsides to using this method. Currently, the default python3 by brew is 3.10.x. Whenever you will install any cask or formula that depends on python#3, it'll invariably install the python#3 formula aka 3.10.x. Installing python3 will make brew symlink 3.10.x into /opt/homebrew/bin.
Python 3.11.x can be used by installing python#3.11 and invoking python3.11. This should drop you into the Python 3.11 interpreter. Append all python executable names with the version like pip will be pip3.11
Trying to force 3.11 over 3.10 links will be complicated and cause instability. It'll only cause frustration during development.
2. Using VirtualEnvs
Your best bet for the most stable and no-headache approach to python is to create virtual envs using either venv or pyenv. use pyenv-virtualenv for max ease of use.
One limitation of venv is that it'll create a virtualenv of the same version that invoked it. Aka if you do brew install python#3 && python3 -m venv <folder>, it'll create virtualenv of python3. For 3.11 you'll have to brew install python#3.11 && python3.11 -m venv <folder>. Pyenv on the other hand can install any version of python.
Go through https://www.dataquest.io/blog/a-complete-guide-to-python-virtual-environments/ and https://github.com/pyenv/pyenv-virtualenv to understand and learn more.
I'm using Ubuntu 19.10, in which there is python of version 3.7. After the release of python 3.8, which I have installed, now I want to uninstall python 3.7 so that whenever I would call python3 in my terminal, it would always call python3.8?
You do not need to uninstall old version for this.
You need to update your update-alternatives , then you will be able to set your default python version.
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.4 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
Then run :
sudo update-alternatives --config python
Set python3.6 as default.
Or use the following command to set python3.6 as default:
sudo update-alternatives --set python /usr/bin/python3.6
This should do it.
I suggest you do not touch your default OS python instalation. Other parts of the system may depend on it and there is no way to know if an upgrade can break something, even if it should not.
I also suggest that you learn (if you haven't already, but I suppose you did not because of the question) about using a python virtual environment, like virtualenv. This allows you to setup specific python environments for each project you write. This means each environment can have its own python version, and, besides the standard python lib for that version, any other third-party python lib you would like to install with pip for that project. This isolates projects from each other. You won't break one because of an upgrade of another.
That said, if you want to keep cutting edge with Python versions, make an install from source, and then install it in the system with the altinstall parameter (see the README.rst of the Python distribution. This way all the installed versions are available with different names (the same for pip) and then you create each of your virtual environments with the wanted version. There is a parameter for virtualenv to apply a specific (older) version if you want.
Quoting the README on the "Installing multiple versions" section:
On Unix and Mac systems if you intend to install multiple versions of Python
using the same installation prefix (--prefix argument to the configure
script) you must take care that your primary python executable is not
overwritten by the installation of a different version. All files and
directories installed using make altinstall contain the major and minor
version and can thus live side-by-side. make install also creates
${prefix}/bin/python3 which refers to ${prefix}/bin/pythonX.Y. If you
intend to install multiple versions using the same prefix you must decide which
version (if any) is your "primary" version. Install that version using make
install. Install all other versions using make altinstall.
For example, if you want to install Python 2.7, 3.6, and 3.8 with 3.8 being the
primary version, you would execute make install in your 3.8 build directory
and make altinstall in the others.
Finally, the other answers are ok to do exactly what you asked, if you still want to.
so that whenever I would call python3 in my terminal, it would always call python3.8
You can simply create an alias for Python 3.8 in your .bashrc file in your /home path.
Open ~/.bashrc with your preferred editor, and go to the last line.
Append this line:
alias python='python3.8'
And whenever you call python in your terminal, it opens Python 3.8.
Note: This doesn't work as expected when creating virtual environments with specific version. You might want to keep that in mind.
After moving over to Fedora (from Windows), I realized that it came with both installations of Python 2.7.5 and Python 3.6.6.
As I familiarized myself with using Python, I learned of the great utilities of virtual environments and how organized they keep everything.
However, my current dilemma is for which Python version should I do pip(2 or 3) install virtualenv virtualenvwrapper.
From my research, I understand that the virtualenvwrapper provides the ability to create a virtual environment using a specified version of Python: mkvirtualenv -p /usr/bin/python(2 or 3) {name}.
Therefore, should I only install virtualenv and virtualenvwrapper on one of the Python versions and use the aforementioned feature? Or should I install virtualenv and virtualenvwrapper on both versions of Python.
Would there be any conflicts?
Edit
More importantly, assuming that I have virtualenv and virtualenvwrapper installed for both Python 2.7.5 and Python 3.6.6, which version's command is called when I run any of the following: workon, mkvirtualenv, rmvirtualenv, etc.?
Would there be any conflicts?
Not until you mistakenly run the default system python command with a script that's using the opposite version as compared to the more specific python2 or python3 commands.
The virtualenvs do not conflict, and must be activated to be used. You can also of course have as many virtualenv's as you wish.
To avoid any problems setting up an environment, its suggested to run python2 -m virtualenv for example, rather than simply virtualenv command itself
For the commands listed at the bottom of the question, it depends on how your PATH is configured. Personally, I use pyenv rather than virtualenv directly, which injects itself into the OS PATH variable
I have two Anaconda installations on my computer. The first one is based on Python 2.7 and the other is based on Python 3.4. The default Python version is the 3.4 though. What is more, I can start Python 3.4 either by typing /home/eualin/.bin/anaconda3/bin/python or just python. I can do the same but for Python 2.7 by typing /home/eualin/.bin/anaconda2/bin/python. My problem is that I don't know how to install new libraries under certain environments (either under Python 2.7 or Python 3.4). For example, when I do pip install seaborn the library gets installed under Python 3.4 by default when in fact I want to install it under Python 2.7. Any ideas?
EDIT
This is what I am doing so far: the ~/.bashrc file contains the following two blocks, of which only one is enabled at any given time.
# added by Anaconda 2.1.0 installer
export PATH="/home/eualin/.bin/anaconda2/bin:$PATH"
# added by Anaconda3 2.1.0 installer
#export PATH="/home/eualin/.bin/anaconda3/bin:$PATH"
Depending of which version I want to work, I open the fie, comment the opposite block and do source ~/.bashrc Then, I install the libraries I want to use one by one. But, is this the recommended way?
You don't need multiple anaconda distributions for different python versions. I would suggest keeping only one.
conda basically lets you create environments for your different needs.
conda create -n myenv python=3.3 creates a new environment named myenv, which works with a python3.3 interpreter.
source activate myenv switches to the newly created environment. This basically sets the PATH such that pip, conda, python and other binaries point to the correct environment and interpreter.
conda install pip is the first thing you may want to do. Afterwards you can use pip and conda to install the packages you need.
After activating your environment pip install <mypackage> will point to the right version of pip so no need to worry too much.
You may want to create environments for different python versions or different sets of packages. Of course you can easily switch between those environments using source activate <environment name>.
For more examples and details you may want to have a look at the docs.
Virtualenv seems like the obvious answer here, but I do want to suggest an alternative that we've been using to great effect lately: Fig - this is particularly effective since we use Docker in production as well, but I imagine that using Fig as a replacement for virtualenv would be quite effective regardless of your production environment.
Using virtualenv is your best option as #Dettorer has mentioned.
I found this method of installing and using virtualenv the most useful.
Check it out:
Proper way to install virtualenv
So I have read this - https://wiki.archlinux.org/index.php/Python
And it is clear from this wiki that I can install Python 2.7.2 via
pacman -S python2
Is it reasonable for me to create a symlink to python2
ln -s python2 /usr/bin/python
if I don't forsee myself switching to python 3.0 any time soon? Or is there a better way of managing multiple python versions like what I usually use on a debian system (update-alternatives --config python) or on a mac os x system (python select)?
CLARIFICATION:
What I am trying to find out is - what is the "best practice" of managing various python versions on an archlinux system?
I am new to archlinux but familiar with ubuntu, debian and mac os x
I would argue you shouldn't create any symlinks like this at all. Especially if you are going to distribute some of your python code, you should not assume a user has python2 or python3 at /usr/bin/python.
If your script requires python2, just use:
#!/usr/bin/env python2
If your script requires python3, use:
#!/usr/bin/env python3
This way your scripts will work fine even through updates to Python. It will also be much more clear what version your script actually needs.
Most unices already have a /usr/bin/python. Overwriting that one is a bad idea, as this is the Python version used by all packages in the system, and changing that one may break them. When installing the Python 2.7 package the executable should be installed as /usr/bin/python2.7 (if not I would claim Archlinux is broken) and it's better to use that when you want to run Python 2.7.
Archlinux is a bit special, since it will use /usr/bin/python for Python 3, despite the default executable name for Python 3 being /usr/bin/python3. This is confusing and can be seen as a bug, but it does mean you can't use that symlink for Python 2, as any other Archlinux script that uses Python 3 will almost certainly break if you do.
So where on other Unices, symlinking /usr/bin/python to Python 2.7 is a bad idea, on Archlinux it is a terrible idea. Instead just install all version you need and call them with /usr/bin/pythonX.X.
There is a nice project on github what helps you with it's called pyenv
What helps you to manage multiple python instances
As others have said, the short answer is "don't do this, it will most likely break things on your system", however, if you mostly use Python 2 you can still set your personal default in your shell (and still have the option of switching to Python 3 at any time). To do this, first become root and install python2-virtualenv:
# pacman -S python2-virtualenv
Then create a virtual environment that uses Python 2 (this will automatically install Python, setuptools, wheel, and pip in the environment):
$ virtualenv -p /usr/bin/python2 --system-site-packages ~/env # (Or wherever you want your environment to live)
If you only want to use locally installed packages (eg. packages you install with pip and not the ones installed by pacman) remove the --system-site-packages option when creating your environment.
Now in your ~/.bash_profile or ~/.profile (or whatever your preferred shells configuration file is), set something like the following:\
source ~/env/bin/activate
This will activate the virtual environment, making your default version Python 2.
This could still break anything that is launched in a shell, but it's not likely that anything will be unless you're explicitly running it from a shell, at which point you can turn the virtual environment off by running:
deactivate
or simply manually run Python 3.
I just stumbled over this post, no necro-bumping intended but I was wondering nobody mentioned virtualenvs. I'm using ArchLinux as well and I use the python packages virtualenv and virtualenvwrapper to create multiple python environments. You can reference to the python 2 or python3 binaries in /usr/bin/ to determine the python version used in the virtual environment.
The benefit is that packages installed in a virtual environment don't mess with the python the system is using and there are many ways to automate project handling.
I know this may be a very old answer, but it took me two days to solve the problem so I'm going to share.
The proper way to manage python versions in your system to work on different projects without them driving you crazy is to use pyenv and its plugins pyenv-virtualenv and pyenv-virtualenvwrapper as described by Henrique Bastos into this blog post. Notice that this way to work is kinda platform independent, since pyenv is a python package and it can be run quite similarly on Windows, Linux and Mac OSx.
The problems start with Arch Linux. The OS doesn't provide a pacman version of pyenv, so you have to install it cloning it from github as described in the installation section of the release. The installation process is the same both for pyenv-virtualenv and pyenv-virtualenvwrapper. Notice that the shell initialization configuration may be different, in my case it didn't work for ~/.bash_profile, but worked for ~/.bashrc .
Running pyenv is not straightforward if your installation is very fresh like the one I'm setting up in these days, since pip requires openSSL and even if you install it via pacman, pyenv doesn't see it. So, if you want to install an older version of Python (namely 3.4.3), you will find the shell complaining about you haven't installed the openSSL plugin, even if you have it. To be honest, I didn't have the right packages the first time I tried to install it; you have to download the following packages
sudo pacman -S openssl
sudo pacman -S openssl-1.0
sudo pacman -S python-pyopenssl
sudo pacman -S python2-pyopenssl
The way I solved the problem is to add the flags as described in the pyenv installation FAQs: that solution eventually led me to install the python version I wanted:
LDFLAGS="-L/usr/lib/openssl-1.0" \
CFLAGS="-I/usr/include/openssl-1.0" \
pyenv install -v 3.4.3
To avoid to go on the FAQs page everytime you want to renew the python installation environment, you can add an alias in ~/.bashrc or whatever is you shell as follows:
echo alias pyenv='LDFLAGS="-L/usr/lib/openssl-1.0" \
CFLAGS="-I/usr/include/openssl-1.0" \
pyenv' >> ~/.bashrc
In this way you can install python properly with a clean pyenv syntax, and manage it via its plugins in the same way (since the syntax is pyenv [COMMAND] [OTHERSTUFF]).
No, there is no better way to do this. The python symlink is part of the Python 3 package.
I guess changing this link won't break anything for now but it might be possible that some packages will depend on it in the future.