Virtualenv doesn't use right version of Python - python

I'm working in Amazon's Cloud9.
ec2-user:~/environment/flask_init $ python -V
Python 2.7.14
ec2-user:~/environment/flask_init $ virtualenv -p python3 venv
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/ec2-user/environment/flask_init/venv/bin/python3
Also creating executable in /home/ec2-user/environment/flask_init/venv/bin/python
Installing setuptools, pip, wheel...done.
ec2-user:~/environment/flask_init $ source venv/bin/activate
(venv) ec2-user:~/environment/flask_init $ python -V
Python 2.7.14
Why is the virtual environment not using Python 3?
Please note that this question is not a duplicate of this one. The issue was specifically to do with the way the Cloud 9 environment sets up Python alias.

I tried your flow on my machine and everything works as expected.
dluzak#Karol-PC:/tmp$ python -V
Python 2.7.12
dluzak#Karol-PC:/tmp$ virtualenv -p python3 venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /tmp/venv/bin/python3
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
dluzak#Karol-PC:/tmp$ source venv/bin/activate
(venv) dluzak#Karol-PC:/tmp$ python -V
Python 3.5.2
(venv) dluzak#Karol-PC:/tmp$
Nonetheless I personally use virtualenv as module when creating venv with python 3: python3 -m virtualenv venv. Maybe this would work.
You provided very little details. Have you installed virtualenv for both Python 2 and 3? Are you sure Python 3 interpreter works fine?
Edit:
After investigation in comments we found out that the problem was in bash settings configured by Amazon. It seams that Amazon configures bash (probably in ~/.bashrc) to replace python calls with an alias. To fix this a call unalias python before enabling venv is needed. It is described in Amazon docs

When I was using virtualenv earlier today, I had the same problem that my env was not using the right version of python.
Instead of activating my environment like this:
source activate
I found that activating it like this actually worked:
source ./activate
Hope this is helpful!

Here is how i create virtualenv on Cloud9
Python 3.4
$ sudo pip install virtualenv
$ virtualenv -p /usr/bin/python3.4 venv
$ source venv/bin/activate
Python 3.6
$ sudo apt update
$ sudo apt install python3.6-venv
$ python3.6 -mvenv venv
$ source venv/bin/activate

I have encountered a similar issue.
In my case did not work because I moved the virtual env folder (but the same thing happens when you rename it).
You can understand which version of python (and thus which module will import) is using by typing
$ which python
If it write something like:
/usr/bin/python
Then it means your virtual env is not being activated.
To solve this issue, instead of creating a new virtual environment, you can simply edit the script activation file in your env:
$ nano venv/bin/activate
And edit the following line with your absolute path of your virtual environment:
VIRTUAL_ENV="/YOUR_ABSOLUT/PATH_TO/venv"
Hope it helps
:)

Related

How to ensure I use the correct python version when I have multiple versions of Python installed? [duplicate]

This question already has answers here:
Dealing with multiple Python versions and PIP?
(28 answers)
Closed 1 year ago.
I have python 3.7.8 and python 3.10.0 installed. I have a question regarding the dependencies of these two. Is it okay to have this two installed at the same time but in different versions?
Does it make my site-packages conflict with one another?
On my command prompt, when I check my python version it says I am using the recent version, 3.10.0.
Having multiple versions on the same machine is perfectly "okay", as long as you understand how to select/use the correct python version, which is the more pertinent and useful question you should be asking yourself.
They would normally be installed in separate locations and would have separate site-packages (from How do I find the location of my Python site-packages directory?):
~$ python3.7 -c 'import site; print(site.getsitepackages())'
['/usr/local/Cellar/python#3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages']
~$ python3.8 -c 'import site; print(site.getsitepackages())'
['/usr/local/Cellar/python#3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages']
~$ python3.9 -c 'import site; print(site.getsitepackages())'
['/usr/local/Cellar/python#3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages']
The recommended way however is to not install to these site-packages folder directly and use a virtual environment instead. There are many variations of virtual env packages/tools, but you can start with the Python docs on Virtual Environments if you are not familiar or using one yet.
With virtual environments, you can usually indicate which version of python to use when creating the virtual env, so that whenever you activate that same env, it would use the same version you used to create it.
tmp$ python3.7 -m venv app1
tmp$ source ./app1/bin/activate
(app1) tmp$ python -V
Python 3.7.12
(app1) tmp$
(app1) tmp$ deactivate
tmp$ python3.8 -m venv app2
tmp$ source ./app2/bin/activate
(app2) tmp$ python -V
Python 3.8.12
(app2) tmp$
(app2) tmp$ deactivate
tmp$ python3.9 -m venv app3
tmp$ source ./app3/bin/activate
(app3) tmp$ python -V
Python 3.9.9
(app3) tmp$
(app3) tmp$ deactivate
In the above example, I created 3 virtual environments for 3 hypothetical apps, each using a different Python version. As long as I activate the correct virtual environment, I don't have to think about which version python refers to, as it will use the same version used to create the env. See also How do I check what version of Python is running my script?.
As for installing packages, again, once you have setup the correct virtual environments, doing pip install would ensure it installs only on the site-packages on that environment, and the app running on that env would be able to import that package.
If not using a virtual environment, as I noted in my comment, the answers at Dealing with multiple Python versions and PIP? provide good suggestions on how to ensure you are installing packages for the correct Python version, namely with
$ </path/or/alias/to/specific/python/installation> -m pip install <packages>
$ python3.7 -m pip install "flake8<=3.6"
$ python3.7 -m pip list | grep flake8
flake8 3.6.0
$ ls -H /usr/local/Cellar/python#3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages | grep flake8
flake8
flake8-3.6.0.dist-info
flake8_quotes
flake8_quotes-3.2.0.dist-info
$ python3.9 -m pip install flake8
$ python3.9 -m pip list | grep flake8
flake8 4.0.1
$ ls -H /usr/local/Cellar/python#3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages | grep flake8
flake8
flake8-4.0.1.dist-info
The above example shows 2 different versions of flake8 installed for 2 different versions of Python, each on their own site-packages folder.
But when I run my command prompt and check my python version it says I am using the recent version. The 3.10.0.
This means the system default for python or python3 on your machine points to the 3.10 installation. Mine is set to point to Python 3.9:
tmp$ python3 -V
Python 3.9.9
tmp$ which python3
/usr/local/bin/python3
tmp$ /usr/local/bin/python3 -V
Python 3.9.9
You can:
Manually change the default version
Setup aliases to the different versions:
tmp$ type python3.7
python3.7 is aliased to `/usr/local/opt/python#3.7/bin/python3'
tmp$ type python3.8
python3.8 is aliased to `/usr/local/opt/python#3.8/bin/python3'
tmp$ type python3.9
python3.9 is aliased to `/usr/local/opt/python#3.9/bin/python3'
Use a version management tool for switching versions, such as pyenv
To add onto Gino Mempin's answer it's also worth noting that in Windows you can quickly switch between versions of Python by editing the PATH environment variable in System Variables under your Environment Variables.
From my experience most people who switch between Python versions are on Linux but for all those people who might be using Windows you can just go to This PC -> Right-click -> Properties -> Advanced System Settings then from System Properties go to the Advanced tab and select Environment Variables to edit PATH. Note you don't have to delete older versions of Python here you can just add other ones and then select to move them above the others to give them priority.
You'll also need to restart your Command Prompt or Powershell after doing this if you had it open and was using Python before editing the environment variables.
After doing this you'll be able to install and manage packages for the current version of Python that has priority.

Make the 'python' phrase refer to Python 3 on Linux?

So whenever I run python -v it says I'm using Python 2, why is python reserved for Python 2 on Ubuntu, and is there a way to make python reserve python 3? The latest version.
For example, Id prefer to write
`python /files/app.py`
To execute my files than write
`python3 /files/app.py`
It seems messy to have to add the 3, you don't on Windows, so why is Ubuntu different?
You can use an alias to do this. Add the following line to your .bashrc:
alias python=python3
Don't forget to reopen your terminal, or do source ~/.bashrc for the changes to take effect.
Duplicate of this post.
Ubuntu does this because you probably have two versions of Python installed, a system variant and a 3.x variant you are using. You can try python --version and python3 --version to see. If they return different things, you have two installations.
For example:
$ python --version
Python 2.7.6
$ python3 --version
Python 3.4.3
$ alias python=python3
$ python --version
Python 3.4.3
To circumvent the alias use the command built-in command:
$ command python --version
Python 2.7.6
Another way to circumvent the alias is to use \ before the command.
$ \python --version
Python 2.7.6
To disable the alias in the current shell use the unalias built-in command:
$ unalias python
$ python --version
Python 2.7.6
As already stated here, you have system python and user python and one option is to create aliases for python and pip on a user level ~/.bashrc
Another option is to create a virtual environment for each project you work on. The advantage of this is that you only install modules relevant to each project.
Create a virtual environment
mkdir someproject
cd someproject/
python3 -m venv venv
Activate environment
source venv/bin/activate
Check python and pip version
$ pip -V
pip 9.0.1 from ~/someproject/venv/lib/python3.6/site-packages (python 3.6)
$ python -V
Python 3.6.7
Deactivate environment
deactivate
Python, pip and any other modules you decide to install is installed in the venv folder instead of system wide.
Only gotcha is to remember to activate your environment.

Using Python 3 in virtualenv

Using virtualenv, I run my projects with the default version of Python (2.7). On one project, I need to use Python 3.4.
I used brew install python3 to install it on my Mac. Now, how do I create a virtualenv that uses the new version?
e.g. sudo virtualenv envPython3
If I try:
virtualenv -p python3 test
I get:
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4'
New python executable in test/bin/python3.4
Also creating executable in test/bin/python
Failed to import the site module
Traceback (most recent call last):
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/site.py", line 67, in <module>
import os
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/os.py", line 634, in <module>
from _collections_abc import MutableMapping
ImportError: No module named '_collections_abc'
ERROR: The executable test/bin/python3.4 is not functioning
ERROR: It thinks sys.prefix is '/Users/user/Documents/workspace/test' (should be '/Users/user/Documents/workspace/test/test')
ERROR: virtualenv is not compatible with this system or executable
simply run
virtualenv -p python3 envname
Update after OP's edit:
There was a bug in the OP's version of virtualenv, as described here. The problem was fixed by running:
pip install --upgrade virtualenv
Python 3 has a built-in support for virtual environments - venv. It might be better to use that instead. Referring to the docs:
Creation of virtual environments is done by executing the pyvenv
script:
pyvenv /path/to/new/virtual/environment
Update for Python 3.6 and newer:
As pawciobiel correctly comments, pyvenv is deprecated as of Python 3.6 and the new way is:
python3 -m venv /path/to/new/virtual/environment
I'v tried pyenv and it's very handy for switching python versions (global, local in folder or in the virtualenv):
brew install pyenv
then install Python version you want:
pyenv install 3.5.0
and simply create virtualenv with path to needed interpreter version:
virtualenv -p /Users/johnny/.pyenv/versions/3.5.0/bin/python3.5 myenv
That's it, check the version:
. ./myenv/bin/activate && python -V
There are also plugin for pyenv pyenv-virtualenv but it didn't work for me somehow.
Install prerequisites.
sudo apt-get install python3 python3-pip virtualenvwrapper
Create a Python3 based virtual environment. Optionally enable --system-site-packages flag.
mkvirtualenv -p /usr/bin/python3 <venv-name>
Set into the virtual environment.
workon <venv-name>
Install other requirements using pip package manager.
pip install -r requirements.txt
pip install <package_name>
When working on multiple python projects simultaneously it is usually recommended to install common packages like pdbpp globally and then reuse them in virtualenvs.
Using this technique saves a lot of time spent on fetching packages and installing them, apart from consuming minimal disk space and network bandwidth.
sudo -H pip3 -v install pdbpp
mkvirtualenv -p $(which python3) --system-site-packages <venv-name>
Django specific instructions
If there are a lot of system wide python packages then it is recommended to not use --system-site-packages flag especially during development since I have noticed that it slows down Django startup a lot. I presume Django environment initialisation is manually scanning and appending all site packages from the system path which might be the reason. Even python manage.py shell becomes very slow.
Having said that experiment which option works better. Might be safe to just skip --system-site-packages flag for Django projects.
virtualenv --python=/usr/bin/python3 <name of env>
worked for me.
This is all you need, in order to run a virtual environment in python / python3
First if virtualenv not installed, run
pip3 install virtualenv
Now Run:
virtualenv -p python3 <env name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Sometime the cmd virtualenv fails, if so use this:
python3 -m virtualenv <env_name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Now activate the virtual env:
source <env_name>/bin/activate
Or:
source `pwd`/<env_name>/bin/activate
Now run
which python
You should see the full path to your dir and <env_name>/bin/python suffix
To exit the virtualenv, run:
deactivate
To troubleshoot Python location got to here
You can specify specific Version of Python while creating environment.
It's mentioned in virtualenv.py
virtualenv --python=python3.5 envname
In some cases this has to be the full path to the executable:
virtualenv --python=/Users/username/.pyenv/versions/3.6.0/bin/python3.6 envname
How -p works
parser.add_option(
'-p', '--python',
dest='python',
metavar='PYTHON_EXE',
help='The Python interpreter to use, e.g., --python=python3.5 will use the python3.5 '
'interpreter to create the new environment. The default is the interpreter that '
'virtualenv was installed with (%s)' % sys.executable)
I had the same ERROR message. tbrisker's solution did not work in my case. Instead this solved the issue:
$ python3 -m venv .env
In addition to the other answers, I recommend checking what instance of virtualenv you are executing:
which virtualenv
If this turns up something in /usr/local/bin, then it is possible - even likely - that you installed virtualenv (possibly using an instance of easy_tools or pip) without using your system's package manager (brew in OP's case). This was my problem.
Years ago - when I was even more ignorant - I had installed virtualenv and it was masking my system's package-provided virtualenv.
After removing this old, broken virtualenv, my problems went away.
The below simple commands can create a virtual env with version 3.5
apt-get install python3-venv
python3.5 -m venv <your env name>
if you want virtual env version as 3.6
python3.6 -m venv <your env name>
Python now comes with its own implementation of virtual environment, by the name of "venv". I would suggest using that, instead of virtualenv.
Quoting from venv - docs,
Deprecated since version 3.6: pyvenv was the recommended tool for
creating virtual environments for Python 3.3 and 3.4, and is
deprecated in Python 3.6.
Changed in version 3.5: The use of venv is now recommended for
creating virtual environments.
For windows, to initiate venv on some project, open cmd:
python -m venv "c:\path\to\myenv"
(Would suggest using double quote around directory path if it contains any spaces. Ex: "C:/My Dox/Spaced Directory/Something")
Once venv is set up, you will see some new folders inside your project directory. One of them would be "Scripts".
To activate or invoke venv you need:
C:\> <venv>\Scripts\activate.bat
You can deactivate a virtual environment by typing “deactivate” in your shell. With this, you are now ready to install your project specific libraries, which will reside under the folder "Lib".
================================ Edit 1 ====================================
The scenario which will be discussed below is not what originally asked, just adding this in case someone use vscode with python extension
In case, you use vs code with its python extension, you might face an issue with its pylint which points to the global installation. In this case, pylint won't be able to see the modules that are installed in your virtual environment and hence will show errors while importing.
Here is a simple method to get past this.
cd Workspace\Scripts
.\Activate.ps1
code .
We are basically activating the environment first and then invoking vs-code so that pylint starts within the environment and can see all local packages.
In python3.6 I tried
python3 -m venv myenv,
as per the documentation, but it was taking so long. So the very simple and quick command is
python -m venv yourenv
It worked for me on python3.6.
On Mac I had to do the following to get it to work.
mkvirtualenv --python=/usr/bin/python3 YourEnvNameHere
If you install python3 (brew install python3) along with virtualenv burrito, you can then do mkvirtualenv -p $(which python3) env_name
Of course, I know virtualenv burrito is just a wrapper, but it has served me well over the years, reducing some learning curves.
virtualenv --python=/usr/local/bin/python3 <VIRTUAL ENV NAME>
this will add python3
path for your virtual enviroment.
It worked for me
virtualenv --no-site-packages --distribute -p /usr/bin/python3 ~/.virtualenvs/py3
For those having troubles while working with Anaconda3 (Python 3).
You could use
conda create -n name_of_your_virtualenv python=python_version
To activate the environment ( Linux, MacOS)
source activate name_of_your_virtualenv
For Windows
activate name_of_your_virtualenv
I tried all the above stuff, it still didn't work. So as a brute force, I just re-installed the anaconda, re-installed the virtualenv... and it worked.
Amans-MacBook-Pro:~ amanmadan$ pip install virtualenv
You are using pip version 6.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting virtualenv
Downloading virtualenv-15.0.3-py2.py3-none-any.whl (3.5MB)
100% |████████████████████████████████| 3.5MB 114kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-15.0.3
Amans-MacBook-Pro:python amanmadan$ virtualenv my_env
New python executable in /Users/amanmadan/Documents/HadoopStuff/python/my_env/bin/python
Installing setuptools, pip, wheel...done.
Amans-MacBook-Pro:python amanmadan$
I wanted to keep python 2.7.5 as default version on Centos 7 but have python 3.6.1 in a virtual environment running alongside other virtual environments in python 2.x
I found the below link the best solution for the newest python version ( python 3.6.1)
https://www.digitalocean.com/community/tutorial_series/how-to-install-and-set-up-a-local-programming-environment-for-python-3.
It shows the steps for different platforms but the basic steps are
Install python3.x (if not present) for your platform
Install python3.x-devel for your platform
Create virtual environment in python 3.x
(for example $ python3.6 -m venv virenv_test_p3/ )
Activate the testenvironment for python 3.x
(for example source virenv_test_p3/bin/activate)
Install the packages which you want to use in your new python 3 virtual environment and which are supported ( for example pip install Django==1.11.2)
On Windows command line, the following worked for me. First find out where your python executables are located:
where python
This will output the paths to the different python.exe on your system. Here were mine:
C:\Users\carandangc\Anaconda3\python.exe
C:\Python27\python.exe
So for Python3, this was located in the first path for me, so I cd to the root folder of the application where I want to create a virtual environment folder. Then I run the following which includes the path to my Python3 executable, naming my virtual environment 'venv':
virtualenv --python=/Users/carandangc/Anaconda3/python.exe venv
Next, activate the virtual environment:
call venv\Scripts\activate.bat
Finally, install the dependencies for this virtual environment:
pip install -r requirements.txt
This requirements.txt could be populated manually if you know the libraries/modules needed for your application in the virtual environment. If you had the application running in another environment, then you can automatically produce the dependencies by running the following (cd to the application folder in the environment where it is working):
pip freeze > requirements.txt
Then once you have the requirements.txt that you have 'frozen', then you can install the requirements on another machine or clean environment with the following (after cd to the application folder):
pip install -r requirements.txt
To see your python version in the virtual environment, run:
python --version
Then voila...you have your Python3 running in your virtual environment. Output for me:
Python 3.7.2
For those of you who are using pipenv and want to install specific version:
pipenv install --python 3.6
I got the same error due to it being a conflict with miniconda3 install so when you type "which virtualenv" and if you've installed miniconda and it's pointing to that install you can either remove it (if your like me and haven't moved to it yet) or change your environment variable to point to the install you want.

Debug installation of pyenv-virtualenvwrapper

I am trying to install and use pyenv-virtualenvwrapper to create a virtual python environment. I am on ubuntu 13.04.
$ pyenv version
2.7.5 (set by /home/evan/.pyenv/version)
$ pip freeze | grep virtual
virtualenv==1.10.1
virtualenv-clone==0.2.4
virtualenvwrapper==4.1.1
$ ls ~/.pyenv/plugins/
pyenv-virtualenvwrapper python-build version-ext-compat
$ pyenv virtualenvwrapper
virtualenvwrapper not installed.
$pyenv which virtualenvwrapper.sh
pyenv: virtualenvwrapper.sh: command not found
$ pyenv whence virtualenvwrapper.sh
$
$ which $HOME/.pyenv/shims/pip
/home/evan/.pyenv/shims/pip
Has anyone else worked their way through this issue? I would like to create a virtual environment. In pythonbrew which I used in prior versions of ubuntu, I would have just..
$ mkvirtualenv dssg
$ workon dssg
(dssg) $
Here:
$ mkvirtualenv dssg
mkvirtualenv: command not found
$ pyenv mkvirtualenv
mkvirtualenv: no such command `mkvirtualenv'
$ pyenv virtualenvwrapper mkvirtualenv
virtualenvwrapper not installed.
$
In order to create a virtual environment in a non-system (virtual) version of python under the control pyenv, you need to use a slightly different syntax. From the documentation of virtualenv plugin
$ pyenv virtualenv venv33
For me this only works if I am in the directory of the non-system version of python ~/.pyenv/versions/2.7.5. I have a tutorial that walks you through this at http://mofj.commons.gc.cuny.edu/2014/01/20/ubuntu-set-up-a-virtual-environment-with-ipython-numpy-and-pandas/
This is actually something really silly an unintuitive about how virtualenvwrapper works on Ubuntu. On most operating systems the following command will let you source virtualenvwrapper.sh:
source `which virtualenvwrapper.sh`
However, for some strange reason, it is instead stored in /etc/bash_completion.d/virtualenvwrapper
So if you want access to the mkvirtualenv command you need to source it in the following way:
source /etc/bash_completion.d/virtualanevwrapper
Depending on how often you make virtualenvs it might be worth making an alias for that just because it's unintuitive, but I usually don't use this enough that it's worth it.
In the future if you're trying to locate something you should read up on the find and locate functions.

Upgrade python in a virtualenv

Is there a way to upgrade the version of python used in a virtualenv (e.g. if a bugfix release comes out)?
I could pip freeze --local > requirements.txt, then remove the directory and pip install -r requirements.txt, but this requires a lot of reinstallation of large libraries, for instance, numpy, which I use a lot.
I can see this is an advantage when upgrading from, e.g., 2.6 -> 2.7, but what about 2.7.x -> 2.7.y?
If you happen to be using the venv module that comes with Python 3.3+, it supports an --upgrade option.
Per the docs:
Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place
python3 -m venv --upgrade ENV_DIR
Did you see this? If I haven't misunderstand that answer, you may try to create a new virtualenv on top of the old one. You just need to know which python is going to use your virtualenv (you will need to see your virtualenv version).
If your virtualenv is installed with the same python version of the old one and upgrading your virtualenv package is not an option, you may want to read this in order to install a virtualenv with the python version you want.
EDIT
I've tested this approach (the one that create a new virtualenv on top of the old one) and it worked fine for me. I think you may have some problems if you change from python 2.6 to 2.7 or 2.7 to 3.x but if you just upgrade inside the same version (staying at 2.7 as you want) you shouldn't have any problem, as all the packages are held in the same folders for both python versions (2.7.x and 2.7.y packages are inside your_env/lib/python2.7/).
If you change your virtualenv python version, you will need to install all your packages again for that version (or just link the packages you need into the new version packages folder, i.e: your_env/lib/python_newversion/site-packages)
Updated again:
The following method might not work in newer versions of virtualenv. Before you try to make modifications to the old virtualenv, you should save the dependencies in a requirement file (pip freeze > requirements.txt) and make a backup of it somewhere else. If anything goes wrong, you can still create a new virtualenv and install the old dependencies in it (pip install -r requirements.txt).
Updated: I changed the answer 5 months after I originally answered. The following method is more convenient and robust.
Side effect: it also fixes the Symbol not found: _SSLv2_method exception when you do import ssl in a virtual environment after upgrading Python to v2.7.8.
Notice: Currently, this is for Python 2.7.x only.
If you're using Homebrew Python on OS X, first deactivate all virtualenv, then upgrade Python:
brew update && brew upgrade python
Run the following commands (<EXISTING_ENV_PATH> is path of your virtual environment):
cd <EXISTING_ENV_PATH>
rm .Python
rm bin/pip{,2,2.7}
rm bin/python{,2,2.7}
rm -r include/python2.7
rm lib/python2.7/*
rm -r lib/python2.7/distutils
rm lib/python2.7/site-packages/easy_install.*
rm -r lib/python2.7/site-packages/pip
rm -r lib/python2.7/site-packages/pip-*.dist-info
rm -r lib/python2.7/site-packages/setuptools
rm -r lib/python2.7/site-packages/setuptools-*.dist-info
Finally, re-create your virtual environment:
virtualenv <EXISTING_ENV_PATH>
By doing so, old Python core files and standard libraries (plus setuptools and pip) are removed, while the custom libraries installed in site-packages are preserved and working, as soon as they are in pure Python. Binary libraries may or may not need to be reinstalled to function properly.
This worked for me on 5 virtual environments with Django installed.
BTW, if ./manage.py compilemessages is not working afterwards, try this:
brew install gettext && brew link gettext --force
Step 1: Freeze requirement & take a back-up of existing env
pip freeze > requirements.txt
deactivate
mv env env_old
Step 2: Install Python 3.7 & activate virutal environment
sudo apt-get install python3.7-venv
python3.7 -m venv env
source env/bin/activate
python --version
Step 3: Install requirements
sudo apt-get install python3.7-dev
pip3 install -r requirements.txt
How to upgrade the Python version for an existing virtualenvwrapper project and keep the same name
I'm adding an answer for anyone using Doug Hellmann's excellent virtualenvwrapper specifically since the existing answers didn't do it for me.
Some context:
I work on some projects that are Python 2 and some that are Python 3; while I'd love to use python3 -m venv, it doesn't support Python 2 environments
When I start a new project, I use mkproject which creates the virtual environment, creates an empty project directory, and cds into it
I want to continue using virtualenvwrapper's workon command to activate any project irrespective of Python version
Directions:
Let's say your existing project is named foo and is currently running Python 2 (mkproject -p python2 foo), though the commands are the same whether upgrading from 2.x to 3.x, 3.6.0 to 3.6.1, etc. I'm also assuming you're currently inside the activated virtual environment.
1. Deactivate and remove the old virtual environment:
$ deactivate
$ rmvirtualenv foo
Note that if you've added any custom commands to the hooks (e.g., bin/postactivate) you'd need to save those before removing the environment.
2. Stash the real project in a temp directory:
$ cd ..
$ mv foo foo-tmp
3. Create the new virtual environment (and project dir) and activate:
$ mkproject -p python3 foo
4. Replace the empty generated project dir with real project, change back into project dir:
$ cd ..
$ mv -f foo-tmp foo
$ cdproject
5. Re-install dependencies, confirm new Python version, etc:
$ pip install -r requirements.txt
$ python --version
If this is a common use case, I'll consider opening a PR to add something like $ upgradevirtualenv / $ upgradeproject to virtualenvwrapper.
Let's consider that the environment that one wants to update has the name venv.
1. Backup venv requirements (optional)
First of all, backup the requirements of the virtual environment:
pip freeze > requirements.txt
deactivate
#Move the folder to a new one
mv venv venv_old
2. Install Python
Assuming that one doesn't have sudo access, pyenv is a reliable and fast way to install Python. For that, one should run
curl https://pyenv.run | bash
and then
exec $SHELL
as suggested here.
If, when one tries to update pyenv
pyenv update
and one gets the error
bash: pyenv: command not found
that is because pyenv path wasn't exported to .bashrc. It can be solved by executing the following commands:
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bashrc
Then restart the shell
exec "$SHELL"
Now one should install the Python version that one wants. Let's say version 3.8.3
pyenv install 3.8.3
One can confirm if it was properly installed by running
pyenv versions
The output should be the location and the versions (in this case 3.8.3)
3. Create the new virtual environment
Finally, with the new Python version installed, create a new virtual environment (let's call it venv)
python3.8 -m venv venv
Activate it
source venv/bin/activate
and install the requirements
pip install -r requirements.txt
Now one should be up and running with a new environment.
I wasn't able to create a new virtualenv on top of the old one. But there are tools in pip which make it much faster to re-install requirements into a brand new venv. Pip can build each of the items in your requirements.txt into a wheel package, and store that in a local cache. When you create a new venv and run pip install in it, pip will automatically use the prebuilt wheels if it finds them. Wheels install much faster than running setup.py for each module.
My ~/.pip/pip.conf looks like this:
[global]
download-cache = /Users/me/.pip/download-cache
find-links =
/Users/me/.pip/wheels/
[wheel]
wheel-dir = /Users/me/.pip/wheels
I install wheel (pip install wheel), then run pip wheel -r requirements.txt. This stores the built wheels in the wheel-dir in my pip.conf.
From then on, any time I pip install any of these requirements, it installs them from the wheels, which is pretty quick.
I just want to clarify, because some of the answers refer to venv and others refer to virtualenv.
Use of the -p or --python flag is supported on virtualenv, but not on venv. If you have more than one Python version and you want to specify which one to create the venv with, do it on the command line, like this:
malikarumi#Tetuoan2:~/Projects$ python3.6 -m venv {path to pre-existing dir you want venv in}
You can of course upgrade with venv as others have pointed out, but that assumes you have already upgraded the Python that was used to create that venv in the first place. You can't upgrade to a Python version you don't already have on your system somewhere, so make sure to get the version you want, first, then make all the venvs you want from it.
This approach always works for me:
# First of all, delete all broken links. Replace my_project_name` to your virtual env name
find ~/.virtualenvs/my_project_name/ -type l -delete
# Then create new links to the current Python version
virtualenv ~/.virtualenvs/my_project_name/
# It's it. Just repeat for each virtualenv located in ~/.virtualenvs
Taken from:
https://github.com/1st/python-on-osx#python-virtualenv
https://gist.github.com/1st/ced02a1c64ac7b82bb27e432eea6b068
If you're using pipenv, I don't know if it's possible to upgrade an environment in place, but at least for minor version upgrades it seems to be smart enough not to rebuild packages from scratch when it creates a new environment. E.g., from 3.6.4 to 3.6.5:
$ pipenv --python 3.6.5 install
Virtualenv already exists!
Removing existing virtualenv…
Creating a v$ pipenv --python 3.6.5 install
Virtualenv already exists!
Removing existing virtualenv…
Creating a virtualenv for this project…
Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv…
⠋Running virtualenv with interpreter /usr/local/bin/python3.6m
Using base prefix '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/python3.6
Also creating executable in /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/python
Installing setuptools, pip, wheel...done.
Virtualenv location: /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD
Installing dependencies from Pipfile.lock (84dd0e)…
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 47/47 — 00:00:24
To activate this project's virtualenv, run the following:
$ pipenv shell
$ pipenv shell
Spawning environment shell (/bin/bash). Use 'exit' to leave.
. /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/activate
bash-3.2$ . /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/activate
(autoscale-aBUhewiD) bash-3.2$ python
Python 3.6.5 (default, Mar 30 2018, 06:41:53)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>>
I moved my home directory from one mac to another (Mountain Lion to Yosemite) and didn't realize about the broken virtualenv until I lost hold of the old laptop. I had the virtualenv point to Python 2.7 installed by brew and since Yosemite came with Python 2.7, I wanted to update my virtualenv to the system python. When I ran virtualenv on top of the existing directory, I was getting OSError: [Errno 17] File exists: '/Users/hdara/bin/python2.7/lib/python2.7/config' error. By trial and error, I worked around this issue by removing a few links and fixing up a few more manually. This is what I finally did (similar to what #Rockalite did, but simpler):
cd <virtualenv-root>
rm lib/python2.7/config
rm lib/python2.7/lib-dynload
rm include/python2.7
rm .Python
cd lib/python2.7
gfind . -type l -xtype l | while read f; do ln -s -f /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/${f#./} $f; done
After this, I was able to just run virtualenv on top of the existing directory.
On OS X or macOS using Homebrew to install and upgrade Python3 I had to delete symbolic links before python -m venv --upgrade ENV_DIR would work.
I saved the following in upgrade_python3.sh so I would remember how months from now when I need to do it again:
brew upgrade python3
find ~/.virtualenvs/ -type l -delete
find ~/.virtualenvs/ -type d -mindepth 1 -maxdepth 1 -exec python3 -m venv --upgrade "{}" \;
UPDATE: while this seemed to work well at first, when I ran py.test it gave an error. In the end I just re-created the environment from a requirements file.
For everyone with the problem
Error: Command '['/Users/me/Sites/site/venv3/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.
You have to install python3.6-venv
sudo apt-get install python3.6-venv

Categories

Resources