I have been through numerous similar questions and I still don't understand how are we activating virtual environment for Django projects. Please mention explanation for how each command works and one more questions is that why do we not need to install python in Django's virtual environment, I am getting confused. Thank you in advance, please help this newbie.
Benefits
You can use any version of python you want for a specific environment
without having to worry about collisions (shoutout to my python 2.7
mac users!)
You can organize your packages much better and know exactly the
packages you need to run your code incase someone else needs to run
it on their machine
Your main python package directory does not get FLOODED with
unnecessary python packages
To create virtual environment
step 1 install environment package (virtualenv) using pip
pip install virtualenv
step 2 create virtualenv
virtualenv env_name #<- env_name is virtualenv name you can set any
step 3 Activate Virtual env
env_name\Scripts\activate #<- for window
step 4 Install pakages you wan to install in virtual env
cmd(env_name): pip install django
Note that python is install in your virtual env automaticaty the
version is same as in your local machine
There is no difference between activating virtual environment for Django or for other purposes. Django on it's own does not differ from any Python library out there.
Python virtual environment allows you to separate your system Python and it's libraries and create self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
On Linux, assuming you already have Python 3 and pip3 installed:
# install virtualenv package (skip if you have it already)
pip3 install virtualenv
# create virtual environment in directory "tutorial-env"
python3 -m venv tutorial-env
# activate virtual environment
source tutorial-env/bin/activate
Upon activation below command should give path to new Python binary:
which python3
Similarly with pip3
which pip3
As long as your environment is activate you can run pip3 install $package_name and it will install it inside virtual environment.
To deactivate your virtual environment:
deactivate
For more info and commands for Windows:
https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/
https://docs.python.org/3/tutorial/venv.html
I am trying to properly install python and set up virtual environments to make project development easier. Im stuck on how to install pip now. My question is this:
Do I need to install pip in a specific way if I'm trying to set up virtual environments?
Here's what I've done so far...
installed pyenv
used pyenv to install python version 3.7, 3.8 and 3.9
set global pyenv to be python 3.9
Now I don't know hot to set up pip.
I did this:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
and then:
python get-pip.py
..however when I go to verify pip was installed i get 'pip command not found'.
How do I properly set up pip so that I can create virtual environments and install packages for different environments?
Do I need to add pip to my PATH? If so, how do I do that?
You can create virtual environment directly from python with venv
(documentation: https://docs.python.org/3/library/venv.html)
Example:
python3 -m venv /path/to/new/virtual/environment
Activate the virtual environment
source /path/to/new/virtual/environment/bin/activate
Then you can install any package with
pip package-to-install
Articles read:
Python Virtual Environments: A Primer,
Pyenv – Install Multiple Python Versions for Specific Project,
How to manage multiple Python versions and virtual environments
Let's suppose we have these directories:
~/Projects/PyA: uses Python 3.4.3 with Django 2.0
~/Projects/PyB: uses Python 3.5.1 with Django 2.1
~/Projects/PyC: uses Python 3.5.6 with Django 2.2
~/Projects/PyD: uses Python 3.2 with python-igraph
The first to do, we install the Python versions needed:
pyenv install 3.4.3
pyenv install 3.5.1
pyenv install 3.5.6
pyenv install 3.2
My questions start here:
Should I do this?
cd ~/Projects/PyA && pyenv local 3.4.3 && python3.4 -m venv proA
cd ~/Projects/PyB && pyenv local 3.5.1 && python3.5 -m venv proB
cd ~/Projects/PyC && pyenv local 3.5.6 && python3.5 -m venv proC
cd ~/Projects/PyD && pyenv local 3.2 && python3.2 -m venv proD
When is a unique directory for virtual environments used? Which option is recommended? Why?
How should I install the per-project packages listed above?
Should I use virtualenvwrapper?
How do I switch between projects (changing Python/virtual-environment in the process) easily or painlessly?
In Ruby, there is a file named Gemfile where I can set which gems (with their respective versions) are installed for the current project, which is a very good idea. Is there something similar for Python?
PS: I use Arch Linux as guest for a Vagrant box.
When is an unique directory for virtual environments used? Which
option is recommended? Why?
Every virtual environment "lives" in its own folder. All packages you install will go there, especially if every environment will have a different Python version.
How should I install per-project packages listed above?
When you switch to the project environment after you created it, see my original answer below. All packages installed will exclusively be installed into that virtual environment you are currently working in.
You can always check which Python interpreter is currently in use by typing
which python
in the terminal you currently have the the project environment activated. In addition you can also check
which pip
to make sure if you are installing using pip install somepackage that you target the correct Python interpreter. If you want to pin the packages, you can do
pip freeze > requirements.txt
any time and the currently installed packages plus their version will be written to the textfile requirements.txt. You can now always create a new environment using
pip install -r requirements.txt
Should I use virtualenvwrapper?
I would always work in a per-project virtual environment, so other projects that may use some pinned version of a special package are not influenced.
How do I switch between projects (changing Python/virtual-environment
in the process) easily or painlessly?
You could define an alias in your ~/.bashrc file or ~/.bash_aliases. In a terminal, open (in my example) the ~/.bashrc with a text editor, e.g., Vim/nano or one of your liking:
nano ~/.bashrc
and somewhere near the end you can add a line with an alias to switch to the project directory and activate the environment at the same time:
alias activate_proj1="cd ~/project_1 && pyenv activate venv_project_1"
so you only type activate_proj1 in the terminal (tab completion also works) and both commands are executed. Don't forget to source the bash-file again after you change something with source ~/.bashrc or just open a new terminal.
Original answer:
pyenv will handle everything you need:
My workflow (for one project to make it more readable) would be the following:
pyenv install 3.5.1
cd python_projects
mkdir myproject
cd myproject
pyenv virtualenv 3.5.1 venv_myproject
After that you can simply activate the virtualenv created by pyenv using
pyenv activate venv_myproject
which will open your distinct environment. Here you can do all the things you want, e.g., install your packages using pip etc.
After you completed setting up the environment, you can freeze the environment and create a requirements file:
pip freeze > requirements.txt
to be able to reconstruct the environment if needed. This way all the overhead that may be needed (setting a PATH etc.) will be handled by pyenv.
If you want to work on different projects, just activate the environment you need and off you go!
Note that you can make pyenv activate the virtualenv when you cd the folder in your terminal by putting its name into your .python-version file as well.
There's a lot going on in this question.
virtualenv workflows are usually pretty simple. You create a directory for your project, cd into it, and run virtualenv venv for a simple virtualenv, but you can also specify which Python executable you'd like in your virtual environment with a -p python3.5 for a Python 3.5 virtual environment, for instance.
There isn’t any magic going on here. You need Python 3.5 installed to create the Python 3.5 virtual environment. To activate this virtual environment, you simply source venv/bin/activate. Once activated, your shell should reflect which virtual environment you're operating in. You can even run which python to see that it's actually directed at the venv directory structure. Simple.
An analog to the Gemfile in Python would be similar to what most projects use as a requirements.txt. These can be generated trivially by running pip freeze > requirements.txt or installed by running pip install -r requirements.txt. Generally, this is done within the context of a virtual environment to avoid disrupting or clobbering your operating system's global Python packages.
Kenneth Reitz released a tool that incorporates virtualenv called pipenv and it looks very nice, but I've had some trouble breaking my habits of using virtualenv, and the truth is, virtualenv hasn't presented me with enough problems to deeply explore this new project, but your mileage may vary. Mr. Reitz's contributions to the Python community are overwhelmingly positive, so it's definitely worth a look.
Using virtualenv, I run my projects with the default version of Python (2.7). On one project, I need to use Python 3.4.
I used brew install python3 to install it on my Mac. Now, how do I create a virtualenv that uses the new version?
e.g. sudo virtualenv envPython3
If I try:
virtualenv -p python3 test
I get:
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4'
New python executable in test/bin/python3.4
Also creating executable in test/bin/python
Failed to import the site module
Traceback (most recent call last):
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/site.py", line 67, in <module>
import os
File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/os.py", line 634, in <module>
from _collections_abc import MutableMapping
ImportError: No module named '_collections_abc'
ERROR: The executable test/bin/python3.4 is not functioning
ERROR: It thinks sys.prefix is '/Users/user/Documents/workspace/test' (should be '/Users/user/Documents/workspace/test/test')
ERROR: virtualenv is not compatible with this system or executable
simply run
virtualenv -p python3 envname
Update after OP's edit:
There was a bug in the OP's version of virtualenv, as described here. The problem was fixed by running:
pip install --upgrade virtualenv
Python 3 has a built-in support for virtual environments - venv. It might be better to use that instead. Referring to the docs:
Creation of virtual environments is done by executing the pyvenv
script:
pyvenv /path/to/new/virtual/environment
Update for Python 3.6 and newer:
As pawciobiel correctly comments, pyvenv is deprecated as of Python 3.6 and the new way is:
python3 -m venv /path/to/new/virtual/environment
I'v tried pyenv and it's very handy for switching python versions (global, local in folder or in the virtualenv):
brew install pyenv
then install Python version you want:
pyenv install 3.5.0
and simply create virtualenv with path to needed interpreter version:
virtualenv -p /Users/johnny/.pyenv/versions/3.5.0/bin/python3.5 myenv
That's it, check the version:
. ./myenv/bin/activate && python -V
There are also plugin for pyenv pyenv-virtualenv but it didn't work for me somehow.
Install prerequisites.
sudo apt-get install python3 python3-pip virtualenvwrapper
Create a Python3 based virtual environment. Optionally enable --system-site-packages flag.
mkvirtualenv -p /usr/bin/python3 <venv-name>
Set into the virtual environment.
workon <venv-name>
Install other requirements using pip package manager.
pip install -r requirements.txt
pip install <package_name>
When working on multiple python projects simultaneously it is usually recommended to install common packages like pdbpp globally and then reuse them in virtualenvs.
Using this technique saves a lot of time spent on fetching packages and installing them, apart from consuming minimal disk space and network bandwidth.
sudo -H pip3 -v install pdbpp
mkvirtualenv -p $(which python3) --system-site-packages <venv-name>
Django specific instructions
If there are a lot of system wide python packages then it is recommended to not use --system-site-packages flag especially during development since I have noticed that it slows down Django startup a lot. I presume Django environment initialisation is manually scanning and appending all site packages from the system path which might be the reason. Even python manage.py shell becomes very slow.
Having said that experiment which option works better. Might be safe to just skip --system-site-packages flag for Django projects.
virtualenv --python=/usr/bin/python3 <name of env>
worked for me.
This is all you need, in order to run a virtual environment in python / python3
First if virtualenv not installed, run
pip3 install virtualenv
Now Run:
virtualenv -p python3 <env name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Sometime the cmd virtualenv fails, if so use this:
python3 -m virtualenv <env_name> # you can specify full path instead <env_name> to install the files in a different location other than the current location
Now activate the virtual env:
source <env_name>/bin/activate
Or:
source `pwd`/<env_name>/bin/activate
Now run
which python
You should see the full path to your dir and <env_name>/bin/python suffix
To exit the virtualenv, run:
deactivate
To troubleshoot Python location got to here
You can specify specific Version of Python while creating environment.
It's mentioned in virtualenv.py
virtualenv --python=python3.5 envname
In some cases this has to be the full path to the executable:
virtualenv --python=/Users/username/.pyenv/versions/3.6.0/bin/python3.6 envname
How -p works
parser.add_option(
'-p', '--python',
dest='python',
metavar='PYTHON_EXE',
help='The Python interpreter to use, e.g., --python=python3.5 will use the python3.5 '
'interpreter to create the new environment. The default is the interpreter that '
'virtualenv was installed with (%s)' % sys.executable)
I had the same ERROR message. tbrisker's solution did not work in my case. Instead this solved the issue:
$ python3 -m venv .env
In addition to the other answers, I recommend checking what instance of virtualenv you are executing:
which virtualenv
If this turns up something in /usr/local/bin, then it is possible - even likely - that you installed virtualenv (possibly using an instance of easy_tools or pip) without using your system's package manager (brew in OP's case). This was my problem.
Years ago - when I was even more ignorant - I had installed virtualenv and it was masking my system's package-provided virtualenv.
After removing this old, broken virtualenv, my problems went away.
The below simple commands can create a virtual env with version 3.5
apt-get install python3-venv
python3.5 -m venv <your env name>
if you want virtual env version as 3.6
python3.6 -m venv <your env name>
Python now comes with its own implementation of virtual environment, by the name of "venv". I would suggest using that, instead of virtualenv.
Quoting from venv - docs,
Deprecated since version 3.6: pyvenv was the recommended tool for
creating virtual environments for Python 3.3 and 3.4, and is
deprecated in Python 3.6.
Changed in version 3.5: The use of venv is now recommended for
creating virtual environments.
For windows, to initiate venv on some project, open cmd:
python -m venv "c:\path\to\myenv"
(Would suggest using double quote around directory path if it contains any spaces. Ex: "C:/My Dox/Spaced Directory/Something")
Once venv is set up, you will see some new folders inside your project directory. One of them would be "Scripts".
To activate or invoke venv you need:
C:\> <venv>\Scripts\activate.bat
You can deactivate a virtual environment by typing “deactivate” in your shell. With this, you are now ready to install your project specific libraries, which will reside under the folder "Lib".
================================ Edit 1 ====================================
The scenario which will be discussed below is not what originally asked, just adding this in case someone use vscode with python extension
In case, you use vs code with its python extension, you might face an issue with its pylint which points to the global installation. In this case, pylint won't be able to see the modules that are installed in your virtual environment and hence will show errors while importing.
Here is a simple method to get past this.
cd Workspace\Scripts
.\Activate.ps1
code .
We are basically activating the environment first and then invoking vs-code so that pylint starts within the environment and can see all local packages.
In python3.6 I tried
python3 -m venv myenv,
as per the documentation, but it was taking so long. So the very simple and quick command is
python -m venv yourenv
It worked for me on python3.6.
On Mac I had to do the following to get it to work.
mkvirtualenv --python=/usr/bin/python3 YourEnvNameHere
If you install python3 (brew install python3) along with virtualenv burrito, you can then do mkvirtualenv -p $(which python3) env_name
Of course, I know virtualenv burrito is just a wrapper, but it has served me well over the years, reducing some learning curves.
virtualenv --python=/usr/local/bin/python3 <VIRTUAL ENV NAME>
this will add python3
path for your virtual enviroment.
It worked for me
virtualenv --no-site-packages --distribute -p /usr/bin/python3 ~/.virtualenvs/py3
For those having troubles while working with Anaconda3 (Python 3).
You could use
conda create -n name_of_your_virtualenv python=python_version
To activate the environment ( Linux, MacOS)
source activate name_of_your_virtualenv
For Windows
activate name_of_your_virtualenv
I tried all the above stuff, it still didn't work. So as a brute force, I just re-installed the anaconda, re-installed the virtualenv... and it worked.
Amans-MacBook-Pro:~ amanmadan$ pip install virtualenv
You are using pip version 6.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting virtualenv
Downloading virtualenv-15.0.3-py2.py3-none-any.whl (3.5MB)
100% |████████████████████████████████| 3.5MB 114kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-15.0.3
Amans-MacBook-Pro:python amanmadan$ virtualenv my_env
New python executable in /Users/amanmadan/Documents/HadoopStuff/python/my_env/bin/python
Installing setuptools, pip, wheel...done.
Amans-MacBook-Pro:python amanmadan$
I wanted to keep python 2.7.5 as default version on Centos 7 but have python 3.6.1 in a virtual environment running alongside other virtual environments in python 2.x
I found the below link the best solution for the newest python version ( python 3.6.1)
https://www.digitalocean.com/community/tutorial_series/how-to-install-and-set-up-a-local-programming-environment-for-python-3.
It shows the steps for different platforms but the basic steps are
Install python3.x (if not present) for your platform
Install python3.x-devel for your platform
Create virtual environment in python 3.x
(for example $ python3.6 -m venv virenv_test_p3/ )
Activate the testenvironment for python 3.x
(for example source virenv_test_p3/bin/activate)
Install the packages which you want to use in your new python 3 virtual environment and which are supported ( for example pip install Django==1.11.2)
On Windows command line, the following worked for me. First find out where your python executables are located:
where python
This will output the paths to the different python.exe on your system. Here were mine:
C:\Users\carandangc\Anaconda3\python.exe
C:\Python27\python.exe
So for Python3, this was located in the first path for me, so I cd to the root folder of the application where I want to create a virtual environment folder. Then I run the following which includes the path to my Python3 executable, naming my virtual environment 'venv':
virtualenv --python=/Users/carandangc/Anaconda3/python.exe venv
Next, activate the virtual environment:
call venv\Scripts\activate.bat
Finally, install the dependencies for this virtual environment:
pip install -r requirements.txt
This requirements.txt could be populated manually if you know the libraries/modules needed for your application in the virtual environment. If you had the application running in another environment, then you can automatically produce the dependencies by running the following (cd to the application folder in the environment where it is working):
pip freeze > requirements.txt
Then once you have the requirements.txt that you have 'frozen', then you can install the requirements on another machine or clean environment with the following (after cd to the application folder):
pip install -r requirements.txt
To see your python version in the virtual environment, run:
python --version
Then voila...you have your Python3 running in your virtual environment. Output for me:
Python 3.7.2
For those of you who are using pipenv and want to install specific version:
pipenv install --python 3.6
I got the same error due to it being a conflict with miniconda3 install so when you type "which virtualenv" and if you've installed miniconda and it's pointing to that install you can either remove it (if your like me and haven't moved to it yet) or change your environment variable to point to the install you want.