Develop Reference

How to choose which python executable to run when using venv?

I have a python installation on my pc (windows 10) which comes from Anaconda. I am a data scientist and using conda as a package manager is very convenient for me.
However, sometimes I want to develop a small app or script to share with my colleagues. In these cases I create a project folder and python -m venv .venv inside it.
This way, I can install only the essential packages I need, and later share the requirements.txt file.
The issue I'm having, is that the python interpreter being used is still the default one, namely, the one that came with Anaconda, even if I activate the virtual environment and deactivate the conda one.
Specifically, if I run python in the terminal, I get this warning message:
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
This is rather inconvenient. My base python installation is 3.7, but if I wanted to use an earlier version, or 3.8, I can't seem to be able to choose.
I would expect that the python executable being used is the one in the current active environment, but this doesn't seem to be the case.
How can I obtain that?

First you have to install the version of python you want to use in your venv. It must already be available somewhere on your system to create a venv using it.
Then instead of just python -m venv .venv you specify which python with the full path : /path/to/pythonX.Y -m venv .venv
You can't have a venv that shares multiple versions of python, to my knowledge at least.

Is there a way to create a virtual environment with an earlier version of python?

I have python 3.8 installed on my pc, but i need 3.7 for a specific task. When i try to set up a virtual environment via
virtualenv -p "my/path/to/python37.exe"
it calls the installer, so i am to install py37 manually, but then it ends up with an error:
Error 0x80070666: Cannot install a product when a newer version is installed
Is there a proper way to implement such a thing?
Thanks!

Have you tried anaconda or miniconda (the lighter version of anaconda)? Having more python versions in different environments and switching between them is quite easy.

I haven't installed Anaconda. If you want to have multiple different environments with pure Python you can do it like this:
1) Install the python versions that you want with the exe installer, ie Python 2.x.x, Python 3.7.x, Python 3.8.x etc, maybe inside a common folder like C:\Python
2) Then edit the System Variables path and pinpoint to the folder for the version you want to create a virtual environment. You need two entries here, one to python folder (for the python.exe) and one to the Scripts folder (for the pip.exe)
3) Open command prompt and hit python. You ll see that it's showing the apporpriate version. Install the virtual environment with "python -m venv name_of_env_you_want"
4) If you want to create a virtual environment with a different version change the paths

How can I use both Anaconda versions (2.7 & 3.5)?

I was using the Anaconda 3.5 distro in a Windows 10 machine. Due to dependencies in libraries that I want to work with, I had to have the 2.7 version installed as well.
The good news is that the libraries I needed can now work with the 2.7 version smoothly and Visual Studio 2015 automagically detected my new Python environment.
The problem comes when using the command line. Upon issuing the command
conda info --envs
I get
root * C:\Users\norah\Anaconda2
i.e. a single environment (to my understanding and search so far, according to this I should see two envs listed). This means I can't use conda to acquire new packages for my Python3.5 installation, at least not at the command line as I used to since conda only refers to Python2.7. The same goes for the GUI version, Anaconda navigator (I'm not very fond of the GUI version but I tried it out).
There's also no way of launching python3 from the command line since
$python
always fires up python2.7 and issuing $python3 or $python3.5 in the command line doesn't seem to work (nor would adding the path of python3 to the system since the actual executable has the same name as python2 i.e. python.exe)
Is my system taken over by Python2.7? Is anyone here using them both smoothly and if so could you please elaborate on how to do that? Was it a "no no" move to install both versions of Anaconda?

You can make Python 3.5 environment with your Anaconda 2.7:
conda create -n py35 python=3.5
Now, activate it:
activate py35
Finally you can install the desired packages:
conda install numpy
or, the whole anaconda:
conda install anaconda
The advantage of this approach is that you can also create Python 3.4 or 3.6 environments. Furthermore, you can create environments with different combinations and versions of libraries.
Actually, it makes sense to create a new environment for each larger project.

I also despise the virtual environment switch that Anaconda tries to force on us. I prefer to have both executables always instantly available from the command line. I'm pretty sure I had this working on a Windows machine once:
Install Anaconda2 and Anaconda3 to the C:\ drive as "C:\Anaconda2\" and "C:\Anaconda3\" respectively.
Edit your "Path" environment variable (Control Panel -> System and Security -> System -> Advanced system settings -> Environment Variables) and make sure that "C:\Anaconda2;C:\Anaconda2\Scripts;C:\Anaconda2\Library\bin" is in front of "C:\Anaconda3;C:\Anaconda3\Scripts;C:\Anaconda3\Library\bin".
Copy and rename the file "C:\Anaconda3\python.exe" to "C:\Anaconda3\python3.exe".
Copy and rename the file "C:\Anaconda3\Scripts\conda.exe" to "C:\Anaconda3\Scripts\conda3.exe"
Copy and rename any other scripts you might use in "C:\Anaconda3\Scripts\", such as "pip.exe" to "pip3.exe", etc.
Now, when you type "python" or "conda" at the command line you will get the python2 version, and when you type "python3" or "conda3", etc. at the command line you will get the python3 version.

The best way to use both Python 2.7x and Python 3.5x together is Jupyter Notebook.
http://jupyter.org/
You will be able to work on your browser with IPython Notebook style interface which is great for working with scripting languages.
I found some of these videos on YouTube very informative:
1) https://www.youtube.com/watch?v=HW29067qVWk
2) https://www.youtube.com/watch?v=e9cSF3eVQv0
Besides Python 2.7 and 3.5, there are a bunch of other languages that you will be able run with your Jupyter Notebook. The various Kernels are available in this link below:
https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
After installation, as you start your Jupyter Notebook, your browser will open up a new page showing your Home Directory. Among the 4 tabs (Files|Running|Clusters|Conda):
1) The Files tab shows all the files in your Home Directory.
2) Running tab shows all the Notebooks that are running.
3) Clusters tab is provided by IPython parallel.
4) Conda tab is where you need to add the Python version 3.5 (and other languages if needed) to your Jupyter Notebook (If Python 2.7 is default).
If you are interested to try C++ with your Jupyter Notebook, there are a couple of Kernels available.
1) JupyRoot - https://github.com/root-mirror/root/tree/master/bindings/pyroot/JupyROOT
2) Cling - https://github.com/root-mirror/cling

How to manage (ana)conda with pycharm 2016.3 in linux

I would like to use Anaconda and the newest Pycharm 2016.3 together. I would like to be able to manage packages in settings->interpreter. If this is not supported, I would like to know the workflow of using these two together. According to another SO question, Pycharm 5 used to have a 'Create conda env' in the interpreter settings, but this seems to be gone now. I have tried:
1) Manually creating a virtual environment with 'conda create --name project numpy' and I add the interpreter ('~/anaconda2/envs/bin/python', the location of python for my created virtual environment. However, pycharm doesn't allow me to add any packages through settings->interpreter. Running an 'import numpy' through the console shows errors that are pointing to /usr/bin/python, not my virtual env python, and an error 'ImportError: cannot import name multiarray'. I'm not sure what package to add using conda from the cli, and the pycharm frontend doesn't add packages
2) I've tried the same as 1) but with my global anaconda python as the interpeter ('~/anaconda2/bin/python') and it doesn't seem to be able to connect to the console.
3) Creating a virtual environment through pycharm directly.
I would like to remove my default pythons (/usr/bin/python2.7/3.5 from the list of interpreters in pycharm) for debugging purposes but it won't let me and it seems to be showing packages that my anaconda virtual env doens't have installed.
Is there a way to manage my VIRTUAL enviornment in Conda using pycharm? If not, what steps do I take to make these two play well together assuming I can't manage it through pycharm interepreters settings.

This is the normal steps that i follow when i use virutalenv with PyCharm
I normally work on ubuntu
First, i always create a separate environment for every project using the command virtualenv "environment_name" from the command line.
Activate the environment using the command - source environment_name/bin/activate in ubuntu.
Suppose if i want to start a django project, i create the project using the command django-admin startproject project_name
Open this project in pycharm.
go to settings-> interpreter in pycharm. choose "add local" interpreter from the settings. It will open a pop-up. Go to the directory of the environment you just created and select the correct python interpreter you want to use.
now if you want to install a new package, you can go to interpreter settings and add package from the pycharm or you can fire up the command line, activate the environment and run pip install package_name. Once the package is installed, it will also show in pycharm.
if you are using Windows OS, use powershell to execute the above commands. The only difference will be in activating the environment. In windows, to activate an env use environment_name/Scripts/activate
EDIT:
Same goes anaconda environments also, the easy way is to manage the environment from the terminal and pycharm will show the packages changes in the interpreter settings.

Here is what I have been doing and it works great. I create a virtual environment separately and when I create a new project in PyCharm, I point the interpreter to the python from the virtual environment. An example is shown in the picture below:

Using (Ana)conda within PyCharm

I've got Pycharm 4 running on my Linux (Ubuntu 14.04) machine. In addition to the system python, I've also got Anaconda installed. Getting the two to play nicely together seems to be a bit of a problem... PyCharm provides some interesting integration for virtualenvs and pip, but the Anaconda Python distribution seems to prefer using its own conda tool for both activities.
Is there a relatively simple/painless way to be able to use conda in conjunction with PyCharm? Not just as an alternative interpreter i.e. point PyCharm at the Anaconda Python binary for a project interpreter, but to be able to create, source/activate and deactivate virtual envs, add/remove packages in those virtual envs, etc.
Or am I going to have to choose between using Anaconda (and having a more recent and up-to-date python than may come with the system), and being able to use PyCharm's features to their fullest extent?

I know it's late, but I thought it would be nice to clarify things: PyCharm and Conda and pip work well together.
The short answer
Just manage Conda from the command line. PyCharm will automatically notice changes once they happen, just like it does with pip.
The long answer
Create a new Conda environment:
conda create --name foo pandas bokeh
This environment lives under conda_root/envs/foo. Your python interpreter is conda_root/envs/foo/bin/pythonX.X and your all your site-packages are in conda_root/envs/foo/lib/pythonX.X/site-packages. This is same directory structure as in a pip virtual environement. PyCharm sees no difference.
Now to activate your new environment from PyCharm go to file > settings > project > interpreter, select Add local in the project interpreter field (the little gear wheel) and hunt down your python interpreter. Congratulations! You now have a Conda environment with pandas and bokeh!
Now install more packages:
conda install scikit-learn
OK... go back to your interpreter in settings. Magically, PyCharm now sees scikit-learn!
And the reverse is also true, i.e. when you pip install another package in PyCharm, Conda will automatically notice. Say you've installed requests. Now list the Conda packages in your current environment:
conda list
The list now includes requests and Conda has correctly detected (3rd column) that it was installed with pip.
Conclusion
This is definitely good news for people like myself who are trying to get away from the pip/virtualenv installation problems when packages are not pure python.
NB: I run PyCharm pro edition 4.5.3 on Linux. For Windows users, replace in command line with in the GUI (and forward slashes with backslashes). There's no reason it shouldn't work for you too.
EDIT: PyCharm5 is out with Conda support! In the community edition too.

Change the project interpreter to ~/anaconda2/python/bin by going to File -> Settings -> Project -> Project Interpreter. Also update the run configuration to use the project default Python interpreter via Run -> Edit Configurations. This makes PyCharm use Anaconda instead of the default Python interpreter under usr/bin/python27.

as per #cyberbikepunk answer pycharm supports Anaconda since pycharm5!
Have a look how easy is to add an environment:

Continuum Analytics now provides instructions on how to setup Anaconda with various IDEs including Pycharm here. However, with Pycharm 5.0.1 running on Unbuntu 15.10 Project Interpreter settings were found via the File | Settings and then under the Project branch of the treeview on the Settings dialog.

this might be repetitive.
I was trying to use pycharm to run flask - had anaconda 3, pycharm 2019.1.1 and windows 10. Created a new conda environment - it threw errors.
Followed these steps -
Used the cmd to install python and flask after creating environment as suggested above.
Followed this answer.
As suggested above, went to Run -> Edit Configurations and changed the environment there as well as in (2).
Obviously kept the correct python interpreter (the one in the environment) everywhere.