Develop Reference

Switching between python virtual environments

I have some noob level virtual environment questions. I've been using virtual environments a little but still have a few questions.
I have created and activated an env which is my main working environment as follows:
virtualenv env
source /path/to/environment/env/bin/activate
Having activated this, I can now see I am in the environment as I have (env) visible on command line.
My first question is, do I need to run the activate command each time I open a terminal session? And therefore each time I turn my laptop on, etc.?
Further, I want to create another environment that runs on an earlier version of python for testing purposes. I was intending to do it as follows:
virtualenv --python=python2.7 env-py2
source /path/to/new/environment/env-py2/bin/activate
Can these virtualenvs be switched easily? So can I activate env-py2 and then easily jump back to activate env again? Or is there an inbetween step required?
Apologies for the very basic questions but I was struggling to find high level information.

Yes you need to run activate command i.e. source each time you open a terminal session.
Switching between two virtual environment is easy. You can run deactivate command and source the other virtual environment.

how do I get python to recognize modules installed in virtual environment?

I'm having trouble getting python in vs code to work properly. There were problems for a while (i dove right in with a workspace folder) so I decided to set up everything from scratch by starting a new project/workspace folder.
so far I've been having a few issues. The latest is this: despite creative a virtual environment as instructed on the vs code website, I can't get python to recognize that I've already installed the modules I want to use.
How do I fix this? (please see attached picture.
towards the bottom you can see that the virtual environment already has matplotlib, but python wouldn't recognize the module anyway

Create your virtual environment (e.g I used venv module to create a virtual env called TEST : $python3 -m venv TEST
From VS code go to Command Palette and type Python: Select Interpreter
And then you can choose the location of the desired interpreter (in my case the one from TEST virtual environment
You can always test your virtual environment from the terminal, e.g TEST env
Bash
From Cmd/Powershell normally you should run:
TEST\Scripts\activate.bat

PyCharm not able to create conda environment

I'm pretty new to programming, so maybe there's something painfully obvious that I've missed, but I've searched a lot around for solutions to this issue without finding any.
I just got a new computer and installed Anaconda and PyCharm for Anaconda. I got it to work fine on my old computer, but when I try to open PyCharm now and create a project, it gets stuck on the creation of the conda environment. Apparently there's a new conda update, so I tried to update to this one in Spyder, which seemed to work fine, but when PyCharm tries creating a new environment, it gets stuck and only displays
"Creating Conda environment"
"$ conda update -n base -c defaults conda"
Does anybody know what's happening? I've tried just waiting it out, but it doesn't seem to be moving forward at all from that point. I've also uninstalled both Anaconda and PyCharm with all their plugins and reinstalling them, but to no avail.
Thanks for any help or input!
Sincerely, confused first year student

It appears that Sophie was able to get the conda environment up and running.
However, other newcomers might come across a similar problem with regards to setting up Anaconda for use within PyCharm. Therefore, I provide a guide below that I recently (today) used in order to accomplish this on a fresh install on a new computer. For this I decided to use PyCharm Community Edition 2018.3 and Anaconda3 version 2019.07. I performed this on Windows 10 Enterprise version 1903. I have also performed this procedure on Windows 7 Home Premium edition (some time ago).
This will enable you to use PyCharm and Anaconda together, so that you can:
Know the prerequisite steps leading up to creating a conda environment, and become familiar with the steps following this action with using Anaconda with PyCharm.
Use the Terminal window in PyCharm with packages that are installed with Anaconda's version of Python. You can then use both regular python and conda commands to view, update or install packages. For instance, depending on your preference, you could use 'pip list' (python command) or 'conda list' to list installed packages.
Use the Anaconda python interpreter as the default Python Console in PyCharm
To accomplish this, please follow these steps:
Install PyCharm, and import any settings file you may have from previous PyCharm installation on other computer (optional). Assuming you are starting fresh, create a new project and set the (default) project folder location.
Install Anaconda. My install location was set to C:\ProgramData\Anaconda3. Note that 'ProgramData' folder is hidden by default. You should enable viewing of hidden folders in windows explorer, if you need to manually browse for it. Upon installation, I decided to have the options Add Anaconda to my PATH environment variable and Register Anaconda as my default Python 3.7 set as deselected.
Open Windows Explorer and paste in C:\ProgramData\Anaconda3 (or your install directory) in the address bar. This should take you to that directory. Navigate to the environments folder named envs. Select the address bar and copy that address. I got C:\ProgramData\Anaconda3\envs. You will need this later.
Open program named Anaconda Navigator (Anaconda3). You should see this from the list of newly installed programs. Alternatively, search for it and open it. This will initialize some stuff (unknown to me) that I have found necessary to be able to complete this guide. For the next steps we dont need this program running. You could close it if you want.
Decide on a name for the (ana)conda environment that we will create. I used py37 for simplicity. You only need to do this once for the python projects you will make. It is possible to make more environments if you should need that for some python projects in the future.
Open program named Anaconda Prompt (Anaconda3), preferentially as administrator. Opening as administrator may prevent complications that could occur on some systems. To do so, search for the program and then right-click it, and choose Run as administrator.
Enter this command: conda create -n py37 python=3.7 anaconda. It should prompt you to install some packages. Enter y to accept and proceed with the setup. This step may take some time to finish. Once finished, it may show you the needed command to activate the environment. Don't activate it just yet, ie. ignore that message for now.
Assuming environment path C:\ProgramData\Anaconda3\envs and environment name py37, run the following command (alter as needed to match your environment path and name): set PATH=C:\ProgramData\Anaconda\envs\py37\Scripts;C:\ProgramData\Anaconda\envs\py37;%PATH%
Run this command: conda activate py37. This will activate your environment.
Navigate to a folder you can easily access, such as the Documents folder on your machine. To do so, execute this command: cd "C:\Users\YourUsername\Documents".
Execute the following command: echo %PATH% > path_value.txt. This will export "PATH" values to a text file that can be found in the Documents folder as path_value.txt. Open this file and copy the content within. This will be needed in the following steps with setting up PyCharm for use with the Anaconda environment.
In PyCharm, navigate to Settings - Tools - Terminal. Check that Start directory field contains the path of your project folder, ie. such as C:/Users/YourUsername/Documents/YourProjectFolderName. Then, select the folder icon for the Environment variables field. Click the plus symbol, and add a new entry with PATH and your path value (as found from the path_value.txt file) in the Name and Value fields, respectively.
In PyCharm, navigate to Settings - Project Interpreter - Add Python Interpreter - Conda Environment - Existing environment. Browse for C:\ProgramData\Anaconda3 (or your install directory). Select the file named python.exe. I chose to edit the Name field of this Anaconda python interpreter as Python 3.7 Anaconda.
In PyCharm, navigate to Settings - Build, Execution, Deployment - Console - Python Console. Check that the Python interpreter field contains Project Default (Python 3.7 Anaconda) or similar entry. The Environment variables input may be empty. Select the folder button next to it and add a new entry with PATH and your path value (as found from the path_value.txt file) in the Name and Value fields, respectively. Exactly the same entries as was inputted in step 12.
[Optional] In PyCharm, navigate to Settings and search for the Run context configuration option. Right click it and select a desired shortcut method. Personally I use a keyboard shortcut which is set to Ctrl+Shift+Less. Once the rest of these steps are followed, this will allow you to run the current python .py file you have open, without having to manually set up configuration options for it (through Add Configuration option, next to green play button, as seen if you have no configurations set up from before). It will force PyCharm to use the default project interpreter (which now is the desired Anaconda-delivered Python 3.7 installation) for whatever project you have open. To use it, just open a .py python file with some code inside, click anywhere inside of it, and then use your shortcut option.
[Note: If you skipped step 15, go to step 17] Run your python file with the method from step 15. This should automatically set up a run configuration option for that file. You may get an error of missing packages. Ignore that for now.
Select Edit Configuration or Add Configuration in the dropdown box next to the green play / run button. Open the Templates tree, then select Python. Check that the Python interpreter option contains Python 3.7 Anaconda or similar. The Environment variables field may contain PYTHONUNBUFFERED=1. Regardless, select the related folder button and add a new entry with PATH and your path value (as found from the path_value.txt file) in the Name and Value fields, respectively. Exactly the same as we did in steps 12 and 14.
Restart PyCharm to re-initialize the Terminal and Python Console windows. Or manually close and reopen those windows.
You have now created and activated an (ana)conda environment, and enabled it's use from within PyCharm. You should also now be able to ie. copy your code over to a fresh new file, and then hit ie. Ctrl+Shift+Less to run the file without having to manually set a configuration file for it.
Hope this helped!
Sources: source1, source2, source3

In my Experience I just opened PyCharm as administrator and the error was gone!

If you use PyCharm to create conda environment, you can choose conda environment not virtualenv.
When you get the error like "can't get the /path/to/", you can open PyCharm with administrator.
If you use "anaconda prompt" to create conda environment, you can use command line like conda create -n py36 python=3.6. Then you can activate environment using conda activate py36.

How to activate anaconda environment within R

I have qiime2 program https://qiime2.org in my directory /home/owner/anaconda3/envs/qiime2-2019.1. In Linux terminal, I run source activate /home/owner/anaconda3/envs/qiime2-2019.1 to start this program.
I tried doing this within R studio as system('source activate /home/owner/anaconda3/envs/qiime2-2019.1'), but it it gives me this error: sh: 1: activate: not found
Warning message:
In system('activate /home/owner/anaconda3/envs/qiime2-2019.1') :
error in running command
Is there a way to activate anaconda env within R or Rstudio?

Yes there are multiple ways I recommend looking into the reticulate package but basically, R Studio preview 1.2 is capable of "finding" your conda environments.
My prefered way is:
library(reticulate)
library(tidyverse)
# Seeing your enviroments
conda_list()
#Using it
conda_list()[[1]][1] %>%
use_condaenv(required = TRUE)
#Checking python
import platform
print(platform.python_version())
Links
Reticulate: https://rstudio.github.io/reticulate/
My blog: https://twosidesdata.netlify.com/2019/03/23/exploratory-data-analysis-basic-pandas-and-dplyr/#how-to-set-up-reticulate

I think it could not be successful to enter the conda environment in R console, but you still can use the environment command by indicating the path.
For instance, my path of qiime is /home/username/miniconda3/envs/qiime2-2019.7/bin/qiime.
If you want to run the code like qiime info, you can use the command:
system("/home/username/miniconda3/envs/qiime2-2019.7/bin/qiime info")

Generally, I do not use Rstudio but from some searches, I can suggest you try by setting python path instead of activating environment by Conda activate.
You can select which python interpreter you are going to use and here it.
library(reticulate)
path_to_python <- "/anaconda3/envs/qiime2-2019.1/python"
use_python(path_to_python, required = TRUE)
here are some answers for the same type of question:
1> https://stackoverflow.com/a/54813273/9071644
2> https://stackoverflow.com/a/45891929/9071644
3> https://stackoverflow.com/a/43411909/9071644

I know this is an old question, but found some ways one can activate. One is using the library reticulate mentioned above, but I use it this way:
library(reticulate)
use_condaenv("py2_env") #activate an environment e.g py2_env
base::system(paste0("sh py2_env_program_to_run.sh")) #run a program that requires that environment
With reticulate im not sure though how you would deactivate the environment.
The second way, I wrote a little bash script called 'env_eval.sh'
#!/bin/bash
PATH=/opt/conda/bin:$PATH
export PATH
eval "$(conda shell.bash hook)"
Then I reference it in my R script everytime i want to use conda e.g:
base::system2(paste0("sh ",run_folder,"/code/bash/env_eval.sh conda activate py2_env && run_py2_env_program.sh"))
I can even deactivate a conda environment with this method:
base::system(paste0("sh ",run_folder,"/code/bash/env_eval.sh conda deactivate && run_program_that_should_not_be_in_any_conda_env.sh"))

If you want to use Python in RStudio, the best way to get it going is to create a separate 'reticulate' environment using Anaconda.
Part of the reason is that so you can use RMarkdown for your output requires PyQt5 which will break your Jupyter/Spyder environments if you overwrite PyQt.
Then you have to make an .Renviron file like this setup. which points R to the proper Python env. Otherwise, the default for RStudio seems to be a miniconda environment.
Once your separate reticulate environment is set and you have .Renviron pointing to it, all of your Python package installs should go into that environment.

PyCharm with Pyenv

How to use Pyenv virtualenv's with Pycharm 2016.3? In the earlier version of Pycharm, I could easily set up local interpreter to point anything installed on my machine.
My first idea was to add .python-version file on the root of the project. I Pyenv virtualenv installed so this will activate & run the project with correct environment automatically. However, Pycharm still doesn't see the correct interpreter causing it to though import and syntax errors.
How can I select my local Pyenv in new PyCharm 2016.3 I used to be able to set the path as variable now I can only browse the path using drop-down menu. It doesn't seem to show hidden files like default path for pyenv ~./.pyenv/versions{project}.

In Pycharm version 2017.3, you can go to
Pycharm -> Preferences -> Project -> Project Interpreter -> <project_name> -> settings button on the right of the python interpreter text box -> Add local
This will open a new window with virtualenv Environment as one of the options on the left. On Selecting it, you will get an option to make a new virtualenv environment or use an existing virtual environment. Here next to the dropdown text box, you can click "..." and browse to your existing virtualenv created using pyenv and select it. It will select this virtualenv when you start terminal from Pycharm and also use the corresponding python interpreter as set while creating that virtualenv.

Get pyenv-virtualenv plugin for more project-specialized environments.
Then, create a new environment for project: (assume that we installed python-3.7.1 with pyenv and we'll use it)
$ pyenv virtualenv 3.7.1 projectName-3.7.1
This command generates folder for our environment.
Open pyCharm (v2018.3.1 used):
Create New Project > Existing Interpreter
Now you can type path of your environment:
~/.pyenv/versions/projectName-3.7.1/bin/python3
Then press Create..
That's all.
If there is already exists project:
File > Settings > Project: projectName > Project: Interpreter
Again, you can type path of the environment as like above. So you will see packages installed on this environment.
If you want to use same version of python and environment on the command line, then you must activate the environment with
$ pyenv activate projectName-3.7.1
command.
Note that pyenv virtualenv can activate that environment when entering the folder within the terminal through putting the name of it into your .python-version file as well.
For more command about pyenv-virtualenv you can look for reference sheet.

Personally, I made the best experiences with using pyenv and pipenv together. So far, I used separate commands for that, rather than using the pyenv-virtualenv plugin, but it should be supported with this hint as well.
My workflow to start a new project:
Create folder and switch into it:
mkdir new_project ; cd new_project
Set desired local pyenv version:
pyenv local 3.8.0
Create an empty pipenv virtual environment, using just that local version:
pipenv --python $(pyenv which python)
Now comes the tricky part: PyCharm is supporting Pipenv as an interpreter, but it doesn't recognize it automatically anymore after the initial interpreter selection (which happens at project initiation / first time opening of the project, automatically).
So - if you just created the new project folder (without PyCharm's .idea/ folder created yet), it will recognize the Pipenv-Virtualenv of the project just fine and set it as a project interpreter, automatically:
If there is already an .idea/ folder, it's not that easy, since PyCharm's GUI just supports to create a new Pipenv environment, but you still have an option:
Close PyCharm, delete .idea/ folder and reopen the project folder in PyCharm.
This will delete other project settings as well, but shouldn't be something too important for a fresh environment.
Open the folder in PyCharm again and it will recognize your Pipenv virtualenv.

You don’t mention what operating system you’re using, and it’s relevant here.
If it’s OS X or macOS, you can press Shift+Cmd+G in the file selection dialog (when you’re choosing the location of a new local interpreter) to enter a path manually. (This is a standard macOS shortcut that works in any native file selection dialog.)

After taking a lead from Mr. Judge regarding the use of pyenv, I stumbled on a way to introduce an interpreter from pyenv to an existing PyCharm (2020.2.2, if it matters) project without blowing away the .idea directory.
Prior to using any other Environment type (Pipenv, Poetry, etc.), first open the Virtual Environment option:
Select Exiting environment and then navigate to one of your pyenv shims using the […] button to the right of the Interpreter: drop-down. Then click Make available to all projects.
You can then go to the Pipenv or Poetry Environemnt (Plugin) to reference that introduced interpreter now.