For my python project, I am using Pipenv. On my local computer I have a bunch of AWS environment variables, that I don't want my virtual environment to use.
Each time my pipenv shell starts, I unset the variables - But for some reason, after each computer restart, upon activating the pipenv shell, I can see that the environment variables from my "parent" are being injected into the virtual environment.
Is there any way to disable this behavior, so that no environment variables are spilled over to my pipenv?
Is there any way to disable this behavior, so that no environment variables are spilled over to my pipenv?
This behavior can't simply be disabled because it is fundamental to how Unix processes work: a child process inherits the environment from the parent process. There are a few ways we can approach your question; the easiest is probably taking advantage of pipenv's support for loading .env files. If a .env file exists in the root of your pipenv project, then pipenv will load it whenever it runs.
So for example if in our parent environment we have:
SOMEVAR=somevalue
ANOTHERVAR=anothervalue
export SOMEVAR ANOTHERVAR
Normally, if we start a pipenv shell we will see those values in the new environment:
$ pipenv shell
$ env | grep -E 'SOMEVAR|ANOTHERVAR'
ANOTHERVAR=anothervalue
SOMEVAR=somevalue
But if we override those with empty values in a .env file:
$ cat > .env <<EOF
> SOMEVAR=
> ANOTHERVAR=
> EOF
Then that will take precedence when we start pipenv shell:
$ pipenv shell
$ env | grep -E 'SOMEVAR|ANOTHERVAR'
ANOTHERVAR=
SOMEVAR=
Related
I currently have this script to show my GitHub branch and virtual env:
setopt PROMPT_SUBST
autoload -Uz vcs_info
precmd() { vcs_info }
zstyle ':vcs_info:git:*' formats '(%b)'
MYPS1=''
MYPS1+='%F{green}'
MYPS1+='${${(%):-%n}:0:1}'
MYPS1+='#'
MYPS1+='${${(%):-%m}:(-4)}' # Get last 4 chars of var machine name
MYPS1+=':'
MYPS1+='%F{yellow}'
MYPS1+='%1~' # Show only the name of the working directory or ~ if it is the home directory
MYPS1+='%F{magenta}'
MYPS1+='${vcs_info_msg_0_}' # Show git branch if any
MYPS1+='%f'
MYPS1+='%# '
PS1=$MYPS1
Sometimes I need to update my .zshrc so I run:
source ~/.zshrc
The problem is, whenever I reload my shell, I cannot see my Python virtual environment anymore even though it's still active.
# After activating virtual env
(my-ve-3.7.13) u#m1:repo-name(github-branch)%
# After reloading my zsh
u#m1:repo-name(github-branch)%
I use pyenv and virtualenvs.
How can I keep the virtual env name in my prompt?
Following #chepner's comment, I figured it out:
Use env to see the list of all env vars. pyenv uses PYENV_VERSION.
Add it to the prompt, use () to have the same look as pyenv does.
...
MYPS1=''
MYPS1+='($PYENV_VERSION) '
MYPS1+='%F{green}'
...
I recently installed anaconda2 on my Mac. By default Conda is configured to activate the base environment when I open a fresh terminal session.
I want access to the Conda commands (i.e. I want the path to Conda added to my $PATH which Conda does when initialised so that's fine).
However I don't ordinarily program in python, and I don't want Conda to activate the base environment by default.
When first executing conda init from the prompt, Conda adds the following to my .bash_profile:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/geoff/anaconda2/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/Users/geoff/anaconda2/etc/profile.d/conda.sh" ]; then
. "/Users/geoff/anaconda2/etc/profile.d/conda.sh"
else
export PATH="/Users/geoff/anaconda2/bin:$PATH"
fi
# fi
unset __conda_setup
# <<< conda initialize <<<
If I comment out the whole block, then I can't activate any Conda environments.
I tried to comment out the whole block except for
export PATH="/Users/geoff/anaconda2/bin:$PATH"
But then when I started a new session and tried to activate an environment, I got this error message:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
This question (and others like it) are helpful, but doesn't ultimately answer my question and is more suited for linux users.
To be clear, I'm not asking to remove the (base) from my $PS1 I'm asking for Conda not to activate base when I open a terminal session.
I have conda 4.6 with a similar block of code that was added by conda. In my case, there's a conda configuration setting to disable the automatic base activation:
conda config --set auto_activate_base false
The first time you run it, it'll create a .condarc in your home directory with that setting to override the default.
This wouldn't de-clutter your .bash_profile but it's a cleaner solution without manual editing that section that conda manages.
There're 3 ways to achieve this after conda 4.6. (The last method has the highest priority.)
Use sub-command conda config to change the setting.
conda config --set auto_activate_base false
In fact, the former conda config sub-command is changing configuration file .condarc. We can modify .condarc directly. Add following content into .condarc under your home directory,
# auto_activate_base (bool)
# Automatically activate the base environment during shell
# initialization. for `conda init`
auto_activate_base: false
Set environment variable CONDA_AUTO_ACTIVATE_BASE in the shell's init file. (.bashrc for bash, .zshrc for zsh)
export CONDA_AUTO_ACTIVATE_BASE=false
To convert from the condarc file-based configuration parameter name to the environment variable parameter name, make the name all uppercase and prepend CONDA_. For example, conda’s always_yes configuration parameter can be specified using a CONDA_ALWAYS_YES environment variable.
The environment settings take precedence over corresponding settings in .condarc file.
References
The Conda Configuration Engine for Power Users
Using the .condarc conda configuration file
conda config --describe
Conda 4.6 Release
The answer depends a little bit on the version of conda that you have installed. For versions of conda >= 4.4, it should be enough to deactivate the conda environment after the initialization, so add
conda deactivate
right underneath
# <<< conda initialize <<<
To disable auto activation of conda base environment in terminal:
conda config --set auto_activate_base false
To activate conda base environment:
conda activate
So in the end I found that if I commented out the Conda initialisation block like so:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
# __conda_setup="$('/Users/geoff/anaconda2/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
# if [ $? -eq 0 ]; then
# eval "$__conda_setup"
# else
if [ -f "/Users/geoff/anaconda2/etc/profile.d/conda.sh" ]; then
. "/Users/geoff/anaconda2/etc/profile.d/conda.sh"
else
export PATH="/Users/geoff/anaconda2/bin:$PATH"
fi
# fi
# unset __conda_setup
# <<< conda initialize <<<
It works exactly how I want. That is, Conda is available to activate an environment if I want, but doesn't activate by default.
If you manage your .bashrc manually and like to keep it simple, all you really need is:
. "$HOME/anaconda2/etc/profile.d/conda.sh"
See Recommended change to enable conda in your shell.
This will make the conda command available without activating the base environment (nor reading your conda config).
Note that this is (of course) not compatible with managing the conda installation with conda init, but other than that, nothing bad is coming from it. You may even experience a significant speedup compared to the conda init generated code, because this solution avoids calling conda to parse your config files on whether to enable the base environment, etc.
It's best to also keep the if/fi lines to avoid error messages if using the same bashrc on several systems where conda may not be installed:
if [ -f "$HOME/anaconda2/etc/profile.d/conda.sh" ]; then
. "$HOME/anaconda2/etc/profile.d/conda.sh"
fi
Finally, if you share your bashrc between several systems where conda may be installed in different paths, you could do as follows:
for CONDA_PREFIX in \
"$HOME/anaconda2" \
"$HOME/miniconda3" \
"/opt/miniconda3" \
do
if [ -f "$CONDA_PREFIX/etc/profile.d/conda.sh" ]; then
. "$CONDA_PREFIX/etc/profile.d/conda.sh"
break
fi
done
Of course, this is now similar in length compared to the conda init generated code, but will still execute much faster, and will likely work better than conda init for users who synchronize their .bashrc between different systems.
for conda 4.12.0 (under WOS) the following worked (where all the previous answers -these included- didn't do the trick):
in your activate.bat file (mine was at ~/miniconda3/Scripts/activate.bat), change the line:
#REM This may work if there are spaces in anything in %*
#CALL "%~dp0..\condabin\conda.bat" activate %*
into
#REM This may work if there are spaces in anything in %*
#CALL "%~dp0..\condabin\conda.bat" deactivate
this line chage/modification doesn't work in the section (of the activate.bat file):
#if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (
#CALL "%~dp0..\condabin\conda.bat" activate
#GOTO :End
)
maybe because it depends on how your miniconda3 (Anaconda Prompt) executable is set up: %windir%\System32\cmd.exe "/K" some-path-to\miniconda3\Scripts\activate.bat some-path-to\miniconda3 (in my case).
caveat: updating conda overwrites this (activate.bat) file; so one has to modify the above line as many times as needed/updated. not much of a deal-breaker if you ask me.
This might be a bug of the recent anaconda. What works for me:
step1: vim /anaconda/bin/activate, it shows:
#!/bin/sh
_CONDA_ROOT="/anaconda"
# Copyright (C) 2012 Anaconda, Inc
# SPDX-License-Identifier: BSD-3-Clause
\. "$_CONDA_ROOT/etc/profile.d/conda.sh" || return $?
conda activate "$#"
step2: comment out the last line: # conda activate "$#"
One thing that hasn't been pointed out, is that there is little to no difference between not having an active environment and and activating the base environment, if you just want to run applications from Conda's (Python's) scripts directory (as #DryLabRebel wants).
You can install and uninstall via conda and conda shows the base environment as active - which essentially it is:
> echo $Env:CONDA_DEFAULT_ENV
> conda env list
# conda environments:
#
base * F:\scoop\apps\miniconda3\current
> conda activate
> echo $Env:CONDA_DEFAULT_ENV
base
> conda env list
# conda environments:
#
base * F:\scoop\apps\miniconda3\current
In the user site .pth files are processed once at interpreter startup:
$ echo 'import sys; sys.stdout.write("hello world\n")' > ~/.local/lib/python3.8/site-packages/hello.pth
$ python3.8 -c ""
hello world
And it's the same behaviour in the system site, e.g. /usr/local/lib/python3.8/site-packages/.
But in venv they are processed twice:
$ rm ~/.local/lib/python3.8/site-packages/hello.pth
$ /usr/local/bin/python3.8 -m venv .venv
$ source .venv/bin/activate
(.venv) $ echo 'import sys; sys.stdout.write("hello world\n")' > .venv/lib/python3.8/site-packages/hello.pth
(.venv) $ python -c ""
hello world
hello world
Why are path configuration files processed twice in a virtual environment?
Looks like it all happens in the site module (not so surprising). In particular in the site.main() function.
The loading of the .pth files happens either in site.addsitepackages() or in site.addusersitepackages(), depending in which folder the file is placed. Well more precisely both these function call site.addpackage(), where it actually happens.
In your first example, outside a virtual environment, the file is placed in the directory for user site packages. So the console output happens when site.main() calls site.addusersitepackages().
In the second example, within a virtual environment, the file is placed in the virtual environment's own site packages directory. So the console output happens when site.main() calls site.addsitepackages() directly and also via site.venv() a couple of lines earlier that also calls site.addsitepackages() if it detects that the interpreter is running inside a virtual environment, i.e. if it finds a pyvenv.cfg file.
So in short: inside a virtual environment site.addsitepackages() runs twice.
As to what the intention is for this behavior, there is a note:
# Doing this here ensures venv takes precedence over user-site
addsitepackages(known_paths, [sys.prefix])
Which, from what I can tell, matters in case the virtual environment has been configured to allow system site packages.
Maybe it could have been solved differently so that the path configuration files are not loaded twice.
Right now I'm using virtualenv and just switching over to Pipenv. Today in virtualenv I load in different environment variables and settings depending on whether I'm in development, production, or testingby setting DJANGO_SETTINGS_MODULE to myproject.settings.development, myproject.settings.production, and myproject.settings.testing.
I'm aware that I can set an .env file, but how can I have multiple versions of that .env file?
I'm far from a Python guru, but one solution I can think of would be to create Pipenv scripts that run shell scripts to change the PIPENV_DOTENV_LOCATION and run your startup commands.
Example Pipfile scripts:
[scripts]
development = "./scripts/development.sh"
development.sh Example:
#!/bin/sh
PIPENV_DOTENV_LOCATION=/path/to/.development_env pipenv run python test.py
Then run pipenv run development
You should create different .env files with different prefixes depending on the environment, such as production.env or testing.env. With pipenv, you can use the PIPENV_DONT_LOAD_ENV=1 environment variable to prevent pipenv shell from automatically exporting the .env file and combine this with export $(cat .env | xargs).
export $(cat production.env | xargs) && PIPENV_DONT_LOAD_ENV=1 pipenv shell would configure your environment variables for production and then start a shell in the virtual environment.
When I was developing and testing my project, I used to use virtualenvwrapper to manage the environment and run it:
workon myproject
python myproject.py
Of course, once I was in the right virtualenv, I was using the right version of Python, and other corresponding libraries for running my project.
Now, I want to use Supervisord to manage the same project as it is ready for deployment. The question is what is the proper way to tell Supervisord to activate the right virtualenv before executing the script? Do I need to write a separate bash script that does this, and call that script in the command field of Supervisord config file?
One way to use your virtualenv from the command line is to use the python executable located inside of your virtualenv.
for me i have my virtual envs in .virtualenvs directory. For example
/home/ubuntu/.virtualenvs/yourenv/bin/python
no need to workon
for a supervisor.conf managing a tornado app i do:
command=/home/ubuntu/.virtualenvs/myapp/bin/python /usr/share/nginx/www/myapp/application.py --port=%(process_num)s
Add your virtualenv/bin path to your supervisord.conf's environment:
[program:myproj-uwsgi]
process_name=myproj-uwsgi
command=/home/myuser/.virtualenvs/myproj/bin/uwsgi
--chdir /home/myuser/projects/myproj
-w myproj:app
environment=PATH="/home/myuser/.virtualenvs/myproj/bin:%(ENV_PATH)s"
user=myuser
group=myuser
killasgroup=true
startsecs=5
stopwaitsecs=10
First, run
$ workon myproject
$ dirname `which python`
/home/username/.virtualenvs/myproject/bin
Add the following
environment=PATH="/home/username/.virtualenvs/myproject/bin"
to the related supervisord.conf under [program:blabla] section.