Reset conda root directory? - python

I recently switched mac computers. My old computer had conda and all the relevant libraries and environments installed. I transferred all the files from the old computer to my new one. However, my new computer has a different username (home directory name) than the old one, so whenever I run conda commands, I get errors like the following:
__conda_exe:2: no such file or directory: /Users/[old_username]/anaconda/bin/conda
I can circumvent this initial error by manually setting $PATH each time, but then I still get errors like this:
__add_sys_prefix_to_path:6: command not found: dirname
__add_sys_prefix_to_path:7: command not found: dirname
__conda_exe:2: no such file or directory: /Users/[old_username]/anaconda/bin/conda
Is there a way to change this so the commands point to my new home directory?

I believe you need to use CPR (conda-prefix-replacement) for this.
However, I'd instead recommend not manually transferring the files, since this usually fails to retain hardlinks. Instead, dump each of your environments to YAML
conda env export -n envname > env.yaml
and recreate them on the new system (after a minimal base install, like Miniconda)
conda env create -n envname -f env.yaml
I would not recommend transferring the old base, but if you must, put it in a new location:
# old system
conda env export -n base > old_base.yaml
# new system
conda env create -n old_base -f old_base.yaml

Related

zsh always show Python virtual env

I currently have this script to show my GitHub branch and virtual env:
setopt PROMPT_SUBST
autoload -Uz vcs_info
precmd() { vcs_info }
zstyle ':vcs_info:git:*' formats '(%b)'
MYPS1=''
MYPS1+='%F{green}'
MYPS1+='${${(%):-%n}:0:1}'
MYPS1+='#'
MYPS1+='${${(%):-%m}:(-4)}' # Get last 4 chars of var machine name
MYPS1+=':'
MYPS1+='%F{yellow}'
MYPS1+='%1~' # Show only the name of the working directory or ~ if it is the home directory
MYPS1+='%F{magenta}'
MYPS1+='${vcs_info_msg_0_}' # Show git branch if any
MYPS1+='%f'
MYPS1+='%# '
PS1=$MYPS1
Sometimes I need to update my .zshrc so I run:
source ~/.zshrc
The problem is, whenever I reload my shell, I cannot see my Python virtual environment anymore even though it's still active.
# After activating virtual env
(my-ve-3.7.13) u#m1:repo-name(github-branch)%
# After reloading my zsh
u#m1:repo-name(github-branch)%
I use pyenv and virtualenvs.
How can I keep the virtual env name in my prompt?
Following #chepner's comment, I figured it out:
Use env to see the list of all env vars. pyenv uses PYENV_VERSION.
Add it to the prompt, use () to have the same look as pyenv does.
...
MYPS1=''
MYPS1+='($PYENV_VERSION) '
MYPS1+='%F{green}'
...

Python application in a container using Podman

I'd like to build a container using Podman which would contains the following:
a Python application
the Python modules I developed but which are not stored at the same place than the Python application
the Python environment (made with miniconda/mambaforge)
a mounted folder for input data
a mounted folder for output data
To do that, I've added a Dockerfile in my home directory. Below is the content of this Dockerfile:
FROM python:3
# Add the Python application
ADD /path/to/my_python_app /my_python_app
# Add the Python modules used by the Python application
ADD /path/to/my_modules /my_modules
# Add the whole mambaforge folder (contains the virtual envs) with the exact same path than the local one
ADD /path/to/mambaforge /exact/same/path/to/mambaforge
# Create a customized .bashrc (contains 'export PATH' to add mamba path and 'export PYTHONPATH' to add my_modules path)
ADD Dockerfile_bashrc /root/.bashrc
Then, I build the container with:
podman build -t python_app .
And run it with:
podman run -i -t -v /path/to/input/data:/mnt/input -v /path/to/output/data:/mnt/output python_app /bin/bash
In the Dockerfile, note I add the whole mambaforge (it is like miniconda). Is it possible to only add the virtual environment? I found I needed to add the whole mambaforge because I need to activate the virtual environment with mamba/conda activate my_env. Which I do in a .bashrc (with the conda initialization) that I put in /root/.bashrc. In this file, I also do export PYTHONPATH="/my_modules:$PYTHONPATH".
I'd also like to add the following line in my Dockerfile to execute automatically the Python application when running the container.
CMD ["python", "/path/to/my_python_app/my_python_app.py"]
However, this doesn't work because it seems the container needs to be run interactively in order to load the .bashrc first.
All of this is kludge and I'd like to know if there is a simpler and better way to do that?
Many thanks for your help!

Python PATH is taken over by anaconda even I've modified .bash_profile [duplicate]

I recently installed anaconda2 on my Mac. By default Conda is configured to activate the base environment when I open a fresh terminal session.
I want access to the Conda commands (i.e. I want the path to Conda added to my $PATH which Conda does when initialised so that's fine).
However I don't ordinarily program in python, and I don't want Conda to activate the base environment by default.
When first executing conda init from the prompt, Conda adds the following to my .bash_profile:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/geoff/anaconda2/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/Users/geoff/anaconda2/etc/profile.d/conda.sh" ]; then
. "/Users/geoff/anaconda2/etc/profile.d/conda.sh"
else
export PATH="/Users/geoff/anaconda2/bin:$PATH"
fi
# fi
unset __conda_setup
# <<< conda initialize <<<
If I comment out the whole block, then I can't activate any Conda environments.
I tried to comment out the whole block except for
export PATH="/Users/geoff/anaconda2/bin:$PATH"
But then when I started a new session and tried to activate an environment, I got this error message:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
This question (and others like it) are helpful, but doesn't ultimately answer my question and is more suited for linux users.
To be clear, I'm not asking to remove the (base) from my $PS1 I'm asking for Conda not to activate base when I open a terminal session.
I have conda 4.6 with a similar block of code that was added by conda. In my case, there's a conda configuration setting to disable the automatic base activation:
conda config --set auto_activate_base false
The first time you run it, it'll create a .condarc in your home directory with that setting to override the default.
This wouldn't de-clutter your .bash_profile but it's a cleaner solution without manual editing that section that conda manages.
There're 3 ways to achieve this after conda 4.6. (The last method has the highest priority.)
Use sub-command conda config to change the setting.
conda config --set auto_activate_base false
In fact, the former conda config sub-command is changing configuration file .condarc. We can modify .condarc directly. Add following content into .condarc under your home directory,
# auto_activate_base (bool)
# Automatically activate the base environment during shell
# initialization. for `conda init`
auto_activate_base: false
Set environment variable CONDA_AUTO_ACTIVATE_BASE in the shell's init file. (.bashrc for bash, .zshrc for zsh)
export CONDA_AUTO_ACTIVATE_BASE=false
To convert from the condarc file-based configuration parameter name to the environment variable parameter name, make the name all uppercase and prepend CONDA_. For example, conda’s always_yes configuration parameter can be specified using a CONDA_ALWAYS_YES environment variable.
The environment settings take precedence over corresponding settings in .condarc file.
References
The Conda Configuration Engine for Power Users
Using the .condarc conda configuration file
conda config --describe
Conda 4.6 Release
The answer depends a little bit on the version of conda that you have installed. For versions of conda >= 4.4, it should be enough to deactivate the conda environment after the initialization, so add
conda deactivate
right underneath
# <<< conda initialize <<<
To disable auto activation of conda base environment in terminal:
conda config --set auto_activate_base false
To activate conda base environment:
conda activate
So in the end I found that if I commented out the Conda initialisation block like so:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
# __conda_setup="$('/Users/geoff/anaconda2/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
# if [ $? -eq 0 ]; then
# eval "$__conda_setup"
# else
if [ -f "/Users/geoff/anaconda2/etc/profile.d/conda.sh" ]; then
. "/Users/geoff/anaconda2/etc/profile.d/conda.sh"
else
export PATH="/Users/geoff/anaconda2/bin:$PATH"
fi
# fi
# unset __conda_setup
# <<< conda initialize <<<
It works exactly how I want. That is, Conda is available to activate an environment if I want, but doesn't activate by default.
If you manage your .bashrc manually and like to keep it simple, all you really need is:
. "$HOME/anaconda2/etc/profile.d/conda.sh"
See Recommended change to enable conda in your shell.
This will make the conda command available without activating the base environment (nor reading your conda config).
Note that this is (of course) not compatible with managing the conda installation with conda init, but other than that, nothing bad is coming from it. You may even experience a significant speedup compared to the conda init generated code, because this solution avoids calling conda to parse your config files on whether to enable the base environment, etc.
It's best to also keep the if/fi lines to avoid error messages if using the same bashrc on several systems where conda may not be installed:
if [ -f "$HOME/anaconda2/etc/profile.d/conda.sh" ]; then
. "$HOME/anaconda2/etc/profile.d/conda.sh"
fi
Finally, if you share your bashrc between several systems where conda may be installed in different paths, you could do as follows:
for CONDA_PREFIX in \
"$HOME/anaconda2" \
"$HOME/miniconda3" \
"/opt/miniconda3" \
do
if [ -f "$CONDA_PREFIX/etc/profile.d/conda.sh" ]; then
. "$CONDA_PREFIX/etc/profile.d/conda.sh"
break
fi
done
Of course, this is now similar in length compared to the conda init generated code, but will still execute much faster, and will likely work better than conda init for users who synchronize their .bashrc between different systems.
for conda 4.12.0 (under WOS) the following worked (where all the previous answers -these included- didn't do the trick):
in your activate.bat file (mine was at ~/miniconda3/Scripts/activate.bat), change the line:
#REM This may work if there are spaces in anything in %*
#CALL "%~dp0..\condabin\conda.bat" activate %*
into
#REM This may work if there are spaces in anything in %*
#CALL "%~dp0..\condabin\conda.bat" deactivate
this line chage/modification doesn't work in the section (of the activate.bat file):
#if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (
#CALL "%~dp0..\condabin\conda.bat" activate
#GOTO :End
)
maybe because it depends on how your miniconda3 (Anaconda Prompt) executable is set up: %windir%\System32\cmd.exe "/K" some-path-to\miniconda3\Scripts\activate.bat some-path-to\miniconda3 (in my case).
caveat: updating conda overwrites this (activate.bat) file; so one has to modify the above line as many times as needed/updated. not much of a deal-breaker if you ask me.
This might be a bug of the recent anaconda. What works for me:
step1: vim /anaconda/bin/activate, it shows:
#!/bin/sh
_CONDA_ROOT="/anaconda"
# Copyright (C) 2012 Anaconda, Inc
# SPDX-License-Identifier: BSD-3-Clause
\. "$_CONDA_ROOT/etc/profile.d/conda.sh" || return $?
conda activate "$#"
step2: comment out the last line: # conda activate "$#"
One thing that hasn't been pointed out, is that there is little to no difference between not having an active environment and and activating the base environment, if you just want to run applications from Conda's (Python's) scripts directory (as #DryLabRebel wants).
You can install and uninstall via conda and conda shows the base environment as active - which essentially it is:
> echo $Env:CONDA_DEFAULT_ENV
> conda env list
# conda environments:
#
base * F:\scoop\apps\miniconda3\current
> conda activate
> echo $Env:CONDA_DEFAULT_ENV
base
> conda env list
# conda environments:
#
base * F:\scoop\apps\miniconda3\current

How to permanently activate desired environment upon login to EC2 instance

I have the following conda list:
➜ ~ conda env list
# conda environments:
#
base * /home/ubuntu/anaconda3
amazonei_mxnet_p36 /home/ubuntu/anaconda3/envs/amazonei_mxnet_p36
aws_neuron_mxnet_p36 /home/ubuntu/anaconda3/envs/aws_neuron_mxnet_p36
aws_neuron_pytorch_p36 /home/ubuntu/anaconda3/envs/aws_neuron_pytorch_p36
aws_neuron_tensorflow_p36 /home/ubuntu/anaconda3/envs/aws_neuron_tensorflow_p36
mxnet_latest_p37 /home/ubuntu/anaconda3/envs/mxnet_latest_p37
mxnet_p36 /home/ubuntu/anaconda3/envs/mxnet_p36
python3 /home/ubuntu/anaconda3/envs/python3
pytorch_latest_p37 /home/ubuntu/anaconda3/envs/pytorch_latest_p37
pytorch_p37 /home/ubuntu/anaconda3/envs/pytorch_p37
tensorflow2_latest_p37 /home/ubuntu/anaconda3/envs/tensorflow2_latest_p37
tensorflow2_p37 /home/ubuntu/anaconda3/envs/tensorflow2_p37
tensorflow_p37 /home/ubuntu/anaconda3/envs/tensorflow_p37
I want to enable my environment to be set to tensorflow2_latest_p37
whenever I login to my AWS account. How can I achieve that?
I tried putting source activate ensorflow2_latest_p37 in .zshrc. But it gave me:
this error message
/home/ubuntu/.zshrc:source:7: no such file or directory: activate
To change the activated environment permanently, there is no method except creating a startup script that runs the conda activate command.
In ubuntu, in .bashrc file, added the following command:
conda activate <env>
and then the specified env is getting activated whenever I open a terminal.
Edited
I created an ec2 instance in aws and used zsh shell. Inserting the bellow 2 lines in .zshrc worked for me:
source ~/anaconda3/bin/activate
conda activate <env_name>
Now when I login to my ec2 instance i get my env activated. I hope this will work for you too.

virtualenvwrapper: how to update project path?

When I move a project folders I have to manually update the project path in the .project file to get the workon command to work. Is it possible to update the path automatically?
According to the docs you can use setvirtualenvproject. This will automatically move you to the project folder if you use the workon command:
bono~$: setvirtualenvproject ~/.virtualenvs/your-virtual-env/ ~/path/to/your/project
Or, as beruic mentioned, it's easier to activate the environment and move to your desired working directory first. Please note that this not always work on my system, but it is a lot easier if it does work for you:
$ workon your-virtual-env
$ cd ~/path/to/your/project
$ setvirtualenvproject
In the future it might also be handy to specify the project path for the virtualenv on creation. You have to specify the project with the -a flag.
The -a option can be used to associate an existing project directory with the new environment.
You can use it something like this:
bono~$: cd ~/your/project
bono~$: mkvirtualenv my-project -a $(pwd)
The next time you use workon you will automatically be moved to your project directory.
Alternative
If you want to automatically detect directory changes and set the correct virtualenvwrapper then and there you can have a look at this post. It's a bit too expansive to go into detail here, but I think you can find what you're looking for there if that's what you meant.
You can just activate your virtual environment, go to the folder you want as project folder and call setvirtualenvproject:
$ workon [your_project]
$ cd [desired_project_folder]
$ setvirtualenvproject
Then the current folder will be set as project folder in the current virtualenv.

Categories

Resources