I tried to create a new environment for anaconda installation that I want to tweak apart from the original install. I found this not working:
$ conda env create --name pandas018numpy111 pandas018test1
Using Anaconda Cloud api site https://api.anaconda.org
Error: Invalid name, try the format: user/package
pandas018test1 does not exist or can't be accessed
environment.yml file not found
There is no requirements.txt
What is wrong here?
What I wanted to accomplish was to create the copy of the original environment, add some modules for testing around, then toss away the test environment (pandas018numpy111).
Ah, I found the answer.
What I need is the conda create command, actually,
$ conda create -n pandas018numpy111 --clone root
Then I have the throw-away environment to modify, etc and later just toss. To switch then I will just use:
$ source activate pandas018numpy111
and to exit from this environment,
$ source deactivate
Related
It often happens that when I create a conda environment, I forgot about activating it, despite the clear conda message. And I end up installing my packages in the conda base environment. (yeah... I'm a bit of a dreamer)
My questions are:
Are there use cases where you create a conda environment but don't activate it right after?
How to create a conda environment and activate it in a single line? (in a Linux prompt shell, and non-interactive)
As a concrete example in answer to your first question, I have used conda as a way to wrap a disposable build environment in some Makefile targets, i.e., I create the environment, and then subsequent commands or targets may make use of the environment via conda run.
Adapting a snippet from one Makefile, you could create a function in a bash startup file:
conda_create_and_run() {
ENV_NAME=$1
CONDA_PY_VER=$2
. ${CONDA_ENV_FILE}
conda config --append envs_dirs ${CONDA_DIR}
conda create -p ${CONDA_DIR}/${ENV_NAME} python=${CONDA_PY_VER} -y
conda activate ${ENV_NAME}
}
Here CONDA_PY_VER is the non-default python version you want the environment to possibly be created with, and CONDA_DIR and CONDA_ENV_FILE are, respectively, the location where conda keeps its environments and the conda environment file you need to source (or have part of your shell init) in order to have the conda commands available.
You would then use it as:
conda_create_and_run myenv 3.8
to create an environment for python3.8 named myenv.
I have a conda environment setup already.
I want to delete a few packages out of it, update few and add a few new packages.
currently, I am doing it manually one after another after source activate <my_exiting_conda>.
conda update x
conda update y
conda remove z --force
conda remove w --force
conda install -c <channel> <package name>
Is there any way using which I can put all these commands in a file and just use it to do all work at once.
Also when I run the above command manually, it asks to hit Y/n for each command? how can I avoid that?
Consider switching to using YAMLs to manage your envs and refrain from using conda update/install/remove commands. When you want to make multiple changes to an env, change them in the YAML, then use:
conda env update -f environment.yaml
This command also has the optional argument --prune which will remove any packages that are not required, i.e., provides the package deletion mechanism you seek.
Note that conda env commands do not provide a transaction review step. For that reason, I would not recommend using it to manage the base env.
Starting from Existing Env
To get a working YAML from an existing env (say foo), try running something like
conda env export -n foo --from-history > foo.yaml
The --from-history argument will only include the explicit specs that you've provided to the env, so the YAML will look closer to what one might make and maintain from scratch.
Note that if there were packages installed through pip they won't appear in the --from-history version. In that case, I would still begin from this version, then export a full YAML to capture any pip specs, and add them to the simpler version.
Yes, you can pass packages in one go
conda update x y
conda remove z w --force
I read that the prefix line in the environment.yaml file is not used by conda env create. Two of the posts on SO pointing to this fact are:
export conda environment without prefix variable which shows local path to executable
Anaconda export Environment file
I have the reverse problem of most of these posts
I want to specify inside the file the actual prefix, so that different users setup their environments in their home directory in a shared machine.
However, as previously mentioned the command for creating environments is completely ignoring the prefix line.
I managed to setup an environment to a specific path using a prefix like this:
conda env create --prefix=<prefix> --file=environment.yaml
but I am trying to figure a way to define the prefix so the user will not have to type it themselves but it will be automatically configured to be their home directory.
I work around the lacking of proper solution by using Makefile target
# project_root/Makefile
# from my experience mamba installs and updates faster than conda
# conda install mamba -n base -c conda-forge
env-create:
mamba env create -p ./envs -f environment.yml
env-update:
mamba env update -p ./envs -f environment.yml
usage:
$ make env-create
# or
$ make env-update
Recently, my anaconda environment got broken due to certain bad conda package upgrade. Due to this back experience, I would like to back up my anaconda environment for future easy recovery.
What I did to back up was to zip up the entire folder at C:\ProgramData\Anaconda3. Is this the correct way?
I am using Windows 10, anaconda python v3.6 64-bit.
It doesn't really make sense to zip up a conda environment for back up purposes since there are other ways to do this which may be more appropriate and use the in-built functions designed to do just this.
You can create a .txt version of the conda environment that details each module and version within, and can then be used to re-create the EXACT environment in the future.
# Create list of the environment
conda list --explicit environment_backup.txt
# Use the newly created text file to recreate the environment
conda create --name my_env_name --file environment_backup.txt
See docs for more information on managing conda environments.
N.B. As an additional point, conda environment directories can be fairly large (often >1GB) whereas the txt file created here is ~25KB, offering a clear advantage when archiving something for safe-keeping.
There are numerous ways you can achieve that as the way the anaconda website has shared. However, if you have limited or no internet accessibility, using a tool named "conda-pack" is recommended (unfortunately with the same OS for now).
Follow steps below:
conda install -c conda-forge conda-pack
And then on your source machine:
# Pack environment my_env into my_env.tar.gz
$ conda pack -n my_env
# Pack environment my_env into out_name.tar.gz
$ conda pack -n my_env -o out_name.tar.gz
# Pack environment located at an explicit path into my_env.tar.gz
$ conda pack -p /explicit/path/to/my_env
Lastly, on your target machine:
# Unpack environment into directory `my_env`
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
# Use python without activating or fixing the prefixes. Most python
# libraries will work fine, but things that require prefix cleanups
# will fail.
$ ./my_env/bin/python
# Activate the environment. This adds `my_env/bin` to your path
$ source my_env/bin/activate
# Run python from in the environment
(my_env) $ python
# Cleanup prefixes from in the active environment.
# Note that this command can also be run without activating the environment
# as long as some version of python is already installed on the machine.
(my_env) $ conda-unpack
# At this point the environment is exactly as if you installed it here
# using conda directly. All scripts should work fine.
(my_env) $ ipython --version
# Deactivate the environment to remove it from your path
(my_env) $ source my_env/bin/deactivate
I want to import an existing virtual env, which I created using anaconda into another anaconda installation on a different distribution.
I've tried creating a new one using the following command in the directory of the venv copied from the other distribution:
conda create -p . python=3.4
That results in:
Error: prefix already exists: /home/xiaolong/development/blog
But anaconda does not know that, when I ask it to list all existing venvs:
conda info --envs
This results in:
# conda environments:
#
firstenv /home/xiaolong/development/anaconda3/envs/firstenv
gtkplus-tool /home/xiaolong/development/anaconda3/envs/gtkplus-tool
testenv /home/xiaolong/development/anaconda3/envs/testenv
tkxld /home/xiaolong/development/anaconda3/envs/tkxld
wxpython-phoenix-tutorial /home/xiaolong/development/anaconda3/envs/wxpython-phoenix-tutorial
root * /home/xiaolong/development/anaconda3
This list is missing my copied venv. How do I add it to that list, so that I can then use source activate blog for example?
-p . cannot work because the directory must not exist already.
You probably want your env in /home/xiaolong/development/blog/env or something similar. So just do conda create -p ./env python=3.4.