I read that the prefix line in the environment.yaml file is not used by conda env create. Two of the posts on SO pointing to this fact are:
export conda environment without prefix variable which shows local path to executable
Anaconda export Environment file
I have the reverse problem of most of these posts
I want to specify inside the file the actual prefix, so that different users setup their environments in their home directory in a shared machine.
However, as previously mentioned the command for creating environments is completely ignoring the prefix line.
I managed to setup an environment to a specific path using a prefix like this:
conda env create --prefix=<prefix> --file=environment.yaml
but I am trying to figure a way to define the prefix so the user will not have to type it themselves but it will be automatically configured to be their home directory.
I work around the lacking of proper solution by using Makefile target
# project_root/Makefile
# from my experience mamba installs and updates faster than conda
# conda install mamba -n base -c conda-forge
env-create:
mamba env create -p ./envs -f environment.yml
env-update:
mamba env update -p ./envs -f environment.yml
usage:
$ make env-create
# or
$ make env-update
Related
Recently, my anaconda environment got broken due to certain bad conda package upgrade. Due to this back experience, I would like to back up my anaconda environment for future easy recovery.
What I did to back up was to zip up the entire folder at C:\ProgramData\Anaconda3. Is this the correct way?
I am using Windows 10, anaconda python v3.6 64-bit.
It doesn't really make sense to zip up a conda environment for back up purposes since there are other ways to do this which may be more appropriate and use the in-built functions designed to do just this.
You can create a .txt version of the conda environment that details each module and version within, and can then be used to re-create the EXACT environment in the future.
# Create list of the environment
conda list --explicit environment_backup.txt
# Use the newly created text file to recreate the environment
conda create --name my_env_name --file environment_backup.txt
See docs for more information on managing conda environments.
N.B. As an additional point, conda environment directories can be fairly large (often >1GB) whereas the txt file created here is ~25KB, offering a clear advantage when archiving something for safe-keeping.
There are numerous ways you can achieve that as the way the anaconda website has shared. However, if you have limited or no internet accessibility, using a tool named "conda-pack" is recommended (unfortunately with the same OS for now).
Follow steps below:
conda install -c conda-forge conda-pack
And then on your source machine:
# Pack environment my_env into my_env.tar.gz
$ conda pack -n my_env
# Pack environment my_env into out_name.tar.gz
$ conda pack -n my_env -o out_name.tar.gz
# Pack environment located at an explicit path into my_env.tar.gz
$ conda pack -p /explicit/path/to/my_env
Lastly, on your target machine:
# Unpack environment into directory `my_env`
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
# Use python without activating or fixing the prefixes. Most python
# libraries will work fine, but things that require prefix cleanups
# will fail.
$ ./my_env/bin/python
# Activate the environment. This adds `my_env/bin` to your path
$ source my_env/bin/activate
# Run python from in the environment
(my_env) $ python
# Cleanup prefixes from in the active environment.
# Note that this command can also be run without activating the environment
# as long as some version of python is already installed on the machine.
(my_env) $ conda-unpack
# At this point the environment is exactly as if you installed it here
# using conda directly. All scripts should work fine.
(my_env) $ ipython --version
# Deactivate the environment to remove it from your path
(my_env) $ source my_env/bin/deactivate
I have Anaconda 5.1 Python distribution installed (by the system admin) on Windows 10, for all users. I can create an environment and then view the available environments:
λ conda create --name py35 python=3.5 anaconda
...
λ conda env list
# conda environments:
#
base * C:\ProgramData\Anaconda3
py35 C:\Users\<my-user-name>\AppData\Local\conda\conda\envs\py35
When I log in as a different user, however, only the base environment is visible/available. How can I create an environment and make it available to all users of the system?
The documentation discusses multi-user installations, but I cannot see how to make environments available to other users.
I would shy away from sharing environments with other users, because if they don't know what they are doing, they could add packages that could conflict with other packages and/or even delete packages that another user might need. The preferred approach is that after you have created an environment, you export it as a yml file:
conda env export > environment.yml
Then you send the users the yml file and have them build their own environment using the yml:
conda env create -f environment.yml
If you really want to use a shared environment where every user can access, then you have to use the -p or --prefix option in your create:
conda create -p C:/full/public/path/to/py35 python=3.5
And then instruct your users to add the public path (C:/full/public/path/to) to their conda config file. Then, they should be able to see the environment when running conda env list.
The key here is adding the path to the folder containing the environment(s) to the user's conda configuration file .condarc. Like this:
envs_dirs:
- C:\full\path\to\environments\folder
This makes all the environments (subfolders within) available to the user. It does not appear to be possible to make a specific, named environment available.
As has been pointed out, you can create an environment in a specific location using the -p flag, and then add the parent directory to the configuration file, but this is not a requirement. This may be useful, however, to avoid permissions errors if sharing environments that exists in protected user areas.
On Windows 10, my user configuration file was in C:\Users\<my-user-name>\, and I just added the above text to the end of it.
The Conda-Cheatseet here lists these following commands.
Save Environment:
conda list --explicit > my-env.txt
Create Environment:
conda env create --file my-env.txt
I tried to create a new environment for anaconda installation that I want to tweak apart from the original install. I found this not working:
$ conda env create --name pandas018numpy111 pandas018test1
Using Anaconda Cloud api site https://api.anaconda.org
Error: Invalid name, try the format: user/package
pandas018test1 does not exist or can't be accessed
environment.yml file not found
There is no requirements.txt
What is wrong here?
What I wanted to accomplish was to create the copy of the original environment, add some modules for testing around, then toss away the test environment (pandas018numpy111).
Ah, I found the answer.
What I need is the conda create command, actually,
$ conda create -n pandas018numpy111 --clone root
Then I have the throw-away environment to modify, etc and later just toss. To switch then I will just use:
$ source activate pandas018numpy111
and to exit from this environment,
$ source deactivate
I am trying to create virtual environment using environment.yml in miniconda (Where environment.yml contains a list of all dependecies.) using the following command:
conda env create -f environment.yml
but I get this error (this is entire output)
Error: prefix already exists: /home/danish/miniconda3/envs/venv
Can someone help me correcting the error?
Thanks in advance :)
The environment.yml specifies that the name of the environment is venv at the top of your file -- i.e.
name: venv
But that environment already exists (you can see it via conda env list). The solution here is to change the name in the environment.yml or use a different name when you are creating the environment. Example:
conda env create -f environment.yml -n new-env-name
Where the new-env-name is an environment name you haven't used yet.
I want to import an existing virtual env, which I created using anaconda into another anaconda installation on a different distribution.
I've tried creating a new one using the following command in the directory of the venv copied from the other distribution:
conda create -p . python=3.4
That results in:
Error: prefix already exists: /home/xiaolong/development/blog
But anaconda does not know that, when I ask it to list all existing venvs:
conda info --envs
This results in:
# conda environments:
#
firstenv /home/xiaolong/development/anaconda3/envs/firstenv
gtkplus-tool /home/xiaolong/development/anaconda3/envs/gtkplus-tool
testenv /home/xiaolong/development/anaconda3/envs/testenv
tkxld /home/xiaolong/development/anaconda3/envs/tkxld
wxpython-phoenix-tutorial /home/xiaolong/development/anaconda3/envs/wxpython-phoenix-tutorial
root * /home/xiaolong/development/anaconda3
This list is missing my copied venv. How do I add it to that list, so that I can then use source activate blog for example?
-p . cannot work because the directory must not exist already.
You probably want your env in /home/xiaolong/development/blog/env or something similar. So just do conda create -p ./env python=3.4.