I am a fairly inexperienced programmer and am struggling with installing Conda into Deepnote. Pip install doesn't work for certain packages. For example, I'm trying to install rdkit, a cheminformatics package, which has rather complex installation instructions or a simple 1 line of code managed through the Anaconda/mini-conda distribution. I really like the Deepnote notebooks and I would really appreciate any help here, please.
So far, I have found this useful code for Conda installation on Google Colab: https://github.com/dataprofessor/code/blob/master/python/google_colab_install_conda.ipynb
! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh
! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
Whilst this successfully works on Google Colab, I'm not sure why it fails as shown below on Deepnote:
--2020-12-04 22:58:34-- https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.130.3, 104.16.131.3, 2606:4700::6810:8203, ...
Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.130.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85055499 (81M) [application/x-sh]
Saving to: ‘Miniconda3-py37_4.8.2-Linux-x86_64.sh’
Miniconda3-py37_4.8 100%[===================>] 81.12M 133MB/s in 0.6s
2020-12-04 22:58:35 (133 MB/s) - ‘Miniconda3-py37_4.8.2-Linux-x86_64.sh’ saved [85055499/85055499]
PREFIX=/usr/local
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 392: /usr/local/conda.exe: Permission denied
chmod: cannot access '/usr/local/conda.exe': No such file or directory
Unpacking payload ...
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 404: /usr/local/conda.exe: No such file or directory
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 406: /usr/local/conda.exe: No such file or directory
I also want to do conda install -c bioconda gromacs, which I cannot find a work around for, so I am hoping someone can help me resolve this query.
Many thanks in advance!
P.S. I am on a Mac OS
You can check this notebook from Daniel Zvara
Using Conda in Deepnote in 3 simple steps
In sum
# 1. Install Conda and make Conda packages available in current environment
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
!chmod +x Miniconda3-latest-Linux-x86_64.sh
!sudo bash ./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
# 2. Package installation
# !sudo conda install -y [EXTRA_OPTIONS] package_name
# So for example Keras from conda-forge channel
!sudo conda install -y -c conda-forge keras
# 3. Package usage
import keras
Also found an alternative solution on the Deepnote community: https://community.deepnote.com/c/custom-environments/custom-environment-for-installing-conda-packages
Can set up a custom environment with the following Dockerfile (just fill in the placeholder for the packages you require).
FROM gcr.io/deepnote-200602/templates/deepnote
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
RUN bash ~/miniconda.sh -b -p $HOME/miniconda
ENV PATH $HOME/miniconda/bin:$PATH
RUN conda install python=3.7 ipykernel -y
RUN conda install <insert packages here> -y
RUN python -m ipykernel install --user --name=conda
ENV DEFAULT_KERNEL_NAME "conda"
And the custom environment in Deepnote is set up through the Environment tab in the sidebar, giving you an option for the Dockerfile. Copy & paste the above, then click build and restart machine - the environment will be set up allowing you to use conda.
You can now choose Anaconda in the built-in environments in Deepnote.
I have two machine that one does not have internet access. I want to install modules with anaconda and copy them to offline computer from the other computer that has internet access.
I tried looking for dependencies and install tar. files manually one by one and sent them to the offline machine but it is very time-consuming.
What is the easiest way? Does miniconda helpful ??
P.S: I forgot to mention that I am using anaconda in both machines. So I guess I need to create an env., install packages then export it for offline computer. Are there any other way to install number of packages to offline comp. from a copy <dir> in the online computer ??
Edit: I tried conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file://C:\Users\myName\Desktop\OfflineInstall\pkgs2 but offline machine still tried to connect internet. I also used --no-deps
Edit2: For those who stuck on the same problem, I solved using conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file:///C:\Users\myName\Desktop\OfflineInstall\pkgs2 --override-channels The tricky way is the file:/// prefix. You need to put ///. Also remember to put --override-channels flag to prevent connection to default channels.
It sounds like Conda-pack is what you are looking for.
Installing:
$ conda install conda-pack
On the source machine:
# Pack environment my_env into my_env.tar.gz
$ conda pack -n my_env
On the target machine:
# Unpack environment into directory `my_env`
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
# Use python without activating or fixing the prefixes. Most python
# libraries will work fine, but things that require prefix cleanups
# will fail.
$ ./my_env/bin/python
# Activate the environment. This adds `my_env/bin` to your path
$ source my_env/bin/activate
# Run python from in the environment
(my_env) $ python
# Cleanup prefixes from in the active environment.
# Note that this command can also be run without activating the environment
# as long as some version of python is already installed on the machine.
(my_env) $ conda-unpack
The caveat being that conda-pack will take the whole environment.
Had this problem the other day, very simple implementation.
First make a .txt file which contains all your python libraries. Now you can just pass this .txt file to whatever machine you want the solution to be installed under and issue the following command :
pip install -r packages.txt
Where "packages" is the name of your .txt file. Hope this helps!
Edit using Conda :
while read requirement; do conda install --yes $requirement; done < requirements.txt
I am currently working on a project in a Git repository that has several different branches. I would like to set up two different conda environments, each using the code from a different branch of this one repository.
I would like each of these environments to be completely independent because I run my project on a compute cluster and would like to queue multiple jobs running on different branches of the repository simultaneously.
More specifically, my repository is organized like
my-repo/
master
issue-fix
additional-feature
and I would like to create one conda environment called something like env1 that uses the code in issue-fix along with some other packages. Then, I would like to create another environment called env2 that uses code in additional-feature along with those same other packages.
With the code in branch issue-fix checked-out, I've tried running the following to create both environments:
conda create -n env1
source activate env1
conda install <my other packages>
pip install -e git+file:///path/to/repo#issue-fix#egg=repo-0.1
source deactivate
conda create -n env2
source activate env2
conda install <my other packages>
pip install -e git+file:///path/to/repo#additional-feature#egg=repo-0.1
However, in environment env2, the code in branch additional-feature was inaccessible. How should I go about setting up these two environments?
This is actually a very interesting problem which I am running into now. I have a repo with two branches in very active development and I need to switch back and forth quite often. The branches have different compiled code (Fortran & C) versions which means that every time I git checkout <branch> I need to recompile code. Not optimal to say the least.
I would like to have two conda environments, with a different branch installed in each env. I would like to have two python kernels running, one started from each conda env and be able to test (run code) in both without doing switching branches (only the window I work in).
What I ended up doing is cloning the repo twice under separate testing directories, both separate from the main repo's main dev dir, checking out different branches in each directory and installing each in a separate conda environment.
As long as I don't touch the master branch in each of those testing directories, I will be able to push upstream and merge when I am ready.
I know this is an old question but I have recently been trying to do the same thing, and found that you can specify the directory for a repo by adding --src <target-repo-directory> to the pip install command. Using different target directories will then allow you to install different branches in different conda virtual environments (which you can confirm with pip list).
For the example above, you would want:
mkdir /path/to/issue-fix-branch
mkdir /path/to/additional-feature-branch
conda create -n env1
source activate env1
conda install <my other packages>
pip install --src /path/to/issue-fix-branch -e git+file:///path/to/repo#issue-fix#egg=repo-0.1
source deactivate
conda create -n env2
source activate env2
conda install <my other packages>
pip install --src /path/to/additional-feature-branch -e git+file:///path/to/repo#additional-feature#egg=repo-0.1
I'm using to Docker to build a Python container with the intention of having a reproducible environment on several machines, which are a bunch of development Macbooks and several AWS EC2 servers.
The container is based on continuumio/miniconda3, i.e. Dockerfile starts with
FROM continuumio/miniconda3
A few days ago on Ubuntu the conda install and conda upgrade commands in the Docker file complained about a new Conda version (4.11) being available:
==> WARNING: A newer version of conda exists. <==
current version: 4.4.10
latest version: 4.4.11
Please update conda by running
$ conda update -n base conda
If I ignore this, package installations quit with an error:
Downloading and Extracting Packages
The command '/bin/sh -c conda install -y pandas=0.22.0 matplotlib
scikit-learn=0.19.1 pathos lazy openpyxl pytables dill pydro psycopg2
sqlalchemy pyarrow arrow-cpp parquet-cpp scipy tensorflow keras
xgboost' returned a non-zero code: 1
When I add this conda update... to the Docker file, things work again.
What's really annoying, however, is that the update that makes things run in Ubuntu does not work on Mac Docker. I get the following error:
CondaEnvironmentNotFoundError: Could not find environment: base .
You can list all discoverable environments with `conda info --envs`.
Note that I get this error when I docker build the same Docker file that works on the Ubuntu machine, which kind of ruins the whole point about using Docker in the first place. On the Mac, the old version of the file (without conda update -n base conda) still runs fine and installs all packages.
Docker / Conda experts, any ideas?
Edit: Here's the full Dockerfile (the one that works in Ubuntu):
# Use an official Python runtime as a parent image
FROM continuumio/miniconda3
WORKDIR /app/dev/predictive.analytics
RUN apt-get update; \
apt-get install -y gcc tmux htop
RUN conda update -y -n base conda
RUN conda config --add channels babbel; \
conda config --add channels conda-forge;
RUN conda install -y pandas=0.22.0 matplotlib scikit-learn=0.19.1 pathos lazy openpyxl pytables dill pydro psycopg2 sqlalchemy pyarrow arrow-cpp parquet-cpp scipy tensorflow keras xgboost
RUN pip install recordclass sultan
RUN conda upgrade -y python
ENV DATA_DIR /host/data
ENV PYTHONPATH /host/predictive.analytics/python
ENV PATH="/host/predictive.analytics:${PATH}"
Perhaps you're using an outdated miniconda on one of the build machine, try doing docker build --pull --no-cache.
Docker doesn't necessarily pull the latest image from the repository, so unless you do a --pull, it is possible that some of your machines may be starting the build with outdated base image.
I installed Python Anaconda on Mac (OS Mavericks). I wanted to revert to the default version of Python on my Mac. What's the best way to do this? Should I delete the ~/anaconda directory? Any other changes required?
Currently when I run which python I get this path:
/Users/username/anaconda/bin/python
From the docs:
To uninstall Anaconda open a terminal window and remove the entire
anaconda install directory: rm -rf ~/anaconda. You may also edit
~/.bash_profile and remove the anaconda directory from your PATH
environment variable, and remove the hidden .condarc file and
.conda and .continuum directories which may have been created in
the home directory with rm -rf ~/.condarc ~/.conda ~/.continuum.
Further notes:
Python3 installs may use a ~/anaconda3 dir instead of ~/anaconda.
You might also have a ~/.anaconda hidden directory that may be removed.
Depending on how you installed, it is possible that the PATH is modified in one of your runcom files, and not in your shell profile. So, for example if you are using bash, be sure to check your ~/.bashrc if you don't find the PATH modified in ~/.bash_profile.
The anaconda installer adds a line in your ~/.bash_profile script that prepends the anaconda bin directory to your $PATH environment variable. Deleting the anaconda directory should be all you need to do, but it's good housekeeping to remove this line from your setup script too.
Package "anaconda clean", available from Anaconda platform, should uninstall safely.
conda activate your_conda_env # activate your conda environment
conda install anaconda-clean # install the package anaconda clean
anaconda-clean --yes # clean all anaconda related files and directories
rm -rf ~/anaconda3 # removes the entire anaconda directory
rm -rf ~/.anaconda_backup # anaconda clean creates a back_up of files/dirs, remove it
# (conda list; cmd shouldn't respond after the clean up)
Refer: https://docs.anaconda.com/anaconda/install/uninstall for more details.
Note: Also, you may want to edit .bashrc (or .bash_profile) & remove the conda path in $PATH environment variable for full proper clean-up
Removing the Anaconda directory helps, but I don't think that's a good idea as you might need to use anaconda sometimes in near future. So, as suggested by mwaskom, anaconda installer automatically adds PATH variable which points to anaconda/bin directory in the ~/.bashrc file.
It looks like this
PATH="/home/linuxsagar/anaconda3/bin:$PATH
So, just comment out the line (add # in the beginning of the line).
Then reload the ~/.bashrc file executing source ~/.bashrc
Now, verify the changes executing which python in the new terminal.
rm -rf ~/anaconda
It was pretty easy. It switched my pointer to Python:
https://docs.continuum.io/anaconda/install#os-x-uninstall
If you're uninstalling Anaconda to be able to use the base Python installation in the system, you could temporarily disable the path by following these steps and not uninstalling Anaconda.
Go to your home directory. Just a cd command will do.
Edit the file .bashrc.
Look for something like export PATH="/home/ubuntu/anaconda3/bin:$PATH" in the file.
Put a # at the beginning to comment it from the script.
#export PATH="/home/ubuntu/anaconda3/bin:$PATH"
Open a new terminal and you should be running the base python installation. This works on Linux systems. Should work on Mac too.
Uninstalling Anaconda
To uninstall Anaconda, you can do a simple remove of the program. This will leave a few files behind, which for most users is just fine. See Option A.
If you also want to remove all traces of the configuration files and directories from Anaconda and its programs, you can download and use the Anaconda-Clean program first, then do a simple remove. See Option B.
Option A.
Use simple remove to uninstall Anaconda:
macOS–Open the Terminal.app or iTerm2 terminal application, and then remove your entire Anaconda directory, which has a name such as anaconda2 or anaconda3, by entering rm -rf ~/anaconda3.
Option B.
Full uninstall using Anaconda-Clean and simple remove.
NOTE: Anaconda-Clean must be run before simple remove.
Install the Anaconda-Clean package from Anaconda Prompt or a terminal window:
conda install anaconda-clean
In the same window, run one of these commands:
Remove all Anaconda-related files and directories with a confirmation prompt before deleting each one:
anaconda-clean
Or, remove all Anaconda-related files and directories without being prompted to delete each one:
anaconda-clean --yes
Anaconda-Clean creates a backup of all files and directories that might be removed, such as .bash_profile, in a folder named .anaconda_backup in your home directory. Also note that Anaconda-Clean leaves your data files in the AnacondaProjects directory untouched.
After using Anaconda-Clean, follow the instructions above in Option A to uninstall Anaconda.
Removing Anaconda path from .bash_profile
If you use Linux or macOS, you may also wish to check the .bash_profilefile in your home directory for a line such as:
export PATH="/Users/jsmith/anaconda3/bin:$PATH"
NOTE: Replace /Users/jsmith/anaconda3/ with your actual path.
This line adds the Anaconda path to the PATH environment variable. It may refer to either Anaconda or Miniconda. After uninstalling Anaconda, you may delete this line and save the file.
by official uninstalling way
rm -rf ~/anaconda3
nano ~/.bashrc
Ctrl+W to search for "Anaconda"
Delete or comment out the following lines:
/home/sammuel/.bashrc
# added by Anaconda3 4.2.0 installer
export PATH="/home/sammuel/anaconda3/bin:$PATH"
When you’re done editing the file, type Ctrl+X to exit and y to save changes.
Anaconda is now removed from your server.
Install the cleaner
me#host:~$ conda install anaconda-clean
Activate the 'base' virtual environment
me#host:~$ source ~/anaconda3/bin/activate
Run the cleaner
(base) me#host:~$ anaconda-clean --yes
Deactivate the 'base' virtual environment
(base) me#host:~$ conda deactivate
Remove the files
me#host:~$ rm -rf ~/anaconda3
me#host:~$ rm -rf ~/.anaconda_backup
Delete lines added by conda from environment file(s)
Open the .bashrc file (and/or .profile and/or .bash_profile)
nano .bashrc
Search for conda:
press CTRL+W
type conda
press ENTER
Remove everything that looks like it has been added by/for anaconda:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/me/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/me/anaconda3/etc/profile.d/conda.sh" ]; then
. "/home/me/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/home/me/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
This was done on Ubuntu 18.04
In case you have multiple version of anaconda,
rm -rf ~/anaconda2 [for version 2]
rm -rf ~/anaconda3 [for version 3]
Open .bashrc file in a text editor
vim .bashrc
remove anaconda directory from your PATH.
export PATH="/home/{username}/anaconda2/bin:$PATH" [for version 2]
export PATH="/home/{username}/anaconda3/bin:$PATH" [for version 3]
I simply:
rm -rf ~/anaconda3
...this removed conda also.
Then:
mousepad ~/.bashrc
...and removed the path line added at the very bottom (clearly identified by Anaconda as 'added by Anaconda'.
Worth noting that anaconda3 created a backup of my .bashrc file before modification, and named it as:
./bashrc-anaconda3.bak
...so I could always have just renamed this and deleted my modified .bashrc
To uninstall anaconda you have to:
1) Remove the entire anaconda install directory with:
rm -rf ~/anaconda2
2) And (OPTIONAL):
->Edit ~/.bash_profile to remove the anaconda directory from your PATH environment variable.
->Remove the following hidden file and folders that may have been created in the home directory:
rm -rf ~/.condarc ~/.conda ~/.continuum
source
I always try to follow the developers advice, since they are usually the ones that now how it would affect your system. Theoretically this should be the safest way:
Install the Anaconda-Clean package from Anaconda Prompt (terminal on Linux or macOS):
conda install anaconda-clean
In the same window, run one of these commands:
Remove all Anaconda-related files and directories with a confirmation prompt before deleting each one:
anaconda-clean
Remove all Anaconda-related files and directories without being prompted to delete each one:
anaconda-clean --yes
Anaconda-Clean creates a backup of all files and directories that might be removed in a folder named .anaconda_backup in your home directory. Also note that Anaconda-Clean leaves your data files in the AnacondaProjects directory untouched.
https://docs.anaconda.com/anaconda/install/uninstall/
To uninstall Anaconda Fully from your System :
Open Terminal
rm -rf ~/miniconda
rm -rf ~/.condarc ~/.conda ~/.continuum
In my case Anaconda3 was not installed in home directory. Instead, it was installed in root. Therefore, I had to do the following to get it uninstalled:
sudo rm -rf /anaconda3/bin/python
Install the Anaconda-Clean package from Anaconda Prompt
conda install anaconda-clean
In the same window, run one of these commands:
TO remove all Anaconda-related files and directories without being prompted to delete each one:
anaconda-clean --yes
Windows:
Use Windows Explorer to delete the envs and pkgs folders prior to running the uninstall in the root of your installation.
In the Control Panel, choose Add or Remove Programs or Uninstall a program, and then select Python 3.6 (Anaconda) or your version of Python.
In macOs
rm -rf ~/opt/anaconda3
rm -rf ~/anaconda3
It was enough
For windows
Install anaconda-clean module using
conda install anaconda-clean
then, run the following command to delete files step by step:
anaconda-clean
Or, just run following command to delete them all-
anaconda-clean --yes
After this Open Control Panel> Programs> Uninstall Program, here uninstall that python for which publisher is Anaconda.
Now, you can remove anaconda/scripts and /anaconda/ from PATH variable.
Hope, it helps.