So I'n new-ish to Docker.
What I'm trying to do:
create a docker image (that puts you into an anaconda enviroment and downloads some libraries via conda), within that enviroment run a setup.py install so that the library CPLEX is also in the image
Build the image. Next I run the image, and set a scratch directory that should have all the python programs in it that I want to run.
This is my docker file:
## syntax=docker/dockerfile:1
from continuumio/miniconda3
RUN conda create --name opt python=3.7
ENV PATH /opt/conda/envs/mro_env/bin:$PATH
RUN /bin/bash -c "source activate opt"
RUN conda install -c conda-forge fenics
RUN conda install -c conda-forge scipy
RUN conda install -c conda-forge numpy
RUN conda install -c conda-forge matplotlib
RUN conda install -c conda-forge ipopt
RUN conda install -c conda-forge glpk
RUN conda install -c conda-forge pyomo
RUN conda install -c conda-forge pandas
RUN conda install -c conda-forge scikit-learn
ENTRYPOINT [ "python" ]
COPY CPLEX_Studio129/ /CPLEX_Studio129/
COPY . /codes/
RUN cd /CPLEX_Studio129/ && python cplex/python/3.7/x86-64_osx/setup.py install
WORKDIR /codes/
RUN cd /codes/
Next I build the image, and then run the image :
sudo docker build -t ex:latest .
docker run --rm -it --entrypoint /bin/bash ex
(base) root#bc9beca93d69:/codes# python test_cplex.py
Traceback (most recent call last):
File "/codes/test_cplex.py", line 1, in <module>
import cplex
ModuleNotFoundError: No module named 'cplex'
(base) root#bc9beca93d69:/codes# ls
For what its worth, if I make /CPLEX_Studio129/ the WORKDIR, and install the library, then start a python in that directory, and try to import cplex it works. But my hope was that it would work when I run a program that imports CPLEX in /codes/, which it cannot find.
Any tips, and a bried explation of where I went wrong would be greatly appreciated.
Related
So I'm trying to install FEniCS from the instructions here. I did the
pip3 install fenics-ffc --upgrade
inside my virtualenv and it worked but when I try to import dolfin I get a ModuleNotFound error. I'm not sure how to get dolfin installed. I did
pip install pybind11
to get pybind11 installed then copied the code for dolfin installation into my cmd
FENICS_VERSION=$(python3 -c"import ffc; print(ffc.__version__)")
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/dolfin
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/mshr
mkdir dolfin/build && cd dolfin/build && cmake .. && make install && cd ../..
mkdir mshr/build && cd mshr/build && cmake .. && make install && cd ../..
cd dolfin/python && pip3 install . && cd ../..
cd mshr/python && pip3 install . && cd ../..
but it just spat out dozens of errors like:
FENICS_VERSION=$(python3 -c"import ffc; print(ffc.version)") 'FENICS_VERSION' is not recognized as an internal or external command, operable program or batch file.
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/dolfin Cloning into 'dolfin'...
fatal: Remote branch $FENICS_VERSION not found in upstream origin
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/mshr Cloning into 'mshr'...
fatal: Remote branch $FENICS_VERSION not found in upstream origin
There were lots more errors after too. Am I not supposed to paste the dolfin code into cmd? I don't know much about this stuff so unsure of how to get the dolfin module. I've previously only used pip to get my packages but this does not work for dolfin as it doesn't appear to be on PyPI.
Do you have cmake? It says in the docs you need it. Also its says to to this to install pybind11 not pip install pybind11
For building optional Python interface of DOLFIN and mshr, pybind11 is needed since version 2018.1.0. To install it:
wget -nc --quiet https://github.com/pybind/pybind11/archive/v${PYBIND11_VERSION}.tar.gz
tar -xf v${PYBIND11_VERSION}.tar.gz && cd pybind11-${PYBIND11_VERSION}
mkdir build && cd build && cmake -DPYBIND11_TEST=off .. && make install
Also what is your os?
So here is how you can install fenics 2019.1 using conda (miniconda):
Install Conda:
First go to https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html
and follow the installation instructions.
Create a conda environment for fenics:
Open a terminal and type:
conda create -n fenics
To activate the created environment "fenics", type:
conda activate fenics
If you want the fenics environment to be activated automatically everytime you open a new terminal, then open you .bashrc file (should be under /home/username/.bashrc) and add the line "source activate fenics" below the ">>> conda initialize >>>" block.
Install fenics:
Type all these commands:
conda install -c conda-forge h5py=*=*mpich*
conda install -c conda-forge fenics
pip install meshio
pip install matplotlib
pip install --upgrade gmsh
conda install -c conda-forge paraview
pip install scipy
The second command will take a while. I added a few nice to have programs like gmsh and paraview which will help you to create meshes and view your solutions.
I'm trying to install torchreid which is a library for person re-identification in PyTorch. I've followed the steps mentioned on the git repository but is getting this error.
#conda install
!wget -c https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
!chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
!bash ./Miniconda3-4.5.4-Linux-x86_64.sh -b -f -p /usr/local
!conda install -q -y --prefix /usr/local python=3.6 ujson
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages')
!git clone https://github.com/KaiyangZhou/deep-person-reid.git
!cd deep-person-reid/
!conda create --name torchreid python=3.7
!conda activate torchreid
!pip install -r /content/deep-person-reid/requirements.txt
!conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
import torchreid
error
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-4faf46a39b5d> in <module>()
----> 1 import torchreid
ModuleNotFoundError: No module named 'torchreid'
Run below command,
# install torchreid (don't need to re-build it if you modify the source code)
python setup.py develop
First deactivate you created conda environment 'torchreid'. Activate it again and try to import.
deactivate torchreid
conda activate torchreid
Note: you are installing torchreid in a specifica conda environment, which in not accessible from outside of that environment.
I have a docker file like :
FROM conda/miniconda3-centos7
WORKDIR /tmp
COPY app/ /tmp
RUN conda install gcc_linux-64
RUN conda install gxx_linux-64
CMD ["python", "Hello_World.py"]
The code gets stuck after the first RUN conda command. The error i get is :
WARNING: The conda.compat module is deprecated and will be removed in a future release.
==> WARNING: A newer version of conda exists. <==
current version: 4.6.11
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda
Removing intermediate container 277edb28a107
---> e6b51d71eac0
Step 7/8 : RUN conda install gxx_linux-64
---> Running in 94166fbfff2a
Traceback (most recent call last):
File "/usr/local/bin/conda", line 12, in <module>
from conda.cli import main
ModuleNotFoundError: No module named 'conda'
The command '/bin/sh -c conda install gxx_linux-64' returned a non-zero code: 1
Can you please suggest?
Adding conda update -n base -c defaults conda in your Dockerfile solves the mentioned problem.
You could also consider using && for optimizing the creation of docker images. Read more about it here.
An optimized Dockerfile would be:
FROM conda/miniconda3-centos7
WORKDIR /arnav
COPY app/ /arnav
RUN conda update -n base -c defaults conda \
&& conda install gcc_linux-64 && conda install gxx_linux-64
CMD ["python", "Hello_World.py"]
I need to install some libraries in an anaconda docker environment. Below are my dockerfile and environment.yml.
I can build my docker without error but when I run my code it seems opencv and imutils not correctly installed. anyone can tell me what should I do???
dockerfile
FROM continuumio/anaconda3
WORKDIR /app
#create environment
COPY environment.yml
RUN conda env create -f environment.yml
#make run commands use the new env
SHELL ["conda", "run", "-n", "myenv", "/bin/bash", "-c"]
#make sure the env is activate
RUN echo "make sure anaconda activate"
RUN python -c "import dlib"
COPY app.py .
ENTRYPOINT ["conda", "run", "-n", "myenv", "python", "app.py"]
environment.yml
name: myenv
channels:
- conda-forge
- defaults
dependencies:
- python=3.8.5
- dlib
- pymongo
- requests
- face_recognition
- opencv
- imutils
this my solution in case somebody has the same problem.
I have 2 issues:
1- I should install imutils with pip in anaconda
2- opencv did not work correctly because I need to install libglu too
so I just change my environment.yml file like below:
name: myenv
channels:
conda-forge
defaults
anaconda
dependencies:
python=3.8.5
libglu
dlib
pymongo
requests
face_recognition
opencv
pip
pip:
imutils
I am a fairly inexperienced programmer and am struggling with installing Conda into Deepnote. Pip install doesn't work for certain packages. For example, I'm trying to install rdkit, a cheminformatics package, which has rather complex installation instructions or a simple 1 line of code managed through the Anaconda/mini-conda distribution. I really like the Deepnote notebooks and I would really appreciate any help here, please.
So far, I have found this useful code for Conda installation on Google Colab: https://github.com/dataprofessor/code/blob/master/python/google_colab_install_conda.ipynb
! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh
! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
Whilst this successfully works on Google Colab, I'm not sure why it fails as shown below on Deepnote:
--2020-12-04 22:58:34-- https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.130.3, 104.16.131.3, 2606:4700::6810:8203, ...
Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.130.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85055499 (81M) [application/x-sh]
Saving to: ‘Miniconda3-py37_4.8.2-Linux-x86_64.sh’
Miniconda3-py37_4.8 100%[===================>] 81.12M 133MB/s in 0.6s
2020-12-04 22:58:35 (133 MB/s) - ‘Miniconda3-py37_4.8.2-Linux-x86_64.sh’ saved [85055499/85055499]
PREFIX=/usr/local
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 392: /usr/local/conda.exe: Permission denied
chmod: cannot access '/usr/local/conda.exe': No such file or directory
Unpacking payload ...
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 404: /usr/local/conda.exe: No such file or directory
./Miniconda3-py37_4.8.2-Linux-x86_64.sh: line 406: /usr/local/conda.exe: No such file or directory
I also want to do conda install -c bioconda gromacs, which I cannot find a work around for, so I am hoping someone can help me resolve this query.
Many thanks in advance!
P.S. I am on a Mac OS
You can check this notebook from Daniel Zvara
Using Conda in Deepnote in 3 simple steps
In sum
# 1. Install Conda and make Conda packages available in current environment
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
!chmod +x Miniconda3-latest-Linux-x86_64.sh
!sudo bash ./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
# 2. Package installation
# !sudo conda install -y [EXTRA_OPTIONS] package_name
# So for example Keras from conda-forge channel
!sudo conda install -y -c conda-forge keras
# 3. Package usage
import keras
Also found an alternative solution on the Deepnote community: https://community.deepnote.com/c/custom-environments/custom-environment-for-installing-conda-packages
Can set up a custom environment with the following Dockerfile (just fill in the placeholder for the packages you require).
FROM gcr.io/deepnote-200602/templates/deepnote
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
RUN bash ~/miniconda.sh -b -p $HOME/miniconda
ENV PATH $HOME/miniconda/bin:$PATH
RUN conda install python=3.7 ipykernel -y
RUN conda install <insert packages here> -y
RUN python -m ipykernel install --user --name=conda
ENV DEFAULT_KERNEL_NAME "conda"
And the custom environment in Deepnote is set up through the Environment tab in the sidebar, giving you an option for the Dockerfile. Copy & paste the above, then click build and restart machine - the environment will be set up allowing you to use conda.
You can now choose Anaconda in the built-in environments in Deepnote.