Develop Reference

Filesize of dockerimage doubles with different python version as baseimage

I build dockerimages for a jupyterlab with some apt and pip packages.
docker image ls:
jupyterlab 3.7.12 9d4cc5c15853 27 minutes ago 2.3GB
jupyterlab 3.9.9 1962a1eb6bff 25 hours ago 1.12GB
The filesize of the images differs quite much, I only took a another python version.
Here are the dockerfiles for comparison:
This dockerfile is build with python 3.7.12 as baseimage:
FROM python:3.7.12-slim-buster
WORKDIR /
COPY wheels/scipy-1.7.3-cp37-cp37m-linux_armv7l.whl ./wheels/scipy-1.7.3-cp37-cp37m-linux_armv7l.whl
## INSTALL WITH APK
RUN apt-get update && apt-get install -y \
g++ \
gcc \
python3-dev \
libjpeg-dev \
zlib1g-dev \
make \
wget \
libatlas-base-dev \
libffi-dev
## INSTALL WITH PIP
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir pillow && \
pip install --no-cache-dir matplotlib && \
pip install --no-cache-dir pandas && \
pip install --no-cache-dir setuptools && \
pip install --no-cache-dir cffi && \
pip install --no-cache-dir GLIBC && \
pip install --no-cache-dir numpy && \
pip install --no-cache-dir /wheels/scipy-1.7.3-cp37-cp37m-linux_armv7l.whl && \
pip install --no-cache-dir jupyterlab
# pip install --no-cache-dir /wheels/numpy-1.21.4-cp39-cp39-linux_armv7l.whl && \
# pip install --no-cache-dir /wheels/scipy-1.7.2-cp39-cp39-linux_armv7l.whl && \
# pip install --no-cache-dir /wheels/jupyterlab-4.0.0a15-py3-none-any.whl
...
This is the one with python 3.9.9 as baseimage:
FROM python:3.9.9-slim-buster
WORKDIR /
COPY wheels ./wheels
## INSTALL WITH APK
RUN apt-get update && apt-get install -y \
g++ \
gcc \
python3-dev \
libjpeg-dev \
zlib1g-dev \
make \
wget \
libatlas-base-dev \
libffi-dev
## INSTALL WITH PIP
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir pillow && \
pip install --no-cache-dir matplotlib && \
pip install --no-cache-dir pandas && \
pip install --no-cache-dir setuptools && \
pip install --no-cache-dir cffi && \
pip install --no-cache-dir GLIBC && \
pip install --no-cache-dir /wheels/numpy-1.21.4-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/scipy-1.7.2-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/jupyterlab-4.0.0a15-py3-none-any.whl
...
The wheels folder and some other files are deleted in both cases. Why is there such a big difference in filesize???

I solved the issue by removing "apt-get python3-dev", it was version 3.7.2. My baseimage is python 3.7.12 so maybe there was a conflict, I can not explain it.
The size of the image:
#docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
jupyterlab 3.7.12_nodev 6112bcaff54d 17 hours ago 1.18GB
jupyterlab 3.9.9 1962a1eb6bff 2 days ago 1.12GB

Unable to locate package python3-pip in Dockerfile while having Universal Repos enabled

I am trying to install python3 and pip.
So, I have a dockerfile
(I also tried with ubuntu:20.04)-
(Note- I have made the Dockerfile short, for full dockerfile you can see here)
FROM ubuntu:latest
WORKDIR /app
COPY . /app
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN add-apt-repository universe
ENV PYTHONUNBUFFERED=1
RUN apt install -y python3
RUN apt-get -y install python3-pip
RUN pip install --no-cache --upgrade pip setuptools
But it always give me error-
---> Running in ba9f299b852b
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package python3-pip
The command '/bin/sh -c apt-get -y install python3-pip' returned a non-zero code: 100
While finding more out in the error, I found out that while the universal repositories were added, python3-pip didnt got installed, while other things like python3-sip got installed.
How do I fix this error and why did it happened?

The software-properties-common meta-packages installs a bunch of softwares that are not required in a docker image and make its the size quite huge.
This simpler Dockerfile does the job for me.
FROM ubuntu:latest
WORKDIR /app
#COPY . /app
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends --quiet \
python3 \
python3-pip \
&& apt-get clean all
RUN pip install --no-cache --upgrade pip setuptools
RUN pip --version # just for test
Output:
STEP 1/5: FROM ubuntu:latest
STEP 2/5: WORKDIR /app
--> Using cache 9f1d34a70ba9e13074601d16456fda97b2de87a0be3b18ddc6963fb65aaee096
--> 9f1d34a70ba
STEP 3/5: RUN apt-get update && apt-get install --assume-yes --no-install-recommends --quiet python3 python3-pip && apt-get clean all
[...]
STEP 4/5: RUN pip install --no-cache --upgrade pip setuptools
Collecting pip
Downloading pip-21.3-py3-none-any.whl (1.7 MB)
Collecting setuptools
Downloading setuptools-58.2.0-py3-none-any.whl (946 kB)
Installing collected packages: pip, setuptools
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'pip'. No files were found to uninstall.
Attempting uninstall: setuptools
Found existing installation: setuptools 45.2.0
Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'setuptools'. No files were found to uninstall.
Successfully installed pip-21.3 setuptools-58.2.0
STEP 5/5: RUN pip --version
pip 21.3 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)
COMMIT staskoverflow-ubuntu
--> 2498ddc1767
Successfully tagged localhost/staskoverflow-ubuntu:latest
2498ddc17672584d2dd38bf23ae63521d9adb43465cf8c7b415c33dc2aa0aa01
The tool pip is updated to the version 21.3 and the OS-provided one is not uninstalled (this is a pip feature).
Anyway this dockefile, more similar to yours, works too:
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends --quiet \
software-properties-common \
&& add-apt-repository universe \
&& apt-get update \
&& apt-get install --assume-yes --no-install-recommends --quiet \
python3 \
python3-pip \
&& apt-get clean all
RUN pip install --no-cache --upgrade pip setuptools
After adding the universe repository, you should run again apt-get update to retrieve the new lists of packages with the new apt configuration. This is the root cause of the problem I think.

Docker installation failed on windows

I am building an image using the following Dockerfile:
FROM ubuntu:18.04
RUN apt-get update \
&& apt-get install -y python3-pip python3-dev \
&& cd /usr/local/bin \
&& ln -s /usr/bin/python3 python \
&& pip3 install --upgrade pip
# Setup the Python's configs
RUN pip install --upgrade pip && \
pip install --no-cache-dir matplotlib==3.0.2 pandas==0.23.4 numpy==1.16.3 && \
pip install --no-cache-dir pybase64 && \
pip install --no-cache-dir scipy && \
pip install --no-cache-dir dask[complete] && \
pip install --no-cache-dir dash==1.6.1 dash-core-components==1.5.1 dash-bootstrap-components==0.7.1 dash-html-components==1.0.2 dash-table==4.5.1 dash-daq==0.2.2 && \
pip install --no-cache-dir plotly && \
pip install --no-cache-dir adjustText && \
pip install --no-cache-dir networkx && \
pip install --no-cache-dir scikit-learn && \
pip install --no-cache-dir tzlocal
# Setup the R configs
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
RUN add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'
RUN apt update
ENV DEBIAN_FRONTEND=noninteractive
RUN apt install -y r-base
RUN pip install rpy2==2.9.4
RUN apt-get -y install libxml2 libxml2-dev libcurl4-gnutls-dev libssl-dev
RUN echo "r <- getOption('repos'); r['CRAN'] <- 'https://cran.r-project.org'; options(repos = r);" > ~/.Rprofile
RUN Rscript -e "install.packages('BiocManager')"
RUN Rscript -e "BiocManager::install('ggplot2')"
RUN Rscript -e "BiocManager::install('DESeq2')"
RUN Rscript -e "BiocManager::install('RColorBrewer')"
RUN Rscript -e "BiocManager::install('ggrepel')"
RUN Rscript -e "BiocManager::install('factoextra')"
RUN Rscript -e "BiocManager::install('FactoMineR')"
RUN Rscript -e "BiocManager::install('apeglm')"
When I build this on Linux I launched the web-app from the container and it's run fine.
But, when I build this on windows using Docker Toolbox although the installation of factoextra and FactoMineR work successfully, when I launch the web-app it's raise an error:
Error in library("factoextra") : there is no package called ‘factoextra’
Do you have any idea what's might cause this problem? It's very strange because when I build the image the installation of these 2 packages seems to work successfully.

Run install python packages on Dockerfile

I'm new to Docker and currently trying to create a Dockerfile with installing the python packages and its libraries as shown here:
FROM balenalib/fincm3-debian-python:latest
# RUN install_packages git
RUN apt-get update && apt-get install python \
&& apt-get install pip3 \
apt-get install libportaudio0 libportaudio2 libportaudiocpp0 portaudio19-dev \
pip3 install pyaudio \
pip3 install numpy \
pip3 install matplotlib \
pip3 install scipy \
pip3 install librosa \
# Set our working directory
WORKDIR /usr/src/app
COPY Recorder.py /usr/src/app
# Recorder.py will run when container starts up on the device
CMD ["python","/usr/src/app/Recorder.py"]
However, while I am trying to push this Dockerfile, the error is generated with
Error: The command '/bin/sh -c apt-get update && apt-get install python && apt-get install pip3 apt-get install libportaudio0 libportaudio2 libportaudiocpp0 portaudio19-dev pip3 install pyaudio
pip3 install numpy pip3 install matplotlib pip3 install scipy pip3 install librosa WORKDIR /usr/src/app' returned a non-zero code: 100

Moving python packages in requirements.txt and installing python3-pip worked with python:3 base image.
# RUN install_packages git
RUN apt-get update \
&& apt-get install -y python \
&& apt-get install -y python3-pip
RUN pip install -r requirements.txt

The package you are looking for is called python3-pip.
Next, you need both && (to separate commands) and \ (to continue the command line). So, in summary, that should be:
FROM balenalib/fincm3-debian-python:latest
RUN apt-get update && apt-get install python && \
apt-get install -y \
python3-pip libportaudio0 libportaudio2 libportaudiocpp0 \
portaudio19-dev && \
pip3 install pyaudio numpy matplotlib \
scipy librosa
# Set our working directory
WORKDIR /usr/src/app
COPY Recorder.py /usr/src/app
# Recorder.py will run when container starts up on the device
CMD ["python","/usr/src/app/Recorder.py"]

I believe you have more than one problem in this Dockerfile, and when you put all commands together with && and \, you don't know which one is triggering the error. I suggest splitting them for debugging purposes, when they all work then you can put then together. Once you understand each individual error is easier to check and solve them. this question has valuable info: how to install pip in docker
Try this:
1- packages are triggers Y/n questions, give -y to guarantee it passes
2- using the backslashes to refer to a new command, you should use &&, backslashes refer to breaking line, you can use \ and then &&
3- pip3 and libportaudio0 packages doesn't exist.
E: Unable to locate package libportaudio0
I found out about the errors dividing the Dockerfile like this and removing the problems mentioned:
RUN apt-get update
RUN apt-get install python -y\
&& apt-get install python3-pip -y
RUN apt-get install libportaudio2 libportaudiocpp0 portaudio19-dev -y
RUN pip3 install pyaudio numpy matplotlib \
scipy librosa
If you want to put the commands together:
RUN apt-get update \
&& apt-get install python -y \
&& apt-get install python3-pip -y \
&& apt-get install libportaudio2 libportaudiocpp0 portaudio19-dev -y \
&& pip3 install pyaudio numpy matplotlib \
scipy librosa
I also suggest adding a pip requirements file, would make things cleaner.

cant install pip in ubuntu 18.04 docker /bin/sh: 1: pip: not found

I am getting the error using pip in my docker image.
FROM ubuntu:18.04
RUN apt-get update && apt-get install -y \
software-properties-common
RUN add-apt-repository universe
RUN apt-get install -y \
python3.6 \
python3-pip
ENV PYTHONUNBUFFERED 1
RUN mkdir /api
WORKDIR /api
COPY . /api/
RUN pip install pipenv
RUN ls
RUN pipenv sync
I installed python 3.6 and pip3 but getting
Step 9/11 : RUN pip install pipenv
---> Running in b184de4eb28e
/bin/sh: 1: pip: not found

To run pip for python3 use pip3, not pip.

Another solution.
You can add this line (after apt-get install). It will upgrade pip to the version you need, for instance:
RUN pip3 install --upgrade pip==20.0.1
and you can then use pip install from requirements file (for instance):
RUN pip install -r requirements.txt