Develop Reference

Pip error with requirements while deployng Django to Digital Ocean [duplicate]

I am baffled by a strange issue that I am facing with docker-compose. Pip install is failing for certain packages within requirements.txt file.
docker version
Client:
Version: 18.09.9
API version: 1.39
Go version: go1.13.4
Git commit: 1752eb3
Built: Sat Nov 16 01:05:26 2019
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.09.9
API version: 1.39 (minimum version 1.12)
Go version: go1.13.4
Git commit: 9552f2b
Built: Sat Nov 16 01:07:48 2019
OS/Arch: linux/amd64
Experimental: false
My docker-compose.yml file is:
version: "3.7"
services:
flask:
build: ./flask
container_name: flask
restart: always
environment:
- APP_NAME=MyFlaskApp
expose:
- 8080
nginx:
build: ./nginx
container_name: nginx
restart: always
ports:
- "80:80"
The contents of Dockerfile inside ./flask directory is:
# Use the Python3.7.5 image
FROM python:3.7.5
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
ADD . /app
# Install the dependencies
RUN pip3 install -r requirements.txt
# run the command to start uWSGI
CMD ["uwsgi", "app.ini"]
My requirements.txt file is (first few lines):
appdirs==1.4.3
apturl==0.5.2
asn1crypto==0.24.0
bcrypt==3.1.6
blinker==1.4
However, when I run docker-compose up command, it fails to install the second package in requirements.txt file.
Building flask
Step 1/6 : FROM python:3.7.5
---> fbf9f709ca9f
Step 2/6 : WORKDIR /app
---> Using cache
---> 39ab3ee34991
Step 3/6 : ADD . /app
---> Using cache
---> 8968809ff844
Step 4/6 : RUN python3 -m pip install --upgrade pip
---> Using cache
---> 15f717de5181
Step 5/6 : RUN pip3 install -r requirements.txt
---> Running in 7068f09498dc
Collecting appdirs==1.4.3
Downloading appdirs-1.4.3-py2.py3-none-any.whl (12 kB)
ERROR: Could not find a version that satisfies the requirement apturl==0.5.2 (from -r requirements.txt (line 2)) (from versions: none)
ERROR: No matching distribution found for apturl==0.5.2 (from -r requirements.txt (line 2))
ERROR: Service 'flask' failed to build: The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1
I have tried a lot of possible options, but of no use. Some examples being:
Running pip upgrade inside the container.
Updating the DNS in docker as per the thread here.

as #WilliamD.Irons already pointed here apturl is a package, you can use :
RUN apt install apturl
RUN pip3 install -r requirements.txt
and remove apturl from your requirements.txt

apturl seems to be a client side Ubuntu program for adding support for links like click within a webpage:
The apturl is a graphical mini-program for installing packages from the repository that a user has. It is pre-installed on Ubuntu since version 7.10, and the Firefox and Pidgin programs come with support for it.
Additionally, there are other suggestions that support for this doesn't exisit outside Ubuntu. As the official python:3.7.5 image is based on Debian GNU/Linux this wouldn't be available anyway.
I'd question why this needs to be in the requirements for your Flask app, as you should just be able to write your own python function to generate compatible links within the app. Anyone hitting these links from a supported client (Any Ubuntu box with Firefox according to the above) should be able to successfully process those links.

If using a base container isn't an issue, try the Ubuntu environment and install python.
apturl is provided by ubuntu, specifically as it uses apt
# Use the ubuntu
FROM ubuntu:lts
# Set the working directory to /app
WORKDIR /app
# install python and pip, and delete cache
RUN apt update && apt install -y python3 python3-pip && rm -rf /var/lib/apt/lists/*
# Copy the current directory contents into the container at /app
ADD . /app
# Install the dependencies
RUN pip3 install -r requirements.txt
# run the command to start uWSGI
CMD ["uwsgi", "app.ini"]

python pip- These packages do not match the hashes from the requirements file installing rasa with docker

I am building this image from this docker file
FROM python:3.8.0
WORKDIR /app
# Install system libraries
RUN apt-get update && \
apt-get install -y git && \
apt-get install -y gcc
# Install project dependencies
COPY ./requirements.txt .
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt --use-deprecated=legacy-resolver
# Don't use terminal buffering, print all to stdout / err right away
ENV PYTHONUNBUFFERED 1
and this is my requirements.txt file
aiohttp==3.6.2
rasa==2.4.1
rasa-sdk==2.4.1
# Rasa requirements
python-dateutil~=2.8
SQLAlchemy==1.3.23
gast==0.3.3
the command to build the image
docker build -f base_Dockerfile -t rasa_base:latest
but this is showing this error
#9 529.6 ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
#9 529.6 tensorflow<2.4,>=2.3 from https://files.pythonhosted.org/packages/10/f9/283e82c10958fc24eb120fea6c40f1d8a66507a85818de420138bbbf88bf/tensorflow-2.3.2-cp38-cp38-manylinux2010_x86_64.whl#sha256=c85d29c7b5c000b196200a25e18a5905cc04f75c989e508f8fd110f3b2c4b4fe (from rasa==2.4.1->-r requirements.txt (line 2)):
#9 529.6 Expected sha256 c85d29c7b5c000b196200a25e18a5905cc04f75c989e508f8fd110f3b2c4b4fe
#9 529.6 Got c29a17cebd72dd06fecffd646fa0990c9b7f7b5781d8746b70d821109125f9c9
#9 529.6
------
executor failed running [/bin/sh -c pip install --no-cache-dir -r requirements.txt --use-deprecated=legacy-resolver]: exit code: 1
I have deleted all the images, and even I am using --no-cache-dir while installing the deps.

The issue was with cache and unused images. This command helped me
docker system prune -a
Remove all dangling images. If -a is specified, will also remove all images not referenced by any container

How to install pip packages in container [duplicate]

This question already has answers here:
Activate python virtualenv in Dockerfile
(6 answers)
Closed 2 years ago.
I am trying to setup a python virtual environment on a docker image running a docker build
The terminal output is ok when I run docker build .. but when I login into my container, no packages are installer in my virtual environment.
#Dockerfile
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python3.8 get-pip.py
RUN pip install virtualenv
RUN virtualenv venv
RUN /home/ubuntu/venv/bin/pip install -r auto/requirements.txt
...
# Docker build command
docker build --no-cache -t auto-server:1.0 .
# Terminal output
Step 27/27 : RUN /home/ubuntu/venv/bin/pip install -r auto/requirements.txt
---> Running in d27dbb9a4c97
Collecting asgiref==3.2.10
Downloading asgiref-3.2.10-py3-none-any.whl (19 kB)
Collecting beautifulsoup4==4.9.1
Downloading beautifulsoup4-4.9.1-py3-none-any.whl (115 kB)
Collecting Django==3.1.1
Downloading Django-3.1.1-py3-none-any.whl (7.8 MB)
...
Successfully installed Django-3.1.1 asgiref-3.2.10 beautifulsoup4-4.9.1 fake-useragent-0.1.11 joblib-0.16.0 numpy-1.19.2 pandas-1.1.2 python-dateutil-2.8.1 pytz-2020.1 scikit-learn-0.23.2 scipy-1.5.2 six-1.15.0 sklearn-0.0 soupsieve-2.0.1 sqlparse-0.3.1 threadpoolctl-2.1.0
Here is what I get when I list pakages in my virtual environment:
$ docker exec -ti auto-server bash
root#9c1f914d1b7b:/home/ubuntu# source venv/bin/activate
(venv) root#9c1f914d1b7b:/home/ubuntu# pip list
Package Version
---------- -------
pip 20.2.2
setuptools 49.6.0
wheel 0.35.1
WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
You should consider upgrading via the '/home/ubuntu/venv/bin/python -m pip install --upgrade pip' command.
Is it the right way to do it? How to make sure packages will be installed?

Finally having a virtual environment is not mandatory in a container development environment, I just setup image python environment as desired.
# Dockerfile
...
RUN python3.8 get-pip.py
RUN pip install -r auto/requirements.txt
This is not exactly what I was looking for but it does the job.

I use simple Dockerfile for dev mode
# pull official base image
FROM python:3.6-alpine
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update \
&& apk add postgresql-dev gcc python3-dev
# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# copy project
COPY . .

docker image build fails at RUN pip install -r requirements.txt in a Flask app

I have a Flask application with the following requirements.txt
chalice
matplotlib
sklearn
numpy
scipy
pandas
flask
flask_restful
and the following Dockerfile:
FROM python:3.6.1-alpine
WORKDIR /project
ADD . /project
RUN pip install -r requirements.txt
CMD ["python","app.py"]
Running the command docker image build -t clf_test .
generates the following error:
You are using pip version 9.0.1, however version 20.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
It seems that the matplotlib can't get installed for some reason.
Running the pip install -r requirements.txt locally doesn't produce any errors

matplotlib must be built from source, and compiling it requires a number of supporting libraries as well as a functioning C compiler. You can figure out what these are and install them so that it builds properly...
...or you can just base your Dockerfile on the non-alpine python:3.6.1 image and then apt-get install python3-matplotlib before installing your other requirements. E.g., this builds without errors:
FROM python:3.6.1
WORKDIR /project
ADD . /project
RUN apt update; apt-get -y install python3-matplotlib
RUN pip install -r requirements.txt
CMD ["python","app.py"]

Not able to install any python package in docker container

I am trying to create an docker image with ubutu 16.04 as base. I want to install few python packages like pandas, flask etc. I have kept all packages in "requirements.txt". But when I am trying to build image, I am getting
Could not find a version that satisfies the requirement requests (from -r requirements.txt (line 1)) (from versions: )
No matching distribution found for requests (from -r requirements.txt (line 1))
Basically, I have not mentioned any version in "requirements.txt". I guess it should take the latest available and compatible version of that package. But for every package same issue I am getting.
My DockerFile is as follows.
FROM ubuntu:16.04
RUN apt-get update -y && \
apt-get install -y python3-pip python3-dev build-essential cmake pkg-config libx11-dev libatlas-base-dev
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /testing/requirements.txt
WORKDIR /testing
RUN pip3 install -r requirements.txt
and requirements.txt is.
pandas
requests
PyMySQL
Flask
Flask-Cors
Pillow
face-recognition
Flask-SocketIO
Where I am doing wrong ? Can anybody help ?

I too ran into the same situation. I observed that, python packages is looking for the network within docker. It is thinking that, it is running in a standalone without network so its not able to locate the package. In these type of situations either
No matching distribution found
or sometimes
Retrying ...
error may occur.
I used a --network option in the docker build command like below to overcome this error where the command insists python to use the host network to download the required packages.
docker build --network=host -t tracker:latest .

Try using this:
RUN python3.6 -m pip install --upgrade pip \
&& python3.6 -m pip install -r requirements.txt
by using it in this way, you are specifying the version of python in which you want to search for those packages.
Change it to python3.7 if you wish to use 3.7 version.

I suggest using the official python image instead. As a result, your Dockerfile will now become:
FROM python:3
WORKDIR /testing
COPY ./requirements.txt /testing/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
... etc ...
Now re: Angular/Node. You have two options from here: 1) Install Angular/Node on the Python image; or 2) Use Docker's multi-stage build feature so you build the Angular and Python-specific images before merging them together. Option 2 is recommended but it would take some work. It would probably look like this:
FROM node:8 as node
# Angular-specific build
FROM python:3 as python
# Python-specific build
# Then copy your data from the Angular image to the Python one:
COPY --from=node /usr/src/app/dist/angular-docker /usr/src/app