Hello im having difficult writing the Dockerfile to run a python spider script which is inside my project directory
The script file is in scrapy_estate/tutorial/tutorial/spiders/ .I think of using the COPY command ,but CMD ["python3","real_estate_spider.py"] still can't find the real_estate_spider.py file
Here is my Dockerfile
FROM ubuntu:18.04
FROM python:3.6-onbuild
RUN apt-get update &&apt-get upgrade -y&& apt-get install python-pip -y
RUN pip install --upgrade pip
RUN pip install scrapy
WORKDIR /usr/local/bin
COPY scrapy_estate/tutorial/tutorial/spiders/real_estate_spider.py
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 80
CMD ["python3","real_estate_spider.py"]
Hope someone can help me :)
It looks like you are forgetting the './' in the copy line for your script file
Related
I want to create a docker image (docker version: 20.10.20)that contains python libraires from a requirement.txt file that contains 50 libraries. Without facing root user permissions how can proceed. Here is the file:
From ubuntu:latest
RUN apt update
RUN apt install python3 -y
WORKDIR /Destop/DS
# COPY requirement.txt ./
# RUN pip install -r requirement.txt
# it contains only pandas==1.5.1
COPY script2.py ./
CMD ["python3", "./script2.py"]
It failed at requiremnt.txt command
*error it takes lot of time while creating image.
because it ask for root permission.
For me the only problem in your Dockerfile is in the line RUN apt install python -y. This is erroring with Package 'python' has no installation candidate.
It is expected since python refers to version 2.x of Python wich is deprecated and no longer present in the default Ubuntu repositories.
Changing your Dockerfile to use Python version 3.x worked fine for me.
FROM ubuntu:latest
RUN apt update
RUN apt install python3 python3-pip -y
WORKDIR /Destop/DS
COPY requirement.txt ./
RUN pip3 install -r requirement.txt
COPY script2.py ./
CMD ["python3", "./script2.py"]
To test I used requirement.txt
pandas==1.5.1
and script2.py
import pandas as pd
print(pd.__version__)
With this building the docker image and running a container from it executed succesfully.
docker build -t myimage .
docker run --rm myimage
This question already has answers here:
WARNING: Running pip as the 'root' user
(4 answers)
Closed 10 months ago.
I have this Dockerfile:
FROM python:3.8-slim
WORKDIR /app
COPY . .
RUN apt-get update
RUN apt-get install -y python3 python3-pip python3-venv
RUN pip freeze > requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
CMD ["python3", "main.py"]
Everything works file until this line:
RUN pip install --no-cache-dir -r requirements.txt
Using docker run --rm -it name bash and pip install -r requirements.txt then I found this error:
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting
behaviour with the system package manager. It is recommended to use a virtual environment
instead: https://pip.pypa.io/warnings/venv
Here, I found solution (which didn't work for me), that it's possible to resolve just by creating new user, but it doesn't seem to be optimal solution. How can I fix this?
In this case the problem was in version of images. Using this Dockerfile I was able to fix this:
FROM python:3.9.3
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python3", "main.py"]
PS. I don't really know if it was about it, but this images has the same python version, that I have on my computer. I could have impact on dependencies.
I am trying to install a package that is not on PyPi. i.e from github. Adding the repo as git+url to the requirements file gives
ERROR: Error [Errno 2] No such file or directory: 'git' while executing command git clone -q https://github.com/Rapptz/discord-ext-menus /tmp/pip-req-build-147rct22
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?
Installing the packages is done with
RUN python3 -m pip install -r requirements.txt
as specified in the docs
I also tried the solutions from this, but the answers mess up my other packages.
The dockerfile is almost directly from the docs
FROM python:3.8-slim-buster
WORKDIR /app
COPY requirements.txt requirements.txt
RUN python3 -m pip install -r requirements.txt
COPY . .
CMD [ "python3", "main.py"]
requirements.txt
asyncpg==0.21.0
git+https://github.com/Rapptz/discord-ext-menus
discord.py==1.7.0
pre-commit==2.10.1
pyclean==2.0.0
pylint==2.6.0
python-dotenv==0.15.0
As the error tells us, we have to simply install git, so that pip can clone the repo and run the setup file.
We can install git with
RUN apt-get update && apt-get install -y git
We also have to build from a python image, the above answer works with python:3.8-slim-buster
I am having lots of problems creating a Docker container using python:3.6-alpine for Plotly. Plotly also uses Pandas and Numpy. When I run my Dockerfile below, the "RUN venv/bin/pip install -r requirements.txt" fails. Anyone have recommendations for this, am I missing requirements?
FROM python:3.6-alpine
RUN adduser -D visualdata
RUN pip install --upgrade pip
WORKDIR /home/visualdata
COPY requirements.txt requirements.txt
RUN python -m venv venv
RUN venv/bin/pip install -r requirements.txt
RUN venv/bin/pip install gunicorn
#RUN venv/bin/pip install install python3-pymysql
COPY app app
COPY migrations migrations
COPY visualdata.py config.py boot.sh ./
RUN chmod a+x boot.sh
ENV FLASK_APP visualdata.py
RUN chown -R visualdata:visualdata ./
USER visualdata
EXPOSE 8000
ENTRYPOINT ["./boot.sh"]
If you look at the Python docker image official repository, there is a Dockerfile example that illustrates the pip step:
RUN pip install --no-cache-dir -r requirements.txt
You should be able to use pip directly instead of venv/bin/pip.
You do not really need to use a virtualenv in a docker container if you are only running one application inside. The container already provides its own isolated environment.
We are trying to create a Docker container for a python application. The Dockerfile installs dependencies using "pip install". The Dockerfile looks like
FROM ubuntu:latest
RUN apt-get update -y
RUN apt-get install -y git wget python3-pip
RUN mkdir /app
COPY . /app
RUN pip3 install asn1crypto
RUN pip3 install cffi==1.10.0
RUN pip3 install click==6.7
RUN pip3 install conda==4.3.16
RUN pip3 install Flask==0.12.2
RUN pip3 install Flask-SSLify==0.1.5
RUN pip3 install Flask-SSLify==0.1.5
RUN pip3 install flask-restful==0.3.6
WORKDIR /app
ENTRYPOINT ["python3"]
CMD [ "X.py", "/app/Y.yml" ]
The docker gets created successfully the issue is on the rebuild time.
If nothing is changed in the dockerfile above
If a line is changed in the dockerfile which is after pip install the docker daemon still runs all the commands in pip install, downloading all the packages though not installing them.
Is there a way to optimize the rebuild?
Thx
Below is what i would like to do momentarily with the Dockerfile for optimization -
FROM ubuntu:latest
RUN apt-get update -y && apt-get install -y \
git \
wget \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY ./requirements.txt .
RUN pip3 install -r requirements.txt
COPY . /app
ENTRYPOINT ["python3"]
CMD [ "X.py", "/app/Y.yml" ]
Reduce the layers by integrating multiple commands into a single one specifically when they are interdependent. This helps reducing the image size.
Always try to use the COPY at the end since a regular source code change may invalidate the next layer caching.
Use a single requirements.txt file for installation through pip. Also define separate steps in case you have lots of packages to install, don't let a normal source code change force packages installation on every build.
Always cleanup the intermediate things which are not required in the final image.
Ref- https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/