I have a python script where various ping check has been performed . The same py file runs perfectly when triggered locally , but when i run the job from jenkins after building docker image the below error comes up and ping check always return false.
sh: 1: ping: not found
Can someone help me why this error is coming, do I need to update dockerfile with some update or some jenkins configuration is required.
Updating docker file
FROM ubuntu:18.04
ADD requirements.txt /requirements.txt
ADD jenkins_bot.py /jenkins_bot.py
ADD src /src
ADD config /config
ADD run_unit_tests.sh /run_unit_tests.sh
ADD utests /utests
WORKDIR /
RUN DEBIAN_FRONTEND=noninteractive && apt-get -qq update && \
apt-get -y install apt-utils && \
apt-get -qqy install ssh && \
apt-get -qqy install build-essential \
python3-dev \
python3-setuptools \
libfreetype6-dev \
libxft-dev && \
apt-get -qqy install python3-pip && \
pip3 install -r /requirements.txt
#RUN sh run_unit_tests.sh
Thanks in advance.
Ping is missing from in the ubuntu base image. You can update your dockerfile as follows to install it:
FROM ubuntu:18.04
ADD requirements.txt /requirements.txt
ADD jenkins_bot.py /jenkins_bot.py
ADD src /src
ADD config /config
ADD run_unit_tests.sh /run_unit_tests.sh
ADD utests /utests
WORKDIR /
RUN DEBIAN_FRONTEND=noninteractive && apt-get -qq update && \
apt-get -y install apt-utils && \
apt-get install -y iputils-ping && \
apt-get -qqy install ssh && \
apt-get -qqy install build-essential \
python3-dev \
python3-setuptools \
libfreetype6-dev \
libxft-dev && \
apt-get -qqy install python3-pip && \
pip3 install -r /requirements.txt
#RUN sh run_unit_tests.sh
Related
I have something like the following in my Dockerfile:
ADD api/requirements.txt /app/
# Install apt and pip packages
RUN set -ex \
&& BUILD_DEPS=" \
build-essential \
gcc \
python3-dev \
libsasl2-dev libldap2-dev libssl-dev \
libpq-dev libffi-dev \
python3 python3-pip python3-wheel python3-setuptools \
zlib1g-dev libjpeg-dev libpng-dev \
" \
&& RUN_DEPS=" \
postgresql-client \
nginx \
libsasl2-dev libldap2-dev libssl-dev python3-dev \
python3 python3-pip python3-setuptools python3-wheel \
wireguard \
gunicorn3 \
curl \
zlib1g-dev libjpeg-dev libpng-dev \
" \
&& apt-get update && apt-get install -y $BUILD_DEPS \
&& pip3 install --no-cache-dir --default-timeout=100000 -r /app/requirements.txt \
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $BUILD_DEPS \
&& apt-get update && apt-get install -y --no-install-recommends $RUN_DEPS \
&& rm -rf /var/lib/apt/lists/*
One pypackage I have in my requirements.txt is six. However, the debian package python3-six is a dependency of one of those packages in BUILD_DEPS. This means that when it runs pip install -r requirements.txt, it skips installing six as it's already supplied by the distro package.
Is there a way to force that pypackage to install anyway?
I have a Python program which is to be executed in the Azure Kubernetes.
Below is my docker file - I have Python installed
#Ubuntu Base image with openjdk8 with TomEE
FROM demo.azurecr.io/ubuntu/tomee/openjdk8:8.0.x
RUN apt-get update && apt-get install -y telnet && apt-get install -y ksh && apt-get install -y python2.7.x && apt-get -y clean && rm -rf /var/lib/apt/lists/*
however I don't know how to install PIP and related dependent libraries (eg: pymssql)?
Best option is installing miniconda on docker image. I used it always when I need to have python on docker image without python or pip.
Here is part for installing minicinda in my simple docker image
FROM debian
RUN apt-get update && apt-get install -y curl wget
RUN rm -rf /var/lib/apt/lists/*
RUN wget \
https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
&& mkdir /root/.conda \
&& bash Miniconda3-latest-Linux-x86_64.sh -b \
&& rm -f Miniconda3-latest-Linux-x86_64.sh
RUN conda --version
I'm using Cloudera Hive ODBC driver in my code and I'm trying to containerize the app.
Below is my Dockerfile,
FROM ubuntu:18.04
FROM continuumio/anaconda3
FROM node:10
RUN conda update -n base -c defaults conda
RUN conda create -n env python=3.7
RUN echo "conda activate env" > ~/.bashrc
ENV PATH /opt/conda/envs/env/bin:$PATH
RUN apt-get update && apt-get install -y \
curl apt-utils apt-transport-https debconf-utils gcc build-essential \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y \
python-pip python-dev python-setuptools \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip
RUN pip install pyyaml pandas numpy pymysql sqlalchemy schedule tornado
RUN apt-get update && apt-get install -y --no-install-recommends git unzip unixodbc unixodbc-dev
RUN conda install -c conda-forge turbodbc=3.1.1
RUN apt-get update && apt-get install -y gettext nano vim -y
RUN yarn install --modules-folder ./static
WORKDIR /app
COPY entry.sh /usr/local/bin/
COPY . /app/
ENV SSH_PASSWD "root:Docker!"
RUN apt-get update \
&& apt-get install -y --no-install-recommends dialog \
&& apt-get update \
&& apt-get install -y --no-install-recommends openssh-server \
&& echo "$SSH_PASSWD" | chpasswd
COPY sshd_config /etc/ssh/
COPY entry.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/entry.sh
EXPOSE 5000 2222 22 80 8000
CMD ["entry.sh"]
Building Image is getting successful, but I see when I run the docker image, I see below error
Traceback (most recent call last):
File "app.py", line 14, in <module>
from abc_scheduler import scheduler_main
File "/app/abc_scheduler.py", line 5, in <module>
from methods import Methods
File "/app/methods.py", line 10, in <module>
from utils import *
File "/app/utils.py", line 2, in <module>
from turbodbc import connect, make_options
ModuleNotFoundError: No module named 'turbodbc'
I have tried many other ODBC's inside my Dockerfile, but no luck. Any help would be great.
As suggested by #DavidMaze, I managed create a successful Dockerfile & is shown below
FROM ubuntu:latest
FROM continuumio/anaconda3
FROM node:10
RUN conda update -n base -c defaults conda
RUN conda create -n env python=3.7
RUN echo 'conda init bash' >/.bashrc
RUN echo "conda activate env" > ~/.bashrc
ENV PATH /opt/conda/envs/env/bin:$PATH
RUN apt-get update && apt-get install -y \
curl apt-utils apt-transport-https debconf-utils gcc build-essential \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y \
python-pip python-dev python-setuptools \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip
# ==================TURBODBC========================
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get dist-upgrade -y
RUN apt-get install -y alien # optional
COPY ClouderaHiveODBC-2.6.1.1001-1.x86_64.rpm /opt/cloudera/
RUN alien /opt/cloudera/ClouderaHiveODBC-2.6.1.1001-1.x86_64.rpm
RUN dpkg -i clouderahiveodbc_2.6.1.1001-2_amd64.deb
# ==================END=============================
RUN conda install --name env -c conda-forge turbodbc=4.1.1 tornado=6.0.4 pyyaml pymysql schedule sqlalchemy pyarrow numpy=1.19.3\
pandas=1.1.4 pybind11 pyarrow
COPY odbc.ini /etc/
RUN apt-get update && apt-get install -y gettext nano vim -y
RUN yarn install --modules-folder ./static
WORKDIR /app
COPY . /app/
ENV SSH_PASSWD "root:Docker!"
RUN apt-get update \
&& apt-get install -y --no-install-recommends dialog \
&& apt-get update \
&& apt-get install -y --no-install-recommends openssh-server \
&& echo "$SSH_PASSWD" | chpasswd
COPY sshd_config /etc/ssh/
COPY entry.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/entry.sh
EXPOSE 9988 2222 22 80 8000
CMD ["entry.sh"]
Keep a copy of ClouderaHiveODBC-2.6.1.1001-1.x86_64.rpm in the current directory
Keep the below files as well :
odbc.ini - which has the DB info
entry.sh - which is shell script and has a command - python app.py
ssh_config - file without any extension has the information as shown below
Port 2222
ListenAddress 0.0.0.0
LoginGraceTime 180
X11Forwarding yes
Ciphers aes128-cbc,3des-cbc,aes256-cbc
MACs hmac-sha1,hmac-sha1-96
StrictModes yes
SyslogFacility DAEMON
PrintMotd no
IgnoreRhosts no
#deprecated option
#RhostsAuthentication no
RhostsRSAAuthentication yes
RSAAuthentication no
PasswordAuthentication yes
PermitEmptyPasswords no
PermitRootLogin yes
I want to expand the answer by showing an approach that works without conda being necessary. In other words, a full-pip minimum viable docker setup for installing turbodbc. I've fully documented the solution in this Github comment in the official turbodbc repo.
I have a Dockerfile that fails on build with the error;
COPY failed: stat /var/lib/docker/tmp/docker-builder476469130/requirements.txt: no such file or directory
The error occurs on the COPY line for the requirments.txt file. I use a pretty standard Dockerfile;
FROM python:3.6.7-slim
# Version: 1.4
# Dockerfile to build the coroner container.
# Install Python and Package Libraries
RUN apt-get update && apt-get upgrade -y && apt-get autoremove && apt-get autoclean
RUN apt-get install -y \
libffi-dev \
libssl-dev \
default-libmysqlclient-dev \
libxml2-dev \
libxslt-dev \
libjpeg-dev \
libfreetype6-dev \
zlib1g-dev \
net-tools \
nano
ARG PROJECT=coroner
ARG PROJECT_DIR=/var/www/${PROJECT}
WORKDIR $PROJECT_DIR
ENV PYTHONUNBUFFERED 1
RUN mkdir -p $PROJECT_DIR
COPY requirements.txt $PROJECT_DIR/requirments.txt
RUN pip install --upgrade pip
RUN pip install -r $PROJECT_DIR/requirements.txt
EXPOSE 8888
STOPSIGNAL SIGINT
ENTRYPOINT ["python", "manage.py"]
CMD ["runserver", "0.0.0.0:8888"]
I am bashing my head against this and have been praying at the church of google for a while now. I have checked the context and it seems to be correct. my build command is;
sudo docker build -t coroner:dev .
Docker Version Docker version 19.03.6, build 369ce74a3c
Can somebody put me out of my misery, please?
You've got a typo in 'requirements.txt' in the destination, you've put 'requirments.txt'.
However, because you're simply copying this to where you've specified your WORKDIR, you can just do:
COPY requirements.txt .
The file will then be copied into your CWD.
I currently have a docker image with Jenkins and Python.
I did something like
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get install -y python python-pip python3-pip && \
... (and more stuff)
... (I also install gcloud sdk)
WORKDIR /opt/app
RUN /usr/bin/env python3 -m pip install --upgrade pip \
&& /usr/bin/env python3 -m pip install pipenv==2018.10.13
RUN /usr/bin/env python -m pip install --upgrade pip \
&& /usr/bin/env python -m pip install pipenv==2018.10.13
RUN chown jenkins /opt/app -R
USER jenkins
But this installs python 3.5.3 ( https://packages.debian.org/stretch/python3 )
I'd need python 3.7 (as well as python 2.7.15).
So, I'm trying my way with multiple FROM as explained here and there. But to no avail.
FROM python:2.7.15-stretch as py2
FROM python:3.7.2-stretch as py3
FROM jenkins/jenkins:lts as jenkins
I'm pretty sure it's not too complicated... once you've played with it once...
So, any help is welcome.
It works! I did like this:
FROM python:3.7.2-stretch as py3
FROM python:2.7.15-stretch as py2
FROM jenkins/jenkins:lts
USER root
COPY --from=py2 /usr/local/lib /usr/local/lib
COPY --from=py2 /usr/local/bin /usr/local/bin
COPY --from=py2 /usr/local/include /usr/local/include
COPY --from=py2 /usr/local/man /usr/local/man
COPY --from=py2 /usr/local/share /usr/local/share
COPY --from=py3 /usr/local/lib /usr/local/lib
COPY --from=py3 /usr/local/bin /usr/local/bin
COPY --from=py3 /usr/local/include /usr/local/include
COPY --from=py3 /usr/local/man /usr/local/man
COPY --from=py3 /usr/local/share /usr/local/share
RUN apt-get update && \
...
TL&DR: This is the best way to get the latest python on the Jenkins provided docker-image.
Description
Create a DockerFile with the following content:
FROM jenkins/jenkins:lts-alpine
USER root
RUN apk add python3 && \
python3 -m ensurepip && \
pip3 install --upgrade pip setuptools && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
RUN apk add pkgconf
RUN apk add build-base
RUN apk add python3-dev
RUN apk add postgresql-dev
RUN apk add postgresql-client
You can use wget to download and install python version that you want.
For example, installing python3.8.12 on jenkins:lts-jdk11
FROM jenkins/jenkins:lts-jdk11
RUN apt-get update -y && apt-get install -y \
# Tools
sudo net-tools wget vim nfs-common arping curl \
# Required libs
build-essential libreadline-gplv2-dev libncursesw5-dev \
libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev zlib1g-dev \
libxml2-dev libxslt1-dev libffi-dev libssl-dev libz-dev && \
# Install python
mkdir install_py && cd install_py && \
sudo wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tgz && \
sudo tar xzf Python-3.8.12.tgz && \
cd Python-3.8.12 && \
sudo ./configure --enable-optimizations && \
sudo make altinstall && \
cd ../.. && rm -r install_py && \
# Set default python path to use python
sudo ln -snf /usr/local/bin/python3.8 /usr/bin/python && \
python -m pip install --upgrade pip
...