I know there are a lot of questions of people who are struggling with this but please read the entire post.
So I just want to create a dockerfile that exposes port 8888 to access a jupyter notebook later on.
This is my dockerfile:
FROM continuumio/anaconda3
ENV DEBIAN_FRONTEND=noninteractive
ARG USERNAME=remote
ARG USER_UID=1000
ARG USER_GID=$USER_UID
COPY environment.yml* noop.txt /tmp/conda-tmp/
RUN apt-get update \
&& apt-get -y install --no-install-recommends apt-utils dialog 2>&1 \
&& apt-get -y install git openssh-client less iproute2 procps iproute2 lsb-release \
&& if [ -f "/tmp/conda-tmp/environment.yml" ]; then /opt/conda/bin/conda env update -n base -f /tmp/conda-tmp/environment.yml; else echo "did not find environment.yml"; fi \
&& rm -rf /tmp/conda-tmp \
&& groupadd --gid $USER_GID $USERNAME \
&& useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \
&& apt-get install -y sudo \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME\
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
ENV DEBIAN_FRONTEND=dialog
EXPOSE 8888
I run the following commands to get the container up and running:
docker build -t intel . (create the image)
docker run -t -d --publish=8888:8888 --name ig intel (start a container based on the image)
Everything up to this moment runs without any problems, but now comes the thing I don't understand:
This runs just fine.
But when I try to go to
localhost:8888?/token=(the token jupyter provides)
or
ipofthecontainer:8888?/token=(the token jupyter provides)
It does nothing(connection timed out) and I have no clue why.
I check the ip of the container with:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name
and I verify that the EXPOSE worked with:
platform: windows
python-version: 3.7.7
conda environment: yes
What am I not seeing here?
EXPOSE is only for documentation; it has no effect.
You need to add --publish=8888:8888 to your docker run so that the container's port is mapped to a host port.
docker run --tty --detach --publish=8888:8888 --name=ig intelligait3d
NB The publish flag does not require the host-port:container-port to match
First modify your docker file in order to start jupyter notebook, add this line at the end of the Dockerfile:
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0", "--allow-root"]
Then build again the image and when you start the container use -p option: --publish , -p Publish a container’s port(s) to the host
docker run -t -d -p 8888:8888 --name ig intelligait3d where the mapping is -p <HOST port>:<CONTAINER port>
Here you can find the reference for docker run command: https://docs.docker.com/engine/reference/commandline/run/
Related
I'm using below docker to use IJava kernel to my jupyter notebook.
FROM ubuntu:18.04
ARG NB_USER="some-user"
ARG NB_UID="1000"
ARG NB_GID="100"
RUN apt-get update || true && \
apt-get install -y sudo && \
useradd -m -s /bin/bash -N -u $NB_UID $NB_USER && \
chmod g+w /etc/passwd && \
echo "${NB_USER} ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
# Prevent apt-get cache from being persisted to this layer.
rm -rf /var/lib/apt/lists/*
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y locales && \
sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8
RUN apt-get install -y \
openjdk-11-jdk-headless \
python3-pip git curl unzip
RUN ln -s /usr/bin/python3 /usr/bin/python & \
pip3 install --upgrade pip
RUN pip3 install packaging jupyter ipykernel awscli jaydebeapi
RUN python -m ipykernel install --sys-prefix
# Install Java kernel
RUN mkdir ijava-kernel/ && cd ijava-kernel && curl -LO https://github.com/SpencerPark/IJava/releases/download/v1.3.0/ijava-1.3.0.zip && \
unzip ijava-1.3.0.zip && \
python install.py --sys-prefix && \
rm -rf ijava-kernel/
RUN jupyter kernelspec list
ENV SHELL=/bin/bash
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64/
WORKDIR /home/$NB_USER
USER $NB_UID
As soon as I run the docker image, inside the container:
some-user#023f579253ec:~$ jupyter kernelspec list ─╯
Available kernels:
python3 /usr/local/share/jupyter/kernels/python3
java /usr/share/jupyter/kernels/java
some-user#023f579253ec:~$
As well as, the console with kernel java is also installed and working as per README.md
jupyter console --kernel java
In [2]: String helloWorld = "Hello world!"
In [3]: helloWorld
Out[3]: Hello world!
But as soon as I run open the jupyter notebook inside the container, I only see Python3 kernel not the Java. see attached image.
can anyone help me out to add the Java Kernel to Notebook GUI?
This is an open issue on IJava's GitHub. The discussion thread mentions a few Docker Images that address your issue: 1, 2, 3.
I have an API build with python and fastapi that works on localhost but when I move it on a Linux machine it loads the image but would not run. I am using: docker run -d -p 10701:8000 fastapi-docker to run it but when I type docker ps it shows that it is not running. Here is the docker file:
FROM python:3.8-slim
WORKDIR /fastapi-app
RUN mkdir -p /app/pdfs
VOLUME /app/pdfs
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY ./app ./app
# COPY ./pdfs ./pdfs
# Installing Oracle instant client
WORKDIR /opt/oracle
RUN apt-get update && apt-get install -y libaio1 wget unzip \
&& wget
https://download.oracle.com/otn_software/linux/instantclient/instantclient-
basiclite-linuxx64.zip \
&& unzip instantclient-basiclite-linuxx64.zip \
&& rm -f instantclient-basiclite-linuxx64.zip \
&& cd /opt/oracle/instantclient* \
&& rm -f *jdbc* *occi* *mysql* *README *jar uidrvci genezi adrci \
&& echo /opt/oracle/instantclient* > /etc/ld.so.conf.d/oracle-
instantclient.conf \
&& ldconfig
WORKDIR /fastapi-app
CMD ["python", "./app/main.py"]
Can someone please tell me why it is not working? Thanks in advance
I tried running it on localhost and the error is: urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/fastapi-app/popriri.xsd'> but I have copied the file and I still get this error: COPY ./popriri.xsd ./fastapi-app
I need a Dockerfile to run my Python script. The script uses Selenium, so I need to load a driver for it to work. An ordinary .exe file - driver is not suitable, so according to the advice of the administrators of the hosting where the script is located I need to create a Dockerfile for the script to work properly.
The main problem is that I simply can not run my script, because I do not understand how to load the required driver on the server.
This is a sample code of what should be in the Dockerfile.
FROM python:3
RUN apt-get update -y
RUN apt-get install -y wget
RUN wget -O $HOME/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/v0.23.0/geckodriver-v0.23.0-linux64.tar.gz
RUN tar xf $HOME/geckodriver.tar.gz -C $HOME
RUN cp $HOME/geckodriver /usr/local/bin/geckodriver
RUN chmod +x /usr/local/bin/geckodriver
RUN rm -f $HOME/geckodriver $HOME/geckodriver.tar.gz
This is the code used in the Python script
options = Options()
options.add_argument('headless')
options.add_argument('window-size=1920x935')
driver = webdriver.Chrome(options=options, executable_path=r"chromedriver.exe")
driver.get(f"https://www.wildberries.ru/catalog/{id}/feedbacks?imtId={imt_id}")
time.sleep(5)
big_stat = driver.find_element(by=By.CLASS_NAME, value="rating-product__numb")
I can redo this snippet of code to make it work on Firefox, if necessary.
This is what the directories of the hosting where all the files are located look like
The directories of the hosting
For getting Selenium to work with Python using a Dockerfile, here's an existing SeleniumBase Dockerfile.
For instructions on using it, see the README.
For building, it's basically this:
Non Apple M1 Mac:
docker build -t seleniumbase .
If running on an Apple M1 Mac, use this instead:
docker build --platform linux/amd64 seleniumbase .
Before building the Dockerfile, you'll need to clone SeleniumBase.
Here's what the Dockerfile currently looks like:
FROM ubuntu:18.04
#=======================================
# Install Python and Basic Python Tools
#=======================================
RUN apt-get -o Acquire::Check-Valid-Until=false -o Acquire::Check-Date=false update
RUN apt-get install -y python3 python3-pip python3-setuptools python3-dev python-distribute
RUN alias python=python3
RUN echo "alias python=python3" >> ~/.bashrc
#=================================
# Install Bash Command Line Tools
#=================================
RUN apt-get -qy --no-install-recommends install \
sudo \
unzip \
wget \
curl \
libxi6 \
libgconf-2-4 \
vim \
xvfb \
&& rm -rf /var/lib/apt/lists/*
#================
# Install Chrome
#================
RUN curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list && \
apt-get -yqq update && \
apt-get -yqq install google-chrome-stable && \
rm -rf /var/lib/apt/lists/*
#=================
# Install Firefox
#=================
RUN apt-get -qy --no-install-recommends install \
$(apt-cache depends firefox | grep Depends | sed "s/.*ends:\ //" | tr '\n' ' ') \
&& rm -rf /var/lib/apt/lists/* \
&& cd /tmp \
&& wget --no-check-certificate -O firefox-esr.tar.bz2 \
'https://download.mozilla.org/?product=firefox-esr-latest&os=linux64&lang=en-US' \
&& tar -xjf firefox-esr.tar.bz2 -C /opt/ \
&& ln -s /opt/firefox/firefox /usr/bin/firefox \
&& rm -f /tmp/firefox-esr.tar.bz2
#===========================
# Configure Virtual Display
#===========================
RUN set -e
RUN echo "Starting X virtual framebuffer (Xvfb) in background..."
RUN Xvfb -ac :99 -screen 0 1280x1024x16 > /dev/null 2>&1 &
RUN export DISPLAY=:99
RUN exec "$#"
#=======================
# Update Python Version
#=======================
RUN apt-get update -y
RUN apt-get -qy --no-install-recommends install python3.8
RUN rm /usr/bin/python3
RUN ln -s python3.8 /usr/bin/python3
#=============================================
# Allow Special Characters in Python Programs
#=============================================
RUN export PYTHONIOENCODING=utf8
RUN echo "export PYTHONIOENCODING=utf8" >> ~/.bashrc
#=====================
# Set up SeleniumBase
#=====================
COPY sbase /SeleniumBase/sbase/
COPY seleniumbase /SeleniumBase/seleniumbase/
COPY examples /SeleniumBase/examples/
COPY integrations /SeleniumBase/integrations/
COPY requirements.txt /SeleniumBase/requirements.txt
COPY setup.py /SeleniumBase/setup.py
RUN find . -name '*.pyc' -delete
RUN find . -name __pycache__ -delete
RUN pip3 install --upgrade pip
RUN pip3 install --upgrade setuptools
RUN pip3 install --upgrade setuptools-scm
RUN cd /SeleniumBase && ls && pip3 install -r requirements.txt --upgrade
RUN cd /SeleniumBase && pip3 install .
#=====================
# Download WebDrivers
#=====================
RUN wget https://github.com/mozilla/geckodriver/releases/download/v0.31.0/geckodriver-v0.31.0-linux64.tar.gz
RUN tar -xvzf geckodriver-v0.31.0-linux64.tar.gz
RUN chmod +x geckodriver
RUN mv geckodriver /usr/local/bin/
RUN wget https://chromedriver.storage.googleapis.com/2.44/chromedriver_linux64.zip
RUN unzip chromedriver_linux64.zip
RUN chmod +x chromedriver
RUN mv chromedriver /usr/local/bin/
#==========================================
# Create entrypoint and grab example tests
#==========================================
COPY integrations/docker/docker-entrypoint.sh /
COPY integrations/docker/run_docker_test_in_firefox.sh /
COPY integrations/docker/run_docker_test_in_chrome.sh /
RUN chmod +x *.sh
COPY integrations/docker/docker_config.cfg /SeleniumBase/examples/
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["/bin/bash"]
I'm trying to run a cronjob inside a docker container, and the logs (created with python logging) from docker logs my_container or from /var/log/cron.log. Neither is working. I tried a bunch of solutions I found in stackoverflow.
This is my Dockerfile:
FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Europe/Minsk
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update && apt-get install -y \
python3-dev \
python3-tk \
python3-pip \
libglib2.0-0\
libsm6 \
postgresql-server-dev-all \
postgresql-common \
openssh-client \
libxext6 \
nano \
pkg-config \
rsync \
cron \
&& \
apt-get clean && \
apt-get autoremove && \
rm -rf /var/lib/apt/lists/*
RUN pip3 install --upgrade setuptools
RUN pip3 install numpy
ADD requirements.txt /requirements.txt
RUN pip3 install -r /requirements.txt && rm /requirements.txt
RUN touch /var/log/cron.log
COPY crontab /etc/cron.d/cjob
RUN chmod 0644 /etc/cron.d/cjob
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV PYTHONUNBUFFERED 1
ADD . /code
WORKDIR /code
COPY ssh_config /etc/ssh/ssh_config
CMD cron -f
and this is how I run it:
nvidia-docker run -d \
-e DISPLAY=unix$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v /media/storage:/opt/images/ \
-v /home/user/.aws/:/root/.aws/ \
--net host \
my_container
I tried different things such as:
Docker ubuntu cron tail logs not visible
See cron output via docker logs, without using an extra file
But I don't get any logs.
Change your chmod code to 755 if you're trying to execute something from there. You might also want to add an -R parameter while at that.
Next, add the following to your Dockerfile before chmod layer.
# Symlink the cron to stdout
RUN ln -sf /dev/stdout /var/log/cron.log
And add this as your final layer
# Run the command on container startup
CMD cron && tail -F /var/log/cron.log 2>&1
Referenced this from the first link that you mentioned. This should work.
I have an application that consists of a docker container with a redis instance, another docker container with a rabbitmq server, and a third container in which I'm trying to start a number of celery workers that can interact with the redis and rabbitmq containers. I'm starting and stopping these workers via a rest api, and have checked that I'm able to do this on my host machine. However, after moving the setup to docker, it seems the workers are not behaving as expected. Whereas on my host machine (windows 10) I was able to see the reply from the rest api and console output from the workers, I can only see the response from the rest api (a log message) and no console output. It also seems that the workers are not accessing the redis and rabbitmq instances.
My docker container is built from a python3.6 (linux) base image. I have checked that everything is installed correctly, and there are no error logs. I build the image with the following dockerfile:
FROM python:3.6
WORKDIR /opt
# create a virtual environment and add it to PATH so that it is applied for
# all future RUN and CMD calls
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
# install msodbcsql17
RUN apt-get update \
&& apt-get install -y curl apt-transport-https gnupg2 \
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
\
&& curl https://packages.microsoft.com/config/debian/9/prod.list >
/etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y msodbcsql17 mssql-tools
# Install Mono for pythonnet.
RUN apt-get update \
&& apt-get install --yes \
dirmngr \
clang \
gnupg \
ca-certificates \
# Dependency for pyodbc.
unixodbc-dev \
&& apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys
3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF \
&& echo "deb http://download.mono-project.com/repo/debian
stretch/snapshots/5.20 main" | tee /etc/apt/sources.list.d/mono-official-
stable.list \
&& apt-get update \
&& apt-get install --yes \
mono-devel=5.20\* \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
COPY src ./src
COPY setup.py ./setup.py
COPY config.json ./config.json
COPY Utility.dll ./Utility.dll
COPY settings_docker.ini ./settings.ini
COPY config.json ./config.json
COPY sql_config.json ./sql_config.json
RUN python3 -m venv $VIRTUAL_ENV \
# From here on, use virtual env's python.
&& venv/bin/pip install --upgrade pip \
&& venv/bin/pip install --no-cache-dir --upgrade pip setuptools wheel \
&& venv/bin/pip install --no-cache-dir -r requirements.txt \
# Dependency for pythonnet.
&& venv/bin/pip install --no-cache-dir pycparser \
&& venv/bin/pip install -U --no-cache-dir "pythonnet==2.5.1" \
# && python -m pip install --no-cache-dir "pyodbc==4.0.25" "pythonnet==2.5.1"
EXPOSE 8081
cmd python src/celery_artifacts/docker_workers/worker_app.py
And then run it with this command:
docker run --name app -p 8081:8081 app
I then attach the container to the same bridge network as the other 2:
docker network connect my_network app
Is there a way to see the same console output from my container as the one on the host?