Develop Reference

Installing Python 3.11.1 on a docker container

I want to use debian:bullseye as a base image and then install a specific Python version - i.e. 3.11.1. At the moment I am just learning docker and linux.
From what I understand I can either:
Download and compile sources
Install binaries (using apt-get)
Use a Python base image
I have come across countless questions on here and articles online. Do I use deadsnakes? What version do I need? Are there any official python distributions (who is deadsnakes anyway)?
But ultimately I want to know the best means of getting Python on there. I don't want to use a Python base image - I am curious in the steps involved. Compile sources - I am far from having that level of knowhow - and one for another day.
Currently I am rolling with the following:
FROM debian:bullseye
RUN apt update && apt upgrade -y
RUN apt install software-properties-common -y
RUN add-apt-repository "ppa:deadsnakes/ppa"
RUN apt install python3.11
This fails with:
#8 1.546 E: Unable to locate package python3.11
#8 1.546 E: Couldn't find any package by glob 'python3.11'
Ultimately - it's not the error - its just finding a good way of getting a specific Python version on my container.

In case you want to install Python 3.11 in debian bullseye you have to compile it from source following the next steps (inside the Dockerfile):
sudo apt update
sudo apt install software-properties-common wget
wget https://www.python.org/ftp/python/3.11.1/Python-3.11.1.tar.xz
sudo tar -xf Python-3.11.1.tar.xz
cd Python-3.11.1
sudo ./configure --enable-optimizations
sudo make altinstall
Another option (easiest) would be to use the official Python Docker image, in your case:
FROM 3.11-bullseye
You have all the versions available in docker hub.
Other option that could be interesting in your case is 3.11-slim-bullseye, that is an image that does not contain the common packages contained in the default tag and only contains the minimal packages needed to run python.

Based on #tomasborella answer, to do this in docker:
Dockerfile
FROM debian:bullseye
RUN apt-get update -y \
&& apt-get upgrade -y \
&& apt-get -y install build-essential \
zlib1g-dev \
libncurses5-dev \
libgdbm-dev \
libnss3-dev \
libssl-dev \
libreadline-dev \
libffi-dev \
libsqlite3-dev \
libbz2-dev \
wget \
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-get purge -y imagemagick imagemagick-6-common
RUN cd /usr/src \
&& wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz \
&& tar -xzf Python-3.11.0.tgz \
&& cd Python-3.11.0 \
&& ./configure --enable-optimizations \
&& make altinstall
RUN update-alternatives --install /usr/bin/python python /usr/local/bin/python3.11 1
update-alternatives - will update the links to allow you to run python as opposed to specifying python3.11 when you want to run it.
It takes a while to compile those sources!

How install PIP & PYMSSQL in docker

I have a Python program which is to be executed in the Azure Kubernetes.
Below is my docker file - I have Python installed
#Ubuntu Base image with openjdk8 with TomEE
FROM demo.azurecr.io/ubuntu/tomee/openjdk8:8.0.x
RUN apt-get update && apt-get install -y telnet && apt-get install -y ksh && apt-get install -y python2.7.x && apt-get -y clean && rm -rf /var/lib/apt/lists/*
however I don't know how to install PIP and related dependent libraries (eg: pymssql)?

Best option is installing miniconda on docker image. I used it always when I need to have python on docker image without python or pip.
Here is part for installing minicinda in my simple docker image
FROM debian
RUN apt-get update && apt-get install -y curl wget
RUN rm -rf /var/lib/apt/lists/*
RUN wget \
https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
&& mkdir /root/.conda \
&& bash Miniconda3-latest-Linux-x86_64.sh -b \
&& rm -f Miniconda3-latest-Linux-x86_64.sh
RUN conda --version

Upgrade Python in Jenkins Docker

I currently have a docker image with Jenkins and Python.
I did something like
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get install -y python python-pip python3-pip && \
... (and more stuff)
... (I also install gcloud sdk)
WORKDIR /opt/app
RUN /usr/bin/env python3 -m pip install --upgrade pip \
&& /usr/bin/env python3 -m pip install pipenv==2018.10.13
RUN /usr/bin/env python -m pip install --upgrade pip \
&& /usr/bin/env python -m pip install pipenv==2018.10.13
RUN chown jenkins /opt/app -R
USER jenkins
But this installs python 3.5.3 ( https://packages.debian.org/stretch/python3 )
I'd need python 3.7 (as well as python 2.7.15).
So, I'm trying my way with multiple FROM as explained here and there. But to no avail.
FROM python:2.7.15-stretch as py2
FROM python:3.7.2-stretch as py3
FROM jenkins/jenkins:lts as jenkins
I'm pretty sure it's not too complicated... once you've played with it once...
So, any help is welcome.

It works! I did like this:
FROM python:3.7.2-stretch as py3
FROM python:2.7.15-stretch as py2
FROM jenkins/jenkins:lts
USER root
COPY --from=py2 /usr/local/lib /usr/local/lib
COPY --from=py2 /usr/local/bin /usr/local/bin
COPY --from=py2 /usr/local/include /usr/local/include
COPY --from=py2 /usr/local/man /usr/local/man
COPY --from=py2 /usr/local/share /usr/local/share
COPY --from=py3 /usr/local/lib /usr/local/lib
COPY --from=py3 /usr/local/bin /usr/local/bin
COPY --from=py3 /usr/local/include /usr/local/include
COPY --from=py3 /usr/local/man /usr/local/man
COPY --from=py3 /usr/local/share /usr/local/share
RUN apt-get update && \
...

TL&DR: This is the best way to get the latest python on the Jenkins provided docker-image.
Description
Create a DockerFile with the following content:
FROM jenkins/jenkins:lts-alpine
USER root
RUN apk add python3 && \
python3 -m ensurepip && \
pip3 install --upgrade pip setuptools && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
RUN apk add pkgconf
RUN apk add build-base
RUN apk add python3-dev
RUN apk add postgresql-dev
RUN apk add postgresql-client

You can use wget to download and install python version that you want.
For example, installing python3.8.12 on jenkins:lts-jdk11
FROM jenkins/jenkins:lts-jdk11
RUN apt-get update -y && apt-get install -y \
# Tools
sudo net-tools wget vim nfs-common arping curl \
# Required libs
build-essential libreadline-gplv2-dev libncursesw5-dev \
libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev zlib1g-dev \
libxml2-dev libxslt1-dev libffi-dev libssl-dev libz-dev && \
# Install python
mkdir install_py && cd install_py && \
sudo wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tgz && \
sudo tar xzf Python-3.8.12.tgz && \
cd Python-3.8.12 && \
sudo ./configure --enable-optimizations && \
sudo make altinstall && \
cd ../.. && rm -r install_py && \
# Set default python path to use python
sudo ln -snf /usr/local/bin/python3.8 /usr/bin/python && \
python -m pip install --upgrade pip
...

How to install graph-tool for Anaconda Python 3.5 on linux-64?

I'm trying to install graph-tool for Anaconda Python 3.5 on Ubuntu 14.04 (x64), but it turns out that's a real trick.
I tried this approach, but run into the problem:
The following specifications were found to be in conflict:
- graph-tool
Use "conda info <package>" to see the dependencies for each package.
Digging through the dependencies led to a dead-end at gobject-introspection
So I tried another approach:
Installed boost with conda, then tried to ./configure, make, and make install graph-tool... which got about as far as ./configure:
===========================
Using python version: 3.5.2
===========================
checking for boostlib >= 1.54.0... yes
checking whether the Boost::Python library is available... yes
checking whether boost_python is the correct library... no
checking whether boost_python-py27 is the correct library... no
checking whether boost_python-py27 is the correct library... (cached) no
checking whether boost_python-py27 is the correct library... (cached) no
checking whether boost_python-py35 is the correct library... yes
checking whether the Boost::IOStreams library is available... yes
configure: error: Could not link against boost_python-py35 !
I know this is something about environment variables for the ./configure command and conda installing libboost to Anaconda's weird place, I just don't know what to do, and my Google-fu is failing me. So this is another dead end.
Can anyone who's had to install graph-tool recently in linux-64 give me a walkthrough? It's a fresh VM running in VMWare Workstation 10.0.7

For those that run into similar issues, try changing the order of conda channels first with:
$ conda config --add channels ostrokach
$ conda config --add channels defaults
$ conda config --add channels conda-forge
then:
$ conda install graph-tool

Installing graph-tool 2.26 for Anaconda Python 3.5, Ubuntu 14.04.
Note: as of me writing this, the ostrokach channel conda install of graph-tool was only at version 2.18.
Here's the docker file I use to install graph-tool 2.26. There's likely a cleaner way, but so far this is the only thing I've managed to cobble together that actually works.
NOTE: If you're unfamiliar with docker files and you'd just like to do the install from the terminal, ignore the first line (starting with FROM), ignore every occurrence of the word RUN, and what you're left with is a series of commands to execute in a terminal.
FROM [your 14.04 base image]
RUN conda upgrade -y conda
RUN conda upgrade -y matplotlib
RUN \
add-apt-repository -y ppa:ubuntu-toolchain-r/test && \
apt-get update -y && \
apt-get install -y gcc-5 g++-5 && \
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5
RUN wget https://github.com/CGAL/cgal/archive/releases/CGAL-4.10.2.tar.gz && \
tar xzf CGAL-4.10.2.tar.gz && \
cd cgal-releases-CGAL-4.10.2/ && \
cmake . && \
make && \
make install
RUN cd /tmp && \
# note: master branch of repo appears relatively stable, has not been updated since 2016
git clone https://github.com/sparsehash/sparsehash.git && \
cd sparsehash && \
./configure && \
make && \
make install
RUN apt-get update
RUN apt-get install -y build-essential g++ python-dev autotools-dev libicu-dev build-essential libbz2-dev libboost-all-dev
RUN apt-get install -y autogen autoconf libtool shtool
# install boost
RUN cd /tmp && \
wget https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.gz && \
tar xzvf boost_1_66_0.tar.gz && \
cd boost_1_66_0 && \
sudo ./bootstrap.sh --prefix=/usr/local && \
sudo ./b2 && \
sudo ./b2 install
# install newer cairo
RUN cd /tmp && \
wget https://cairographics.org/releases/cairo-1.14.12.tar.xz && \
tar xf cairo-1.14.12.tar.xz && \
cd cairo-1.14.12 && \
./configure && \
make && \
sudo make install
RUN cd /tmp && \
wget https://download.gnome.org/sources/libsigc++/2.99/libsigc++-2.99.10.tar.xz && \
tar xf libsigc++-2.99.10.tar.xz && \
cd libsigc++-2.99.10 && \
./configure && \
make && \
sudo make install && \
sudo cp ./sigc++config.h /usr/local/include/sigc++-3.0/sigc++config.h
RUN cd /tmp && \
wget https://www.cairographics.org/releases/cairomm-1.15.5.tar.gz && \
tar xf cairomm-1.15.5.tar.gz && \
cd cairomm-1.15.5 && \
./configure && \
make && \
sudo make install && \
sudo cp ./cairommconfig.h /usr/local/include/cairomm-1.16/cairomm/cairommconfig.h
RUN conda install -y -c conda-forge boost pycairo
RUN conda install -y -c numba numba=0.36.2
RUN conda install -y -c libboost py-boost && \
conda update -y cffi dbus expat pycairo pandas scipy numpy harfbuzz setuptools boost
RUN apt-get install -y apt-file dbus libdbus-1-dev && \
apt-file update
RUN apt-get install -y graphviz
RUN conda install -y -c conda-forge python-graphviz
RUN sudo apt-get install -y valgrind
RUN apt-get install -y libcgal-dev libcairomm-1.0 libcairomm-1.0-dev libcairo2-dev python-cairo-dev
RUN conda install -y -c conda-forge pygobject
RUN conda install -y -c ostrokach gtk
RUN cd /tmp && \
wget https://git.skewed.de/count0/graph-tool/repository/release-2.26/archive.tar.bz2 && \
bunzip2 archive.tar.bz2 && \
tar -xf archive.tar && \
cd graph-tool-release-2.26-b89e6b4e8c5dba675997d6f245b301292a5f3c59 && \
# Fix problematic parts of the graph-tool configure.ac file
sed -i 's/PKG_INSTALLDIR/#PKG_INSTALLDIR/' ./configure.ac && \
sed -i 's/AM_PATH_PYTHON(\[2\.7\])/AM_PATH_PYTHON(\[3\.5\])/' ./configure.ac && \
sed -i 's/\${PYTHON}/\/usr\/local\/anaconda3\/bin\/python/' ./configure.ac && \
sed -i '$a ACLOCAL_AMFLAGS = -I m4' ./Makefile.am && \
sudo ./autogen.sh && \
sudo ./configure CPPFLAGS="-I/usr/local/include -I/usr/local/anaconda3/pkgs/pycairo-1.15.4-py35h1b9232e_1/include -I/usr/local/include/cairo -I/usr/local/include/sigc++-3.0 -I/usr/include/freetype2" \
LDFLAGS="-L/usr/local/include -L/usr/local/lib/cairo -L/usr/local/include/sigc++-3.0 -L/usr/include/freetype2" \
PYTHON="/usr/local/anaconda3/bin/python" \
PYTHON_VERSION=3.5 \
sudo make && \
sudo make install
Warning: makeing graph-tool might take a couple hours and require >7 GB of ram.

Error installing pyV8 from source on ubuntu

when trying to install PyV8 in ubuntu, and type the command:
python setup.py build
then it display this error:
error: command 'c++' failed with exit status 1
anybody have solution about this?

Here is what I have in my Dockerfile. The following is tested and runs in production on top of Debian Stretch. I recommend using exactly the PyV8 / V8 setup that I'm using - I've spent at least a week to figure out which combination doesn't lead to memory leaks. I also recommend reading through the discussion and the JSContext fix here and here.
In short, support for PyV8 is almost non-existent - either you use it just as a toy, or you follow exactly this recipe, or you spend a significant amount of time and effort to fork the repo and make it better. If starting fresh, I recommend using Node-JS instead and communicate through some IPC method with Python.
ENV MY_HOME /home/forge
ENV MY_LIB $FORGE_HOME/lib
# preparing dependencies for V8 and PyV8
ENV V8_HOME $MY_LIB/v8
RUN apt-get update && \
apt-get install -y libboost-thread-dev \
libboost-all-dev \
libboost-dev \
libboost-python-dev \
autoconf \
libtool \
systemtap \
scons
# compiling an older version of boost, required for this version of V8
RUN mkdir -p $MY_LIB/boost && cd $MY_LIB/boost && \
wget http://sourceforge.net/projects/boost/files/boost/1.54.0/boost_1_54_0.tar.gz && tar -xvzf boost_1_54_0.tar.gz && cd $MY_LIB/boost/boost_1_54_0 && \
./bootstrap.sh && \
./b2 install --prefix=/usr/local --with-python --with-thread && \
ldconfig && \
ldconfig /usr/local/lib
# preparing gcc 4.9 - anything newer will lead to errors with the V8 codebase
ENV CC "gcc-4.9"
ENV CPP "gcc-4.9 -E"
ENV CXX "g++-4.9"
ENV PATH_BEFORE_V8 "${MY_HOME}/bin:${PATH}"
ENV PATH "${MY_HOME}/bin:${PATH}"
RUN echo "deb http://ftp.us.debian.org/debian/ jessie main contrib non-free" >> /etc/apt/sources.list && \
echo "deb-src http://ftp.us.debian.org/debian/ jessie main contrib non-free" >> /etc/apt/sources.list && \
apt-get update && \
apt-get install -y gcc-4.9 g++-4.9 && \
mkdir -p ${MY_HOME}/bin && cd ${MY_HOME}/bin && \
ln -s /usr/bin/${CC} ${MY_HOME}/bin/gcc && \
ln -s /usr/bin/${CC} ${MY_HOME}/bin/x86_64-linux-gnu-gcc && \
ln -s /usr/bin/${CXX} ${MY_HOME}/bin/g++ && \
ln -s /usr/bin/${CXX} ${MY_HOME}/bin/x86_64-linux-gnu-g++
# compiling a specific version of V8 and PyV8, since older combos lead to memory leaks
RUN git clone https://github.com/muellermichel/V8_r10452.git $V8_HOME && \
git clone https://github.com/muellermichel/PyV8_r429.git $MY_LIB/pyv8 && \
cd $MY_LIB/pyv8 && python setup.py build && python setup.py install
# cleaning up
RUN PATH=${PATH_BEFORE_V8} && \
head -n -2 /etc/apt/sources.list > ${MY_HOME}/sources.list.temp && \
mv ${MY_HOME}/sources.list.temp /etc/apt/sources.list && \
apt-get update
ENV PATH "${PATH_BEFORE_V8}"
ENV CC ""
ENV CPP ""
ENV CXX ""
older version that depends on the now defunct googlecode and was made for Ubuntu 12.04:
export MY_LIB_FOLDER=[PUT-YOUR-DESIRED-INSTALL-PATH-HERE]
apt-get install -y libboost-thread-dev
apt-get install -y libboost-all-dev
apt-get install -y libboost-dev
apt-get install -y libboost-python-dev
apt-get install -y git-core autoconf libtool systemtap
apt-get install -y subversion
apt-get install -y wget
mkdir -p $MY_LIB_FOLDER/boost && cd $MY_LIB_FOLDER/boost && wget http://sourceforge.net/projects/boost/files/boost/1.54.0/boost_1_54_0.tar.gz && tar -xvzf boost_1_54_0.tar.gz
cd $MY_LIB_FOLDER/boost/boost_1_54_0 && ./bootstrap.sh && ./b2 install --prefix=/usr/local --with-python --with-thread && ldconfig && ldconfig /usr/local/lib
svn checkout -r10452 http://v8.googlecode.com/svn/trunk/ $MY_LIB_FOLDER/v8
export V8_HOME=$MY_LIB_FOLDER/v8
svn checkout -r429 http://pyv8.googlecode.com/svn/trunk/ $MY_LIB_FOLDER/pyv8
git clone https://github.com/taguchimail/pyv8-linux-x64.git $MY_LIB_FOLDER/pyv8-taguchimail && cd $MY_LIB_FOLDER/pyv8-taguchimail && git checkout origin/stable
apt-get install -y scons
cd $MY_LIB_FOLDER/pyv8 && patch -p0 < $MY_LIB_FOLDER/pyv8-taguchimail/patches/pyv8.patch && python setup.py build && python setup.py install

Had the same problem and this worked for me:
export LIB=~
apt-get install -y curl libboost-thread-dev libboost-all-dev libboost-dev libboost-python-dev git-core autoconf libtool
svn checkout -r19632 http://v8.googlecode.com/svn/trunk/ $LIB/v8
export V8_HOME=$LIB/v8
svn checkout http://pyv8.googlecode.com/svn/trunk/ $LIB/pyv8 && cd $LIB/pyv8 && python setup.py build && python setup.py install
Solution found in comments here - https://code.google.com/p/pyv8/wiki/HowToBuild

I'm using a Debian based distro. Here's how I installed PyV8 (you'll need to
have git installed):
cd /usr/share
sudo git clone https://github.com/emmetio/pyv8-binaries.git
cd pyv8-binaries/
sudo unzip pyv8-linux64.zip
sudo cp -a PyV8.py _PyV8.so /usr/bin