Develop Reference

python compilation does not find ssl

I have compiled and run without an issue python3 on a rpi 3. Now It fails after the compilation to use pip on an rpi 4
These are the steps of the compilation
version=3.9.5
mkdir ~/temp
wget -O ~/temp/Python-$version.tar.xz https://www.python.org/ftp/python/$version/Python-$version.tar.xz
cd ~/temp
tar xf Python-$version.tar.xz
cd Python-$version
./configure --enable-optimizations --with-openssl=/usr/lib/ssl
make -j -l 4
sudo make altinstall
and whereis ssl.h shows
ssl: /usr/lib/ssl /etc/ssl /usr/share/man/man7/ssl.7ssl.gz
but still configure complains
checking for openssl/ssl.h in /usr/lib/ssl... no
checking whether compiling and linking against OpenSSL works... no
checking for --with-ssl-default-suites... python
checking for --with-builtin-hashlib-hashes... md5,sha1,sha256,sha512,sha3,blake2
I can confirm later pip complains about ssl.
Any idea why python compilation should not include properly ssl ?

if some one comes across this
./configure --enable-optimizations --enable-shared
make -j -l 4
sudo make altinstall
sudo ldconfig
did work for me. However still not sure if that is a valid solution

how to link to libssl.so.1.1 "./python: error libssl.so.1.1: cannot open shared object file: No such file or directory"

I am working on a shared server without root access. I have spent hours to get openssl and libffi working with python on the server. I figured it out once, but then it broke because I moved the directories, so I decided to start it over. Here is the documentation I wrote for myself to set this up:
// Pthon3.8.2 setup on bluehost -currently libffi is still broken
mkdir install-workspaces
cd install-workspaces
mkdir openssl
cd openssl
wget https://ftp.openssl.org/source/old/1.1.1/openssl-1.1.1.tar.gz
tar xvf openssl-1.1.1.tar.gz
cd openssl-1.1.1
//in order to run wsgi we might need to say "shared" instead of "no-shared"
./config --prefix=/home/user/local/lib/openssl --openssldir=/home/user/local/lib/openssl no-shared zlib-dynamic
make
make test
make install
cd ../../
mkdir libffi
cd libffi
wget ftp://sourceware.org/pub/libffi/libffi-3.2.1.tar.gz
tar xzf libffi-3.2.1.tar.gz
cd libffi-3.2.1
//I dont know if --disable-docs does anything, but I used the command and the install worked
./configure --prefix=/home/user/local/lib/libffi/ --disable-docs
make
make install
cd ../../
mkdir python
cd python
wget http://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz
tar -xvf Python-3.8.2.tar.xz
cd Python-3.8.2
vim Modules/Setup
//press [i]
/* //remove the comments on the lines below and change the SSL= to be your openssl directory
SSL=/home/user/local/lib/openssl
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto
*/
//press [esc] type ":wq" and press enter
export LD_LIBRARY_PATH=/home/user/local/lib/openssl/lib
PKG_CONFIG_PATH=/home/user/local/lib/libffi/lib/pkgconfig
./configure --with-openssl=/home/user/local/lib/openssl --prefix="/home/user/local/lib/python" --enable-optimizations LDFLAGS='-L/home/user/local/lib/libffi/lib64 -R/home/user/local/lib/libffi/lib64' --enable-shared
make
make altinstall //<- important for _ctypes
Is there any way I can path to libssl.so.1.1 when using ./configure in python?

ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)

ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version) django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later
I met with this problem at the django project, mismatching version for the sqlite3 and the django. this occurs at the centos7 env, and I also want a convenient solution that works in the container env.

I solved this issue by upgrading my version of sqlite3 using this command:
cd ~ && wget https://www.sqlite.org/2020/sqlite-autoconf-3320100.tar.gz && tar xvfz sqlite-autoconf-3320100.tar.gz && cd sqlite-autoconf-3320100 && ./configure && make && make install
I am using ElasticBeanstalk for my setup, so I added a .config file to the .ebextensions folder and put this in it:
option_settings:
aws:elasticbeanstalk:application:environment:
LD_LIBRARY_PATH: "/usr/local/lib"
commands:
01_upgrade_sqlite:
command: "cd ~ && wget https://www.sqlite.org/2020/sqlite-autoconf-3320100.tar.gz && tar xvfz sqlite-autoconf-3320100.tar.gz && cd sqlite-autoconf-3320100 && ./configure && make && make install"
Many thanks to Bejür for adding the LD_LIBRARY_PATH environment variable here in order to get it to work.

In my case, upgrading sqlite with kloddant's instructions helped only partially. SQLite was installed, but python was still referring to the old sqlite version.
An additional step to fix this issue was setting the environment variable LD_LIBRARY_PATH="/usr/local/lib"
Add it to .ebextensions/*.config
option_settings:
"aws:elasticbeanstalk:application:environment":
LD_LIBRARY_PATH: "/usr/local/lib"

You have an outdated version of SQLite,you can try this:
python -m pip install -U sqlite

I now find a usable solution: change the version of Django from 'Django==3.0.4' to 'Django==2.1.8' and it worked.
pip install django==2.1.8 -i https://mirrors.aliyun.com/pypi/simple/

Building Python 3.6.4 on Linux from scratch

I'm trying to build Python 3.6.4 from LFS 8.2-systemd so I run the configure command:
./configure --prefix=/usr \
--enable-shared \
--with-system-expat \
--with-system-ffi \
--with-ensurepip=yes
followed by make -j.
However, at this point the module "pyexpat" is not found by Python, but the file exists in /usr/lib/libexpat.so.
After reading building Python from source with zlib support, I created a symlink:
ln -s /usr/lib /usr/lib/x86_64-gnu-linux
If i run make install, I get an error:
ModuleNotFoundError: No module named pyexpat
My expat lib version is 2.2.5.
I'm doing the compilation inside env -i chroot /mnt bash
and my environment just contains a valid PATH and LX_ALL=POSIX variables.

I ran into this same problem for python 3.6.8 , when I initially configured using:
./configure --prefix=/opt/python-3.6/ --enable-optimizations
However, when I retried using the command in the BLFS book:
./configure --prefix=/opt/python-3.6/ --enable-shared --with-system-expat --with-system-ffi --with-ensurepip=yes
My pyexpat started working.
That being said, I think it may be helpful to just retry, since my second command is functionally identical to yours.

sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6
To make python3 use the new installed python 3.6 instead of the default 3.5 release, run following 2 commands:
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2
Finally switch between the two python versions for python3 via command:
sudo update-alternatives --config python3
After selecting version 3.6:
python3 -V

Can CPython be compiled with Clang?

I'm trying to build CPython using Clang, with very specific requirements:
Python 2.7.14
CentOS 6.9 (but settling for Ubuntu 16.04), x64
LLVM 5.0.0
I tried setting env variable CC to my clang executable location (i.e. /opt/llvm/5/bin/clang, but the ./configure command fails with the following:
configure: error: C compiler cannot create executables
What flags should I set to make this build work?

As the commenters on OP's post pointed out, you can compile cpython with clang. Here are reproducible instructions, in the form of a Dockerfile.
FROM ubuntu:16.04
WORKDIR /opt/cpython-2.7.14
RUN apt-get update -qq \
&& apt-get install --yes build-essential curl \
# Install clang
&& curl -fsSL https://releases.llvm.org/5.0.0/clang+llvm-5.0.0-linux-x86_64-ubuntu16.04.tar.xz \
| tar xJ -C /usr/local --strip-components 1 \
&& curl -fsSL https://www.python.org/ftp/python/2.7.14/Python-2.7.14.tgz \
| tar xz --strip-components 1 \
&& CC=/usr/local/bin/clang ./configure --prefix /usr/local/cpython-2.7.14 \
&& make \
&& make install
ENTRYPOINT ["/usr/local/cpython-2.7.14/bin/python"]
docker build --tag cpython:2.7 .
docker run --rm cpython:2.7 --version
# Python 2.7.14
It is difficult to say what OP's original issue was, because it seemed like a problem with the clang installation. Looking at the configure logs would provide more information.
Disclaimer: Python 2 has reached its end of life, and ubuntu 16.04 reaches its end of life in April 2021.

tried setting env variable CC to my clang executable location
On Linux, define an alias in Bash as follows:
alias cc="clang"
On my system I set it as alias cc="clang-11"
Also, be sure to have installed all clang and llvm packages on the system.