Develop Reference

RHEL PyEnv Python Install without Root / Sudo access

I am looking to install Python 3.10.9 and some packages behind a firewall on RHEL 7 machine.
I do not have administrative rights, nor access to elevated privileges. Some core requirements are missing to build items from source. I've been unable to build with OpenSSL, and thus can't pip install packages.
I prefer to stick to packaged solutions as much as possible, so I've pursued a path that allows installing dependencies with yum.
Relevant related questions
Preparing _tkinter and sqlite3 for Python installation (no admin rights)
Relevant lessons: How to install packages in Linux (CentOS) without root user with automatic dependency handling?
# Install prerequisites
mkdir ~/rpm
mkdir ~/rhel
yumdownloader --destdir ~/rpm --resolve zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel openssl-devel tk-devel libffi-devel xz-devel libffi
cd ~/rhel && for i in ~/rpm/*.rpm; do rpm2cpio $i | cpio -idv; done  
# Modify ~/.bashrc with necessary flags and includes. See .bashrc below
source ~/.bashrc
#Download pyenv and install
curl https://pyenv.run | bash
pyenv install 3.10.9
The part I seem to be getting stuck on, is correctly referencing the various libraries that are installed.
Rather than build dependencies from source, utilized yum to install:
Current ~/.bashrc
# For building custom YUM locally
export PATH="$HOME/rhel/usr/sbin:$HOME/rhel/usr/bin:$HOME/rhel/bin:$PATH"
export MANPATH="$HOME/rhel/usr/share/man:$MANPATH"
L='/lib:/lib64:/usr/lib:/usr/lib64'
export LD_LIBRARY_PATH="$HOME/rhel/usr/lib:$HOME/rhel/usr/lib64:$L"
# PyEnv install https://www.mpietruszka.com/install-pyenv-ce-rhel8.html
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
#export PKG_CONFIG_PATH=$HOME/rhel/usr/lib/pkgconfig:$PKG_CONFIG_PATH
#LDFLAGS
export LDFLAGS="-L$HOME/rhel/usr/lib -L$HOME/rhel/usr/lib64 -L/usr/lib -L/usr/lib64 -L/lib -L/lib64"
#CPPFLAGS
export CPPFLAGS="-I$HOME/rhel/usr/lib -I$HOME/rhel/usr/lib64 -I/usr/lib -I/usr/lib64 -I/lib -I/lib64"
#LD_Library
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/rhel/usr/include"
The errors I'm currently getting
pyenv install 3.10.9
Downloading Python-3.10.9.tar.xz...
-> https://www.python.org/ftp/python/3.10.9/Python-3.10.9.tar.xz
Installing Python-3.10.9...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "~/.pyenv/versions/3.10.9/lib/python3.10/ctypes/__init__.py", line 8, in <module>
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
WARNING: The Python ctypes extension was not compiled. Missing the libffi lib?
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'readline'
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "~/.pyenv/versions/3.10.9/lib/python3.10/ssl.py", line 99, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?
Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems
BUILD FAILED (RedHatEnterpriseServer 7.7 using python-build 20180424)
Inspect or clean up the working tree at /tmp/python-build.20230208180402.153250
Results logged to /tmp/python-build.20230208180402.153250.log
Last 10 log lines:
$ensurepip --root=/ ; \
fi
Looking in links: /tmp/tmp540mosi3
Processing /tmp/tmp540mosi3/setuptools-65.5.0-py3-none-any.whl
Processing /tmp/tmp540mosi3/pip-22.3.1-py3-none-any.whl
Installing collected packages: setuptools, pip
WARNING: The scripts pip3 and pip3.10 are installed in '~/.pyenv/versions/3.10.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.3.1 setuptools-65.5.0
make: warning: Clock skew detected. Your build may be incomplete.
I'd appreciate some criticism on correctly calling these libraries...

How can I use the Python language bindings for RPM / RPM5 on MacOS?

On MacOS there is a homebrew formula to install the RPM development package from rpm5.org. However, this installs only the command line tools (rpm, rpmlint, rpmbuild, etc) and does not install any of the language bindings which are supported.
I would like to use the Pascal language bindings. However, when I build them and attempt to import the rpm package into Python 2.7 I get this error:
$ python -c "import rpm._rpm"
Traceback (most recent call last):
File "", line 1, in
File
"/usr/local/lib/python2.7/venv-default/lib/python2.7/site-packages/rpm/init.py",
line 7, in
from _rpm import * ImportError: dlopen(/usr/local/lib/python2.7/venv-default/lib/python2.7/site-packages/rpm/_rpmmodule.so,
2): Symbol not found: _sqlite3_enable_load_extension Referenced
from: /usr/local/Cellar/rpm/5.4.15_1/lib/librpmio-5.4.dylib Expected
in: flat namespace in
/usr/local/Cellar/rpm/5.4.15_1/lib/librpmio-5.4.dyl
To build the Python bindings I re-installed the rpm package with homebrew using these commands:
brew install -v --keep-tmp --build-from-source rpm 2>&1 | tee brew_install.log
cd /tmp/rpm-20170408-18245-1u8nsbs/rpm-5.4.15
./configure --prefix=/usr/local/Cellar/rpm/5.4.15_1 --localstatedir=/usr/local/var --with-path-cfg=/usr/local/etc/rpm --with-path-magic=/usr/local/share/misc/magic --with-path-sources=/usr/local/var/lib/rpmbuild --with-libiconv-prefix=/usr --disable-openmp --disable-nls --disable-dependency-tracking --with-db=external --with-sqlite=external --with-file=external --with-popt=external --with-beecrypt=internal --with-libtasn1=external --with-neon=internal --with-uuid=external --with-pcre=internal --with-lua=internal --with-syck=internal --without-apidocs varprefix=/usr/local/var --with-python
cd python
make
make install
Note the ./configure command is the same one as Homebrew used with the --with-python switch appended.
How can I use the cross platform rpm5.org based source code to do Python language development on MacOS?

Your error depends on how sqlite3 and rpm are built on OS/X with brew.
sqlite3 can be built to add _sqlite3_enable_load_extension
or RPM can be patched to remove the need for _sqlite3_enable_load_extension .
If patching RPM, see rpmio/rpmsql.c near line 2881.

How to install pyatspi?

I'm using ldtp and while I import it ,my idle jump out with a error:
from ldtp import *
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/ldtpd/__init__.py", line 38, in <module>
from xmlrpc_daemon import XMLRPCLdtpd
File "/usr/local/lib/python2.7/site-packages/ldtpd/xmlrpc_daemon.py",line 25, in <module>
import core
File "/usr/local/lib/python2.7/site-packages/ldtpd/core.py", line 24, in <module>
from pyatspi import findDescendant, Registry
ImportError: No module named pyatspi
And then I yum install pyatspi but found
No package pyatspi available.
Then I download pyatspi-2.20.0.tar.xz
During ./configure ,an error occurs as
configure: error: Package requirements (pygobject-3.0 >= 2.90.1) were not met:
No package 'pygobject-3.0' found
So I go back and try to install pygobject and found:
configure: error: Package requirements (glib-2.0 >= 2.24.0
gobject-introspection-1.0 >= 1.29.0
) were not met:
No package 'gobject-introspection-1.0' found
Excuse me ???
Is that the same thing we all have to suffer during install pyatspi???
What should I do to jump out from this nightmare?
Thanks for any help !

pyatspi is a part of GNOME. What you got involved into by building pyatspi from source was leading you into building a significant part of GNOME from source. No easy task at all.
Your best bet is to find AT SPI package for your distribution. You did not specify what linux distribution do you have. But for most common distributions such packages exist.
First make sure you have got python-gobject installed
For rpm-based distributions
rpm -q python-gobject
For dpkg-based:
dpkg -s python-gobject
Install it if necessary.
Next install a package containing pyatspi for your distribution.
For instance for OpenSUSE you need to install python-atspi
zypper in python-atspi
For Ubuntu and derivatives it is python-pyatspi:
apt-get install python-pyatspi
For RedHat, Fedora and derivatives it is at-spi-python
yum install at-spi-python
Having got that you should be able to type
python -c "import gi,pyatspi"

ImportError: No module named distutils

Attempt to install psutils resulted a big headache...
$ python -V
Python 2.4.2
$ cat /etc/SuSE-release
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 4
$ cd psutil-2.1.1/
$ python setup.py install
Traceback (most recent call last):
File "setup.py", line 17, in ?
from distutils.core import setup, Extension
ImportError: No module named distutils.core
Next - I try to install setuptools to use easy_install:
$ which easy_install
which: no easy_install
$ cd ../setuptools-1.4/
$ python setup.py install
Traceback (most recent call last):
File "setup.py", line 12, in ?
from distutils.util import convert_path
ImportError: No module named distutils.util
Trying install distutils from ez_setup.py:
$ python ez_setup.py
Traceback (most recent call last):
File "ez_setup.py", line 278, in ?
main(sys.argv[1:])
File "ez_setup.py", line 210, in main
egg = download_setuptools(version, delay=0)
File "ez_setup.py", line 139, in download_setuptools
from distutils import log
ImportError: No module named distutils
So - how can I install it?
P.S. No, I haven't root on this machine and can't use package manager.

you need to run this (if Error happens on python3) ==> sudo apt-get install python3-distutils --reinstall
you need to run this (if Error happens on python2) ==> sudo apt-get install python2-distutils --reinstall

I have an answer here but I will copy it here
AskUbuntu answer:
Debian has decided that distutils is not a core python package, so it is not included in the last versions of debian and debian-based OSes. You should be able to do sudo apt install python3-distutils and it should work.
However, it did not work for me. I use Parrot OS, which is, as Ubuntu, Debian based. I upgraded my system and pip stopped working for python3.7, and I also got the error ModuleNotFoundError: No module named 'distutils.util'
I tried a lot of stuff to fix it and to reinstall distutils, and I found out by pure luck, that pip3, for python3.8 did work. I then tried python3.7 -m pip3 -V, got /usr/bin/python3.7: No module named pip3 so I decided to have a look in the /usr/lib files.
I looked at /usr/lib/python3/dist-packages and everything looked fine. Then I looked at /usr/lib/python3.7 and saw the folder distutil.
I opened it, and saw the __pycache__, the __init__.py file and a version.py file. I had no idea how many files should be in there, or what the code should be, but I knew that those two files were either wrong or missing another file.
Then I had a look at what was inside /usr/lib/python3.8/distutil and it was totally different. I found the following files:
command Folder
__pycache__ Folder
archive_util.py Python script
bcppcompiler.py Python script
cmd.py Python script
config.py Python script
core.py Python script
cygwinccompiler.py Python script
debug.py Python script
dep_util.py Python script
errors.py Python script
extension.py Python script
fancy_getopt.py Python script
filelist.py Python script
file_util.py Python script
__init__.py Python script
log.py Python script
msvc9compiler.py Python script
_msvccompiler.py Python script
msvccompiler.py Python script
README Plain text file
spawn.py Python script
sysconfig.py Python script
text_file.py Python script
unixccompiler.py Python script
util.py Python script
version.py Python script
versionpredicate.py Python script
This was a lot more promising, and since pip3 did work, I assumed that this distutils worked too, and I tried to copy it to the python3.7 folder by running this command:
sudo cp -r /usr/lib/python3.8/distutil /usr/lib/python3.7/distutil
Then I tried again python3.7 -m pip -V and got
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.7)
Then I tried installing some modules and everything works fine.
I hope this is helpful.

#ciro
I need to do even more to get virtualenv running again (upgraded from 18.04 to 20.04):
sudo cp /usr/lib/python3.8/_sysconfigdata__* /usr/lib/python3.6/
cd /usr/lib/python3.6
sudo ln -s _sysconfigdata_m_linux_x86_64-linux-gnu.py _sysconfigdata_m_x86_64-linux-gnu.py

Python 2.7 No module named _sqlite3 (no root access to machine, python is locally installed)?

There might be other questions similar to this but, in my particular case, I don't have super user (sudo) access to the machine and I have locally installed Python 2.7.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/spicmacay/.local/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/spicmacay/.local/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named _sqlite3
>>>
UPDATE: When I run ./configure&& make, I get:
make
running build
running build_ext
building dbm using gdbm
INFO: Can't locate Tcl/Tk libs and/or headers
Python build finished, but the necessary bits to build these modules were not found:
_sqlite3 _tkinter bsddb185
dl imageop sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
running build_scripts

Came to this issue also.. fixed by compiling Python and passing sqlite3 headers/libs during ./configure
Python 2.7.3 _sqlite3 module is not being built after passing headers/libraries
Posted just in case anyone else has this issue in the future and does not have sudo access to a node.

This happened me recently. You need to apt-get install libsqlite3-dev (on debian - sqlite-devel possibly elsewhere) and recompile python.

If you first ./configure sqlite3 with a --prefix option, then make and make install it and then use the same --prefix when compiling Python, the Python installation will be able to magically find and use the sqlite3 you just installed.
cd sqlite-autoconf-3080100
./configure --prefix=/home/xdanek7/appscale/local
make
make install
cd ../Python-2.7.6
./configure --prefix=/home/xdanek7/appscale/local
make
make install

Try adding sudo before the command apt-get install libsqlite3-dev to get rid of "NO root access"
i.e on terminal, write:
sudo apt-get install libsqlite3-dev