Develop Reference

Unable to downgrade version of Python package by `pip install -I`

To temporary workaround a compatibility issue, I tried to downgrade version of Python package arrow from 0.17.0 to 0.13.2.
On a particular server I was unable to, as the below command calls to install 0.13.2 but ends up install back 0.17.0. If remove the virtual environment, and generate a new one and freshly install 0.13.2, it works OK.
I also tested on a virtual machine, and the downgrade works OK without this symptom.
I'm wondering why, and any inputs will be highly appreciated.
(venv3.7) [user#host freeze]$ pip install -I arrow==0.13.2
Collecting arrow==0.13.2
Using cached arrow-0.13.2-py2.py3-none-any.whl (37 kB)
Collecting python-dateutil
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting six>=1.5
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, python-dateutil, arrow
Successfully installed arrow-0.17.0 python-dateutil-2.8.1 six-1.15.0
(venv3.7) [user#host freeze]$

Use --force-reinstall instead of -I, --ignore-installed.
-I can break existing installs, according to the docs:
-I, --ignore-installed Ignore the installed packages, overwriting them. This can break your system if the existing package is of a different version or was installed with a different package manager!
If you want to install a version older than what you currently have installed, --force-reinsall is a better fit:
--force-reinstall Reinstall all packages even if they are already up-to-date.
A demonstration of --force-reinstall in action:
$ pip install arrow==0.17.0
Collecting arrow==0.17.0
<... snip ...>
Successfully installed arrow-0.17.0
$ pip install --force-reinstall arrow==0.13.2
Collecting arrow==0.13.2
<... snip ...>
Attempting uninstall: arrow
Found existing installation: arrow 0.17.0
Uninstalling arrow-0.17.0:
Successfully uninstalled arrow-0.17.0
Successfully installed arrow-0.13.2 python-dateutil-2.8.1 six-1.15.0
$ pip freeze | grep arrow
arrow==0.13.2

Another option from luasoftware.com
% pip install arrow
Collecting arrow
Downloading arrow-1.2.1-py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 110 kB/s
Installing collected packages: six, python-dateutil, arrow
Successfully installed arrow-1.2.1
% pip install --upgrade arrow==0.13.2
Collecting arrow==0.13.2
Downloading arrow-0.13.2-py2.py3-none-any.whl (37 kB)
Installing collected packages: arrow
Attempting uninstall: arrow
Found existing installation: arrow 1.2.1
Uninstalling arrow-1.2.1:
Successfully uninstalled arrow-1.2.1
Successfully installed arrow-0.13.2
% pip list | grep arrow
arrow 0.13.2

Why do I get so many wheel files when I download a Python package using pip

I executed this:
$ pip download virtualenv
Collecting virtualenv
Using cached virtualenv-20.0.31-py2.py3-none-any.whl (4.9 MB)
Saved d:\test\gits\virtualenv-20.0.31-py2.py3-none-any.whl
Collecting appdirs<2,>=1.4.3
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Saved d:\test\gits\appdirs-1.4.4-py2.py3-none-any.whl
Collecting six<2,>=1.9.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Saved d:\test\gits\six-1.15.0-py2.py3-none-any.whl
Collecting distlib<1,>=0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Saved d:\test\gits\distlib-0.3.1-py2.py3-none-any.whl
Collecting filelock<4,>=3.0.0
Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Saved d:\test\gits\filelock-3.0.12-py3-none-any.whl
Successfully downloaded virtualenv appdirs six distlib filelock
Although I attempted to download one package, I got these wheel files:
appdirs-1.4.4-py2.py3-none-any.whl
six-1.15.0-py2.py3-none-any.whl
distlib-0.3.1-py2.py3-none-any.whl
virtualenv-20.0.31-py2.py3-none-any.whl
filelock-3.0.12-py3-none-any.whl
Now my questions are:
Why are there so many wheel files,and not just one file called virtualenv-20.0.31-py2.py3-none-any.whl?
Now once I have this downloaded and stored somewhere on my computer or network location, how do I do the install?
What happens if I just have the virtualenv-20.0.31-py2.py3-none-any.whl and no other wheel file when I attemp the install?

1 . Using pip download also downloads the dependencies, and those extra files that you saw are precisely the deps of the virtualenv distribution:
$ johnnydep virtualenv
name summary
---------------------- -------------------------------------------------------------------------------------------------
virtualenv Virtual Python Environment builder
├── appdirs<2,>=1.4.3 A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".
├── distlib<1,>=0.3.1 Distribution utilities
├── filelock<4,>=3.0.0 A platform independent file lock.
└── six<2,>=1.9.0 Python 2 and 3 compatibility utilities
If you only want the package itself without dependencies, use:
pip download --no-deps virtualenv
2 . You can install a wheel file directly with pip
pip install ./virtualenv-20.0.31-py2.py3-none-any.whl
3 . If you install with --no-deps the app won't work properly, probably it will crash with some ImportError due to missing dependencies.
$ pip install -q --no-deps virtualenv
$ virtualenv .venv
Traceback (most recent call last):
...
ModuleNotFoundError: No module named 'appdirs'
Otherwise it will just collect and install the dependencies from PyPI, or whatever index your pip is configured at - see pip config list.

Those additional wheels are dependencies, including recursive (transitive) dependencies, i.e. dependencies of dependencies.
pip install --find-links /path/to/download/dir/ virtualenv-20.0.31-py2.py3-none-any.whl
pip tries to download dependencies from the configured index server, default is PyPI. Failed to download any dependency pip fails to install anything. It exits with an error message and an error code.

pip: downloading dependencies to specific platform including non binaries

I'm trying to download the dependencies of paramiko from a linux host to a windows target which has no internet access .
After reading the example on pip's documentation I've used to following command in order to download the dependencies recursively to a 64 bit windows platform:
pip3 download --only-binary=:all: --platform win_amd64 --implementation cp paramiko
Was able to recursively download the dependencies until reaching pycparser. That is not surprising since I've used the --only-binary=:all: flag. Thing is - pip forces the usage of this flag when --platform flag is passed:
ERROR: --only-binary=:all: must be set and --no-binary must not be set (or must be set to :none:) when restricting platform and interpreter constraints using --python-version, --platform, --abi, or --implementation.
Terminal produced the following output:
Collecting paramiko
Downloading paramiko-2.3.0-py2.py3-none-any.whl (182kB)
100% |████████████████████████████████| 184kB 340kB/s
Saved ./paramiko-2.3.0-py2.py3-none-any.whl
Collecting pynacl>=1.0.1 (from paramiko)
Using cached PyNaCl-1.1.2-cp35-cp35m-win_amd64.whl
Saved ./PyNaCl-1.1.2-cp35-cp35m-win_amd64.whl
Collecting cryptography>=1.5 (from paramiko)
Using cached cryptography-2.0.3-cp35-cp35m-win_amd64.whl
Saved ./cryptography-2.0.3-cp35-cp35m-win_amd64.whl
Collecting pyasn1>=0.1.7 (from paramiko)
Using cached pyasn1-0.3.5-py2.py3-none-any.whl
Saved ./pyasn1-0.3.5-py2.py3-none-any.whl
Collecting bcrypt>=3.1.3 (from paramiko)
Using cached bcrypt-3.1.3-cp35-cp35m-win_amd64.whl
Saved ./bcrypt-3.1.3-cp35-cp35m-win_amd64.whl
Collecting cffi>=1.4.1 (from pynacl>=1.0.1->paramiko)
Using cached cffi-1.11.0-cp35-cp35m-win_amd64.whl
Saved ./cffi-1.11.0-cp35-cp35m-win_amd64.whl
Collecting six (from pynacl>=1.0.1->paramiko)
Using cached six-1.11.0-py2.py3-none-any.whl
Saved ./six-1.11.0-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=1.5->paramiko)
Using cached asn1crypto-0.22.0-py2.py3-none-any.whl
Saved ./asn1crypto-0.22.0-py2.py3-none-any.whl
Collecting idna>=2.1 (from cryptography>=1.5->paramiko)
Using cached idna-2.6-py2.py3-none-any.whl
Saved ./idna-2.6-py2.py3-none-any.whl
Collecting pycparser (from cffi>=1.4.1->pynacl>=1.0.1->paramiko)
Could not find a version that satisfies the requirement pycparser (from cffi>=1.4.1->pynacl>=1.0.1->paramiko) (from versions: )
No matching distribution found for pycparser (from cffi>=1.4.1->pynacl>=1.0.1->paramiko)
Is there a way of overcoming this issue? Will I have to manually install non-binary packages (and their dependencies)?
Thanks,
Joey.

You have two options:
run the download operation on the same platform (be careful to be the same)
fix the internet access on your host
Don't try other fancy method or you will shut yourself in the foot: some dependencies will need to compile!

You can use the --prefer-binary option in pip. That'll make pip consider wheels as more important, even if they're an older version than an existing sdist (sdist is short for source distribution). An sdist would be selected if no wheels are found to be compatible.
This was released in pip 18.0 (so that's early 2018, pip's using CalVer now).

#sorin is right, your only real option is to use the exact same environment to download the dependencies as you'll be installing them on.
My solution is to use Docker to build a wheel that matches the target platform. In my case it is Debian 10, but it will work just the same for any operating system and version as long as there is a Docker image available.
Example Dockerfile to build a wheel with the dependencies in requirements.txt for Debian 10 with cPython 3.9:
FROM python:3.9-slim-buster
COPY requirements.txt requirements.txt
RUN set -eux; \
apt-get update && \
apt-get install -y build-essential && \
python3 -m venv .venv --without-pip
ENV VIRTUAL_ENV=.venv
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"
RUN set -eux; \
curl --silent https://bootstrap.pypa.io/get-pip.py | python && \
pip download --prefer-binary --upgrade setuptools wheel setuptools-rust -d deps && \
pip download --prefer-binary -r requirements.txt -d deps && \
mkdir -p main-wheel && \
pip wheel --wheel-dir=main-wheel -r requirements.txt
Build the image and extract the wheel:
docker build -t buildwheel -f Dockerfile
mkdir -p artifacts
CONTAINER=$(docker create buildwheel || exit 1)
docker cp "${CONTAINER}":main-wheel artifacts/. || exit 1
docker rm "${CONTAINER}"
docker image rm buildwheel
Congrats, you now have a wheel specifically for Debian 10 with cPython 3.9 inside the directory artifacts/main-wheel. Copy it to the target machine and do pip install --no-index --find-links=artifacts/main-wheel -r requirements.txt and everything should work.
PS: You might need to add build-time dependencies to the apt-get install inside the Dockerfile.

in python3 you can download the dependencies like mentioned below
while running this be inside the folder you want to save
pip download -r requirements.txt
once you downloaded the files move these to the machine you want to you can install
then run this command
pip install -r req.txt --no-index --find-links="/path/to/downloaded/files"

How to get mod_wsgi-express from pip using Dockerfile

I can't recall how I installed mod_wsgi-express, but I'm almost certain that I used pip.
I've got this line in my requirements.txt file:
mod_wsgi-express==4.5.15
Collecting mod_wsgi-express==4.5.15 (from -r /usr/src/app/requirements.txt (line 16))
Could not find a version that satisfies the requirement mod_wsgi-express==4.5.15 (from -r /usr/src/app/requirements.txt (line 16)) (from versions: )
No matching distribution found for mod_wsgi-express==4.5.15 (from -r /usr/src/app/requirements.txt (line 16))
You are using pip version 9.0.0, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Is the naming different here? Or am I just not seeing that package?
$ pip list | grep mod
mod-wsgi (4.5.15)
mod-wsgi-httpd (2.4.23.2)

In requirements.txt: mod_wsgi==4.5.15
The package information can be found on PyPi at:
https://pypi.python.org/pypi/mod_wsgi

pip install ok, but "insecure and unverifiable files" on travis.ci

I created and uploaded a package callled pdfminer.six on PyPi
It installs correctly through "pip install pdfminer.six" on my PC (Windows)
and passes tests on travis-ci
Since I need this package for my other project (Goulib), I added "pdfminer.six" it to its requirements.txt, but Travis.ci fails importing this package ) :
Downloading/unpacking pdfminer.six (from -r requirements.txt (line 7))
Could not find any downloads that satisfy the requirement pdfminer.six
(from -r requirements.txt (line 7)) Some insecure and unverifiable
files were ignored (use --allow-unverified pdfminer.six to allow).
Cleaning up... No distributions at all found for pdfminer.six (from -r
requirements.txt (line 7))
Tried to add --allow-unverified and version number ==20140915 in all possible ways, no change ...
What should I do to make pdfminer.six "secure and verifiable" on Travis-ci ?