Develop Reference

Meeting a pip requirement using an alternative module

I'm trying to install a python module, 'pyAudioProcessing' (https://github.com/jsingh811/pyAudioProcessing) on my Linux Mint distribution, and one of the items in requirements.txt is causing issues: python-magic-bin==0.4.14. When I run pip3 install -e pyAudioInstaller, I get an error:
ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from pyAudioProcessing==1.1.5) (from versions: none)
ERROR: No matching distribution found for python-magic-bin==0.4.14 (from pyAudioProcessing==1.1.5)
The same error appears if I try to manually install the module using pip3 install python-magic-bin. The module installs without issues on my windows machine.
pypi.org lets me download files for it manually, however only Windows and MacOS .whl files are available. I tried simply removing the requirement from the list, but that resulted in a large number of other errors to appear, so I assume the module is legitimately required.
Thee is another module called python-magic-debian-bin that I can download. Is there a simple way to convince pyAudioInstaller to use this other module instead of the original? Like can I somehow rename python-magic-debian-bin to python-magic-bin and hope it works out?

python-magic-bin 0.4.14 provides wheels for OSX, w32 and w64, but not for Linux. And there is no source code at PyPI.
You need to install it from github:
pip install git+https://github.com/julian-r/python-magic.git
As for pyAudioProcessing I can see 2 ways to install it:
Clone the repository and edit requirements/requirements.txt, replace python-magic-bin==0.4.14 with pip install git+https://github.com/julian-r/python-magic.git#egg=python-magic;
Install requirements manually and then install pyAudioProcessing without dependencies:
pip install --no-deps pyAudioProcessing
or
pip install --no-deps git+https://github.com/jsingh811/pyAudioProcessing.git

The library has updated the requirements very recently for it to work on Linux.
pip install -U pyAudioProcessing
Should get it all set up for you.
Alternatively, https://github.com/jsingh811/pyAudioProcessing the readme describes other getting started methods as well.

Install package with pip offline [duplicate]

This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I want to install a python3 package (in this specific case is torch and torchvision) on a cluster that is not connected to the external world. I have tried to do pip3 download <package> and then I move the file to the cluster and run pip3 install <downloaded-file> -t /custom/folder (because I am installing in one of the folders in my personal account, and not on the entire cluster).
When I run the install command, pip3 is trying to download numpy package -- for some reason (it is actually already installed and working on ipython3, I do not know why it is trying to download that).
Of course, the installation fails because there is no connection to the external world. Can I force python to download everything with any flags?
Moreover, I would like to be sure I am downloading the proper thing: the location from which I am downloading the file is different from the location where I want to install, the first it is a regular CPU and the second runs GPU and has CUDA installed.

Formal answer here. Pip is attempting to install a different version of numpy specified in your package's requirements. You can easily compare the version installed with pip freeze and the version it is attempting to download (before it fails).
To make the download and installation fool-proof you probably should create a requirements.txt file and then run pip download -r requirements.txt in the environment that is connected to the internet and then transfer to the non-connected environment and carry on with the installation.
Regarding the GPU/CPU difference between the packages, it is hard to tell without actually knowing which package this is, but if it's similar to tensorflow where you have completely different package names when using pip the lack of GPU should not affect the download process. See pip download documentation for more information.

How to install python packages in a private network [duplicate]

This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I'm an experienced programmer, but very new to python. My company requires us to do development on a private network for some of our projects. There is a pypi index on the private network which can be used to install packages using pip. Recently, while needing to install a package, the pypi index when down and was down for several hours. Although it did come back up eventually, the situation begs the question, how do I install packages (maybe manually without pip) in the absense of an index? I've tried to google this, but came up empty. I'm sure there's a way, but I'm probably not searching for the right phrase. Thanks for any help with.

You can manually install Python packages if you have read access to the package repositories. Every Python package has a setup.py file in the root directory and you can do something like
python setup.py sdist
This creates a subdirectory called dist which contains a compressed archived file, tar.gz or .zip depending in your OS. You can pass this archived file to pip and install the package
pip3 install some-python-package.tar.gz

I would download the wheel and install that. For this to you do need to install the wheel package:
pip install wheel
You can then tell pip to install the project (and it'll download the wheel if available), or the wheel file directly:
pip install project_name # download and install
pip install wheel_file.whl # directly install the wheel
The wheel module is also runnable from the command line and you can use it to install already-downloaded wheels:
python -m wheel install wheel_file.whl

There are a few ways you can get around this issue. The two that I know of are:
Use a proxy to get to the standard PyPI. If your company permits it, then you can tunnel your traffic through their proxy and install packages from PyPA's standard locations.
Use a locally hosted index. All you need is a directory structured like https://pypi.org/simple/, and you can then pip install -i ~/my/personal/index/path and packages will be installed from there.

Why is pip installing an old version of my package?

I've just uploaded a new version of my package to PyPi (1.2.1.0-r4): I can download the egg file and install it with easy_install, and the version checks out correctly. But when I try to install using pip, it installs version 1.1.0.0 instead. Even if I explicitly specify the version to pip with pip install -Iv tome==1.2.1.0-r4, I get this message: Requested tome==1.2.1.0-r4, but installing version 1.1.0.0, but I don't understand why.
I double checked with parse_version and confirmed that the version string on 1.2.1 is greater than that on 1.1.0 as shown:
>>> from pkg_resources import parse_version as pv
>>> pv('1.1.0.0') < pv('1.2.1.0-r4')
True
>>>
So any idea why it's choosing to install 1.1.0 instead?

This is an excellent question. It took me forever to figure out. This is the solution that works for me:
Apparently, if pip can find a local version of the package, pip will prefer the local versions to remote ones. I even disconnected my computer from the internet and tried it again -- when pip still installed the package successfully, and didn't even complain, the source was obviously local.
The really confusing part, in my case, was that pip found the newer versions on pypi, reported them, and then went ahead and re-installed the older version anyway ... arggh. Also, it didn't tell me what it was doing, and why.
So how did I solve this problem?
You can get pip to give verbose output using the -v flag ... but one isn't enough. I RTFM-ed the help, which said you can do -v multiple times, up to 3x, for more verbose output. So I did:
pip install -vvv <my_package>
Then I looked through the output. One line caught my eye:
Source in /tmp/pip-build-root/ has version 0.0.11, which satisfies requirement <my_package>
I deleted that directory, after which pip installed the newest version from pypi.

Try forcing download the package again with:
pip install --no-cache-dir --upgrade <package>

Thanks to Marcus Smith, who does amazing work as a maintener of pip, this was fixed in version 1.4 of pip which was released on 2013-07-23.
Relevant information from the changelog for this version
Fixed a number of issues (#413, #709, #634, #602, and #939) related to
cleaning up and not reusing build directories. (Pull #865, #948)

I found here that there is a known bug in pip that it won't check the version if there's a build directory with unpacked sources. I have checked this on my troubling package and after deleting its sources from build directory pip installed the required version.

If you are using a pip version that comes with some distribution packages (ex. Ubuntu python-pip), you may need to install a newer pip version:
Update pip to latest version:
sudo pip install -U pip
In case of "virtualenv", skip "sudo":
pip install -U pip
Following command may be required, if your shell report something like -bash: /usr/bin/pip: No such file or directory after pip update:
hash -d pip
Now install your package as usual:
pip install -U foo
or
pip install foo==package.version.here

Got the same issue to update pika 0.9.5 to 0.9.8. The only working way was to install from tarball: pip install https://pypi.python.org/packages/source/p/pika/pika-0.9.8.tar.gz.

In my case the python version used (3.4) didn't satisfy Django 2.1 dependencies requirements (python >= 3.5).

For my case I had to delete the .pip folder in my home directory and then I was able to get later versions of multiple libraries. Note that this was on linux.
pip --version
pip 18.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
virtualenv --version
15.1.0

Just in case that anyone else hassles with upgrading torchtext (or probably any other torch library):
Although https://pypi.org/project/torchtext/ states that you could run pip install torchtext I had to install it similiar to torch by specifying --find-links aka -f:
pip install torchtext===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
What irritated me was that PyCharm pointed me to the new version, but couldn't find it when attempting to upgrade to it. I guess that PyCharm uses its own mechanism to spot new versions. Then, when invoking pip under the hood, it didn't find the new version without the --find-links option.

In my case I am pip installing a .tar.gz package from Artifactory that I make a lot of updates to. In order to overwrite my cached Python files and always grab/install the latest I was able to run:
pip install --no-cache-dir --force-reinstall <path/to/tar.gz>
You should see this re-download any necessary files and install those, instead of using your local cache.

10 years on and pip still fails to work as expected 😖.
I wasted a couple of hours now banging my head against the wall trying to find out why pip won't install a development version of my package. In my case, there are versions 0.0.4 and 0.0.5.dev1 in a private gitlab.com package registry (hence the --extra-index-url argument below), but I believe that's not relevant to the problem.
Following a lot of the advice on this page, I create a test venv in a far away folder, clear the pip cache, uninstall the package in question, etc. first to rule out the most common problems:
$ pip cache purge && \
pip uninstall --yes my-package && \
pip install --extra-index-url "https://_:${GITLAB_PASSWORD_TOOLS_VAULTTOOLS}#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple" \
--no-cache-dir \
--pre \
--upgrade my-package
output (using empty lines to separate output for commands):
WARNING: No matching packages
Files removed: 0
Found existing installation: my-package 0.0.4
Uninstalling my-package-0.0.4:
Successfully uninstalled my-package-0.0.4
Looking in indexes: https://pypi.org/simple, https://_:****#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple
Collecting my-package
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/f07 ... 397/my_package-0.0.5.dev1-py3-none-any.whl (16 kB)
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/775 ... 70e/my_package-0.0.4-py3-none-any.whl (16 kB)
...
Successfully installed my-package-0.0.4
So pip does see the dev package version, but chooses the earlier one nonetheless.
In an attempt to figure out what's going on, I published a 0.0.5 version: Error persists, pip sees all three versions, but still installs 0.0.4.
In a further, increasingly desperate attempt, I removed any versions prior to 0.0.5* from the gitlab.com package registry.
Only now, pip would bother to actually display some useful information:
$ (same command as above)
... (similar output as above) ...
ERROR: Cannot install my-package==0.0.5 and my-package==0.0.5.dev1 because these package versions have conflicting dependencies.
The conflict is caused by:
my-package 0.0.5 depends on my-other-package<0.2.5 and >=0.2.4
my-package 0.0.5.dev1 depends on my-other-package<0.2.5 and >=0.2.4
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
OK, so there is something wrong with my package dependencies. Thanks for letting me know.
Seriously - I tried hard for a couple of hours using all kinds of pip ... -vvv and/or fixed versions such as e.g. my-package==0.0.5.dev1 - but I did not manage to get any useful output out of pip - until I wiped the entire history from my package registry 🤬.
Hope this at least helps someone in the same situation.

I found that if you use microversions, pip doesn't seem to recognize them. For example, we couldn't get version 1.9.9.1 to upgrade.

In my case, someone had published the latest version of a package with python2, so attempting to pip3 install it grabbed an older version that had been built with python3.
Handy things to check when debugging this:
If pip install claims to not be able to find the version, see whether pip search can see it.
Take a look at the "Download Files" section on the pypi repo -- the filenames might suggest what's wrong (in my case i saw -py2- there clear as day).
As suggested by others, try running pip install --no-cache-dir in case pip isn't bothering to ask the internet because it already has your answer locally.

I had hidden unversioned files under the Git tab in PyCharm that were being installed with pip install . even though I didn't see the files anywhere else.
Took a long time to find it for me, posting this in hope that it'll help somebody else.

if you need the path for your package do pip -v list. Example see related post when using pip -e Why is an old version of a package of my python library installing by itself with pip -e?

pip uses incorrect cached package version, instead of the user-specified version

I need to install psycopg2 v2.4.1 specifically. I accidentally did:
pip install psycopg2
Instead of:
pip install psycopg2==2.4.1
That installs 2.4.4 instead of the earlier version.
Now even after I pip uninstall psycopg2 and attempt to reinstall with the correct version, it appears that pip is re-using the cache it downloaded the first time.
How can I force pip to clear out its download cache and use the specific version I'm including in the command?

If using pip 6.0 or newer, try adding the --no-cache-dir option (source).
If using pip older than pip 6.0, upgrade it with pip install -U pip.

Clear the cache directory where appropriate for your system
Linux and Unix
~/.cache/pip # and it respects the XDG_CACHE_HOME directory.
OS X
~/Library/Caches/pip
Windows
%LocalAppData%\pip\Cache
UPDATE
With pip 20.1 or later, you can find the full path for your operating system easily by typing this in the command line:
pip cache dir
Example output on my Ubuntu installation:
➜ pip3 cache dir
/home/tawanda/.cache/pip

With pip 20.1 or later, you can do:
pip cache remove matplotlib: removes all wheel files related to matplotlib from pip's cache.
pip cache purge: to clear all wheel files from pip's cache.
pip cache dir: to get the location of the cache.
If you want to not use the pip cache for some reason (which is a bad idea, according the official docs), your options are:
pip install --no-cache-dir <package>: install a package without using the cache, for just this run.
pip config set global.no-cache-dir false: configure pip to not use the cache "globally" (in all commands).
Some history around this question (puts on pip maintainer hat):
The specific issue of "installing the wrong version due to caching" issue mentioned in the question was fixed in pip 1.4, back in 2013!)
Fix a number of issues related to cleaning up and not reusing build directories. (#413, #709, #634, #602, #939, #865, #948)
Since pip 6.0 (back in 2014!), pip install, pip download and pip wheel commands can be told to avoid using the cache with the --no-cache-dir option. (e.g. pip install --no-cache-dir <package>)
Back then, yes, passing --no-cache-dir was the only option to avoid this bug. So... it's a bit unfortunate that this is the top search result on "pip cache remove". :)
Since pip 10.0 (back in 2018!), a pip config command was added, which can be used to configure pip to always ignore the cache. This was always possible by manually editing the relevant files, but this surfaced that ability to the command line. Details on pip's configuration mechanisms is available here.
Since pip 20.1, pip has a pip cache command to manage the contents of pip's cache.

From documentation at https://pip.pypa.io/en/latest/reference/pip_install.html#caching:
Starting with v6.0, pip provides an on-by-default cache which
functions similarly to that of a web browser. While the cache is on by
default and is designed do the right thing by default you can disable
the cache and always access PyPI by utilizing the --no-cache-dir
option.

pip can install a package ignoring the cache, like this
pip --no-cache-dir install scipy

Since pip 20.1b1, which was released on 21 April 2020 and "added pip cache command for inspecting/managing pip’s wheel cache", it is possible to issue this command:
pip cache purge
The reference guide is here:
https://pip.pypa.io/en/stable/reference/pip_cache/
The corresponding pull request is here.

On Ubuntu, I had to delete /tmp/pip-build-root.

If you like to set the --no-cache-dir option by default, you can put this into pip.conf:
[global]
no-cache-dir = false
Note 1: It's confusing, but to enable the no-cache-dir option you actually have to set it to false. Pretty silly if you ask me... but that's how it is. There is a github issue to fix this.
Note 2: The location of pip.conf depends on your OS. See the documentation for more info.

I just had a similar problem and found that the only way to get pip to upgrade the package was to delete the $PWD/build (%CD%\build on Windows) directory that might have been left over from a previously unfinished install or a previous version of pip (it now deletes the build directories after a successful install).

On archlinux pip cache is located at ~/.cache/pip, I could solve my issue by removing the http folder inside it.

On my mac I had to remove the cache directory ~/Library/Caches/pip/

Simply
rm -d -r "$(pip cache dir)"

On Windows 7, I had to delete %HOMEPATH%/pip.

If using virtualenv, look for the build directory under your environments root.

I had to delete %TEMP%\pip-build On Windows 7

(pyvenv.d) jdoe$ pip --version # pip version for this answer (or newer).
pip 21.1.1
(pyvenv.d) jdoe$ pip cache --help # Review all options available to you.
(pyvenv.d) jdoe$ pip cache dir # Cache-directory for pip(1).
/home/jdoe/.cache/pip
(pyvenv.d) jdoe$ pip cache purge # Purge cache-directory (by example).
Files removed: 621 # If cache-directory is already empty, the
# output will be: "ERROR: No matching packages".

On Mac OS (Mavericks), I had to delete /tmp/pip-build/

A better way to do it is to delete the cache and rebuild it. In this way, if you install it again for other virtualenv, it will use the cache instead of building every time when you install it.
For example, when you install it, it will say it uses cached wheel,
Processing <some_prefix>/Library/Caches/pip/wheels/d0/c4/e4/e49fd07bca8dda00dd6b4bbc606aa05a25aacb00d45747a47a/horovod-0.19.3-cp37-cp37m-macosx_10_9_x86_64.wh
Just delete that one and restart your install.

(...) it appears that pip is re-using the cache (...)
I'm pretty sure that's not what's happening. Pip used to (wrongly) reuse build directory not cache. This was fixed in version 1.4 of pip which was released on 2013-07-23.