Develop Reference

pip: uninstalling a disutils installed package

I am trying to install psutil with the command pip install -U psutil and that gives me the error:
Cannot uninstall 'psutil'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
It seems like this is a known issue in pip with versions > 10, and I understand that part (I currently have pip 18). But I just found that I can solve it by directly doing a pip install psutil without using the Upgrade flag. I was wondering if there is a reasoning behind that. My initial sense is that in the first case, where pip tries to upgrade, it first tries to remove the package, which it cannot, but in the latter case it tries to install directly, and hence does not get the error. My question is does it still not have to remove the package first and install (when not using the Upgrade flag), or why specifically is it that pip gives an error with an Upgrade flag but no error without it.
EDIT: So, I tried to run pip install -v psutil as hoefling suggested, and I got a whole bunch of text, as opposed to saying that requirements already met, which means that psutil didn't get installed in the first place. I tried to figure this a bit, and this is what I understand so far: I was running inside a python virtualenv and installing it by means of pip -U -r requirements.txt where requirements.txt contains a bunch of packages including psutil. When I remove the -U flag, it skips installing psutil, and jumps over to other packages. Which raises another question, whether this is how pip is supposed to behave when there is no -U flag. Its interesting that the first time, when its installing the packages with the -U flag, it looks inside the main python installation instead of the virtual environment one, and when the -U flag is removed it doesn't do that and skips entirely.

There are some setups where you have a bunch of packages installed somewhere that isn't the normal install location for setuptools, and comes after the normal install location on sys.path.
Probably the most common of these setups is Apple's pre-installed Python 2.7, so I'll use it as an example. Even if that isn't your setup, it will be hopefully still be instructive.
Apple includes an Extras directory (at /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), with a bunch of third-party packages that either Apple's own tools need, or that Apple thought you might want (back when Apple cared about providing the best Python experience of any platform).
For example, on macOS 10.13, that directory will include NumPy 1.8.0.
These packages are all installed as distribute-style eggs.
(Some linux distros do, or at least used to do, similar things, with Python packages built as RPM/DEB/etc. packages, which go into adistutils directory, unlike things you install via pip or manually, which go into a setuptools directory. The details are a bit different, but the effects, and the workaround, end up being the same.)
If you install pip, and then try to pip install -U numpy or pip uninstall numpy, pip will see the distribute-style numpy-1.8.0rc1-py2.7.egg-info file and refuse to touch it for fear of breaking everything.
If you just pip install numpy, however, it will look only in the standard site-packages installation location used by setuptools, /Library/Python/2.7/site-packages, see nothing there, and happily install a modern version of NumPy for you.
And, because /Library/Python/2.7/site-packages comes before /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python on your sys,path, the new NumPy will hide the ancient NumPy, and everything will just work as intended.
There can be a few problems with this. Most notably, if you try to install something which isn't included in Extras itself, but which has a dependency that is included in Extras, it may fail with mysterious and hard-to-debug errors. For example, on macOS 10.12, pip install pandas will throw a bunch of errors at you about not being able to upgrade dateutil, which you didn't even know you were trying to do. The only thing you can do is look at the dependencies for pandas, see which ones are pre-installed in Extras, and manually pip install shadowing versions of all of them.
But, for the most part, it works.

Cannot uninstall 'numpy'

I've been trying to install astropy and at the end of the installation I get this message:
Cannot uninstall 'numpy'. It is a distutils installed project and thus
we cannot accurately determine which files belong to it which would
lead to only a partial uninstall.
I have tried: pip uninstall numpy and then I get the same message.
I have Python 2.7.10 in a macOS High Sierra version 13.10.5

This doesn't directly answer your question, but that's because you're asking the wrong question.
Astropy requires Python 3.5 or 3.6. Trying to get it working with Apple's pre-installed Python 2.7 is a waste of time. You might be able to get an old version working this way, but not by using the installation instructions on astropy.org, and it won't be supported even if you do.
The easy solution is to just Install the latest Anaconda 5.x with Python 3.6, because it comes with Astropy built in.
The almost-as-easy solution is to install Python 3.6 from either a python.org binary installer, or Homebrew, and then use pip3 or, better, python3 -m pip to install everything, as explained on the Astropy install page.
Either way, before doing anything else, you want to get back to a clean slate. In particular, you do not want pip, or any other scripts, attached to Apple's Python 2.7; they will only cause confusion. Assuming you can't reinstall macOS from scratch, the best way to do this is:
Look in /Library/Python/2.7/site-packages and delete everything there except for README and Extras.pth.
Look in /usr/local/bin for symlinks to anything in that site-packages. (If you don't know much about using Unix, try this command: ls -l /usr/local/bin | grep 2.7.) You'll probably have pip and pip2.7 here, and probably nothing else. But whatever you have here, delete it.
Now, when you install Python 3.6, the only thing named pip anywhere will be that Python 3.6's pip. You still want to use pip3 or python3 -m pip, but if you screw up and type pip by accident, it won't break anything.
Also, you should strongly consider using a virtual environment. See the Python Packaging Authority's User Guide (or the Anaconda docs, if you went that way) for more on this.

One simple solution I found:
sudo -H pip install astropy --ignore-installed numpy

I also had this issue and couldn't get around it in a clean way, but this is what I did:
Inside the Lib folder search "numpy" for an egg_info file (eg. numpy-1.11.0.dev0_2329eae.egg-info).
In my case, this is what Pip was looking at to determine if a current version already exists. I deleted it, then ran normal
pip install numpy
and installed the newest version.
I don't recommend this because I don't understand what it's doing under the hood and it doesn't properly uninstall the old version which could be a recipe for trouble down the line, but if you're desperate like I was then maybe this is a solution for you.

How to install NumPy for IronPython in 2017?

I have found the old answer to this question, but the instructions provided in the mentioned post are no longer working... Essentially the URL where NumPy for IronPython was stored is broken.
How to install NumPy for IronPython in 2015/2016

You can download the required wheel file from https://pypi.org/project/numpy.
Update the pip to the latest version based on your OS.
If you are unable to use pip, please add pip to your PATH by editing
environment variables.
pip install --upgrade pip
After upgrading pip, you can use pip to install the local *.whl file you downloaded
pip install C:\path\to\whl_file.whl
For more customized install, you can look here: https://pip.pypa.io/en/stable/

Paste this in to your terminal
pip install numpy
If it say: pip is not recognized as an internal or external command, then try reinstalling Python and remember to add it to PATH (Easiest step. There is more ways to fix this but
this is the easiest one).
Then, it will work perfectly :)

pip says modules "weren't found" to uninstall, but pip list shows them

With pip list I'm shown a listing of the installed modules, which includes scipy, numpy, and pandas.
If I pip install -U... any of those, it checks it, sees its up-to-date, and returns to prompt. But if I pip uninstall pandas or any of the items I can see in the list, it says
Can't uninstall numpy. No files were found to uninstall
I installed everything that has to do with Python through Enthought's Canopy Stack.

Apparently there is some extra care required when installing/package managing with Canopy - for example, devs/staff # Enthought say (eg https://support.enthought.com/entries/22914233-Using-non-EPD-package-installers-such-as-pip)
"The best practice, when you wish to install an EPD package from another source, is first to uninstall it (enpkg --remove some_package), then to install it using whatever technique you prefer" such as (pip install some_package)
It's worth noting that the 'sklearn' module is letting me import datasets after having used the Canopy Package Manager to update (33) out-of-date packages. NOTE: THESE WERE PACKAGES PIP SAID WERE UP-TO-DATE!
To sum up: if you installed Canopy, use the package manager (IMHO, always, even though there is some work-around listed in the link above) to keep updated and then work from the prompt if you choose. FOR SOME REASON PIP CANNOT SEE (?) THE PACKAGES BEING MANAGED BY CANOPY!!!

Why is pip installing an old version of my package?

I've just uploaded a new version of my package to PyPi (1.2.1.0-r4): I can download the egg file and install it with easy_install, and the version checks out correctly. But when I try to install using pip, it installs version 1.1.0.0 instead. Even if I explicitly specify the version to pip with pip install -Iv tome==1.2.1.0-r4, I get this message: Requested tome==1.2.1.0-r4, but installing version 1.1.0.0, but I don't understand why.
I double checked with parse_version and confirmed that the version string on 1.2.1 is greater than that on 1.1.0 as shown:
>>> from pkg_resources import parse_version as pv
>>> pv('1.1.0.0') < pv('1.2.1.0-r4')
True
>>>
So any idea why it's choosing to install 1.1.0 instead?

This is an excellent question. It took me forever to figure out. This is the solution that works for me:
Apparently, if pip can find a local version of the package, pip will prefer the local versions to remote ones. I even disconnected my computer from the internet and tried it again -- when pip still installed the package successfully, and didn't even complain, the source was obviously local.
The really confusing part, in my case, was that pip found the newer versions on pypi, reported them, and then went ahead and re-installed the older version anyway ... arggh. Also, it didn't tell me what it was doing, and why.
So how did I solve this problem?
You can get pip to give verbose output using the -v flag ... but one isn't enough. I RTFM-ed the help, which said you can do -v multiple times, up to 3x, for more verbose output. So I did:
pip install -vvv <my_package>
Then I looked through the output. One line caught my eye:
Source in /tmp/pip-build-root/ has version 0.0.11, which satisfies requirement <my_package>
I deleted that directory, after which pip installed the newest version from pypi.

Try forcing download the package again with:
pip install --no-cache-dir --upgrade <package>

Thanks to Marcus Smith, who does amazing work as a maintener of pip, this was fixed in version 1.4 of pip which was released on 2013-07-23.
Relevant information from the changelog for this version
Fixed a number of issues (#413, #709, #634, #602, and #939) related to
cleaning up and not reusing build directories. (Pull #865, #948)

I found here that there is a known bug in pip that it won't check the version if there's a build directory with unpacked sources. I have checked this on my troubling package and after deleting its sources from build directory pip installed the required version.

If you are using a pip version that comes with some distribution packages (ex. Ubuntu python-pip), you may need to install a newer pip version:
Update pip to latest version:
sudo pip install -U pip
In case of "virtualenv", skip "sudo":
pip install -U pip
Following command may be required, if your shell report something like -bash: /usr/bin/pip: No such file or directory after pip update:
hash -d pip
Now install your package as usual:
pip install -U foo
or
pip install foo==package.version.here

Got the same issue to update pika 0.9.5 to 0.9.8. The only working way was to install from tarball: pip install https://pypi.python.org/packages/source/p/pika/pika-0.9.8.tar.gz.

In my case the python version used (3.4) didn't satisfy Django 2.1 dependencies requirements (python >= 3.5).

For my case I had to delete the .pip folder in my home directory and then I was able to get later versions of multiple libraries. Note that this was on linux.
pip --version
pip 18.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
virtualenv --version
15.1.0

Just in case that anyone else hassles with upgrading torchtext (or probably any other torch library):
Although https://pypi.org/project/torchtext/ states that you could run pip install torchtext I had to install it similiar to torch by specifying --find-links aka -f:
pip install torchtext===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
What irritated me was that PyCharm pointed me to the new version, but couldn't find it when attempting to upgrade to it. I guess that PyCharm uses its own mechanism to spot new versions. Then, when invoking pip under the hood, it didn't find the new version without the --find-links option.

In my case I am pip installing a .tar.gz package from Artifactory that I make a lot of updates to. In order to overwrite my cached Python files and always grab/install the latest I was able to run:
pip install --no-cache-dir --force-reinstall <path/to/tar.gz>
You should see this re-download any necessary files and install those, instead of using your local cache.

10 years on and pip still fails to work as expected 😖.
I wasted a couple of hours now banging my head against the wall trying to find out why pip won't install a development version of my package. In my case, there are versions 0.0.4 and 0.0.5.dev1 in a private gitlab.com package registry (hence the --extra-index-url argument below), but I believe that's not relevant to the problem.
Following a lot of the advice on this page, I create a test venv in a far away folder, clear the pip cache, uninstall the package in question, etc. first to rule out the most common problems:
$ pip cache purge && \
pip uninstall --yes my-package && \
pip install --extra-index-url "https://_:${GITLAB_PASSWORD_TOOLS_VAULTTOOLS}#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple" \
--no-cache-dir \
--pre \
--upgrade my-package
output (using empty lines to separate output for commands):
WARNING: No matching packages
Files removed: 0
Found existing installation: my-package 0.0.4
Uninstalling my-package-0.0.4:
Successfully uninstalled my-package-0.0.4
Looking in indexes: https://pypi.org/simple, https://_:****#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple
Collecting my-package
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/f07 ... 397/my_package-0.0.5.dev1-py3-none-any.whl (16 kB)
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/775 ... 70e/my_package-0.0.4-py3-none-any.whl (16 kB)
...
Successfully installed my-package-0.0.4
So pip does see the dev package version, but chooses the earlier one nonetheless.
In an attempt to figure out what's going on, I published a 0.0.5 version: Error persists, pip sees all three versions, but still installs 0.0.4.
In a further, increasingly desperate attempt, I removed any versions prior to 0.0.5* from the gitlab.com package registry.
Only now, pip would bother to actually display some useful information:
$ (same command as above)
... (similar output as above) ...
ERROR: Cannot install my-package==0.0.5 and my-package==0.0.5.dev1 because these package versions have conflicting dependencies.
The conflict is caused by:
my-package 0.0.5 depends on my-other-package<0.2.5 and >=0.2.4
my-package 0.0.5.dev1 depends on my-other-package<0.2.5 and >=0.2.4
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
OK, so there is something wrong with my package dependencies. Thanks for letting me know.
Seriously - I tried hard for a couple of hours using all kinds of pip ... -vvv and/or fixed versions such as e.g. my-package==0.0.5.dev1 - but I did not manage to get any useful output out of pip - until I wiped the entire history from my package registry 🤬.
Hope this at least helps someone in the same situation.

I found that if you use microversions, pip doesn't seem to recognize them. For example, we couldn't get version 1.9.9.1 to upgrade.

In my case, someone had published the latest version of a package with python2, so attempting to pip3 install it grabbed an older version that had been built with python3.
Handy things to check when debugging this:
If pip install claims to not be able to find the version, see whether pip search can see it.
Take a look at the "Download Files" section on the pypi repo -- the filenames might suggest what's wrong (in my case i saw -py2- there clear as day).
As suggested by others, try running pip install --no-cache-dir in case pip isn't bothering to ask the internet because it already has your answer locally.

I had hidden unversioned files under the Git tab in PyCharm that were being installed with pip install . even though I didn't see the files anywhere else.
Took a long time to find it for me, posting this in hope that it'll help somebody else.

if you need the path for your package do pip -v list. Example see related post when using pip -e Why is an old version of a package of my python library installing by itself with pip -e?