When I am trying to install some packages with pip, sometimes some dependency fails to install, and I need to sort out why. A fundamental question is "why did I need to install package X at all?", but I cannot find any way to answer this from the pip install output, even with -vvv. Pip tells me what it is installing, but it doesn't say why it is doing it.
There are various tools to introspect dependency trees of packages, e.g. pipdeptree, especially when they are already installed, but these don't help me when the installation has failed. And it seems that internally pip must already have solved these dependencies and know why it has chosen to install a particular package. So how can I get it to share this information with me at install time?
Edit: It already shows this information when telling you what dependencies are already satisfied, e.g.
Requirement already satisfied: pillow>=6.2.0 in /data2/users/bfarmer/envs/bfarmer_dev_py 39/lib/python3.9/site-packages (from matplotlib>=1.3.1->stf-modelling) (8.0.1)
Requirement already satisfied: certifi>=2020.06.20 in /data2/users/bfarmer/envs/bfarmer_ dev_py39/lib/python3.9/site-packages (from matplotlib>=1.3.1->stf-modelling) (2020.12.5)
(at least I assume that it is what it is telling me at the end of those lines). But when I need this information the most, i.e. when something goes wrong, I get nothing:
Collecting PIL
Downloading http://ehp.bom.gov.au/ehprepo/pypi/simple/pil/PIL-1.1.7.tar.gz (506 kB)
|████████████████████████████████| 506 kB 8.1 MB/s
ERROR: Command errored out with exit status 1:
...
<traceback etc. follows>
In this example I am left wondering why the heck some package wanted PIL when pillow is already there. PIL is basically dead, so I need to update whatever package has a dependency on PIL to use pillow instead. But I have no idea what package that might be, and cannot figure out any way to find out. This seems like basic information, there must surely be a way to get it.
It kind of seems like no, pip cannot do this. I found this issue about it here:
https://github.com/pypa/pip/issues/53
It sounds like they are working on it, but nothing exists currently. I am still interested in workaround/third party solutions though, or advice from other developers about what they typically do in this situation. It kind of seems like I just have to manually trawl through all the dependencies of my dependencies, which just seems stupid. Maybe I can hack something in to the pip source...
Related
I've been trying to install IMGAUG package for an ML project. But the installation gets stuck when it tries to install scikit-image
My input: pip install scikit-image
output:
Collecting imgaug
Using cached
https://files.pythonhosted.org/...
Requirement already satisfied: scipy in
c:\users\*<username>*\appdata\local\programs\python\python37\lib\site-
packages (from imgaug) (1.1.0)
Collecting scikit-image>=0.11.0 (from imgaug)
Using cached https://files.pythonhosted.org/packages/...
Complete output from command python setup.py egg_info:
----------------------------------------
Command "python setup.py egg_info" failed with error code 3221225477 in
C:\Users\<name>~1.<name2>\AppData\Local\Temp\pip-install-qmdp6ysz\scikit-image\
Note: I've already tried installing other versions of it, upgrading setuptools and pip. Error persists.
PS: Now it's showing up on everything I try to install.
(Scroll down to a horizontal line to skip explanation and go straight to the suggested solution if you wish)
3221225477 is 0xC0000005 which is NTSTATUS STATUS_ACCESS_VIOLATION; the corresponsing error message is The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s..
In Windows, a process usually quits with this exit code if it tries to access an invalid memory address and Windows terminates it as a result. If you install Visual Studio, you'll be able to pinpoint the exact module at fault as shown on the link.
Now, this error means a bug in or an incompatibility between some of your installed extension modules (or in Python engine itself, but this is very unlikely in comparison).
The easiest way to fix is to clean up any problems with the involved modules' installation and (if that isn't enough) update them to the latest versions, hoping that whatever is causing that is fixed in them.
In particular, scipy in c:\users\*<username>*\appdata\local\programs\python\python37\lib\site-packages looks suspicious: you aren't using --user in your pip command
which suggests that your didn't pay attention to this flag when using pip before (it's official that it CAN lead to version conflicts), and some of your installed packages are installed both into %ProgramFiles%\Python37\Lib\site-packages and %APPDATA%\Python\Python37\ib\site-packages, with different versions in these two locations.
I hereby suggest you to:
decide where you want your 3rd-party modules to be
%ProgramFiles% is system-wide and requires elevation to manage, %APPDATA% is per-user and doesn't require elevation
Unless you don't have administrative rights at your machine (you do, judging by the command you gave) or have special needs, keep everything in the system-wide location for simplicity
uninstall all the modules in the other location (pip uninstall <name(s)> with or without --user)
reinstall them to the desired location, updating existing versions (-U pip flag)
if that wasn't enough to solve the problem (very unlikely), update all packages to the latest versions
This happened to me also. However I resolved it by uninstalling the package (pip uninstall ), then installing it using conda rather than pip (conda install ).
This question already has answers here:
Pip install from pypi works, but from testpypi fails (cannot find requirements)
(2 answers)
Closed 2 years ago.
TL;DR Even though I've specified dependencies with install_requires in setup.py, the install through pip fails because some dependencies can't be found.
I've developed a package which I intend to distribute via PyPi. I've created a built distribution wheel and uploaded it to testPyPI to see if everything is working with the upload and if the package can be installed from a user perspective.
However, when I try to pip install the package inside a vanilla python 2.7 environment, the installation process fails while installing the dependencies.
My package depends on these packages (which I added to the setup.py file accordingly):
...
install_requires=['numpy','gdal','h5py','beautifulsoup4','requests','tables','progress'],
...
So when I run pip install, everything looks normal for a moment, until I receive this error:
Could not find a version that satisfies the requirement progress (from #NAME#) (from versions: )
No matching distribution found for progress (from #NAME#)
When I remove the progress dependency (I could live without it), same thing happens for pytables:
Could not find a version that satisfies the requirement tables (from #NAME#) (from versions: )
No matching distribution found for tables (from #NAME#)
If I run pip install tables and pip install progress manually beforehand, everything works as expected.
So how can I assure that if someone downloads my package, all missing dependencies are installed with it?
Related bonus question:
Can I include a wheel file in my package (maybe through MANIFEST.in) and install it as dependency if the module is not available? If so, how?
And I think I've found the answer to my question myself.
When installing a package from testPyPI, the dependencies are also installed from there. And it seems, that while there are many packages available, pytables and progress are apparently missing. This caused the installation to fail.
Naturally, manually installing with pip install gets the package from the "normal" PyPi, which of course works. This obviously added to my confusion.
Here's a look at the output from pip install when installing the package from the testPyPi:
Downloading https://test-files.pythonhosted.org/packages/4f/96/b3329750a04fcfc316f15f658daf6d81acc3ac61e3db390abe8954574c18/nump
y-1.9.3.tar.gz (4.0MB)
while installing the wheel directly, it looks slightly different:
Downloading https://files.pythonhosted.org/packages/2e/91/504e434d3b95d943caab926f33dee5691768fbb622bc290a0fa6df77e1d8/numpy-1.1
4.2-cp27-none-win32.whl (9.8MB)
Additionally, running
pip install --index-url https://test.pypi.org/simple/ tables
produces the same error as described in my question.
I want to use networkx (python 2.7.10).
Therefore I have installed networkx with pip. But when I actually tried to import it, I got this error No module named networkx
When I check pip freeze, it return networkx==1.11
Although I tried to reinstall it, it returned
Requirement already satisfied: networkx in /usr/local/lib/python2.7/site-packages
Requirement already satisfied: decorator>=3.4.0 in /usr/local/lib/python2.7/site-packages (from networkx)
I can't figure out what I should do to get it to work.
My Mac version is 10.11.6
It would be greatly appreciated if you could explain the details.
I would really not mess with your systems python.
If you are going to be doing data science work. I would suggest you get a Python distribution like Anaconda and run it inside your home directory so you don't do something you will probably regret.
It has an easy installer. The key during the install to make this problem go away is to say "YES" to changing your path.
After that you will have a nice GUI tool for installing packages.
I referring url to profile data and to profile the data we need pyxplorer inside of python interpreter but when i try to install pyxplorer package it gives me error like:
Collecting pyxplorer
Could not find a version that satisfies the requirement pyxplorer (from versions: ) No matching distribution found for pyxplorer
command to install package is:
pip install pyxplorer
I know below link only about data profiling(pyxplorer)
1) https://github.com/grundprinzip/pyxplorer
2) http://nbviewer.jupyter.org/github/grundprinzip/pyxplorer/blob/master/pyxplorer_stuff.ipynb
The links which I have already tried are:
1)pip cannot install anything
2) Could not find any downloads that satisfy the requirement newrelic-plugin-agent
Thanks in advance.
It looks like the pyxplorer package on PyPI is invalid and doesn't actually contain any release data. Have a look at the releases key of the JSON for pyxplorer - it's an empty array, but normal packages look more like this.
The best solution would be to install directly from GitHub, like so:
pip install git+https://github.com/grundprinzip/pyxplorer
(You may need to use sudo on Unix-like systems or Run as Administrator on Windows)
It might also be wise to file an issue on the pyxplorer bug tracker so they know about this.
After hours of finding out why there are missing some documented functions in my Babel installation, I learned there are two branches of Babel development:
Babel has two separate development paths (0.9.x branch and trunk) in
parallel for about 4 years now despite very few developers working on
the project. We try to resolve that situation by releasing a stable
1.0 version but well, real live is not always friendly to open source contribution.
Babel's FAQ confirms that. I want to use Flask-Babel in my project. It's dependency in setup.py says I need just Babel. It means my pip takes any version installed in my environment or searches PyPI for the newest version, where is version 0.9.6. Unlogically, Flask-Babel uses functions which are not present in 0.9.x branch. Maybe I am missing something, maybe I am just confused, but how can I easily install the trunk version, where is the most of new features? And how can I enforce using such a version in my setup.py? How it all works for the people who use Flask-Babel? (I know, the last question is rather Flask-specific and should go here, but all the other questions can answer anyone else.)
Thank you for any suggestions. The bold questions are the most important, the rest is rather Flask-Babel-specific "nice to have".
Have you tried using pip with the url to the branch that you need?
$ sudo pip install http://svn.edgewall.org/repos/babel/trunk
After that, pip should be happy with the dependency:
$ sudo pip install Flask-Babel
...
Requirement already satisfied (use --upgrade to upgrade): Babel in /usr/local/lib/python2.7/dist-packages (from Flask-Babel)
...
Regarding how to force a dependency in you setup.py. Since you're already using pip, you can give a try to a requirements file.