I created and maintain a package seg-metrics. But I found that I can not install all of its dependencies when I use pip install seg-metrics to install it, even though I have already included all necessary dependencies into the requirements.txt.
Can you tell me why and how to solve it?
You can try to reproduce it into your own pc.
conda create -n test_env python=3.8 pip # install pip as well
pip install seg-metrics # install the package
pip list # shown the installed package, you can see seg-metrics, but you can not see the dependencies like: SimpleITK and medutils. This can be verified by the following commands.
python # enter the python interpreter
import seg_metrics.seg_metrics as sm # Error occured here !!!
Thanks to #phd, I found that I should set install_requires in setup.py. After that, this issue was fixed.
Related
I know this question has been asked and answered a number of times but none of those solutions have worked for me. I have installed Python 2.7 into a local directory and added it to my path. When I try to install numpy i get the following error:
ImportError: No module named setuptools
I cannot simply sudo apt-get install python-setuptools because i don't have root access.
I need to install numpy and ideally have pip working for future applications.
0) Try to install packages that are isolated to the current user, use the --user flag:
pip install --user SomeProject
1a) I agree with #Pi Marillion here, use an isolated conda environment if you don't have root access. This way you keep your path clean.
To install conda:
Since I don't know about your OS, go to https://docs.conda.io/en/latest/miniconda.html
After installation, update your conda (just in case):
conda update conda
To list the installed packages, you can do
conda list
You should see python installed. you can start an interpreter by typing python in the terminal.
There's conda cheat sheet that I found incredibly helpful:
https://docs.conda.io/projects/conda/en/latest/user-guide/cheatsheet.html
b) Now try installing via pip and I think you might need python 3.x for setuptools.
https://packaging.python.org/tutorials/installing-packages/#id13
2) If this does not work you can still try
https://packaging.python.org/guides/installing-stand-alone-command-line-tools/
Hope this helps :)
First try easy_install --user setuptools pip. If that doesn't work you need to install things manually.
Download setuptools-*.zip from https://pypi.org/project/setuptools/#files. Unzip the archive, cd into the new directory and run python2.7 setup.py install.
Then try pip install. If it still doesn't work reinstall pip: download get-pip.py and run python get-pip.py --user.
I am dealing with a legacy Dockerfile. Here is a very simplified version of what I am dealing with:
FROM ubuntu:14.04
RUN apt-get -y update && apt-get -y install \
python-pip \
python-numpy # ...and many other packages
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt # includes e.g., numpy==1.13.0
RUN pip install -r /tmp/requirements2.txt
RUN pip install -r /tmp/requirements3.txt
First, several packages are installed using apt, and then several packages are installed using pip. pip version 10 has been released, and part of the release is this new restriction:
Removed support for uninstalling projects which have been installed using distutils. distutils installed projects do not include metadata indicating what files belong to that install and thus it is impossible to actually uninstall them rather than just remove the metadata saying they've been installed while leaving all of the actual files behind.
This leads to the following problem in my setup. For example, first apt installs python-numpy. Later pip tries to install a newer version of numpy from e.g., /tmp/requirements1.txt, and tries to uninstall the older version, but because of the new restriction, it cannot remove this version:
Installing collected packages: numpy
Found existing installation: numpy 1.8.2
Cannot uninstall 'numpy'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Now I know at this point there are several solutions.
I could not install python-numpy through apt. However, this causes issues because python-numpy installs a few different packages as requirements, and I do not know if another part of the system relies on these packages. And in reality, there are several apt packages installed through the Dockerfile, and each one I remove seems to reveal another Cannot uninstall X error, and removes a number of other packages along with it, that our app may or may not rely on.
I could also use the --ignore-installed option when I try to pip install things that have already been installed through apt, but then again I have the same problem of every --ignore-installed argument revealing yet another thing that needs to be ignored.
I could pin pip at an older version that does not have this restriction, but I don't want to be stuck using an outdated version of pip forever.
I have been going around in circles trying to come up with a good solution that involves minimal changes to this legacy Dockerfile, and allows the app we deploy with that file to continue to function as it has been. Any suggestions as to how I can safely get around this problem of pip 10 not being able to install newer versions of distutils packages? Thank you!
UPDATE:
I did not realize that --ignore-installed could be used without a package as an argument to ignore all installed packages. I am considering whether or not this might be a good option for me, and have asked about it here.
This is the solution I ended up going with, and our apps have been running in production without any issues for close to a month with this fix in place:
All I had to do was to add
--ignore-installed
to the pip install lines in my dockerfile that were raising errors. Using the same dockerfile example from my original question, the fixed dockerfile would look something like:
FROM ubuntu:14.04
RUN apt-get -y update && apt-get -y install \
python-pip \
python-numpy # ...and many other packages
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt --ignore-installed # don't try to uninstall existing packages, e.g., numpy
RUN pip install -r /tmp/requirements2.txt
RUN pip install -r /tmp/requirements3.txt
The documentation I could find for --ignore-installed was unclear in my opinion (pip install --help simply says "Ignore the installed packages (reinstalling instead)."), and I asked about the potential dangers of this flag here, but have yet to get satisfying answer. However, if there are any negative side effects, our production environment has yet to see the effects of them, and I think the risk is low/none (at least that has been our experience). I was able to confirm that in our case, when this flag was used, the existing installation was not uninstalled, but that the newer installation was always used.
Update:
I wanted to highlight this answer by #ivan_pozdeev. He provides some information that this answer does not include, and he also outlines some potential side-effects of my solution.
This is what worked for me--
pip install --ignore-installed <Your package name>
or
sudo pip install --ignore-installed <Your package name>
or (inside jupyter notebook)
import sys
!{sys.executable} -m pip install --ignore-installed <Your package name>
For windows
write
conda update --all
pip install --upgrade <Your package name>
OR
conda update --all
pip install <Your package name>
OR
pip install wrapt --upgrade --ignore-installed
pip install <Your package name>
from ERROR: Cannot uninstall 'wrapt'. during upgrade
You can just remove numpy manually but keep the other dependencies installed by apt. Then use pip as before to install the latest version of numpy.
#Manually remove just numpy installed by distutils
RUN rm /usr/lib/python2.7/dist-packages/numpy-1.8.2.egg-info
RUN rm -r /usr/lib/python2.7/dist-packages/numpy
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt
The location of numpy should be the same. But if you want to confirm the location you can run the container without running the requirements.txt files and issue the following commands in the python console inside the container.
>>> import numpy
>>> print numpy.__file__
/usr/lib/python2.7/dist-packages/numpy/__init__.pyc
I am trying to run this tutorial
https://learn.microsoft.com/en-US/azure/data-factory/quickstart-create-data-factory-python
but I fail to install the packages. I tried several installations but I keep getting the error No module named 'azure.mgmt.datafactory' when trying to run from azure.mgmt.datafactory import DataFactoryManagementClient.
I am using anaconda and windows 10.
I tried running the recommended anaconda packages https://anaconda.org/anaconda/azure and https://anaconda.org/clinicalgraphics/azure-mgmt-resource under a python 3.5 environment and I also tried to manually install everything from github (https://github.com/Azure/azure-sdk-for-python) using
git clone git://github.com/Azure/azure-sdk-for-python.git
cd azure-sdk-for-python
python setup.py install
In both the normal (Python 3.6) and the new (Python 3.5, using Anaconda version with Python 3.5) environment. None of this worked.
What am I missing?
(Note that from azure.mgmt.resource import ResourceManagementClient worked fine with the anaconda installation)
EDIT
After the first response, I ran the following commands from the powershell
pip install azure-mgmt-resource
pip install azure-mgmt-datafactory
pip install azure-mgmt
which resulted in ModuleNotFoundError: No module named 'azure.mgmt'
Uninstalling the three packages and installing azure-mgmt as a first one did not solve the issue either. However, I don't know how to uninstall the manually installed package from python setup.py install, which still might be an issue.
Have you tried pip install in powershell/cmd?
pip install azure-mgmt-datafactory
Update (Jan's answer):
pip freeze > requirements.txt
pip uninstall -r requirements.txt
python -m pip install azure-common
python -m pip install azure-mgmt
python -m pip install azure-mgmt-datafactory (this might not be needed as it comes with azure-mgmt)
Ok, this is how I got the required azure libraries to work (thx to Saul Cruy, who gave me the idea)
Using this post What is the easiest way to remove all packages installed by pip?, I created a requirements file in PowerShell
pip freeze > requirements.txt
In this file, I manually kept only the entries with azure.
Then, I deleted all packages in the file
pip uninstall -r requirements.txt
The steps above were repeated twice, as upon first delete, some azure packages survived.
Then, I ran (all in PowerShell, in that order)
python -m pip install azure-common
python -m pip install azure-mgmt
python -m pip install azure-mgmt-datafactory
The reason might(!) be that installing packages in the anaconda console using the conda commands causes confusion in the dependencies (I tried a similar approach in a conda environment as it seemed like a good idea to seperate the azure packages from the other ones, but without success).
When I run pip install django-pgjsonb as a regular user it fails with setuptools must be installed to install from a source distribution.
But setuptools is installed as can be seen from the fact that pip list includes the following packages:
Django (1.8.4)
pip (7.1.2)
setuptools (18.3.2)
wheel (0.24.0
Any ideas what the error is?
UPDATE I cannot reproduce the problem in a fresh venv. So I'm just recreating a venv with the same stuff.
However I'm leaving the question open in case anyone has anything to fix.
Once you have activated a virtual environment try the following. It worked for me.
pip install --user -Iv django-pgjsonb
I have dutifully uninstalled all the Python packages I installed with sudo pip install and installed them with pip --user install instead. Yay me :)
On Ubuntu, I know I can find the relevant binaries at /home/<USERNAME>/.local/bin and the packages themselves at /home/<USERNAME>/.local/lib/python2.7/site-packages ... but navigating there is not as simple as good old pip freeze.
How can I pip freeze and get only the packages I installed with pip --user install rather than all the Python packages, including those installed via apt?
Currently pip does not have any such options. So with default pip its not possible. (and I submitted a feature request and now there is a working PR too!)
However I wrote a little script, which does solve your problem:
# pip_user_installs.py
import sys
import pkg_resources
for module in pkg_resources.working_set:
if sys.argv[1] in module.location:
print module.project_name
usage:
$ python pip_user_installs.py $HOME
It's fairly easy in recent versions of pip (the PR in the other answer is now part of pip).
pip freeze --user
This will output a list of packages currently installed to the user's site-packages.