Correspondence between Python and pip versions - python

I have to install Tensorflow and the object detection API on a Jetson NANO.
I generally manage everything with virtual environments, but for a bunch of reasons this time I had to install everything in the base environment, and I assume I might've messed
something up.
The process starts, but it's getting stuck on solving some dependencies issues, and after running for the whole night it returns an error regarding the space left on the device (I assume it's downloading multiple versions of different packages and running out of space). So, I'm wondering if I'm installing them correctly.
I tried running the command:
python3 -m pip3 install --use-feature=2020-resolver .
but I'm getting the error:
/usr/bin/python3: No module named pip3
If I run simply
python3 -m pip install --use-feature=2020-resolver .
it goes, but as I was saying it gets stuck, so I was wondering if it is installing the files in the right locations.
I tried checking the outputs of:
which python3
which pip3
and I got
/usr/bin/python3
/usr/local/bin/pip3
So, the command pip3 does exist, and refers to /usr/local/bin/pip3...at this point, I don't understand why it returns the error when trying to use python3 -m pip3 install.
As a matter of fact, I used pip3 for all the previous steps in the object detection API installation, like:
sudo pip3 install -U pip testresources setuptools==49.6.0
sudo pip3 install -U numpy==1.16.1 future==0.18.2 mock==3.0.5 h5py==2.10.0 keras_preprocessing==1.1.1 keras_applications==1.0.8 gast==0.2.2 futures protobuf pybind11
sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v45 tensorflow
(I generally DON'T use sudo with pip, just following the official instructions for the Jetson NANO)

Related

Need to install Python Pint Package using MSYS

I am very new to using MSYS and I need it to run a Fortran based code through python interface. I followed the whole installation procedure in this link
https://github.com/SINTEF/thermopack/blob/main/addon/pycThermopack/README.md
and when I tried to install Python Pint package I kept getting this error:
MSYS Error
It seems that this package is very important to run the GUI correctly. What did I miss here? Is there any other way to install this package?
Thanks a lot in advance.
Solution
I managed to install it by installing Pip command using
$ pacman -S python3-pip
$ pip3 install --upgrade pip
pip install Pint

PyTorch Installation - PiP - Erorr

When trying to install PyTorch, by entering the command:
pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio===0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
I get the following Error
Error - "The Package index page used does not have a proper HTML doctype declaration."
it also says to contact the owner of the package index, to get this fixed. https://github.com/pypa/pip/issues/10825
Any solutions for this? I want to install PyTorch
EDIT: I downloaded the latest version of pip, pip 22.0.3
Seems like it's an ongoing issue with pip==22.0.{0,1,2,3} (confirm your pip version using pip3 --version or pip --version). They started a migration process to remove an HTML parser 1, but it seems like PyTorch didn't notice and now they're trying to solve it (this is the GitHub issue where they're tracking the progress on this matter).
At the moment, it seems like you have these options:
Try to use the pip install command by adding this flag: --use-deprecated=html5lib. For example:
pip3 install --use-deprecated=html5lib torch==1.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
Downgrade pip to a "non-22.0.*" version. It seems like the recommended way is to use a version that comes with your Python version 2:
python -m pip uninstall pip && python -m ensurepip
Then, use the pip install ... command normally.
Install PyTorch using a different technique.

pip3 SSL Error while installing new package

I am using Kali Linux, and I am trying to install some requirements of one packet for test purposes, but I keep getting the below error. I have tried many solutions but they are all not working and the error still persists
# pip3 install -r requirements.txt
or
pip3 install name_of_package-py
Generally, the only command you would need is:
pip install name_of_package
if it doesn't work you could try:
sudo pip install name_of_package
sudo should work, but you have to be careful regarding the package because with sudo you could end up installing a package with some malware in it. Therefore be certain of the reliability of the package.
also depending on the version you are using could be pip or pip3

pip 10 and apt: how to avoid "Cannot uninstall X" errors for distutils packages

I am dealing with a legacy Dockerfile. Here is a very simplified version of what I am dealing with:
FROM ubuntu:14.04
RUN apt-get -y update && apt-get -y install \
python-pip \
python-numpy # ...and many other packages
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt # includes e.g., numpy==1.13.0
RUN pip install -r /tmp/requirements2.txt
RUN pip install -r /tmp/requirements3.txt
First, several packages are installed using apt, and then several packages are installed using pip. pip version 10 has been released, and part of the release is this new restriction:
Removed support for uninstalling projects which have been installed using distutils. distutils installed projects do not include metadata indicating what files belong to that install and thus it is impossible to actually uninstall them rather than just remove the metadata saying they've been installed while leaving all of the actual files behind.
This leads to the following problem in my setup. For example, first apt installs python-numpy. Later pip tries to install a newer version of numpy from e.g., /tmp/requirements1.txt, and tries to uninstall the older version, but because of the new restriction, it cannot remove this version:
Installing collected packages: numpy
Found existing installation: numpy 1.8.2
Cannot uninstall 'numpy'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Now I know at this point there are several solutions.
I could not install python-numpy through apt. However, this causes issues because python-numpy installs a few different packages as requirements, and I do not know if another part of the system relies on these packages. And in reality, there are several apt packages installed through the Dockerfile, and each one I remove seems to reveal another Cannot uninstall X error, and removes a number of other packages along with it, that our app may or may not rely on.
I could also use the --ignore-installed option when I try to pip install things that have already been installed through apt, but then again I have the same problem of every --ignore-installed argument revealing yet another thing that needs to be ignored.
I could pin pip at an older version that does not have this restriction, but I don't want to be stuck using an outdated version of pip forever.
I have been going around in circles trying to come up with a good solution that involves minimal changes to this legacy Dockerfile, and allows the app we deploy with that file to continue to function as it has been. Any suggestions as to how I can safely get around this problem of pip 10 not being able to install newer versions of distutils packages? Thank you!
UPDATE:
I did not realize that --ignore-installed could be used without a package as an argument to ignore all installed packages. I am considering whether or not this might be a good option for me, and have asked about it here.
This is the solution I ended up going with, and our apps have been running in production without any issues for close to a month with this fix in place:
All I had to do was to add
--ignore-installed
to the pip install lines in my dockerfile that were raising errors. Using the same dockerfile example from my original question, the fixed dockerfile would look something like:
FROM ubuntu:14.04
RUN apt-get -y update && apt-get -y install \
python-pip \
python-numpy # ...and many other packages
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt --ignore-installed # don't try to uninstall existing packages, e.g., numpy
RUN pip install -r /tmp/requirements2.txt
RUN pip install -r /tmp/requirements3.txt
The documentation I could find for --ignore-installed was unclear in my opinion (pip install --help simply says "Ignore the installed packages (reinstalling instead)."), and I asked about the potential dangers of this flag here, but have yet to get satisfying answer. However, if there are any negative side effects, our production environment has yet to see the effects of them, and I think the risk is low/none (at least that has been our experience). I was able to confirm that in our case, when this flag was used, the existing installation was not uninstalled, but that the newer installation was always used.
Update:
I wanted to highlight this answer by #ivan_pozdeev. He provides some information that this answer does not include, and he also outlines some potential side-effects of my solution.
This is what worked for me--
pip install --ignore-installed <Your package name>
or
sudo pip install --ignore-installed <Your package name>
or (inside jupyter notebook)
import sys
!{sys.executable} -m pip install --ignore-installed <Your package name>
For windows
write
conda update --all
pip install --upgrade <Your package name>
OR
conda update --all
pip install <Your package name>
OR
pip install wrapt --upgrade --ignore-installed
pip install <Your package name>
from ERROR: Cannot uninstall 'wrapt'. during upgrade
You can just remove numpy manually but keep the other dependencies installed by apt. Then use pip as before to install the latest version of numpy.
#Manually remove just numpy installed by distutils
RUN rm /usr/lib/python2.7/dist-packages/numpy-1.8.2.egg-info
RUN rm -r /usr/lib/python2.7/dist-packages/numpy
RUN pip install -U pip
RUN pip install -r /tmp/requirements1.txt
The location of numpy should be the same. But if you want to confirm the location you can run the container without running the requirements.txt files and issue the following commands in the python console inside the container.
>>> import numpy
>>> print numpy.__file__
/usr/lib/python2.7/dist-packages/numpy/__init__.pyc

Python on anaconda cannot find azure.mgmt.datafactory

I am trying to run this tutorial
https://learn.microsoft.com/en-US/azure/data-factory/quickstart-create-data-factory-python
but I fail to install the packages. I tried several installations but I keep getting the error No module named 'azure.mgmt.datafactory' when trying to run from azure.mgmt.datafactory import DataFactoryManagementClient.
I am using anaconda and windows 10.
I tried running the recommended anaconda packages https://anaconda.org/anaconda/azure and https://anaconda.org/clinicalgraphics/azure-mgmt-resource under a python 3.5 environment and I also tried to manually install everything from github (https://github.com/Azure/azure-sdk-for-python) using
git clone git://github.com/Azure/azure-sdk-for-python.git
cd azure-sdk-for-python
python setup.py install
In both the normal (Python 3.6) and the new (Python 3.5, using Anaconda version with Python 3.5) environment. None of this worked.
What am I missing?
(Note that from azure.mgmt.resource import ResourceManagementClient worked fine with the anaconda installation)
EDIT
After the first response, I ran the following commands from the powershell
pip install azure-mgmt-resource
pip install azure-mgmt-datafactory
pip install azure-mgmt
which resulted in ModuleNotFoundError: No module named 'azure.mgmt'
Uninstalling the three packages and installing azure-mgmt as a first one did not solve the issue either. However, I don't know how to uninstall the manually installed package from python setup.py install, which still might be an issue.
Have you tried pip install in powershell/cmd?
pip install azure-mgmt-datafactory
Update (Jan's answer):
pip freeze > requirements.txt
pip uninstall -r requirements.txt
python -m pip install azure-common
python -m pip install azure-mgmt
python -m pip install azure-mgmt-datafactory (this might not be needed as it comes with azure-mgmt)
Ok, this is how I got the required azure libraries to work (thx to Saul Cruy, who gave me the idea)
Using this post What is the easiest way to remove all packages installed by pip?, I created a requirements file in PowerShell
pip freeze > requirements.txt
In this file, I manually kept only the entries with azure.
Then, I deleted all packages in the file
pip uninstall -r requirements.txt
The steps above were repeated twice, as upon first delete, some azure packages survived.
Then, I ran (all in PowerShell, in that order)
python -m pip install azure-common
python -m pip install azure-mgmt
python -m pip install azure-mgmt-datafactory
The reason might(!) be that installing packages in the anaconda console using the conda commands causes confusion in the dependencies (I tried a similar approach in a conda environment as it seemed like a good idea to seperate the azure packages from the other ones, but without success).

Categories

Resources