I have created an Azure function and I am trying to build a Azure pipeline. The pipeline fails at install application dependencies with the below error.
ERROR: Cannot install -r requirements.txt (line 8) and azure-storage-blob==2.1.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested azure-storage-blob==2.1.0
azure-storage-file-datalake 12.7.0 depends on azure-storage-blob<13.0.0 and >=12.12.0
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict.
Below is the code in the install dependencies.
python -m venv worker_venv
source worker_venv/bin/activate
pip install setuptools
pip install -r requirements.txt
In my requirements.txt file the azure-storage-blob version is 2.1.0. Should I remove the version part in the requirements.txt file and trying running the pipeline. Is there any other way to fix this issue.
Thank you.
I just removed the version in the requirements.txt file and it was successful.
Should I remove the version part in the requirements.txt file and
trying running the pipeline. Is there any other way to fix this issue.
Yes, of course you have other way to fix this issue. But you need to make sure your code is backward compatible, if your code must be based on azure-storage-blob version 2.1.0, then you can't use azure-storage-file-datalake version 12.7.0.
If your code is compatible with 'azure-storage-blob<13.0.0 and >=12.12.0', then you can specify any version within this range in the requirements.txt file.
For example, you can do this:
requirements.txt
azure-functions
azure-storage-blob==12.12.0
azure-storage-file-datalake==12.7.0
Now your direct remove version is valid because the current latest version of azure-storage-blob is 12.12.0, which can meet the dependency requirements of azure-storage-file-datalake 12.7.0:
https://pypi.org/project/azure-storage-blob/12.12.0/#history
But if you use the 12.7.0 version of azure-storage-file-datalake, you will still have version conflicts in the near future. Because if you do not specify the version number, the pip tool will install the latest version by default. If the latest version of azure-storage-blob is greater than or equal to 13.0.0 in the future, you will encounter an error again.
I've been trying to install a package through pip on my rpi 3 model B
my operating system is raspbian. Debian based pip version is 21.0.1 and python version is 3.7.4
the command I'm using is:
python3 -m pip install librosa
the problem is that the dependency resolver takes way too long to resolve the conflicts.
and after a few hours, it keeps repeating this line over and over again for hours ( I even left the installation running for 2 days overnights )
INFO: pip is looking at multiple versions of <Python from requires-Python> to determine which version is compatible with other requirements. this could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run you can press ctrl + c to do so.
I've tried using a stricter constraint such as adding "numpy > 1.20.0" and other stuff but now the popped up and I have no clue what I can do now.
So as of pip 20.3, a new (not always working) resolver has been introduced. As of pip 21.0 the old (working) resolver is unsupported and slated for removal dependent on pip team resources.
Changes to the pip dependency resolver in 20.3
I have hit the same issue trying to build jupyter, my solution was to pin pip back to the 20.2 release which is the last release with the old resolver. This got past the point my builds were choking at using the new resolver under pip 21.1.1.
A second approach that might work (untested) is to use the flag:
--use-deprecated=legacy-resolver
which appears to have been added when 20.3 switched over to the new resolver. This would allow the benefits of newer pip releases, until the backtracking issue is resolved, assuming it works.
What is happening, according to the devs on this Github issue, is "pip downloads multiple versions [of each package] because the versions it downloaded conflict with other packages you specified, which is called backtracking and is a feature. The versions need to be downloaded to detect the conflicts." But it takes a very long time to download all of these versions. Pip explains this in detail, along with ways to resolve it or speed it up, at https://pip.pypa.io/en/latest/topics/dependency-resolution/.
If you run
pip install -r requirements.txt --use-deprecated=legacy-resolver
you will not get this backtracking behavior, but your install will complete, and you will see an error at the end that is useful for troubleshooting:
ERROR: pip's legacy dependency resolver does not consider dependency conflicts when selecting packages. This behaviour is the source of the following dependency conflicts.
apache-airflow-providers-amazon 2.6.0 requires boto3<1.19.0,>=1.15.0, but you'll have boto3 1.9.253 which is incompatible.
package_xyz 0.0.1 requires PyJWT==2.1.0, but you'll have pyjwt 1.7.1 which is incompatible.
Upgrading my pip to 21.3.1 worked
python.exe -m pip install --upgrade pip
I have two versions of my python build:
16.1206.43542
17.0817.221945+f4cc396
The only really difference I can see is the ending metadata. When I run pip install package, the version 16.1206.43542 is installed and not the latest. Is this the proper behavior? I would have thought pip would have honored the metadata, and installed the later package?
Thoughts? Ideas? Any would be welcomed. For transparency, I am adding the sha from a git build into the version.
I looked at this, and found that the correct answer is that anything following the normal Semantic Versioning, it labels that build as a pre-release build and must then be explicitly called to be installed, normally identified with a '-'. https://semver.org/ (Topic 9)
For pip install The version must be a NON pre-release build to be automatically installed.
I am managing several modules on an HPC, and want to install some requirements for a tool using pip.
I won't use virtualenv because they don't work well with our module system. I want to install module-local versions of packages and will set PYTHONPATH correctly when the module is loaded, and this has worked just fine when the packages I am installing are not also installed in the default python environment.
What I do not want to do is uninstall the default python's versions of packages while I am installing module-local versions.
For example, one package requires numpy==1.6, and the default version installed with the python I am using is 1.8.0. When I
pip install --install-option="--prefix=$RE_PYTHON" numpy==1.6
where RE_PYTHON points to the top of the module-local site-packages directory, numpy==1.6 installs fine, then pip goes ahead and starts uninstalling 1.8.0 from the tree of the python I am using (why it wants to uninstall a newer version is beyond me but I want to avoid this even when I am doing a local install of e.g. numpy==1.10.1).
How can I prevent pip from doing that? It is really annoying and I have not been able to find a solution that doesn't involve virtualenv.
You have to explicitly tell pip to ignore the current installed package by specifying the -I option (or --ignore-installed). So you should use:
PYTHONUSERBASE=$RE_PYTHON pip install -I --user numpy==1.6
This is mentioned in this answer by Ian Bicking.
I've just uploaded a new version of my package to PyPi (1.2.1.0-r4): I can download the egg file and install it with easy_install, and the version checks out correctly. But when I try to install using pip, it installs version 1.1.0.0 instead. Even if I explicitly specify the version to pip with pip install -Iv tome==1.2.1.0-r4, I get this message: Requested tome==1.2.1.0-r4, but installing version 1.1.0.0, but I don't understand why.
I double checked with parse_version and confirmed that the version string on 1.2.1 is greater than that on 1.1.0 as shown:
>>> from pkg_resources import parse_version as pv
>>> pv('1.1.0.0') < pv('1.2.1.0-r4')
True
>>>
So any idea why it's choosing to install 1.1.0 instead?
This is an excellent question. It took me forever to figure out. This is the solution that works for me:
Apparently, if pip can find a local version of the package, pip will prefer the local versions to remote ones. I even disconnected my computer from the internet and tried it again -- when pip still installed the package successfully, and didn't even complain, the source was obviously local.
The really confusing part, in my case, was that pip found the newer versions on pypi, reported them, and then went ahead and re-installed the older version anyway ... arggh. Also, it didn't tell me what it was doing, and why.
So how did I solve this problem?
You can get pip to give verbose output using the -v flag ... but one isn't enough. I RTFM-ed the help, which said you can do -v multiple times, up to 3x, for more verbose output. So I did:
pip install -vvv <my_package>
Then I looked through the output. One line caught my eye:
Source in /tmp/pip-build-root/ has version 0.0.11, which satisfies requirement <my_package>
I deleted that directory, after which pip installed the newest version from pypi.
Try forcing download the package again with:
pip install --no-cache-dir --upgrade <package>
Thanks to Marcus Smith, who does amazing work as a maintener of pip, this was fixed in version 1.4 of pip which was released on 2013-07-23.
Relevant information from the changelog for this version
Fixed a number of issues (#413, #709, #634, #602, and #939) related to
cleaning up and not reusing build directories. (Pull #865, #948)
I found here that there is a known bug in pip that it won't check the version if there's a build directory with unpacked sources. I have checked this on my troubling package and after deleting its sources from build directory pip installed the required version.
If you are using a pip version that comes with some distribution packages (ex. Ubuntu python-pip), you may need to install a newer pip version:
Update pip to latest version:
sudo pip install -U pip
In case of "virtualenv", skip "sudo":
pip install -U pip
Following command may be required, if your shell report something like -bash: /usr/bin/pip: No such file or directory after pip update:
hash -d pip
Now install your package as usual:
pip install -U foo
or
pip install foo==package.version.here
Got the same issue to update pika 0.9.5 to 0.9.8. The only working way was to install from tarball: pip install https://pypi.python.org/packages/source/p/pika/pika-0.9.8.tar.gz.
In my case the python version used (3.4) didn't satisfy Django 2.1 dependencies requirements (python >= 3.5).
For my case I had to delete the .pip folder in my home directory and then I was able to get later versions of multiple libraries. Note that this was on linux.
pip --version
pip 18.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
virtualenv --version
15.1.0
Just in case that anyone else hassles with upgrading torchtext (or probably any other torch library):
Although https://pypi.org/project/torchtext/ states that you could run pip install torchtext I had to install it similiar to torch by specifying --find-links aka -f:
pip install torchtext===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
What irritated me was that PyCharm pointed me to the new version, but couldn't find it when attempting to upgrade to it. I guess that PyCharm uses its own mechanism to spot new versions. Then, when invoking pip under the hood, it didn't find the new version without the --find-links option.
In my case I am pip installing a .tar.gz package from Artifactory that I make a lot of updates to. In order to overwrite my cached Python files and always grab/install the latest I was able to run:
pip install --no-cache-dir --force-reinstall <path/to/tar.gz>
You should see this re-download any necessary files and install those, instead of using your local cache.
10 years on and pip still fails to work as expected 😖.
I wasted a couple of hours now banging my head against the wall trying to find out why pip won't install a development version of my package. In my case, there are versions 0.0.4 and 0.0.5.dev1 in a private gitlab.com package registry (hence the --extra-index-url argument below), but I believe that's not relevant to the problem.
Following a lot of the advice on this page, I create a test venv in a far away folder, clear the pip cache, uninstall the package in question, etc. first to rule out the most common problems:
$ pip cache purge && \
pip uninstall --yes my-package && \
pip install --extra-index-url "https://_:${GITLAB_PASSWORD_TOOLS_VAULTTOOLS}#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple" \
--no-cache-dir \
--pre \
--upgrade my-package
output (using empty lines to separate output for commands):
WARNING: No matching packages
Files removed: 0
Found existing installation: my-package 0.0.4
Uninstalling my-package-0.0.4:
Successfully uninstalled my-package-0.0.4
Looking in indexes: https://pypi.org/simple, https://_:****#gitlab.com/api/v4/projects/<project-id>/packages/pypi/simple
Collecting my-package
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/f07 ... 397/my_package-0.0.5.dev1-py3-none-any.whl (16 kB)
Downloading https://gitlab.com/api/v4/projects/<project-id>/packages/pypi/files/775 ... 70e/my_package-0.0.4-py3-none-any.whl (16 kB)
...
Successfully installed my-package-0.0.4
So pip does see the dev package version, but chooses the earlier one nonetheless.
In an attempt to figure out what's going on, I published a 0.0.5 version: Error persists, pip sees all three versions, but still installs 0.0.4.
In a further, increasingly desperate attempt, I removed any versions prior to 0.0.5* from the gitlab.com package registry.
Only now, pip would bother to actually display some useful information:
$ (same command as above)
... (similar output as above) ...
ERROR: Cannot install my-package==0.0.5 and my-package==0.0.5.dev1 because these package versions have conflicting dependencies.
The conflict is caused by:
my-package 0.0.5 depends on my-other-package<0.2.5 and >=0.2.4
my-package 0.0.5.dev1 depends on my-other-package<0.2.5 and >=0.2.4
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
OK, so there is something wrong with my package dependencies. Thanks for letting me know.
Seriously - I tried hard for a couple of hours using all kinds of pip ... -vvv and/or fixed versions such as e.g. my-package==0.0.5.dev1 - but I did not manage to get any useful output out of pip - until I wiped the entire history from my package registry 🤬.
Hope this at least helps someone in the same situation.
I found that if you use microversions, pip doesn't seem to recognize them. For example, we couldn't get version 1.9.9.1 to upgrade.
In my case, someone had published the latest version of a package with python2, so attempting to pip3 install it grabbed an older version that had been built with python3.
Handy things to check when debugging this:
If pip install claims to not be able to find the version, see whether pip search can see it.
Take a look at the "Download Files" section on the pypi repo -- the filenames might suggest what's wrong (in my case i saw -py2- there clear as day).
As suggested by others, try running pip install --no-cache-dir in case pip isn't bothering to ask the internet because it already has your answer locally.
I had hidden unversioned files under the Git tab in PyCharm that were being installed with pip install . even though I didn't see the files anywhere else.
Took a long time to find it for me, posting this in hope that it'll help somebody else.
if you need the path for your package do pip -v list. Example see related post when using pip -e Why is an old version of a package of my python library installing by itself with pip -e?