Develop Reference

Version mismatch between scipy and poetry

I'm using the poetry dependency manager for some of my development (RTL-SDR application). However, when I try to add scipy to the environment (calling poetry add scipy inside Windows 11 Powershell), I get the following output:
Using version ^1.10.0 for scipy
Updating dependencies
Resolving dependencies...
The current project's Python requirement (>=3.11,<4.0) is not compatible with some of the required packages Python requirement:
- scipy requires Python <3.12,>=3.8, so it will not be satisfied for Python >=3.12,<4.0
Because no versions of scipy match >1.10.0,<2.0.0
and scipy (1.10.0) requires Python <3.12,>=3.8, scipy is forbidden.
So, because sdr1 depends on scipy (^1.10.0), version solving failed.
• Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
For scipy, a possible solution would be to set the `python` property to ">=3.11,<3.12"
https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
https://python-poetry.org/docs/dependency-specification/#using-environment-markers
However,using py -V, I verify that my python version is 3.11.0. So, everything should work, right?
Suggestions on resolving this would be most appreciated.

The solution I came up with isn't pretty, and it doesn't answer my questions, but it does work. As Imre_G suggested, I looked at [tool.poetry.dependencies] in pyproject.toml. There, I changed the Python requirement from
python="^3.11.0"
to
python="==3.11.0"
Apparently, Poetry accepted this as effectively excluding version 3.12 where "^3.11.0" did not.
Thanks for the help.

Where it is getting this "Python >=3.12,<4.0" I don't know.
Specifying python="^3.11.0" is equivalent to >=3.11,<4.0.
This indicates that the project is expected to work for every version in that range, not just any.
Since Scipy does not support 3.12, the project cannot support 3.12, and therefore cannot meet the requirement.

Python packaging. Specify python version requirements after multiple releases

I Have a package published on pypi and versioned as 3.0.0.
setup.py has never mentioned python_requires directive.
In release 2.5.0 there was a change which made package to be incompatible with python 2, which went unnoticed, until now.
Since 2.5.0 there were a plenty of releases of a package published on pypi.
Now, if I want to install the package using python2 - pip will install the latest release 3.0.0 which won't work.
I need pip to install version 2.4.0 - which has no compatibility issues. But how exactly can I accomplish that? (without prior knowledge of pip install package==2.4.0 - something like using pip's backtracking mechanism)
If I specify the directive python_requires=">=3.6" in release 3.1.0 pip will backtrack to release 3.0.0 installing package which won't work.
I can think of:
cx_Oracle way. Raising exception in setup.py if minimal version does not match required for installation and specifying how to install correct one.
Create 2 new releases. One, which is essentialy 2.4.0 versioned as 3.1.0 with python_requires=">=2.7,<3.6" and one which is 3.0.0 versioned as 3.1.1 with python_requires=">=3.6"
Is there a better way?

There is a relatively new feature on PyPI: you could "yank" the release(s) which are incompatible with Python 2 but not correctly specifying that in the metadata.
A yanked release is a release that is always ignored by an installer, unless it is the only release that matches a version specifier (using either == or ===).
See PEP 592 -- Adding "Yank" Support to the Simple API for more information. In fact, what you've described is the main scenario described in the motivation section of the PEP.

Why does Pip claim that a version of Python is not in a given range?

I am using Python 3.8.7 and the latest version of Pip, v21.0.1. I have written a package that contains the following Python version constraint:
python_requires='>=3.6, <3.9',
When I try to install the package, Pip refuses and prints the following error:
ERROR: Package 'my-package' requires a different Python: 3.8.7 not in '<3.9,>=3.6'
I have seen a similar error when installing other packages with similar version constraints:
ERROR: Package 'my-other-package' requires a different Python: 3.8.7 not in '!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<3.9,>=2.7'
Clearly Python 3.8.7 is greater than 3.6 and less than 3.9, so why does Pip claim that a different version of Python is required?
Some investigation suggests that this may be an inaccurate error message. I have run an experiment uses packages with the following dependent relationship:
package-a requires Python >= 3.6, and depends on package-b;
package-b requires Python >= 3.6 and < 3.9.
I have found that when installing package-a using Python 3.9, I receive the following error:
ERROR: Package 'package-a' requires a different Python: 3.9.0 not in '>=3.6'
So basically, package-a cannot be installed with Python 3.9 because package-b does not work with Python 3.9; however, Pip erroneously states that package-a requires a "different" Python, and then erroneously prints out package-a's version specifier, rather than package-b.
I will continue to investigate, but this may be a bug in Pip itself.

This was a bug in pip and has been fixed in version 21.1 (see #9541). Specifically, it included the wrong Python constraint (the one of a dependency) in the error message when resolution failed.

Why isn't pip installing the latest version of a package, even when a newer version is on PyPI?

I was trying to upgrade to the latest version of a package I had installed with pip, but for some reason it won't get the latest version. I've tried uninstalling the package in question, or even reinstalling pip entirely, but it still refuses to get the latest version from PyPI. When I try to pin the package version (e.g. pip install package==0.10.0) it says that it "Could not find a version that satisfies the requirement package==0.10.0 (from versions: ...)"
pip search package even acknowledges that the installed version isn't the latest, labeling the two versions for me.
I've seen other questions with external files or local versions, but I've tried the respective solutions (--allow-external doesn't exist anymore, and --no-cache-dir doesn't help) and I'm still stuck on the older version.

I was trying to upgrade Quart. Maybe other packages have something else going on.
In this particular case, Quart had dropped support for Python 3.6 (the version I had installed) and only supported 3.7 or later. (This was a fairly recent change to the project, so I just didn't see the news.)
However, when attempting to install a package only supported by a later Python, pip doesn't really explain why it couldn't find a version to satisfy the requirement - instead, it just lists all the versions that should work with the current Python, without indicating that more exist and just can't be installed.
The only real options to fix are:
Update your Python to meet the package's requirements
Ask/help the maintainer to backport the package to the version you have.

setup.py: restrict the allowable version of the python interpreter

I have a Python library. Unfortunately I have not updated it to work with Python 3 yet.
In its setup.py, I added
install_requires=['python<3'],
My intent was to not allow this package to be installed/used under Python 3, because I know it doesn't (yet) work. I don't think this is the right way to do it, because pip then tries to download and install python 2.7.3 (which is already the installed version!).
How should I specify my library dependency on a particular range of Python interpreter versions? Should I add a Programming Language :: Python :: 2 :: Only tag? Will this actually prevent installation under Python 3? What if I also want to restrict the minimum version to Python 2.6?
I'd prefer a solution that works everywhere, but would settle for one that only works in pip (and hopefully doesn't cause easy_install to choke).

As of version 9.0.1 pip will honor a new python_requires string, specifying the Python version required for installation, e.g, for example if one wishes to enforce minimum Python version of 3.3:
setup(
...,
python_requires=">=3.3"
)
See here for more details. See also this answer on SO.

A possible solution is to test for the Python version, since pip can't satisfy the Python version except for the version it's currently running in (it installs in the current Python environment):
import sys
if not sys.version_info[0] == 2:
sys.exit("Sorry, Python 3 is not supported (yet)")
setup(...

After commenting in the answer above and receiving feedback, I thought to turn my comment into an answer. Note that the answers above are all fine, yet from my experience, I found one thing that is "missing" in these answers, that needs to be pointed out, so here I will illustrate this issue.
For simplicity and completeness of illustration, I have composed a very minimal and simple Python 3 project. The only 3rd party package it uses, is the famous SSH client package paramiko (it's official PyPi page can be found here).
The Python interpreter in the virtual environment of my project is of version 3.6.9
Now, in order to check the python_requires attribute "in action", I have added it to the project's setup.py script, which looks as follows:
from setuptools import setup, find_packages
setup(name='mySampleProject',
version='1.0',
description='Sample project in Python 3',
author='Guy Avraham',
license='MIT',
packages=find_packages(),
include_package_data=True,
python_requires='>=3.8',
install_requires=['paramiko'])
Note that I "required" that the Python version will be 3.8+. This of course should NOT work with the current Python version in the project's virtual environment which is 3.6.9.
Now, when I build the project using the "normal" use in the setup.py, meaning by running: python3 setup.py install, the project was built successfully. See the following output of the pip3 list command after running the python3 setup.py install command:
(mySampleProject_env) guya#ubuntu:~/mySampleProject$ pip3 list
DEPRECATION: The default format will switch to columns in the future. You can use --
format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bcrypt (3.2.0)
cffi (1.14.3)
cryptography (3.1.1)
mySampleProject (1.0)
paramiko (2.7.2)
pip (9.0.1)
pkg-resources (0.0.0)
pycparser (2.20)
PyNaCl (1.4.0)
setuptools (39.0.1)
six (1.15.0)
As you can see, the project, along with all its "sub dependencies" was installed EVEN though I was NOT expecting it to.
On the other hand, when I installed the project using the command: pip3 install -e . (note the . to indicate the "current working directory"), I got the following output:
(mySampleProject_env) guya#ubuntu:~/mySampleProject$ pip3 install -e .
Obtaining file:///home/guya/mySampleProject
mySampleProject requires Python '>=3.8' but the running Python is 3.6.9
Which now, indeed, "considers" the python_requires attribute, thus "failing" the build of the project.
It is detailed in the very first paragraph in the tutorial in this page
and also during minutes ~09:00 - 11:00 in this video
NOTE: I did NOT check all the above for Python 2 (or pip for Python 2).