Develop Reference

Pre-release version number in mixed python/rust project with maturin

I am working on a mixed Rust/Python project being build with maturin. I am having trouble figuring out how I can specify a pre-release version that can than be installed by pip.
The start of my Cargo.toml is
[package]
version = "0.0.1-test.1"
When I do maturin build --release, it creates my_project-0.0.1_test.1-cp39-cp39-macosx_10_7_x86_64.whl. Note that 0.0.1-test.1 in the version was changed to 0.0.1_test.1 in the name of the wheel file. (The dash became an underscore.)
If I publish this to my Nexus server and try to install it in a separate project, pip gives an error:
$ pip install my_project==0.0.1_test.1
Looking in indexes: ...
ERROR: Could not find a version that satisfies the requirement my_project==0.0.1_test.1 (from versions: 0.0.1-test.1, 0.0.1.dev1)
If I try it with a dash instead of an underscore:
pip install py_mod_query_builder==0.0.1-test.1
Looking in indexes: ...
ERROR: Exception:
<stack trace>
pip._vendor.packaging.version.InvalidVersion: Invalid version: '0.0.1-test.1'
If I change the version in Cargo.toml to 0.0.1_troy.1, then Cargo fails to parse the build file:
$ maturin build --release
🍹 Building a mixed python/rust project
💥 maturin failed
Caused by: Cargo metadata failed. Do you have cargo in your PATH?
Caused by: Error during execution of `cargo metadata`: error: failed to parse manifest at `/Users/me/my-project/Cargo.toml`
Caused by:
unexpected character '_' after patch version number for key `package.version`
I did have some success with pip install path/to/my_project*.whl. However, that is difficult to incorporate into building a docker image. The wheel is outside the project with the Dockerfile, so docker will not be able to find it without jumping through a bunch of hoops, while downloading it from the server during the docker build is simple, but fails due to the dash/underscore problem.
Additionally, while both projects are being developed, I would like the CI pipeline for the development branch to use the pre-release version. Getting access to the wheel file (without having pip download it) is nearly impossible in that situation.
It seems that Cargo insists on the format being #.#.#-string and pip insists on it being #.#.#_string. Is there a way to make both of them happy?
Versions:
Installing project
pip 22.1
python 3.9.12
Building project
pip 20.0.4
python 3.9.12
maturin 0.8.3
rustc 1.56.1 (59eed8a2a 2021-11-01)

How can I test a package with pycairo as a dependency with tox on MacOS?

I am trying to use tox to test a graphics package I am working on. One of its dependencies is pycairo, so when I set up my tox.ini file, I specify it under deps like so:
[testenv]
deps =
pycairo
...(some other packages)
and while my tests work fine on Windows, when I try testing the package on MacOS, the test always fails with the following error when I try to pip-install pycairo:
pip3 install pycairo
Collecting pycairo
Using cached pycairo-1.20.1.tar.gz (344 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting pygame
Downloading pygame-2.0.1-cp39-cp39-macosx_10_9_intel.whl (6.9 MB)
Building wheels for collected packages: pycairo
Building wheel for pycairo (PEP 517): started
Building wheel for pycairo (PEP 517): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /Users/appveyor/projects/cpython-cmu-graphics-0l7rb/.tox/py39/bin/python /Users/appveyor/projects/cpython-cmu-graphics-0l7rb/.tox/py39/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/5s/g225f6nd6jl4g8tshbh1ltk40000gn/T/tmpnqn0c3o6
cwd: /private/var/folders/5s/g225f6nd6jl4g8tshbh1ltk40000gn/T/pip-install-1vu11s7g/pycairo_6159cae3f6b14ec3a8681d1238fa6919
Complete output (12 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.15-x86_64-3.9
creating build/lib.macosx-10.15-x86_64-3.9/cairo
copying cairo/__init__.py -> build/lib.macosx-10.15-x86_64-3.9/cairo
copying cairo/__init__.pyi -> build/lib.macosx-10.15-x86_64-3.9/cairo
copying cairo/py.typed -> build/lib.macosx-10.15-x86_64-3.9/cairo
running build_ext
Requested 'cairo >= 1.15.10' but version of cairo is 1.12.14
Command '['pkg-config', '--print-errors', '--exists', 'cairo >= 1.15.10']' returned non-zero exit status 1.
----------------------------------------
ERROR: Failed building wheel for pycairo
Failed to build pycairo
ERROR: Could not build wheels for pycairo which use PEP 517 and cannot be installed directly
I established that the main reason I'm getting this error is because wheels and Cairo binary files are not provided for the pip installation of pycairo on MacOS. (It's worth noting that I'm running my MacOS tests via a remote VM) As such, I tried to install cairo first using Homebrew like so:
brew install cairo
However, whenever I retry the tests, I still get the same error message. I read on another SO post that you should brew install pkg-config as well, so in addition to the brew installation above, I also did:
brew install pkg-config
And still ended up with the same error message when I retried the tests. Frustrated, I once again took to Stack Overflow and discovered that you can directly install pycairo (as well as its dependencies, like cairo) with one single brew install command:
brew install py3cairo
Now, whenever I SSH'd into the Mac VM, running the test files worked, but because tox runs tests inside of virtual environments, it can't access this version of pycairo.
Now, one nasty, probably-horrible-practice, brute-force solution I found was to print out the path of the pycairo directory using this small Python script:
import os
import cairo
print(os.path.dirname(cairo.__file__))
And then I cp'd that directory into a virtual environment and found that it actually allows you to run import cairo without getting an error.
cp -r <path>/cairo venv3.9/lib/python3.9/site-packages
Python 3.9.1 (default, Dec 26 2020, 00:12:24)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairo
>>>
However, not surprisingly, this doesn't seem to work with any other Python minor version that I'm testing, and I wouldn't be surprised if this breaks the library in other ways I haven't discovered yet. So that's not really an acceptable solution either.
What can I do to make my tests run properly? In my tests I just want to simulate an environment that already has all the package dependencies installed, but with pycairo it doesn't seem like there's a way for me to access the package.
I just need this to work in tox for testing purposes only. I don't anticipate anyone using our package inside of a virtual environment, so our users should just be able to install py3cairo via brew directly to their system in the worst case.
Most likely, it looks like I need a way to install cairo and pkg-config such that pip inside of a virtual environment can access those files and still install the Python bindings. But I'm also open to any other suggestions that would just allow my tox tests to run. Does anyone have any thoughts on how to fix this?

Requested 'cairo >= 1.15.10' but version of cairo is 1.12.14
Your issue is not about package discoverability but an out-of-date version. If the brew-installed version of cairo is newer than 1.15.10 then you might have a separate cairo installation lying around which gets preferred over your brew-installed version.
To reproduce the issue, I did the following:
brew install cairo
python -m venv cairo
source cairo/bin/activate
pip install pycairo
which worked as expected (Python 3.9.1, pip 20.2.3).

Compiling GTK+3 by Using Meson

GTK+3 v3.24 is downloaded from Gitlab and extracted from zip file.
After running Meson, it gives the following error when meson setup --prefix C:\Users\user\Desktop\gtk\gtk-gtk-3-24 builddir command is run:
meson.build:359:0: ERROR: Git program not found.
No Git programs are installed on my machine (Windows 10 x64). But source code is downloaded. Meson and Ninja are installed by using pip command.
What is the problem?

The problem that in addition to GTK sources you need to provide all required dependencies: some dependency is not found -> meson tries to use "fallback", i.e. download some subproject from git, probably this is glib dependency:
glib_dep = dependency('glib-2.0', version: glib_req,
fallback : ['glib', 'libglib_dep'])
Here you can find the list of some of required packages:
You will also need various dependencies, based on the platform you are
building for:
GLib
GdkPixbuf
GObject-Introspection
...
But instead of painful process of setting up all these, why not just trying to use tool that is recommended to setup GTK on Windows? Check gvsbuild project or MSYS2, both described here.

Limit package to CPython - setup.cfg

I'm using pyproject.toml and setup.cfg per PEP-517.
I have my setup.cfg as follows:
[metadata]
name = Package
version = 1.0.0
[options]
py_modules = ...
python_requires = >=3
I wish to limit the package to work only on CPython, as it solves an implementation-specific issue.
I've tried using the environment markers, but unfortunately they don't work on the python_requires field:
python_requires = >=3; implementation_name == "cpython"
Is there any way to achieve what I wish without resorting to the deprecated setup.py?

Unfortunately, you can't restrict the implementation via environment markers and the package metadata doesn't define any suitable field. However, the implementation can be restricted via the language implementation tag in wheel metadata; the format of the tag is defined in PEP 425. The CPython implementation is abbreviated with cp, python_requires = >=3 from your config implies cp3 for the full tag. Set the tag in a [bdist_wheel] section in setup.cfg:
[metadata]
...
[options]
...
[bdist_wheel]
python_tag=cp3
This is not special to bdist_wheel; any distutils subcommand can have a section in setup.cfg where one can persist its command line options. Read through Writing the Setup Configuration File for more details on that.
To answer your question from the comments:
To be honest I haven't even found the online documentation for the python-tag command. Am I missing something?
This is indeed somewhat hidden; wheel's docs don't provide a reference for the bdist_wheel's options. Usually, you list them via python setup.py bdist_wheel --help (just as with any other subcommand), but since you don't have the setup script, you can fake one:
$ python -c "from setuptools import setup; setup()" bdist_wheel --help
Options for 'bdist_wheel' command:
...
--python-tag Python implementation compatibility tag (default: 'py3')
A built wheel file will now have the cp3-none-any suffix (and the cp3-none-any tag in wheel metadata), and will be rejected by other implementations:
$ pypy3 -m pip install meowpkg-0.0.1-cp3-none-any.whl
ERROR: meowpkg-0.0.1-cp3-none-any.whl is not a supported wheel on this platform.
This will also work with source dists since pip will always build a wheel from PEP 517-compliant sdists (and not resort to legacy setup.py install), so this is also safe to use with source dists. Example:
$ python -m build
$ pip uninstall -y wheel # for testing only
WARNING: Skipping wheel as it is not installed.
$ pip install dist/meowpkg-0.0.1.tar.gz
Processing ./dist/meowpkg-0.0.1.tar.gz
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: meowpkg
Building wheel for meowpkg (PEP 517) ... done
Created wheel for meowpkg: filename=meowpkg-0.0.1-cp3-none-any.whl size=1005 sha256=d87571afcdeda6be74a182b9e3e1ce6035a4aea4bb173c291900a85e53232983
Stored in directory: /home/oleg.hoefling/.cache/pip/wheels/1a/8c/43/90d6b484adcf2515d25503b0c47f14565cadf0e891a597e23e
Successfully built meowpkg
Installing collected packages: meowpkg
Attempting uninstall: meowpkg
Found existing installation: meowpkg 0.0.1
Uninstalling meowpkg-0.0.1:
Successfully uninstalled meowpkg-0.0.1
Successfully installed meowpkg-0.0.1

I can't install python package, although required libraries exist

I am trying to install a Python package and I get a dependency error but I am sure I have fulfilled that requirement.
It says that it can't find libdickinson.so, but this library is already installed (system wide) and its files are in /user/local/lib/. What am I doing wrong?
This is my console output:
(iwidget)chris#mint-desktop ~ $ pip install pthelma
Downloading/unpacking pthelma
Downloading pthelma-0.7.2.tar.gz (50kB): 50kB downloaded
Running setup.py egg_info for package pthelma
libdickinson.so: cannot open shared object file: No such file or directory
Please make sure you have installed dickinson
(see http://dickinson.readthedocs.org/).
Complete output from command python setup.py egg_info:
libdickinson.so: cannot open shared object file: No such file or directory
Please make sure you have installed dickinson
(see http://dickinson.readthedocs.org/).
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /home/chris/.virtualenvs/iwidget/build/pthelma
Storing complete log in /home/chris/.pip/pip.log
(iwidget)chris#mint-desktop ~ $ ls /usr/local/lib/
libdickinson.a libdickinson.la libdickinson.so libdickinson.so.0 libdickinson.so.0.0.0 python2.7/ python3.2/ site_ruby/
(iwidget)chris#mint-desktop ~ $

Also try the above command as superuser:
sudo pip install pthelma
and just go through the thread given below:
Why can't Python find shared objects that are in directories in sys.path?

Try building it yourself and installing from the GIT repo:
git clone https://github.com/openmeteo/pthelma.git
Also, try running it as super user (pip).
sudo pip install pthelma
It looks like it can't see the libdickinson.so file but if you're confident it's installed and setup correctly you can, as I said, try cloning the source and building it that way.