Develop Reference

Install pyarrow in VS Code for Windows

I am trying to install pyarrow v10.0.0 to a Python 3.11.0 project in both IntelliJ and VS Code. I am using v1.73.0 of VS Code on WIndows 11.
We use a custom JFrog instance to pull all the libraries. To pull the libraries we use the pip manager extension.
I have version 0.38.0 of wheel.
At the top of the output log is the following error message.
Using cached https://xxxjrog.com/artifactory/api/pypi/ENT_WDA_python_virtual/packages/packages/bd/3d/9594c09e1e2fe2e5ed7ef5c22e4347fee2ea243bccd960442e2c97731fd2/pyarrow-10.0.0.tar.gz (994 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: numpy>=1.16.6 in c:\users\app1dwg\appdata\local\programs\python\python311\lib\site-packages (from pyarrow) (1.23.4)
Building wheels for collected packages: pyarrow
Building wheel for pyarrow (pyproject.toml): started
Building wheel for pyarrow (pyproject.toml): finished with status 'error'
Failed to build pyarrow
ERROR: Command errored out with exit status 1:
At the bottom of the output is the following error. I noticed it was using Visual Studio 15 2017 Win64", but I have upgraded to the most recent version of the MS build tools as was suggested in another post.
-- Running CMake for PyArrow C++ cmake -DARROW_BUILD_DIR=build -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=C:\Users\app1dwg\AppData\Local\Temp\pip-install-399aazc6\pyarrow_6bd82e73181f4c8985e52c102dc8bb5e\build\dist
-DPYTHON_EXECUTABLE=C:\Users\app1dwg\AppData\Local\Programs\Python\Python311\python.exe
-DPython3_EXECUTABLE=C:\Users\app1dwg\AppData\Local\Programs\Python\Python311\python.exe
-DPYARROW_CXXFLAGS= -DPYARROW_WITH_DATASET=off -DPYARROW_WITH_PARQUET_ENCRYPTION=off -DPYARROW_WITH_HDFS=off -G "Visual Studio 15 2017 Win64"
C:\Users\app1dwg\AppData\Local\Temp\pip-install-399aazc6\pyarrow_6bd82e73181f4c8985e52c102dc8bb5e\pyarrow/src error: command 'cmake' failed: None
---------------------------------------- ERROR: Failed building wheel for pyarrow
ERROR: Could not build wheels for pyarrow, which is required to
install pyproject.toml-based projects
Any idea what I am doing wrong?
Thanks,
David

Try to use Python version below 3.11.0. According to the content mentioned in github, the package pyarrow does not support Python version 3.11.0 at present

As a pip install user, am I supposed to have wheel installed?

Consider the usual scenario - I want to create a virtual environment and install some packages. Say
python3 -m venv venv
source venv/bin/activate
pip install databricks-cli
During the installation, I get an error
Building wheels for collected packages: databricks-cli
Building wheel for databricks-cli (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/paulius/Documents/wheeltest/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-maxix98x
cwd: /tmp/pip-install-m7jmyh1m/databricks-cli/
Complete output (8 lines):
/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py:24: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for databricks-cli
While it is benign (the installation actually works), it is still annoying.
I know that pip install wheel resolves this, but wheel does not come with the virtual environment by default. So should I always add it to my requirements.txt, or maybe this is something that can be solved by the package maintainer (in this case databricks-cli) and hence I should open an issue in their Github?
Update: note that the wheel is not necessary to install wheels, in this example bunch of dependencies get successfully downloaded and installed as wheels. The only databricks-cli package gets the error, as it does not have a wheel, but for some reason, pip tries to build it.

Update 3:
To prevent it from the maintainer's perspective use:
setup_requires=["wheel"]
it looks like you're on Linux and using the pre-installed or otherwise modified Python and its setuptools.
I've experienced the same thing due to Debian cutting off parts of the packages in not quite a sane way and I have mainly had issues with the prebuilt python-setuptools and likes. Check if the version matches and if not, install setuptools from pip, that might help in the future.
I have setuptools 45.2.0 and had no issues installing the package as a wheel. Then I uninstalled wheel and dropped the cache dir and it installed it properly even from the tar.gz source.
Update 2:
It might or might not be resolved by requiring wheel package. If the setup.py expects bdist_wheel to be present prior to the installation (most likely), adding it to the setup() function will not help and a manual check for the package within the setup script (+ perhaps a reference in README) are necessary so an end-user can install it properly.
For example if it's not present on the system, just print a warning and call an exit(). That would be the least a maintainer should do.
Update:
Yes, in the case where you encounter bdist_wheel command missing you need to install wheel with pip instal wheel.
It's not required, but it's recommended. Pip will work just fine without wheels, but you'll be installing from source (tar.gz, .zip or .egg).
See the packaging discussion for whether to use wheel or egg (or source).

This was a pip bug, and the solution is to upgrade the pip. With the newest version things look fine:
(venv) paulius#xps:~/Documents/wheeltest$ pip install databricks-cli
Collecting databricks-cli
Using cached databricks-cli-0.14.3.tar.gz (54 kB)
Collecting click>=6.7
Using cached click-8.0.1-py3-none-any.whl (97 kB)
Collecting requests>=2.17.3
Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six>=1.10.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tabulate>=0.7.7
Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting idna<4,>=2.5
Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.1-py3-none-any.whl (35 kB)
Using legacy 'setup.py install' for databricks-cli, since package 'wheel' is not installed.
Installing collected packages: urllib3, idna, charset-normalizer, certifi, tabulate, six, requests, click, databricks-cli
Running setup.py install for databricks-cli ... done
Successfully installed certifi-2021.5.30 charset-normalizer-2.0.1 click-8.0.1 databricks-cli-0.14.3 idna-3.2 requests-2.26.0 six-1.16.0 tabulate-0.8.9 urllib3-1.26.6
Note the Using legacy 'setup.py install' ... line.
This is a related issue in the pip github https://github.com/pypa/pip/issues/8302. Not exactly that, but there is an explanation in the comments on what's the wheel building logic supposed to by.

Beeware 'briefcase create' asks for cairo >= 1.15.10

I'm following Beeware tutorial and unable to 'briefcase create'.
At some point it shows this:
Collecting pygobject>=3.14.0
Downloading PyGObject-3.38.0.tar.gz (712 kB)
|████████████████████████████████| 712 kB 6.9 MB/s
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.8 /usr/local/lib/python3.8/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-6esqaemw/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel pycairo
cwd: None
Complete output (36 lines):
WARNING: The directory '/home/brutus/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting setuptools
Downloading setuptools-50.3.2-py3-none-any.whl (785 kB)
Collecting wheel
Downloading wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Collecting pycairo
Downloading pycairo-1.20.0.tar.gz (344 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Building wheels for collected packages: pycairo
Building wheel for pycairo (PEP 517): started
Building wheel for pycairo (PEP 517): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.8 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp426eh9du
cwd: /tmp/pip-install-rmj9v5en/pycairo
Complete output (12 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/cairo
copying cairo/__init__.py -> build/lib.linux-x86_64-3.8/cairo
copying cairo/__init__.pyi -> build/lib.linux-x86_64-3.8/cairo
copying cairo/py.typed -> build/lib.linux-x86_64-3.8/cairo
running build_ext
Requested 'cairo >= 1.15.10' but version of cairo is 1.14.6
Command '['pkg-config', '--print-errors', '--exists', 'cairo >= 1.15.10']' returned non-zero exit status 1.
----------------------------------------
ERROR: Failed building wheel for pycairo
Failed to build pycairo
ERROR: Could not build wheels for pycairo which use PEP 517 and cannot be installed directly
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.8 /usr/local/lib/python3.8/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-6esqaemw/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel pycairo Check the logs for full command output.
Unable to install dependencies. This may be because one of your dependencies is invalid, or because pip was unable to connect to the PyPI server.
I believe the main problem is
Requested 'cairo >= 1.15.10' but version of cairo is 1.14.6
But what I don't understand is were is cairo 1.14.6 if I have only cairo 1.20.0 installed.
I tried to update docker, reinstall cairo and pycairo, updated python packages but the error is still there.

The beeware build process is trying to install the pycairo package. This package is just a python interface to the cairo graphics library (libcairo2).
pycairo changelog shows, newest version 1.20.0 for pycairo, which requires cairo (libcairo2) version 1.15.10+.
If you are lucky, you can simply update your cairo package to a version which satisfies the requirement. Info is on the official site cairographics.org/download/.
I have the same problem as you. In my case I am again reminded to upgrade my OS to newer version of Ubuntu, since the cairo package libcairo2 is only available in version 14.6 in Ubuntu 16.04 LTS official PPA. In Ubuntu 20.04 LTS the available libcairo2 is 16.0. I suspect you have a simmilar OS, because your installed cairo version is the same as mine.

You can build the package from source to bypass the error:

Multiple errors trying to install jq

%pip install jq
returns the following error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\myacn\anaconda3\python.exe' 'C:\Users\myacn\anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\myacn\AppData\Local\Temp\tmpygza7t_7'
cwd: C:\Users\myacn\AppData\Local\Temp\pip-install-l6yh1hi9\jq
Complete output (7 lines):
running bdist_wheel
running build
running build_ext
Downloading https://github.com/kkos/oniguruma/releases/download/v6.9.4/onig-6.9.4.tar.gz
Downloaded https://github.com/kkos/oniguruma/releases/download/v6.9.4/onig-6.9.4.tar.gz
Executing: ./configure CFLAGS=-fPIC --prefix=C:\Users\myacn\AppData\Local\Temp\pip-install-l6yh1hi9\jq\_deps\onig-install-6.9.4
error: [WinError 2] The system cannot find the file specified
----------------------------------------
ERROR: Failed building wheel for jq
ERROR: Could not build wheels for jq which use PEP 517 and cannot be installed directly
Looking online this link resolves for mac and linux, I'm not entirely familiar with the right installation for windows, trying %pip install autoconf automake libtool gives
Collecting autoconf
Using cached autoconf-0.7.1-py3-none-any.whl (10 kB)
Note: you may need to restart the kernel to use updated packages.
ERROR: Could not find a version that satisfies the requirement automake (from versions: none)
ERROR: No matching distribution found for automake
Collecting jq
Using cached jq-1.1.1.tar.gz (69 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Building wheels for collected packages: jq
Building wheel for jq (PEP 517): started
Building wheel for jq (PEP 517): finished with status 'error'
Failed to build jq
Note: you may need to restart the kernel to use updated packages.
LOAD DURATION: 0:00:11.548563
ERROR: Command errored out with exit status 1:
command: 'C:\Users\myacn\anaconda3\python.exe' 'C:\Users\myacn\anaconda3\lib\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\myacn\AppData\Local\Temp\tmpygza7t_7'
cwd: C:\Users\myacn\AppData\Local\Temp\pip-install-l6yh1hi9\jq
Complete output (7 lines):
running bdist_wheel
running build
running build_ext
Downloading https://github.com/kkos/oniguruma/releases/download/v6.9.4/onig-6.9.4.tar.gz
Downloaded https://github.com/kkos/oniguruma/releases/download/v6.9.4/onig-6.9.4.tar.gz
Executing: ./configure CFLAGS=-fPIC --prefix=C:\Users\myacn\AppData\Local\Temp\pip-install-l6yh1hi9\jq\_deps\onig-install-6.9.4
error: [WinError 2] The system cannot find the file specified
----------------------------------------
ERROR: Failed building wheel for jq
ERROR: Could not build wheels for jq which use PEP 517 and cannot be installed directly
Seems like automake was the issue.
Reading another thread here, the error message is slightly different: the link talks about bottleneck, but mine does not. Going through that thread I first tried downgrading pip pip install pip==18.1, restarted kernel,
Here's the new errror:
Failed building wheel for jq
Command "C:\Users\myacn\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\myacn\\AppData\\Local\\Temp\\pip-install-m3mejes3\\jq\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\myacn\AppData\Local\Temp\pip-record-_38z9t11\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\myacn\AppData\Local\Temp\pip-install-m3mejes3\jq\
You are using pip version 18.1, however version 20.3b1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
I then downloaded Visual Studio, specifically checking C++ install tools like the steps indicated; I tried re-running with updated pip and 18.1, and get the same errors.

ERROR: Failed building wheel for cryptography

Hey guys I was installing "channels"
"pip install channels"
but I received this Error.
Failed building wheel for cryptography
Running setup.py clean for cryptography
Failed to build cryptography
Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
What should I do now?