Develop Reference

Error building pygdal: Unknown distribution option: 'use_2to3_fixers' and 'use_2to3_exclude_fixers'

Solution provided below:
Started getting errors when building pygdal on Ubuntu 20.04:
# Command
python3 -m pip install --no-cache-dir pygdal==3.0.4.*
produces the following output
Collecting pygdal==3.0.4.*
Downloading pygdal-3.0.4.6.tar.gz (458 kB)
|████████████████████████████████| 458 kB 3.7 MB/s
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-we7e68ge/pygdal/setup.py'"'"'; __file__='"'"'/tmp/pip-install-we7e68ge/pygdal/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-we7e68ge/pygdal/pip-egg-info
cwd: /tmp/pip-install-we7e68ge/pygdal/
Complete output (5 lines):
/usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'use_2to3_fixers'
warnings.warn(msg)
/usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'use_2to3_exclude_fixers'
warnings.warn(msg)
error in pygdal setup command: use_2to3 is invalid.
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The steps to reproduce the error are:
# Commands to run inside of running Ubuntu 20.04 docker instance
apt update
# install python3.8 and link python3 to this version - these next 2 steps may not be needed for your scenario
apt install -y python3.8
ln -s /usr/bin/python3.8 /usr/bin/python3
# done installing python3.8
apt install -y python3-pip
apt-get install -y python3-gdal gdal-bin libgdal-dev gcc g++ python3.8-dev git
python3 -m pip install --upgrade --no-cache-dir setuptools==58.0.4
python3 -m pip install --upgrade --no-cache-dir numpy wheel requests
python3 -m pip install --no-cache-dir pygdal==3.0.4.*
The solution is to install an older version of setuptools. Specifying version 58.0.2 and earlier resulted in a successful build. I only went back to 58.0.0 for this issue, but this is the first time I've seen this issue so earlier versions should work as well.
# Replace the above installation of setuptools with the following
python3 -m pip install --upgrade --no-cache-dir setuptools==58.0.2

For installing GDAL version 3.0.4. In my case, I've downgrading the setuptools from 58.2.0 to 57.5.0 version worked.
Using the command:
pip3 install setuptools==57.5.0

Replace the above installation of setuptools with the following
python3 -m pip install --upgrade --no-cache-dir setuptools==58.0.2
depending upon which version of pygdal you are using, you may need to use an earlier version of setuptools

Failed to build wheel with autopy

I was bored recently so I decided to do some programming and follow this tutorial which seemd interesting: https://www.youtube.com/watch?ev=8gPONnGIPgw&t=244s
During it, he installs the autopy module directly from pycharm, however when I try it directly myself from the settings in pycharm it doesn't work and I receive an error.
To combat this I decided to follow the instructions on the main site: https://pypi.org/project/autopy/
I tried the first
pip install autopy
command directly and got a massive error where it first said it failed to build the wheel, and then when it ran the second part setup.py it also gave a massive error saying it also failed.
From there I went to the second option:
rustup default nightly-2019-10-05
pip install -U setuptools-rust
pip install -U autopy
by installing rustup and running the other commands but it still fails and gives the "failed to build wheel" and setup.py failed error messages, the exact same ones as before.
Finally I tried the third option:
git clone git://github.com/autopilot-rs/autopy-rs.git
cd autopy
make
make install
by downloading the repository itself and then going into it and using make(that I installed via chocolatey in the powershell) and running these commands but it also fails.
I've tried all 3 methods and the internet is yielding minimum results, any help here or resources that may help is greatly appreciated.
EDIT:
my apologies I thought I added the error, here it is
Collecting autopy
Using cached autopy-4.0.0.tar.gz (20 kB)
Building wheels for collected packages: autopy
Building wheel for autopy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\moham\appdata\local\programs\python\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"'; __file__='"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\moham\AppData\Local\Temp\pip-wheel-bt_24b82'
cwd: C:\Users\moham\AppData\Local\Temp\pip-install-9plfhi_h\autopy_f9372250d2954cabaa93abb9058afa97\
Complete output (21 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib
creating build\lib\autopy
copying autopy\__init__.py -> build\lib\autopy
running build_ext
running build_rust
error: no override and no default toolchain set
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
----------------------------------------
ERROR: Failed building wheel for autopy
Running setup.py clean for autopy
ERROR: Command errored out with exit status 1:
command: 'c:\users\moham\appdata\local\programs\python\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"'; __file__='"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
cwd: C:\Users\moham\AppData\Local\Temp\pip-install-9plfhi_h\autopy_f9372250d2954cabaa93abb9058afa97
Complete output (18 lines):
running clean
removing 'build\lib' (and everything under it)
'build\bdist.win-amd64' does not exist -- can't clean it
'build\scripts-3.9' does not exist -- can't clean it
removing 'build'
running clean_rust
error: no override and no default toolchain set
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
----------------------------------------
ERROR: Failed cleaning build dir for autopy
Failed to build autopy
Installing collected packages: autopy
Running setup.py install for autopy ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\moham\appdata\local\programs\python\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"'; __file__='"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\moham\AppData\Local\Temp\pip-record-lgc9_ekn\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\moham\appdata\local\programs\python\python39\Include\autopy'
cwd: C:\Users\moham\AppData\Local\Temp\pip-install-9plfhi_h\autopy_f9372250d2954cabaa93abb9058afa97\
Complete output (21 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\autopy
copying autopy\__init__.py -> build\lib.win-amd64-3.9\autopy
running build_ext
running build_rust
error: no override and no default toolchain set
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\moham\appdata\local\programs\python\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"'; __file__='"'"'C:\\Users\\moham\\AppData\\Local\\Temp\\pip-install-9plfhi_h\\autopy_f9372250d2954cabaa93abb9058afa97\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\moham\AppData\Local\Temp\pip-record-lgc9_ekn\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\moham\appdata\local\programs\python\python39\Include\autopy' Check the logs for full command output.

Looking at the repository for AutoPy, I've found this issue here
Have you tried to use pip install autopy3 for python 3 and above?
Furthermore, autopy uses rust cargo, you can try to clone the following repo and try to build it with Rust (cargo build) on your platform?

Shap installation

I am facing an issue when trying to install shop in a python virtual environment. In particular when
pip install shap
the following error appears:
ERROR: Command errored out with exit status 1:
command: /Users/federiconutarelli/Desktop/Riccaboni_python/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/setup.py'"'"'; __file__='"'"'/private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-wheel-nnajc046
cwd: /private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/
Complete output (224 lines):
The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME to enable CUDA
Error building cuda module: TypeError('cannot unpack non-iterable NoneType object')
WARNING: Could not compile cuda extensions
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/datasets.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/links.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap ...
(it is longer than that but I cut it for readability). Also, conda installation gets stuck at executing transaction. As far as I have understood, the problem lies on the fact that Shapiros depends on CUDA. However also after setting CUDA to the path
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64
PATH=${CUDA_HOME}/bin:${PATH}
export PATH
the error remains. I am working on a python virtual environment. I know that anaconda should provide CUDA but anaconda seems to get stuck as well (every command get sucked at "Solving Environment".
How can I solve the issue?
Thank you all.

Try install python3 dev.
For ubuntu
sudo apt -y install python3-dev
For Centos
sudo yum -y install python3-devel
Edit: If you are using a specific version of python
sudo apt -y install python3.7-dev
Make sure that the dev version and python version are the same

How to install psycopg2 on Python 3.9 on Windows?

I have been trying to install psycopg2 (pip install psycopg2), but I keep getting this error.
I have also tried with: pip install psycopg2-binary but I'm getting the same error.
Here is the error message:
ERROR: Command errored out with exit status 1:
command: 'c:\users\viktor\pycharmprojects\wemport\venv\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Viktor\\AppData\\Local\\Temp\\pip-install-015ceiei\\psycop
g2\\setup.py'"'"'; __file__='"'"'C:\\Users\\Viktor\\AppData\\Local\\Temp\\pip-install-015ceiei\\psycopg2\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'
"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl'
cwd: C:\Users\Viktor\AppData\Local\Temp\pip-install-015ceiei\psycopg2\
Complete output (23 lines):
running egg_info
creating C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl\psycopg2.egg-info
writing C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl\psycopg2.egg-info\PKG-INFO
writing dependency_links to C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl\psycopg2.egg-info\dependency_links.txt
writing top-level names to C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl\psycopg2.egg-info\top_level.txt
writing manifest file 'C:\Users\Viktor\AppData\Local\Temp\pip-pip-egg-info-3shb13sl\psycopg2.egg-info\SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
How to fix that?

The following will install prebuilt binaries for windows. It's worked for me on 3.9. You can uninstall pipwin after install psycopg2.
pip install pipwin
pipwin install psycopg2
Optional after psycopg2 installed:
pip uninstall pipwin

if u use linux
You need to make sure that you install wheel model
and Try that
sudo ./venv/bin/python -m pip install wheel
after that install psycopg2 and psycopg2-binary
sudo ./venv/bin/python -m pip install psycopg2
sudo ./venv/bin/python -m pip install psycopg-binary

I can't install Psycopg2 with Python3 on Mac. I've installed Python3 and pip3

I've tried to type this command:
pip3 install --user psycopg2 psycopg2-binary
Bash answers:
ERROR: running install
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-install-i0kg4ysj/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-record-lkzzuu55/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-install-i0kg4ysj/psycopg2/

You're not being able to install psycopg2 as the gcc compiler in your device is either outdated and not compatible to install and run it, or it does not exist.
if you just run pip install psycopg2-binary, then it will work on your development and testing environment, however it is not recommended for production.
To install psycopg2, make sure that you have gcc installed in your device by running-
gcc --version
gcc where
If it is not installed, then install it by referring to the guide here-
Installing GCC in Mac
I hope this helps. :)

after successfully installed python and pip, this command should work:
pip install psycopg2