Develop Reference

Using pip to install package in AWS

I am new to using Python packages, and I am trying to install a package called RSeQC into a Linux AWS instance.
I started by running:
sudo yum install python3-pip
then running:
sudo yum install python3-devel
and Finally:
sudo pip3 install RSeQC
However, I get the following long error:
compilation terminated.
In file included from libBigWig/bwWrite.c:6:0:
libBigWig/bigWig.h:4:10: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^~~~~~~~
compilation terminated.
error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DNOCURL -DWITHNUMPY -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -IlibBigWig -I/usr/include/python3.7m -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c pyBigWig.c -o build/temp.linux-x86_64-3.7/pyBigWig.o" failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-
kv90ovvh/pyBigWig/setup.py';f=getattr(tokenize, 'open', open).
(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__,
'exec'))" install --record /tmp/pip-smel45e_-record/install-record.txt --single-version-
externally-managed --compile" failed with error code 1 in /tmp/pip-build-kv90ovvh/pyBigWig/
Would greatly appreciate any advice in debugging this as I am stumped. Thanks in advance!

Why is python giving me a fatal error upon downloading a package?

When trying to install Marine-traffic-api im getting an error:
Installing collected packages: ujson, Marine-Traffic-API
Running setup.py install for ujson: started
Running setup.py install for ujson: finished with status 'error'
Complete output from command /home/benjamin/PycharmProjects/ShipTracking/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pycharm-packaging/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ys30b_z8/install-record.txt --single-version-externally-managed --compile --install-headers /home/benjamin/PycharmProjects/ShipTracking/venv/include/site/python3.7/ujson:
Warning: 'classifiers' should be a list, got type 'filter'
running install
running build
running build_ext
building 'ujson' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/python
creating build/temp.linux-x86_64-3.7/lib
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/home/benjamin/PycharmProjects/ShipTracking/venv/include -I/usr/include/python3.7m -c ./python/ujson.c -o build/temp.linux-x86_64-3.7/./python/ujson.o -D_GNU_SOURCE
In file included from ./python/ujson.c:39:0:
./python/py_defines.h:39:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Install the needed packages:
sudo apt-get install build-essential python3 python-dev python3-dev
After that, run:
pip install --no-cache-dir ujson

How to fix gcc error produced by pip install?

I tried to install a python package from PyPI with the following command
sudo pip3 install switcheo
The installation fails and show this error message:
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -DHAVE_CONFIG_H -DHAVE_SYSCTL_HW_USERMEM=1 -Iscrypt-1.2.1 -Iscrypt-1.2.1/lib -Iscrypt-1.2.1/lib/scryptenc -Iscrypt-1.2.1/lib/crypto -Iscrypt-1.2.1/lib/util -Iscrypt-1.2.1/libcperciva/cpusupport -Iscrypt-1.2.1/libcperciva/alg -Iscrypt-1.2.1/libcperciva/util -Iscrypt-1.2.1/libcperciva/crypto -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c scrypt-1.2.1/libcperciva/util/asprintf.c -o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/asprintf.o
gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.7/src/scrypt.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/crypto/crypto_scrypt_smix_sse2.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/crypto/crypto_scrypt_smix.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/crypto/crypto_scrypt.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/scryptenc/scryptenc.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/scryptenc/scryptenc_cpuperf.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/lib/util/memlimit.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/alg/sha256.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/crypto/crypto_aes_aesni.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/crypto/crypto_aes.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/crypto/crypto_aesctr.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/crypto/crypto_entropy.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/entropy.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/insecure_memzero.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/warnp.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/humansize.o build/temp.macosx-10.9-x86_64-3.7/scrypt-1.2.1/libcperciva/util/asprintf.o -lcrypto -o build/lib.macosx-10.9-x86_64-3.7/_scrypt.cpython-37m-darwin.so
ld: library not found for -lcrypto
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-install-2cof4cvu/scrypt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/tmp/pip-record-1ecvxjh1/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-install-2cof4cvu/scrypt/

Install the libssl-dev package
On Ubuntu, it's as simple as sudo apt-get install libssl-dev
On mac, you need to install openssl: brew install openssl
If that doesn't work, you need to install Homebrew. You should use Homebrew, it makes installing much easier.
If that still doesn't work, you can install libssl-dev and make a symlink:
ln -s /usr/local/Cellar/openssl/1.0.2p/lib/libssl.dylib /usr/local/lib/

Python issue with pycurl and openssl

I am trying to run code that uses pycurl but when I do i get the following error:
uilleann-pipes-160:data_collection Jim$ python xlsxDownloader.py
Traceback (most recent call last):
File "xlsxDownloader.py", line 21, in <module>
from import_hedgehogs import Link
File "/Users/Jim/Documents/Hedgehogs/hedgehogs/data_collection/import_hedgehogs.py", line 8, in <module>
import pycurl
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
When I do "which pycurl" in terminal it gives no output so I then tried installing pycurl using "pip install pycurl". However, When I do that I get the following error:
uilleann-pipes-160:data_collection Jim$ pip install pycurl
Collecting pycurl
Using cached pycurl-7.43.0.1.tar.gz
Installing collected packages: pycurl
Running setup.py install for pycurl ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/zw/hmv25ltn4hxcr3t9lrx7fc9r0000gn/T/pip-build-gqCefl/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/zw/hmv25ltn4hxcr3t9lrx7fc9r0000gn/T/pip-Vbtxf4-record/install-record.txt --single-version-externally-managed --compile:
Using curl-config (libcurl 7.54.0)
running install
running build
running build_py
creating build
creating build/lib.macosx-10.13-intel-2.7
creating build/lib.macosx-10.13-intel-2.7/curl
copying python/curl/__init__.py -> build/lib.macosx-10.13-intel-2.7/curl
running build_ext
building 'pycurl' extension
creating build/temp.macosx-10.13-intel-2.7
creating build/temp.macosx-10.13-intel-2.7/src
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DPYCURL_VERSION="7.43.0.1" -DHAVE_CURL_OPENSSL=1 -DHAVE_CURL_SSL=1 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/docstrings.c -o build/temp.macosx-10.13-intel-2.7/src/docstrings.o
In file included from src/docstrings.c:4:
src/pycurl.h:164:13: fatal error: 'openssl/ssl.h' file not found
# include <openssl/ssl.h>
^~~~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/zw/hmv25ltn4hxcr3t9lrx7fc9r0000gn/T/pip-build-gqCefl/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/zw/hmv25ltn4hxcr3t9lrx7fc9r0000gn/T/pip-Vbtxf4-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/zw/hmv25ltn4hxcr3t9lrx7fc9r0000gn/T/pip-build-gqCefl/pycurl/

Install the first two packages
sudo apt-get install libcurl4-openssl-dev
python -m pip install pybind11
These are optional
sudo apt install libgmp-dev libmpfr-dev libmpc-dev
sudo apt install libssl-dev
sudo apt-get install unixodbc unixodbc-dev

Building Python kerberos extension on opensuse fails

I am trying to build kerberos on opensuse 13.2 inside a docker container. I have already installed the required libraries:
bash-4.2# cat /etc/os-release
NAME=openSUSE
VERSION="13.2 (Harlequin)"
VERSION_ID="13.2"
PRETTY_NAME="openSUSE 13.2 (Harlequin) (x86_64)"
ID=opensuse
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:opensuse:13.2"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://opensuse.org/"
ID_LIKE="suse"
bash-4.2#
bash-4.2# zypper search --installed krb5
S | Name | Summary | Type
--+------------+---------------------------------------------+--------
i | krb5 | MIT Kerberos5 Implementation--Libraries | package
i | krb5-devel | MIT Kerberos5 - Include Files and Libraries | package
bash-4.2# pip install kerberos
Downloading/unpacking kerberos
Downloading kerberos-1.2.3.tar.gz
Running setup.py (path:/tmp/pip_build_root/kerberos/setup.py) egg_info for package kerberos
Installing collected packages: kerberos
Running setup.py install for kerberos
building 'kerberos' extension
gcc -pthread -fno-strict-aliasing -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DOPENSSL_LOAD_CONF -fPIC -I/usr/include/python2.7 -c src/base64.c -o build/temp.linux-x86_64-2.7/src/base64.o sh: krb5-config: command not found
gcc: error: sh:: No such file or directory
gcc: error: krb5-config:: No such file or directory
gcc: error: command: No such file or directory
gcc: error: not: No such file or directory
gcc: error: found: No such file or directory
error: command 'gcc' failed with exit status 1
Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/kerberos/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8u80ki-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'kerberos' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DOPENSSL_LOAD_CONF -fPIC -I/usr/include/python2.7 -c src/base64.c -o build/temp.linux-x86_64-2.7/src/base64.o sh: krb5-config: command not found
gcc: error: sh:: No such file or directory
gcc: error: krb5-config:: No such file or directory
gcc: error: command: No such file or directory
gcc: error: not: No such file or directory
gcc: error: found: No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/kerberos/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8u80ki-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/kerberos
Storing debug log for failure in /root/.pip/pip.log
I tried various things such as finding krb5-config or other krb5 libraries. But, on openSUSE everything is inside krb5-devel package. Do you have any pointers?
My core intention is to build Ansible Windows support on openSUSE

Python kerberos package seems to be difficult to compile, this could be at least a temporary workaround
pip install kerberos==1.1.1
Later versions seem to miss required header files, so there might be some packaging problem.

Try 1.2.4; not sure what went wrong with the headers in 1.2.3 but they are there in 1.2.4.