In my current project, I'm extensively using Cython. I have many separated setup.py files to build Cython code (*.pyx) with no issues at all, (python 3.8 and gcc 8.3) with the command:
python setup.py build_ext --inplace
Here is a straightforward example:
# =============================================================
# Imports:
# =============================================================
import os
import sys
from distutils.core import setup
import setuptools
from distutils.extension import Extension
from Cython.Build import build_ext
from Cython.Build import cythonize
import numpy
import os.path
import io
import shutil
# =============================================================
# Modules:
# =============================================================
ext_modules = [
Extension("cc1", ["cc1.pyx"],
extra_compile_args=['-O3','-w','-fopenmp'],
extra_link_args=['-fopenmp','-ffast-math','-march=native'],
include_dirs=[numpy.get_include()],
language='c++'),
Extension("cc2", ["cc2.pyx"],
extra_compile_args=['-O3','-w','-fopenmp'],
extra_link_args=['-fopenmp','-ffast-math','-march=native'],
include_dirs=[numpy.get_include()],
language='c++'),
]
# =============================================================
# Class:
# =============================================================
class BuildExt(build_ext):
def build_extensions(self):
if '-Wstrict-prototypes' in self.compiler.compiler_so:
self.compiler.compiler_so.remove('-Wstrict-prototypes')
super().build_extensions()
# =============================================================
# Main:
# =============================================================
for e in ext_modules:
e.cython_directives = {'embedsignature': True,'boundscheck': False,'wraparound': False,'linetrace': True, 'language_level': "3"}
setup(
name='jackProject',
version='0.1.0',
author='Jack',
ext_modules=ext_modules,
cmdclass = {'build_ext': BuildExt},
)
Everything works fine, and while the script is in execution, I can check the status on my console, as instance:
running build_ext
cythoning cc1.pyx to cc1.cpp
cythoning cc2.pyx to cc2.cpp
building 'cc1' extension
creating build
creating build/temp.linux-x86_64-3.8
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc1.cpp -o build/temp.linux-x86_64-3.8/cc1.o -O3 -w -fopenmp
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc1.cpp -o build/temp.linux-x86_64-3.8/cc1.o -O3 -w -fopenmp
g++ -pthread -shared -B /home/anaconda3/compiler_compat -L/home/anaconda3/lib -Wl,-rpath=/home/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/cc1.o -o /home/Desktop/DRAFT_CODING/Python/return_many_values_with_cyt/cc1.cpython-38-x86_64-linux-gnu.so -fopenmp -ffast-math -march=native
building 'cc2' extension
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc2.cpp -o build/temp.linux-x86_64-3.8/cc2.o -O3 -w -fopenmp
g++ -pthread -shared -B /home/anaconda3/compiler_compat -L/home/anaconda3/lib -Wl,-rpath=/home/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/cc2.o -o /home/Desktop/DRAFT_CODING/Python/return_many_values_with_cyt/cc2.cpython-38-x86_64-linux-gnu.so -fopenmp -ffast-math -march=native
Now, that I have transferred my project into my Raspberry Pi 4, everything works fine as well, but when I start the setuptools, by launching the same command as usual on the remote terminal, for the same setup.py, this time nothing is written on the terminal, while the script is cythoninzing the source code.
Launch python with the 'verbose' option is not the best...
In this case my virtualenv is based on Python 3.9. with gcc 10.21.
Can anyone tell me what is happening or which is the culprit?
How can I capture again the classic output messages I have always seen until today?
It is not the end of the world, but it was convenient to follow the status of the execution, especially when creating many shared objects at the same time.
I coerced Cython to use clang by specifying the CC environment variable:
import os
os.environ['CC'] = 'clang'
I have a standard build:
EXT_MODULES = [Extension('example.src.ex',
sources=['example/src/ex.pyx'])]
setup_info = dist(
...
ext_modules=cythonize(EXT_MODULES,
compiler_directives={'language_level': '3'}),
...
)
setup(**setup_info)
However it seems like Cython is somehow using both clang and gcc in different parts of the build step, in particular it's using gcc to build the shared libraries:
running build_ext
building 'example.src.ex' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/example
creating build/temp.linux-x86_64-3.8/example/src
clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.8 -c example/src/ex.c -o build/temp.linux-x86_64-3.8/example/src/ex.o
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/example
creating build/lib.linux-x86_64-3.8/example/src
gcc -shared -Wl,--strip-all build/temp.linux-x86_64-3.8/example/src/ex.o -L/usr/local/lib -o build/lib.linux-x86_64-3.8/example/src/ex.cpython-38-x86_64-linux-gnu.so
How do I get Cython to use clang on both steps?
I realized distutils needs an override of the linker as well
os.environ['LDSHARED'] = 'clang -shared'
I have problems building pywt from source on Ubuntu.
When I run python setup.py build, I get the following error:
running build
running build_py
running build_ext
Cython is not installed. Using compiled file: src/_pywt.pyx
building 'pywt._pywt' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPY_EXTENSION -Isrc -I/usr/local/include/python2.7 -c src/_pywt.c -o build/temp.linux-x86_64-2.7/src/_pywt.o
gcc: error: src/_pywt.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 4
When I run cython --version I get the following output:
Cython version 0.21.1
pywt can be installed as an Ubuntu package using apt-get:
sudo apt-get install python-pywt
I'm using Mac, and I am trying to write my own c code to use it from Python. I'm new to this, and trying out a sample example from http://www.tutorialspoint.com/python/python_further_extensions.htm.
But, when I run the setup.py file to install my c file, it gives me an error, and I'm not sure what to do. Below is the resulting error message. Thanks a lot!
$ sudo python3 setup.py install
Password:
running install
running build
running build_ext
building 'sample' extension
clang -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/opt/sqlite/include -DFOO=1 -UBAR -I/some/dir -I/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c pysample.c -o build/temp.macosx-10.10-x86_64-3.4/pysample.o
clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.10-x86_64-3.4/pysample.o -L/usr/local/lib -lsample -o build/lib.macosx-10.10-x86_64-3.4/sample.so
ld: library not found for -lsample
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'clang' failed with exit status 1
Good afternoon all,
I am having trouble using the bsdiff module with Python. While I can use shell scripts I would prefer to have a cross-platform solution.
I have downloaded bsdiff4-1.1.4 and attempt to run setup.py as follows:
Files list:
bsdiff4 build CHANGELOG.txt do.sh examples Makefile README.rst setup.py
I run:
python3 setup.py build
And I get this error:
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
running build
running build_py
running build_ext
building 'bsdiff4.core' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c bsdiff4/core.c -o build/temp.linux-x86_64-3.4/bsdiff4/core.o
bsdiff4/core.c:8:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I want to be able to use python to apply patches to binary files.
Any help appreciated.
José
You need to install apt-get install python3-dev.
ubuntu python3-dev
header files and a static library for Python
If it is your system python you will need sudo