I've just begun to learn cython, and I have not found any good command or instruction by googling. I've tried those commands:
gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing \ 1 тип
-I/usr/include/python3.5 -o rrrattack.cpython-39-x86_64-linux-gnu.so rrrattack.c
cythonize -a -i yourmod.pyx
cythonize gives me .so file, gcc returns error ""python.h" does not exists". What I do not such as a way?
Related
In my current project, I'm extensively using Cython. I have many separated setup.py files to build Cython code (*.pyx) with no issues at all, (python 3.8 and gcc 8.3) with the command:
python setup.py build_ext --inplace
Here is a straightforward example:
# =============================================================
# Imports:
# =============================================================
import os
import sys
from distutils.core import setup
import setuptools
from distutils.extension import Extension
from Cython.Build import build_ext
from Cython.Build import cythonize
import numpy
import os.path
import io
import shutil
# =============================================================
# Modules:
# =============================================================
ext_modules = [
Extension("cc1", ["cc1.pyx"],
extra_compile_args=['-O3','-w','-fopenmp'],
extra_link_args=['-fopenmp','-ffast-math','-march=native'],
include_dirs=[numpy.get_include()],
language='c++'),
Extension("cc2", ["cc2.pyx"],
extra_compile_args=['-O3','-w','-fopenmp'],
extra_link_args=['-fopenmp','-ffast-math','-march=native'],
include_dirs=[numpy.get_include()],
language='c++'),
]
# =============================================================
# Class:
# =============================================================
class BuildExt(build_ext):
def build_extensions(self):
if '-Wstrict-prototypes' in self.compiler.compiler_so:
self.compiler.compiler_so.remove('-Wstrict-prototypes')
super().build_extensions()
# =============================================================
# Main:
# =============================================================
for e in ext_modules:
e.cython_directives = {'embedsignature': True,'boundscheck': False,'wraparound': False,'linetrace': True, 'language_level': "3"}
setup(
name='jackProject',
version='0.1.0',
author='Jack',
ext_modules=ext_modules,
cmdclass = {'build_ext': BuildExt},
)
Everything works fine, and while the script is in execution, I can check the status on my console, as instance:
running build_ext
cythoning cc1.pyx to cc1.cpp
cythoning cc2.pyx to cc2.cpp
building 'cc1' extension
creating build
creating build/temp.linux-x86_64-3.8
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc1.cpp -o build/temp.linux-x86_64-3.8/cc1.o -O3 -w -fopenmp
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc1.cpp -o build/temp.linux-x86_64-3.8/cc1.o -O3 -w -fopenmp
g++ -pthread -shared -B /home/anaconda3/compiler_compat -L/home/anaconda3/lib -Wl,-rpath=/home/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/cc1.o -o /home/Desktop/DRAFT_CODING/Python/return_many_values_with_cyt/cc1.cpython-38-x86_64-linux-gnu.so -fopenmp -ffast-math -march=native
building 'cc2' extension
gcc -pthread -B /home/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anaconda3/lib/python3.8/site-packages/numpy/core/include -I/home/anaconda3/include/python3.8 -c cc2.cpp -o build/temp.linux-x86_64-3.8/cc2.o -O3 -w -fopenmp
g++ -pthread -shared -B /home/anaconda3/compiler_compat -L/home/anaconda3/lib -Wl,-rpath=/home/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/cc2.o -o /home/Desktop/DRAFT_CODING/Python/return_many_values_with_cyt/cc2.cpython-38-x86_64-linux-gnu.so -fopenmp -ffast-math -march=native
Now, that I have transferred my project into my Raspberry Pi 4, everything works fine as well, but when I start the setuptools, by launching the same command as usual on the remote terminal, for the same setup.py, this time nothing is written on the terminal, while the script is cythoninzing the source code.
Launch python with the 'verbose' option is not the best...
In this case my virtualenv is based on Python 3.9. with gcc 10.21.
Can anyone tell me what is happening or which is the culprit?
How can I capture again the classic output messages I have always seen until today?
It is not the end of the world, but it was convenient to follow the status of the execution, especially when creating many shared objects at the same time.
Please help.
Trying to install python cryptography package:
pip install trigger
gcc -pthread -B /home/dan/.conda/envs/py27/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dan/.conda/envs/py27/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o
build/temp.linux-x86_64-2.7/_openssl.c:493:10: fatal error: openssl/opensslv.h: No such file or directory
493 | #include
using conda with python 2.7 virtual env
I am trying to build Python directly from source code, with no admin rights, and after running:
export CPPFLAGS='-I/opt/sqlite-3.7.16.2/include -I/opt/tk8.6.0/include
-I/opt/tcl8.6.0/include/'
export LDFLAGS='-L/opt/sqlite-3.7.16.2/lib -L/opt/tk8.6.0/lib/
-L/opt/tcl8.6.0/lib/ ./configure --prefix=/path_to_python-2.7.4 --enable-shared'
and then
make
I get the following:
building '_tkinter' extension
gcc -pthread -fPIC -fno-strict-aliasing
-g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11/include -I. -IInclude -I./Include -I/opt/sqlite/sqlite-3.7.16.2/include -I/opt/tk8.6.0/include -I/opt/tcl8.6.0/include -I/usr/local/include -I/opt/python/src/Python-2.7.4/Include -I/opt/python/src/Python-2.7.4 -c /opt/python/src/Python-2.7.4/Modules/_tkinter.c -o build/temp.linux-x86_64-2.7/opt/python/src/Python-2.7.4/Modules/_tkinter.o
gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11/include -I. -IInclude -I./Include -I/opt/sqlite/sqlite-3.7.16.2/include -I/opt/tk8.6.0/include -I/opt/tcl8.6.0/include -I/usr/local/include -I/opt/python/src/Python-2.7.4/Include -I/opt/python/src/Python-2.7.4 -c /opt/python/src/Python-2.7.4/Modules/tkappinit.c -o build/temp.linux-x86_64-2.7/opt/python/src/Python-2.7.4/Modules/tkappinit.o
gcc -pthread -shared -L/opt/sqlite/sqlite-3.7.16.2/lib
-L/opt/tk8.6.0/lib/ -L/opt/tcl8.6.0/lib/ -L/opt/sqlite/sqlite-3.7.16.2/lib -L/opt/tk8.6.0/lib/ -L/opt/tcl8.6.0/lib/ -I. -IInclude -I./Include -I/opt/sqlite/sqlite-3.7.16.2/include -I/opt/tk8.6.0/include -I/opt/tcl8.6.0/include build/temp.linux-x86_64-2.7/opt/python/src/Python-2.7.4/Modules/_tkinter.o
build/temp.linux-x86_64-2.7/opt/python/src/Python-2.7.4/Modules/tkappinit.o
-L/usr/X11/lib -L/opt/sqlite/sqlite-3.7.16.2/lib -L/opt/tk8.6.0/lib/ -L/opt/tcl8.6.0/lib/ -L/usr/local/lib -L. -ltk8.6 -ltcl8.6 -lX11 -lpython2.7 -o build/lib.linux-x86_64-2.7/_tkinter.so
* WARNING: renaming "_tkinter" since importing it failed: libtk8.6.so: cannot open shared object file: No such file or directory
The odd thing is that I can see libtk8.6.so. It is actually right there under /opt/tcl8.6.0/lib as I specified with LDFLAGS.
Why did that compilation fail?
This problem takes place during installation in setup.py, when Python tries to import _tkinter. If you look at the function build_extension, there is a block that says:
imp.load_dynamic(ext.name, ext_filename)
This line tries a dynamic load of _tkinter (which uses the dynamic shared library libtk8.6.so) . So even though the compilation/linking worked, when Python tests the module, it uses the dynamic library, and I didn't have tcl/lib nor tk/lib in LD_LIBRARY_PATH. Once I added these, it all worked fine.
In summary:
I had to add the following include paths through CPPFLAGS
/path_to/sqlite3/include
/path_to/tcl/include
/path_to/tk/include
the following lib paths through LDFLAGS
/path_to/sqlite3/lib
/path_to/tcl/lib
/path_to/tk/lib
and the following lib paths through LD_LIBRARY_PATH:
/path_to/sqlite3/lib
/path_to/tcl/lib
/path_to/tk/lib
with all this, everything worked.
I want use Twisted in Python, but when I installing ,in comes this error, how to handle it?
....
running build_ext
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
building 'twisted.runner.portmap' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c twisted/runner/portmap.c -o build/temp.linux-i686-2.7/twisted/runner/portmap.o
twisted/runner/portmap.c:10:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
You are missing the python development headers, needed to build packages that need to compile extensions.
If you are building on OSX, make sure you either install a prebuilt mac python package or if building python from source, use the framework flag when configuring. And also make sure you have XCODE installed so that you have a compiler.
If you are building on Linux, you probably need to install the python devel headers. For instance on Ubuntu you would need: apt-get install build-essential python-dev.
Once you have the python development headers, twisted should be able to find them when you build with that python interpreter.
I'm trying to cross-compile the pycrypto package, and I'm getting closer and closer however, I've hit an issue I just can't figure out.
I want distutils to use the cross-compile specific gcc- so I set the CC env var and it seems to respect the setting for the first invocation of the compiler, but thats it.
export CC="/opt/teeos/buildroot/output/host/usr/bin/i586-linux-gcc"
/opt/teeos/buildroot/output/host/usr/bin/i586-linux-gcc -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 --sysroot=/opt/teeos/buildroot/output/staging -I/opt/teeos/buildroot/output/staging/usr/include/python2.7 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.7 -c src/_fastmath.c -o build/temp.linux-i686-2.7/src/_fastmath.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-i686-2.7/src/_fastmath.o -lgmp -o build/lib.linux-i686-2.7/Crypto/PublicKey/_fastmath.so
unable to execute gcc: No such file or directory
I temporarily moved my systems gcc so it can't be found.
How do I make distutils respect the CC=/opt/buildroot... option for every invocation of the compiler / set the path to the GCC / LD I want distutils to use?
This sounds similar to another answer I recently gave for customizing the distutils compiler. You'll also need to define LDSHARED which is the command used to produce the final shared object. See if this works:
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('LDSHARED')
'gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions'
>>> sysconfig.get_config_var('CC')
'gcc -pthread'
Then replace gcc with your desired compiler and options in the CC and LDSHARED environment variables:
% LDSHARED="i586-linux-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions" \
CC="i586-linux-gcc -pthread" python setup.py build