I want to wrap a test project containing C++ and OpenMP code with Cython, and build it with distutils via a setup.py file. The content of my file looks like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
modules = [Extension("Interface",
["Interface.pyx", "Parallel.cpp"],
language = "c++",
extra_compile_args=["-fopenmp"],
extra_link_args=["-fopenmp"])]
for e in modules:
e.cython_directives = {"embedsignature" : True}
setup(name="Interface",
cmdclass={"build_ext": build_ext},
ext_modules=modules)
The -fopenmp flag is used with gcc to compile and link against OpenMP. However, if I just invoke
cls ~/workspace/CythonOpenMP/src $ python3 setup.py build
this flag is not recognized, because the compiler is clang:
running build
running build_ext
skipping 'Interface.cpp' Cython extension (up-to-date)
building 'Interface' extension
cc -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c Interface.cpp -o build/temp.macosx-10.8-x86_64-3.3/Interface.o -fopenmp
clang: warning: argument unused during compilation: '-fopenmp'
cc -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c Parallel.cpp -o build/temp.macosx-10.8-x86_64-3.3/Parallel.o -fopenmp
clang: warning: argument unused during compilation: '-fopenmp'
Parallel.cpp:24:10: warning: unknown pragma ignored [-Wunknown-pragmas]
#pragma omp parallel for
^
1 warning generated.
c++ -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib build/temp.macosx-10.8-x86_64-3.3/Interface.o build/temp.macosx-10.8-x86_64-3.3/Parallel.o -o build/lib.macosx-10.8-x86_64-3.3/Interface.so -fopenmp
ld: library not found for -lgomp
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'c++' failed with exit status 1
I've unsucessfully tried to specify gcc:
cls ~/workspace/CythonOpenMP/src $ python3 setup.py build --compiler=g++-4.7
running build
running build_ext
error: don't know how to compile C/C++ code on platform 'posix' with 'g++-4.7' compiler
How can I tell distutils to use gcc?
Try setting the "CC" environment variable from inside the setup.py with os.environ.
I just took a look at the distutils source, and the --compiler option expects "unix", "msvc", "cygwin", "mingw32", "bcpp", or "emx". It checks the compiler name you want by checking the CC environment variable. Try calling build like this:
CC=gcc python setup.py build
You don't need to set CXX, it doesn't check for that.
Just in case some others are facing the same problem under Windows (where CC environment variable wouldn't have any effect) :
Create file "C:\Python27\Lib\distutils\distutils.cfg" and write this inside :
Code :
[build]
compiler = mingw32
Remove all instances of "-mno-cygwin" gcc option from file "C:\Python27\Lib\distutils\cygwinccompiler.py" :
This :
self.set_executables(compiler='gcc -mno-cygwin -O -Wall',
compiler_so='gcc -mno-cygwin -mdll -O -Wall',
compiler_cxx='g++ -mno-cygwin -O -Wall',
linker_exe='gcc -mno-cygwin',
linker_so='%s -mno-cygwin %s %s'
% (self.linker_dll, shared_option,
entry_point))
Becomes this :
self.set_executables(compiler='gcc -O -Wall',
compiler_so='gcc -mdll -O -Wall',
compiler_cxx='g++ -O -Wall',
linker_exe='gcc',
linker_so='%s %s %s'
% (self.linker_dll, shared_option,
entry_point))
The second point can be necessary in case you are using a recent version of gcc, where the deprecated option -mno-cygwin has been removed.
Hope this will help even if it is not directly related to the OP real needs (but still related to the question's title...)
According to this wiki, Python versions after 3.4 do not support MinGW anymore.
CPython 3.7 for Windows is compiled with MSC v.1916. When I try to use above-mentioned method with distutils.cfg, I then get an error from distutils: Unknown MS Compiler Version 1916. Looks like it has a hardcoded table of msvcr libraries in its cygwincompiler.py file (which is also responsible for MinGW), and last version known to that file is 1600 from VS2010 / MSVC 10.0.
Try this:
http://mail.python.org/pipermail/distutils-sig/2002-August/002944.html
In short, it appears that you should try: python setup.py build --compiler=g++ first.
On linux while using distutils.ccompiler do
os.environ('CC')='gcc' and then call distutils.sysconfig.customize_compiler(compiler)
It will do the job.
Related
I'm trying to cross-compile numpy for arm (BeagleBone & RPi). I am running on debian buster, with a 32bit cross-compilation toolchain, including i386 (32bit) libraries and all necessary arm toolchains.
When I run:
export CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar RANLIB=arm-linux-gnueabihf-ranlib; \
python3 setup.py build_ext --inplace
I get the following error:
C compiler: arm-linux-gnueabihf-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -m32 -I/usr/include/i386-linux-gnu/ -m32 -I/usr/include/i386-linux-gnu/ -fPIC
compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -c'
arm-linux-gnueabihf-gcc: _configtest.c
arm-linux-gnueabihf-gcc: error: unrecognized command line option ‘-m32’; did you mean ‘-mbe32’?
I've traced through numpy's setup.py file and found that ultimately these settings seem to be generated in the distutils.core._setup_distribution.get_command_obj("build") call.
The error is quite self explanatory, and trying to run the command without the -m32 flags seems to work correctly.
Edit: further digging has revealed that these parameters come to the build script via:
sysconfig.get_config_var("LDSHARED")
I'm still stumped because the following still doesn't work:
$LDSHARED="gcc -pthread -shared" python3 -c "import sysconfig; print(sysconfig.get_config_var('LDSHARED'))"
gcc -pthread -shared -m32 -m32
Turns out this LDSHARED parameter is coming from the Makefile that was used during the build of the python host itself. Under most scenarios it makes sense to have it this way, but a combination of cross-compiling and the arm gcc not accepting that flag made it problematic...
This may very well be unresolvable and I have instead resorted to capturing the parameter from the gcc itself via a symlink/bash script.
I am trying to install biopython 1.65 in debian. I have the dependencies Numpy and Scipy.
When I try to build it, it fails:
python setup.py build
running build
running build_py
running build_ext
building 'Bio.cpairwise2' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c Bio/cpairwise2module.c -o build/temp.linux-x86_64-2.7/Bio/cpairwise2module.o
Bio/cpairwise2module.c:12:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Somebody know how I can solve it?
Many thanks
#include "Python.h"
tells the preprocessor to search a local file, and if it doesn't exists there, the preprocessor changes it to
#include <Python.h>
which should be located in /usr/include/python2.7 (which is passed as argument to gcc). Many Linux distros don't have header files installed by default, so you have to install it manually.
Header files for python2.7 are shipped with package libpython2.7-dev
You can find which package to install by searching it with aptitude, synaptic or apt-cache search adding dev after the package name (in that case python dev); the name could be different than the installed one.
Had the same issue on Fedora, what helped was:
yum install python-devel
Good afternoon all,
I am having trouble using the bsdiff module with Python. While I can use shell scripts I would prefer to have a cross-platform solution.
I have downloaded bsdiff4-1.1.4 and attempt to run setup.py as follows:
Files list:
bsdiff4 build CHANGELOG.txt do.sh examples Makefile README.rst setup.py
I run:
python3 setup.py build
And I get this error:
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
running build
running build_py
running build_ext
building 'bsdiff4.core' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c bsdiff4/core.c -o build/temp.linux-x86_64-3.4/bsdiff4/core.o
bsdiff4/core.c:8:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I want to be able to use python to apply patches to binary files.
Any help appreciated.
José
You need to install apt-get install python3-dev.
ubuntu python3-dev
header files and a static library for Python
If it is your system python you will need sudo
I want to embed python in C. But I find that the version of python interpreter which is embedded in my program is 2.7 (The default version on mac).
How could I specify particular version of python interpreter when I compile the c codes in mac os x. The gcc in os x is definitely different from in linux.
I have already installed python3 through HomeBrew.
Thanks a lot.
UPDATE:
I try to use python3.4-config --cflags and python3.4-config --ldflags to find out the required compiler and linker flags. Then I get these recommended flags when compiling & linking:
-I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -Wno-unused-result -Werror=declaration-after-statement -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/opt/sqlite/include
and
-L/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m -ldl -framework CoreFoundation -lpython3.4m
After this, I assemble these flags along with source file into gcc, and obtain an error:
Undefined symbols for architecture x86_64:
"_PyUnicodeUCS2_FromString", referenced from:
_main in py2-5d8da5.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The C code which I test here comes from Python Documentation
I got the same errors when trying to do this tutorial on OSX. You don't need all of the flags that the config utility spits out. You definitely don't need the corefoundation framework if you're just doing the embedding tutorial. Just use the include directory for headers:
-I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m
, and the library to link to:
-L/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m -lpython3.4m
so here's a one-liner to compile and link:
gcc -I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/include/python3.4m -L/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m -lpython3.4m /path/to/main.c -o /path/to/output/executable
I have mpich2 installed and python 2.7 on Ubuntu 11.04.
I configured without any extra arguments.
When I go to build mpi4py from source, I get this error:
$ python setup.py build
running build
running build_py
running build_ext
MPI C compiler: /usr/bin/mpicc
MPI C++ compiler: /usr/bin/mpicxx
MPI linker: /usr/bin/mpicc
checking for MPI compile and link ...
/usr/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict prototypes -I/usr/include/python2.7 -c _configtest.c -o _configtest.o
/usr/bin/mpicc _configtest.o -o _configtest
/usr/bin/ld: cannot find -lcr
collect2: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
error: Cannot compile/link MPI programs. Check your configuration!!!
Any ideas as to what's going on?
Looks like you are missing libcr-dev.
sudo apt-get install libcr-dev
Another package to try is lam4-dev