Develop Reference

PyCharm/Cython cross-compile on Windows? [duplicate]

I have simple python + cython project (hello world example from http://docs.cython.org/src/tutorial/cython_tutorial.html) on my ubuntu 16 x86_64. I can build this project with cython for x86_64.
How can I build the project for armv7 version of ubuntu 15 without using real armv7 board/cpu?
I have arm-linux-gnueabihf-gcc (http://packages.ubuntu.com/xenial/devel/gcc-arm-linux-gnueabihf) and it can compile simple C programs for armv7. How can I change settings of cython to use cross compiler for building shared objects for arm?

Architecture dependent libraries and headers files are needed for cross compiling.
When testing if python3.5-dev package and others could be installed after dpkg --add-architecture armhf and apt-get update (after some modification to sources.list), the result was basically.
python3.5-dev:armhf : Depends: python3.5:armhf (= 3.5.1-10) but it is not going to be installed
apt-get install python3.5:armhf is something that doesn't work, see
The existing proposals allow for the co-installation of libraries and
headers for different architectures, but not (yet) binaries.
One possible solution that does not require "full" virtual machine is provided by QEMU and chroot. A suitable directory for chroot can be created by debootstrap command. After creation, schroot can give access to that environment.
Substitute <DIRECTORY> and <USER> in the following commands:
apt-get install -y debootstrap qemu-user-static binfmt-support schroot
debootstrap --arch=armhf --foreign --include=gcc,g++,python3.5-dev xenial <DIRECTORY>
cp /usr/bin/qemu-arm-static <DIRECTORY>/usr/bin
chroot <DIRECTORY>
/debootstrap/debootstrap --second-stage
echo "deb http://ports.ubuntu.com/ubuntu-ports xenial universe" >> /etc/apt/sources.list
echo "deb http://ports.ubuntu.com/ubuntu-ports xenial multiverse" >> /etc/apt/sources.list
apt-get update
apt-get install -y cython cython3
exit
cat <<END > /etc/schroot/chroot.d/xenial-armhf
[xenial-armhf]
description=Ubuntu xenial armhf
type=directory
directory=/home/xenial-armhf
groups=sbuild,root
root-groups=sbuild,root
users=root,<USER>
END
The environment should be accessible by
schroot -c chroot:xenial-armhf
and for root user session (the user must be in a group listed in root-groups) ,
schroot -c chroot:xenial-armhf -u root
After this, it is also possible to cross compile a cython module:
hello.pyx:
print("hello world")
compiling (python3.5-config --cflags and python3.5-config --libs in chroot for options, note -fPIC):
cython hello.pyx
arm-linux-gnueabihf-gcc --sysroot <DIRECTORY> -I/usr/include/python3.5m -I/usr/include/python3.5m -Wno-unused-result -Wsign-compare -g -fstack-protector-strong -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c hello.c
arm-linux-gnueabihf-gcc --shared --sysroot <DIRECTORY> -lpython3.5m -lpthread -ldl -lutil -lm hello.o -o hello.so
The module can be then tested
schroot -c chroot:xenial-armhf
python3
import hello
Cross compiling cython based python modules may also work. With setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import os
os.environ['CC'] = 'arm-linux-gnueabihf-gcc'
os.environ['LDSHARED'] = 'arm-linux-gnueabihf-gcc -shared'
sysroot_args=['--sysroot', '/path/to/xenial-armhf']
setup(cmdclass = {'build_ext': build_ext},
ext_modules= [ Extension("hello", ["hello.pyx"],
extra_compile_args=sysroot_args,
extra_link_args=sysroot_args) ])
Building a simple hello world module was possible this way. The file name for the module was wrong, in this case it was hello.cpython-35m-x86_64-linux-gnu.so. After renaming it as hello.so it was possible to import it.

Failed to install pykaldi on ubuntu 18.04

I followed the instructions and ran the following to commands to install pykaldi:
git clone https://github.com/pykaldi/pykaldi.git
cd pykaldi
sudo apt-get install autoconf automake cmake curl g++ git graphviz libatlas3-base libtool make pkg-config subversion unzip wget zlib1g-dev
sudo apt install intel-mkl-64bit-2020.4-912
python3.7 -m pip install --upgrade pip setuptools
python3.7 -m pip install numpy pyparsing ninja==1.10.0
cd tools
sudo torify ./check_dependencies.sh /usr/bin/python3.7
sudo torify ./install_protobuf.sh /usr/bin/python3.7
sudo torify ./install_clif.sh /usr/bin/python3.7
sudo torify ./install_kaldi.sh
cd ..
python3.7 -m pip install setuptools
sudo apt-get install ninja-build
sudo python3.7 setup.py install
when it comes to the last line, I get the following error:
running install
running bdist_egg
running egg_info
writing pykaldi.egg-info/PKG-INFO
writing dependency_links to pykaldi.egg-info/dependency_links.txt
writing requirements to pykaldi.egg-info/requires.txt
writing top-level names to pykaldi.egg-info/top_level.txt
reading manifest file 'pykaldi.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'pykaldi.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
Using PYCLIF: /usr/local/bin/pyclif
Using CLIF_MATCHER: /usr/clang/bin/clif-matcher
-- Configuring done
-- Generating done
-- Build files have been written to: /home/soroushh/KhodnevisProjects/wav2vec-u/pykaldi/build
[6/505] Building CXX object kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o
FAILED: kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o
/usr/bin/c++ -D_matrix_ext_EXPORTS -I../kaldi/lib -I../kaldi -Ikaldi -I../tools/kaldi/src -I/usr/include/python2.7 -I/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include -std=c++11 -I.. -isystem /home/soroushh/KhodnevisProjects/wav2vec-u/pykaldi/tools/kaldi/tools/openfst-1.6.7/include -O1 -Wall -Wno-sign-compare -Wno-unused-local-typedefs -Wno-deprecated-declarations -Winit-self -DKALDI_DOUBLEPRECISION=0 -DHAVE_EXECINFO_H=1 -DHAVE_CXXABI_H -DHAVE_MKL -I/opt/intel/mkl/include -m64 -msse -msse2 -pthread -g -fPIC -Wno-maybe-uninitialized -fPIC -MD -MT kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o -MF kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o.d -o kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o -c ../kaldi/matrix/matrix-ext.cc
In file included from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:4:0,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from ../kaldi/matrix/matrix-ext.cc:8:
/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/npy_common.h:386:9: error: ‘Py_hash_t’ does not name a type
typedef Py_hash_t npy_hash_t;
^~~~~~~~~
In file included from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:0,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from ../kaldi/matrix/matrix-ext.cc:8:
/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:655:9: error: ‘npy_hash_t’ does not name a type; did you mean ‘npy_half’?
npy_hash_t hash;
^~~~~~~~~~
npy_half
[11/505] Building CXX object kaldi/chain/CMakeFiles/_chain_generic_numerator.dir/chain-generic-numerator-clifwrap.cc.o
ninja: build stopped: subcommand failed.
Command '['ninja', '-j', '6']' returned non-zero exit status 1.
I also tried different versions of PyKaldi, i.e. 1.8.0 and 1.10.0. But it shows the same error.
Edit 1: when I execute /usr/bin/ninja -j 6 I get the following error: ninja: error: loading 'build.ninja': No such file or directory

Linking Boost Python on OSX 10.10

I am trying to build the project from: https://github.com/TNG/boost-python-examples which is a set of boost python examples on Yosemite.
First I installed Boost and Boost python and cmake
brew install boost
brew install boost-pyton
brew install cmake
I've run into the following error:
/usr/local/Cellar/cmake/3.0.2/bin/cmake -H/Users/demo/devel/boost-python-examples -B/Users/demo/devel/boost-python-examples/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/Cellar/cmake/3.0.2/bin/cmake -E cmake_progress_start /Users/demo/devel/boost-python-examples/build/CMakeFiles /Users/demo/devel/boost-python-examples/build/CMakeFiles/progress.marks
/Applications/Xcode.app/Contents/Developer/usr/bin/make -f CMakeFiles/Makefile2 all
/Applications/Xcode.app/Contents/Developer/usr/bin/make -f 01-HelloWorld/CMakeFiles/hello.dir/build.make 01-HelloWorld/CMakeFiles/hello.dir/depend
cd /Users/demo/devel/boost-python-examples/build && /usr/local/Cellar/cmake/3.0.2/bin/cmake -E cmake_depends "Unix Makefiles" /Users/demo/devel/boost-python-examples /Users/demo/devel/boost-python-examples/01-HelloWorld /Users/demo/devel/boost-python-examples/build /Users/demo/devel/boost-python-examples/build/01-HelloWorld /Users/demo/devel/boost-python-examples/build/01-HelloWorld/CMakeFiles/hello.dir/DependInfo.cmake --color=
/Applications/Xcode.app/Contents/Developer/usr/bin/make -f 01-HelloWorld/CMakeFiles/hello.dir/build.make 01-HelloWorld/CMakeFiles/hello.dir/build
Linking CXX shared module hello.so
cd /Users/demo/devel/boost-python-examples/build/01-HelloWorld && /usr/local/Cellar/cmake/3.0.2/bin/cmake -E cmake_link_script CMakeFiles/hello.dir/link.txt --verbose=1
/usr/bin/c++ -g -bundle -Wl,-headerpad_max_install_names -o hello.so CMakeFiles/hello.dir/hello.cpp.o /usr/local/lib/libboost_python-mt.dylib
Undefined symbols for architecture x86_64:
"_PyString_Type", referenced from:
boost::python::to_python_value<char const* const&>::get_pytype() const in hello.cpp.o
"__Py_NoneStruct", referenced from:
boost::python::api::object::object() in hello.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [01-HelloWorld/hello.so] Error 1
make[1]: *** [01-HelloWorld/CMakeFiles/hello.dir/all] Error 2
make: *** [all] Error 2
It looks like its calling:
/usr/bin/c++ -stdlib=libstdc++ -g -bundle -Wl,-headerpad_max_install_names -o hello.so CMakeFiles/hello.dir/hello.cpp.o /usr/local/lib/libboost_python-mt.dylib
I've tried the following:
Removing the -stdlib=libstdc++
Using gcc instead of c++
Using -m32 (bad idea)
As far as i can tell the dynlib is good:
/usr/local/lib/libboost_python-mt.dylib: Mach-O 64-bit dynamically linked shared library x86_64
So I'm not sure where my issue currently is. I'm assuming its some sort of build flag i'm missing here.
Thanks!

You also need to link in the Python library.
In your CMakeLists.txt you can add
include(FindPythonLibs)
This will define a variable PYTHON_LIBRARIES which you can then use like this
target_link_libraries(your_exe ${PYTHON_LIBRARIES ...)

Try to force cmake to use the right directories:
cmake -DBOOST_ROOT=xxx -DPYTHON_LIBRARY=xxx -DPYTHON_INCLUDE_DIR=xxx ..

Pygame - smpeg does not find Python headers

I've been trying to install Pygame for days now, and the only dependency missing is smpeg. I get the following output:
brew install --HEAD smpeg
==> Installing dependencies for smpeg: gobject-introspection, gdk-pixbuf,
==> Installing smpeg dependency: gobject-introspection
==> Building source; bottle blocked by python requirement
==> Downloading http://ftp.gnome.org/pub/GNOME/sources/gobject-introspection/1.4
Already downloaded: /Library/Caches/Homebrew/gobject-introspection-1.40.0.tar.xz
==> ./configure --prefix=/usr/local/Cellar/gobject-introspection/1.40.0
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/sitepackages
checking for headers required to compile python extensions... ./configure: line 14647: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/pytho n-config: No such file or directory
not found
configure: error: Python headers not found
I've also tried to brew edit smpeg, adding include.install Dir["*.h"] above the two end at the end of the code, but to no avail.
My brew doctor says:
Warning: Python is installed at /Library/Frameworks/Python.framework
Homebrew only supports building against the System-provided Python or a
brewed Python. In particular, Pythons installed to /Library can interfere
with other software installs.
My brew list says:
autoconf git libgcrypt mpfr sdl_mixer
automake glib libgpg-error pkg-config sdl_ttf
cloog gmp libmpc portmidi tree
cmake gnu-getopt libpng pwgen webp
freetype isl libtiff sdl xz
gettext jpeg libtool sdl_gfx
gfortran libffi mercurial sdl_image
brew --config:
==> Configuration
HOMEBREW_VERSION: 0.9.5
HEAD: 7448fd1532ae1c1709fe2f03ffe0dc188a134b3e
CPU: quad-core 64-bit ivybridge
OS X: 10.9.2-x86_64
Xcode: 5.1.1
CLT: 5.1.0.0.1.1396320587
X11: 2.7.5 => /opt/X11
brew was updated and upgraded
I have Python 2.7.6
Later I've managed to install smpeg.h (and MPEGfilter.h) in Python.framework/Versions/2.7/include/python and now when I run python setup.py install from cd pygame-1.9.1release, I get:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ddarwin -I/Library/Frameworks/SDL.framework/Versions/Current/Headers -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/movie.c -o build/temp.macosx-10.3-fat-2.7/src/movie.o
In file included from src/movie.c:26:
In file included from src/pygame.h:106:
In file included from /Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL.h:30:
In file included from /Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_main.h:26:
In file included from /Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_stdinc.h:30:
In file included from /Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_config.h:34:
In file included from /Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_config_macosx.h:29:
/usr/include/AvailabilityMacros.h:110:14: warning: Building for Intel with Mac
OS X Deployment Target < 10.4 is invalid. [-W#warnings]
#warning Building for Intel with Mac OS X Deployment Target ...
^
1 warning generated.
/usr/bin/clang -bundle -undefined dynamic_lookup -arch i386 -g build/temp.macosx-10.3-fat-2.7/src/movie.o -ls -lm -lp -le -lg -o build/lib.macosx-10.3-fat-2.7/pygame/movie.so -framework SDL -F/Library/Frameworks/
ld: library not found for -ls
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/bin/clang' failed with exit status 1

got it. did this: inside 'pygame-1.9.1release', 'cd src, emacs scale_mmx64.c'. there I looked for all occurrences of 'movsxl' and replaced them (two) with with 'movslq'. then 'sudo pip install hg+http://bitbucket.org/pygame/pygame worked'.

Fatal error while compiling PyQt5: Python.h does not exist

I'm trying to install PyQt5 on my Ubuntu 12.04 box. So after downloading it from here I untarred it, ran python configure.py and make. Make however, results in the following:
cd qpy/ && ( test -f Makefile || /opt/qt5/bin/qmake /home/kram/Downloads/PyQt-gpl-5.0/qpy/qpy.pro -o Makefile ) && make -f Makefile
make[1]: Map '/home/kram/Downloads/PyQt-gpl-5.0/qpy' is entered
cd QtCore/ && ( test -f Makefile || /opt/qt5/bin/qmake /home/kram/Downloads/PyQt-gpl-5.0/qpy/QtCore/QtCore.pro -o Makefile ) && make -f Makefile
make[2]: Map '/home/kram/Downloads/PyQt-gpl-5.0/qpy/QtCore' is entered
g++ -c -pipe -fno-strict-aliasing -O2 -Wall -W -fPIC -D_REENTRANT -DQT_NO_DEBUG -DQT_CORE_LIB -I/opt/qt5/mkspecs/linux-g++ -I. -I. -I../../QtCore -I/usr/local/include/python2.7 -I/opt/qt5/include -I/opt/qt5/include/QtCore -I. -o qpycore_chimera.o qpycore_chimera.cpp
qpycore_chimera.cpp:21:20: fatal error: Python.h: File or folder does not exist
compilation terminated.
make[2]: *** [qpycore_chimera.o] Error 1
make[2]: Map '/home/kram/Downloads/PyQt-gpl-5.0/qpy/QtCore' is left
make[1]: *** [sub-QtCore-make_first] Error 2
make[1]: Map '/home/kram/Downloads/PyQt-gpl-5.0/qpy' is left
make: *** [sub-qpy-make_first-ordered] Error 2
(I translated some parts of the error message from Dutch to English, so some words may be a bit off from the normal wording..)
Does anybody what the problem is here? Where could the relevant Python.h file be?

The problem is that the include path for all python headers in every Makefile will be pointing to /usr/local/include/python2.7 , which should have been /usr/include/python2.7
There are 2 possible solutions for this. Either you can change all the occurrence of this in every Makefile or else you can create a symlink to that location
sudo ln -s /usr/include/python2.7 /usr/local/include/python2.7
Now you can run make

sudo apt-get install python-dev
Your missing the python header files.

The problem you're having is that PyQt assumes you're not using your distro's managed python, and instead defaults to looking for sip in /usr/local/include/python2.7.
Luckily, configure.py provides options to override the python and sip include locations:
python configure.py --sip-incdir /usr/include/python2.7 py_inc_dir=/usr/include/python2.7
This solution should preferred to symlinking /usr/include/python2.7 into /usr/local/include/python2.7 as that will enable manually installed software to pollute (or corrupt packages installed to) distro-managed paths.

It is better to add existing header files to the project directory in both QTCreator and Anjuta IDE.