Develop Reference

Preventing spurious -m32 flag when running running setup.py build_ext

I'm trying to cross-compile numpy for arm (BeagleBone & RPi). I am running on debian buster, with a 32bit cross-compilation toolchain, including i386 (32bit) libraries and all necessary arm toolchains.
When I run:
export CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar RANLIB=arm-linux-gnueabihf-ranlib; \
python3 setup.py build_ext --inplace
I get the following error:
C compiler: arm-linux-gnueabihf-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -m32 -I/usr/include/i386-linux-gnu/ -m32 -I/usr/include/i386-linux-gnu/ -fPIC
compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -c'
arm-linux-gnueabihf-gcc: _configtest.c
arm-linux-gnueabihf-gcc: error: unrecognized command line option ‘-m32’; did you mean ‘-mbe32’?
I've traced through numpy's setup.py file and found that ultimately these settings seem to be generated in the distutils.core._setup_distribution.get_command_obj("build") call.
The error is quite self explanatory, and trying to run the command without the -m32 flags seems to work correctly.
Edit: further digging has revealed that these parameters come to the build script via:
sysconfig.get_config_var("LDSHARED")
I'm still stumped because the following still doesn't work:
$LDSHARED="gcc -pthread -shared" python3 -c "import sysconfig; print(sysconfig.get_config_var('LDSHARED'))"
gcc -pthread -shared -m32 -m32

Turns out this LDSHARED parameter is coming from the Makefile that was used during the build of the python host itself. Under most scenarios it makes sense to have it this way, but a combination of cross-compiling and the arm gcc not accepting that flag made it problematic...
This may very well be unresolvable and I have instead resorted to capturing the parameter from the gcc itself via a symlink/bash script.

boost python library linking issue -- undefined symbol

I am using boost-python built for python3 to expose a simple hello-world program. The example can be found here : https://github.com/TNG/boost-python-examples/blob/master/01-HelloWorld/hello.cpp
I ran the following commands to get the shared object:
g++ -fPIC -c -I/usr/include/python3.4m -I/usr/include/python3.4m -Wno-unused-result -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -L/usr/lib/x86_64-linux-gnu/libboost_python-py34 hello.cpp
g++ -shared hello.o -o hello.so
After this, I run the python3 -c 'import hello' command and I get the following error:
Traceback (most recent call last):
File "", line 1, in
ImportError: hello.so: undefined symbol: _ZTIN5boost6python7objects21py_function_impl_baseE
I partly understand this issue may be because my boost-python installation may be built for an alternative python version (for instance python2.7). When I run the command:
ls /usr/lib/x86_64-linux-gnu/libboost_python*.so
There are three .so files:
1. libboost_python-py27.so
2. libboost_python-py34.so
3. libboost_python.so
How can this issue be circumvented?

use pkg-config to retrieve ldflags and cflags of your boost library

Installing miniconda might be an option for you. https://conda.io/miniconda.html
This will provide a complete, isolated, python environment. You can then
conda install boost
I've tested this on my system and it worked well. I modified the Makefile from http://www.shocksolution.com/python-basics-tutorials-and-examples/linking-python-and-c-with-boostpython/
My Makefile can be found here:
https://github.com/grelleum/boost-python-with-anaconda

OSX: pyosmium setup error

I'm running a setup Python script for Pyosmium on OSX 10.11, Python version is 2.7:
python setup.py install
running install
running build
running build_py
running build_ext
building 'osmium._osmium' extension
gcc -fno-strict-aliasing -I/Users/stephen/anaconda/include -arch x86_64
-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include
-I../libosmium/include -I/Users/stephen/anaconda/include/python2.7 -c
lib/osmium.cc -o build/temp.macosx-10.5-x86_64-2.7/lib/osmium.o -std=c++0x
-stdlib=libstdc++ -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
In file included from lib/osmium.cc:1:
../libosmium/include/osmium/visitor.hpp:36:10: fatal error: 'type_traits' file not found
As far as possible I've most available dependencies using Brew. And added required directories for supporting developments.
Though it's clear I am dealing with a level here that I am not at all familiar with.
My guess at the moment is the compiler that's been called - gcc, should this be Any ideas would be much appreciated!
Thank you,
Stephen.

Failed to build the bsdiff module

Good afternoon all,
I am having trouble using the bsdiff module with Python. While I can use shell scripts I would prefer to have a cross-platform solution.
I have downloaded bsdiff4-1.1.4 and attempt to run setup.py as follows:
Files list:
bsdiff4 build CHANGELOG.txt do.sh examples Makefile README.rst setup.py
I run:
python3 setup.py build
And I get this error:
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
running build
running build_py
running build_ext
building 'bsdiff4.core' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c bsdiff4/core.c -o build/temp.linux-x86_64-3.4/bsdiff4/core.o
bsdiff4/core.c:8:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I want to be able to use python to apply patches to binary files.
Any help appreciated.
José

You need to install apt-get install python3-dev.
ubuntu python3-dev
header files and a static library for Python
If it is your system python you will need sudo

Install Twisted in python failed with 'No such file'

I want use Twisted in Python, but when I installing ,in comes this error, how to handle it?
....
running build_ext
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
building 'twisted.runner.portmap' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c twisted/runner/portmap.c -o build/temp.linux-i686-2.7/twisted/runner/portmap.o
twisted/runner/portmap.c:10:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

You are missing the python development headers, needed to build packages that need to compile extensions.
If you are building on OSX, make sure you either install a prebuilt mac python package or if building python from source, use the framework flag when configuring. And also make sure you have XCODE installed so that you have a compiler.
If you are building on Linux, you probably need to install the python devel headers. For instance on Ubuntu you would need: apt-get install build-essential python-dev.
Once you have the python development headers, twisted should be able to find them when you build with that python interpreter.