Develop Reference

Compiling source code that uses Boost, wxWidgets and Python with MingW

I have been trying to compile a source code but every time I use the make command, it gives me the error:
In file included from ../src/MadEditFrame.cpp:60:0:
../src/EmbeddedPython.hpp:11:20: fatal error: Python.h: No such file or directory
#include <Python.h>
I had been able to pass the Boost and wxWidget part. This Python part is what makes no sense. I have Python 3.8 installed, it is recognised by system when in command line i use the Python --version command as 3.8.2 g++ too is recognised as 6.3.0 with Mingw.
Update:
The code has three makefiles with terminations: install, mac and mingw. I'm using the one with mingw by copying and deleting termination. The content of said file is too long, but there is a part that goes this:
...
./Release/nsUTF8Prober.obj \
$(RES)
LINKOBJ = $(OBJ)
#WXWIN = D:\wxWidgets28_SVN
WXWIN = C:\wxWidgets-3.1.5
#XPRESSIVE_DIR = N:/boost
XPRESSIVE_DIR = ../xpressive
#BOOST_DIR = D:/boost
BOOST_DIR = C:\Programs\boost_1_77_0\boost
LIBS = -L$(WXWIN)/lib/gcc_lib -L"../libunicows" -mwindows -lwxmsw28u_core -lwxbase28u -lunicows -lkernel32 -luser32 -lgdi32 -lcomdlg32 -lwinspool -lwinmm -lshell32 -lcomctl32 -lole32 -loleaut32 -luuid -lrpcrt4 -ladvapi32 -lwsock32 -lodbc32 -lopengl32 -lgmon -s
#-g -pg
CXXINCS = -I$(XPRESSIVE_DIR) -I$(BOOST_DIR) -I"../charset-detector/include" -I"../charset-detector/src" -I"../charset-detector/src/tables" -I$(WXWIN)/include -I$(WXWIN)/lib/gcc_lib/mswu
BIN = Release/MadEdit.exe
CXXFLAGS = $(CXXINCS) -D_UNICODE -D__WXMSW__ -D__GNUWIN32__ -D__WIN95__ -DWIN32 -O2
#-g -pg
RM = rm -f
.PHONY: all all-before all-after clean clean-custom
all: all-before Release/MadEdit.exe all-after
...
I added the paths under the commented ones for WXWIN and BOOSTDIR

waf check_python_headers fails with python installed with pyenv

I'm trying to build a project using the waf build tool.
The build fails in the configure step when checking the python headers.
I narrowed the problem down to this minimal example of my wscript:
def options(ctx):
ctx.load('compiler_c')
def configure(ctx):
ctx.load('compiler_c python')
ctx.check_python_version((2,7,0))
if int(ctx.env.PYTHON_VERSION[0]) == 2:
print ('→ Configuring for python2')
else:
print ('→ Configuring for python3')
ctx.check_python_headers(features='pyext')
I use python versions (3.6.12 & 3.7.9) that I installed with pyenv on Ubuntu 20.04.
I run that like this and it fails with an error that I cannot make sense of:
$ python waf configure
Setting top to : /home/myuser/waf-test
Setting out to : /home/myuser/waf-test/build
Checking for 'gcc' (C compiler) : /usr/bin/gcc
Checking for program 'python' : /home/myuser/.pyenv/versions/3.7.9/bin/python
Checking for python version >= 2.7.0 : 3.7.9
→ Configuring for python3
python-config : /home/myuser/.pyenv/versions/3.7.9/bin/python-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /home/myuser/waf-test/build/config.log)
$ tail -n 20 build/config.log
[1/2] Compiling build/.conf_check_d3b505aa7ab58576b6d76a2fc3091b1f/test.c
['/usr/bin/gcc', '-fPIC', '-g', '-fwrapv', '-O3', '-I../../../../.pyenv/versions/3.7.9/include/python3.7m', '-DPYTHONDIR="/usr/local/lib/python3.7/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.7/site-packages"', '-DNDEBUG', '../test.c', '-c', '-o/home/myuser/waf-test/build/.conf_check_d3b505aa7ab58576b6d76a2fc3091b1f/testbuild/test.c.1.o']
[2/2] Linking build/.conf_check_d3b505aa7ab58576b6d76a2fc3091b1f/testbuild/testprog.cpython-37m-x86_64-linux-gnu.so
['/usr/bin/gcc', '-shared', 'test.c.1.o', '-o/home/myuser/waf-test/build/.conf_check_d3b505aa7ab58576b6d76a2fc3091b1f/testbuild/testprog.cpython-37m-x86_64-linux-gnu.so', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/home/myuser/.pyenv/versions/3.7.9/lib/python3.7/config-3.7m-x86_64-linux-gnu', '-L/home/myuser/.pyenv/versions/3.7.9/lib', '-lpython3.7m', '-lcrypt', '-lpthread', '-ldl', '-lutil', '-lm', '-lpython3.7m', '-lcrypt', '-lpthread', '-ldl', '-lutil', '-lm']
err: /usr/bin/ld: /home/myuser/.pyenv/versions/3.7.9/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a(pylifecycle.o): relocation R_X86_64_PC32 against symbol `Py_VerboseFlag' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
from /home/myuser/waf-test: Test does not build: Traceback (most recent call last):
File "/home/myuser/waf-test/.waf3-2.0.20-36f5354d605298f6a89c09e0c7ef6c1d/waflib/Configure.py", line 335, in run_build
bld.compile()
File "/home/myuser/waf-test/.waf3-2.0.20-36f5354d605298f6a89c09e0c7ef6c1d/waflib/Build.py", line 176, in compile
raise Errors.BuildError(self.producer.error)
waflib.Errors.BuildError: Build failed
-> task in 'testprog' failed with exit status 1 (run with -v to display more information)
Could not build python extensions
from /home/myuser/waf-test: The configuration failed
Note that this problem does not appear with python 3.8.6 that was also installed with pyenv. I can configure without errors.
My questions: How can it be that the python versions in pyenv differ in a way that waf checks fail in one version and not in another?
Since I simply install the versions with pyenv install 3.6.12, what do I need to change in the install so my build succeeds?

Well, the error is that, when creating the test program to check if pyext is correctly set, the compiler complains about the fact that pylifecycle.o included in libpython3.7m.a is non relocatable ie non compiled with the -fPIC flag.
As the code fragment generated by waf is compiled with -fPIC, it seems that the problem is that pylifecycle.o in libpython3.7m.a is not compiled with -fPIC, which makes sense as it is in a static library. Maybe the problem is that it uses the .a archive lib instead of the shared object one ie the .so.
Check if you have the shared object version of libpython ie libpython3.7m.so.

PyCaffe build fails, lboost_python not found

I am on MacOS 10.13.3 (17D47) with Python 2.7.14. I am in the process of building caffe w/ python. The project is CPU only. I am able to build caffe through make run test however, after attempting make pycaffe I get an error. This is the result:
touch python/caffe/proto/__init__.py
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
PROTOC (python) src/caffe/proto/caffe.proto
In file included from python/caffe/_caffe.cpp:17:
In file included from ./include/caffe/caffe.hpp:12:
./include/caffe/net.hpp:41:5: warning: unused typedef 'INVALID_REQUESTED_LOG_SEVERITY' [-Wunused-local-typedef]
LOG_EVERY_N(WARNING, 1000) << "DEPRECATED: ForwardPrefilled() "
^
/usr/local/include/glog/logging.h:943:30: note: expanded from macro 'LOG_EVERY_N'
INVALID_REQUESTED_LOG_SEVERITY); \
^
1 warning generated.
ld: library not found for -lboost_python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [python/caffe/_caffe.so] Error 1
I have a Makefile.config that calls out boost-python from the brew install set as:
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/local/Cellar/boost-python/1.67.0/lib
My two initial questions are:
1)Am I not allowed to link to the boost library using the system python? 2)What is the correct way to tell make to grab boost-python?

I was actually having the same problem and couldn't find any answers for days. However, I found this question: Build caffe with Python ( cannot find -lboost_python3 ) .
That question is for python3, but I'm using python2.7 . Basically, I went into the Makefile (not Makefile.config), searched for boost_python, and changed it to boost_python27 . I did the following steps:
vim Makefile
use vim command :?boost_python (there should only be 1 occurrence)
changed PYTHON_LIBRARIES ?= boost_python python2.7
to PYTHON_LIBRARIES ?= boost_python27 python2.7
ran the command sudo make pycaffe
I hope this helps you!

Goto Makefile (and not the .config file) and change the name of the ld library to python27.

Compile mod_wsgi with non-default Apache Webserver?

I am trying to install mod_wsgi for a updated version of Apache.
I currently have Apache 2.4 installed via /opt/rh/httpd24/root/etc/httpd.
I am wanting to compile with it using 2.4 not 2.2. Any help on the syntax here? I can't find any reference to that specific argument.
[root#bmograba mod_wsgi-4.4.13]# ./configure -help
`configure' configures this package to adapt to many kinds of systems.
Usage: ./configure [OPTION]... [VAR=VALUE]...
To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE. See below for descriptions of some of the useful variables.
Defaults for the options are specified in brackets.
Configuration:
-h, --help display this help and exit
--help=short display options specific to this package
--help=recursive display the short help of all the included packages
-V, --version display version information and exit
-q, --quiet, --silent do not print `checking ...' messages
--cache-file=FILE cache test results in FILE [disabled]
-C, --config-cache alias for `--cache-file=config.cache'
-n, --no-create do not create output files
--srcdir=DIR find the sources in DIR [configure dir or `..']
Installation directories:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local]
--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
[PREFIX]
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc. You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
For better control, use the options below.
Fine tuning of the installation directories:
--bindir=DIR user executables [EPREFIX/bin]
--sbindir=DIR system admin executables [EPREFIX/sbin]
--libexecdir=DIR program executables [EPREFIX/libexec]
--sysconfdir=DIR read-only single-machine data [PREFIX/etc]
--sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
--localstatedir=DIR modifiable single-machine data [PREFIX/var]
--libdir=DIR object code libraries [EPREFIX/lib]
--includedir=DIR C header files [PREFIX/include]
--oldincludedir=DIR C header files for non-gcc [/usr/include]
--datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
--datadir=DIR read-only architecture-independent data [DATAROOTDIR]
--infodir=DIR info documentation [DATAROOTDIR/info]
--localedir=DIR locale-dependent data [DATAROOTDIR/locale]
--mandir=DIR man documentation [DATAROOTDIR/man]
--docdir=DIR documentation root [DATAROOTDIR/doc/PACKAGE]
--htmldir=DIR html documentation [DOCDIR]
--dvidir=DIR dvi documentation [DOCDIR]
--pdfdir=DIR pdf documentation [DOCDIR]
--psdir=DIR ps documentation [DOCDIR]
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
--enable-framework enable mod_wsgi framework link
--disable-embedded disable mod_wsgi embedded mode
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-apxs=NAME name of the apxs executable [[apxs]]
--with-python=NAME name of the python executable [[python]]
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
LIBS libraries to pass to the linker, e.g. -l<library>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
you have headers in a nonstandard directory <include dir>
Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.
Report bugs to the package provider.
Compiled plane with no arguments:
[root#bmograba mod_wsgi-4.4.13]# ./configure
checking for apxs2... no
checking for apxs... /usr/sbin/apxs
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for prctl... yes
checking Apache version... 2.2.15
checking for python... /usr/bin/python
configure: creating ./config.status
config.status: creating Makefile
Just to see what package output would be looking for:
[root#bmograba mod_wsgi-4.4.13]# ./configure --with-PACKAGE
configure: WARNING: you should use --build, --host, --target
configure: WARNING: unrecognized options: --with-PACKAGE
checking for apxs2... no
checking for apxs... /usr/sbin/apxs
checking for httpd-gcc... no
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for prctl... yes
checking Apache version... 2.2.15
checking for python... /usr/bin/python
configure: creating ./config.status
config.status: creating Makefile
configure: WARNING: unrecognized options: --with-PACKAGE

When they say, --with-PACKAGE, PACKAGE is just a placeholder for a package name. You aren't supposed to literally put --with-PACKAGE.

Find where the 'apxs' script is inside of the alternate Apache installation. Then run:
./configuure --with-apxs /opt/rh/httpd24/bin/apxs
where that argument is the location of that 'apxs' script.
https://code.google.com/p/modwsgi/wiki/QuickInstallationGuide#Configuring_The_Source_Code

How to remove a compiler option that python disutils includes (language code=pic32 not recognized)

I'm trying to compile the msgpack-python python module with gcc (v4.7) on solaris 10. The python installed is 2.6.8. Distutils is automatically picking up a incorrect compiler option (-xcode=pic32) that I want to remove from command.
The full command that distutils is putting together is:
/opt/csw/bin/gcc-4.7 -DNDEBUG -O -O2 -pipe -mcpu=v9 -I/opt/csw/include -xcode=pic32 -I/opt/csw/include/python2.6 -c msgpack/_msgpack.c -o build/temp.solaris-2.10-sun4v-2.6/msgpack/_msgpack.o
but produces this error:
gcc-4.7: error: language code=pic32 not recognized
then fails. If I remove that -xcode=pic32 option and manually execute the above command the module compiles successfully.
I need to be able to do this in an automated fashion though (using a buildfarm to produce the packages). The question is, Without modifying or changing the current python or distutils, is there a way to "remove" this option that distutils is picking up, so I can have the python setup.py process build the module appropriately (i.e. without the pic32 option)?
Thanks

Do not compile with that gcc. -xcode=pic32 is Sun Studio complier command line parameter. It will lead to linking problems too, even if you compile OK. Compile with SUN CoolTools gcc which can understand such parameter, or use Oracle Solaris Studio for SPARC.
Some hints:
GCC produce very slow code for SPARC, that's for why SUN created Cool Tools.
You haven't to remove -xcode=pic32, but change for -m32 -fpic, when you insist on gcc-4.7
To get mature setup of OSS tool I'm using pkgsrc compiling with Studio Express to particular CPU (-xtarget=native)

You may also find luck by setting the follow env vars:
export CC=$gcc_dir_path # Example: /usr/bin/gcc
export CXX=$gxx_dir_path # Example: /usr/bin/g++
export CFLAGS=''
export CPPFLAGS=''
export CXXFLAGS=''
export LDFLAGS=''
Note: There is a difference between unset env var, and set-as-empty env var. I had build bugs with Python packages when my *FLAGS env vars were unset. (Calling gcc with option -xO2 was the cause.) Setting as empty did the trick.