Develop Reference

How to fix: "Testing pyext configuration : Could not build python extensions"

I am trying to install wxPython but the wheel build fails. The error message is not helpful in indicating what to do or where to look to fix this. Can anyone please help me understand how to build this wheel correctly?
Machine: Linux on Power (this is not x86)
OS: RHEL Server, 7.5 (Maipo)
python version: Python 3.6.4
pip3 version: pip 19.3.1
I noticed this stack overflow post, which is also not helpful because my linux release is not on the list of the ones provided.
Following links above I tried wxPython download page and the following install with pip but in step 5 basically tells you "look at the log and figure it out"....not helpful.
I tried to manually hack the wxPython package using my very limited competence and removed some dependency.....still nothing.
<...>
Finished command: build_wx (1m56.907s)
Running command: build_py
Checking for /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build
Setting top to : /tmp/pip-req-build-dgnp13sp
Setting out to : /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3/config.log)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (0m6.991s)
Finished command: build (2m3.899s)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" -u build.py build' failed with exit code 1.
Building wheel for wxPython (setup.py): finished with status 'error'
ERROR: Failed building wheel for wxPython
<...>
----Update 12/2 (After Robin Dunn's feedback)
Thank you Robin for the directives. Following is the last portion of the config.log from a different run using the build command. Also the error message is slightly different (below, following the config.log), because I had previously used the explicit wheel-build command: pip wheel -v wxPython-4.0.7.post1.tar.gz 2>&1 | tee build.log. Does this log below confirm your theory regarding the Python built with the --enable-shared configure flag?
Testing pyext configuration
==>
#include <Python.h>
#ifdef __cplusplus
extern "C" {
#endif
void Py_Initialize(void);
void Py_Finalize(void);
#ifdef __cplusplus
}
#endif
int main(int argc, char **argv)
{
(void)argc; (void)argv;
Py_Initialize();
Py_Finalize();
return 0;
}
<==
[1/2] Compiling [32mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/test.cpp[0m
['/bin/g++', '-fPIC', '-g', '-fwrapv', '-O3', '-I../../../../../../../../../../../../../../cte/tools/python/vol2/.3.6.4-linux-ppc64le/include/python3.6m', '-I/opt/xsite/cte/tools/python/common2018/include', '-DPYTHONDIR="/usr/local/lib/python3.6/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.6/site-packages"', '-DNDEBUG', '../test.cpp', '-c', '-o/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/test.cpp.1.o']
[2/2] Linking [33mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so[0m
['/bin/g++', '-shared', '-Xlinker', '-export-dynamic', 'test.cpp.1.o', '-o', '/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm']
err: /bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_keyword':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7767:(.text.unlikely+0x608): call to `_Py_keyword' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_comprehension':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7419:(.text.unlikely+0x9f4): call to `_Py_comprehension' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_alias':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7802:(.text.unlikely+0xbec): call to `_Py_alias' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_withitem':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7837:(.text.unlikely+0xdd4): call to `_Py_withitem' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
from /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1: Test does not build: Traceback (most recent call last):
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Configure.py", line 324, in run_build
bld.compile()
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Build.py", line 176, in compile
raise Errors.BuildError(self.producer.error)
waflib.Errors.BuildError: Build failed
-> task in 'testprog' failed with exit status 1 (run with -v to display more information)
Could not build python extensions
from /.....: The configuration failed
and this is the error message that I get this new run, slightly different
msgfmt --verbose -c -o zh_TW.mo zh_TW.po
1710 translated messages, 82 fuzzy translations, 61 untranslated messages.
make[1]: Leaving directory `/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale'
Setting top to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1
Setting out to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/config.log)
Will build using: "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3"
3.6.4 (default, Feb 12 2018, 16:08:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Python's architecture is 64bit
cfg.VERSION: 4.0.7.post1
Running command: build
Running command: build_wx
wxWidgets build options: ['--wxpython', '--unicode', '--gtk3']
Configure options: ['--enable-unicode', '--with-gtk=3', '--enable-sound', '--enable-graphics_ctx', '--enable-display', '--enable-geometry', '--enable-debug_flag', '--enable-optimise', '--disable-debugreport', '--enable-uiactionsim', '--enable-autoidman', '--with-sdl']
/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/configure --enable-unicode --with-gtk=3 --enable-sound --enable-graphics_ctx --enable-display --enable-geometry --enable-debug_flag --enable-optimise --disable-debugreport --enable-uiactionsim --enable-autoidman --with-sdl
make --jobs=128
Building message catalogs in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale
make allmo
Finished command: build_wx (12m36.623s)
Running command: build_py
Checking for /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build
Command '"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (2m7.118s)
Finished command: build (14m43.742s)

It looks like you might be using a custom Python build. Was your Python built with the --enable-shared configure flag? On Linux platforms the waf build tool assumes that Python was built that way, and will try to link with the Python shared library, and will fail if it is not present.
If that doesn't help then you can get more details from waf's configure log. When using pip to do the build then it will have removed the temporary build folders before you can get at them. So if you unpack the source archive and do the build with python3 build.py build you'll be able to find the log in ./build/waf/3.6/gtk3/config.log and hopefully find some useful clues there.

f2py: command not found

Mac OS Sierra.
In terminal, while trying to execute a .sh file that involves a Python script with Fortran portions, I got this message:
lenscov-master Roger$ ./run_covariances.sh recompile
rm *.o *.so
rm: *.so: No such file or directory
make: [clean] Error 1 (ignored)
gfortran -fPIC -c *.f -lgfortran -lifcore
f2py-2.7 -c loop_lensing.f90 *.o -m loop_lensing --opt=-ffixed-line-length-
none --opt=-O3
/bin/sh: f2py-2.7: command not found
make: *** [loop_lensing] Error 127
However, it is fine just test-running the .f90 file:
f2py -c loop_lensing.F90 -m test
returns a bunch of normal-looking information.
Does anyone maybe know what is going on?

it is fine just test-running the .f90 file
means you have f2py installed. But the script use f2py-2.7
You need to check the version of installed f2py, and make sure f2py-2.7 redirects to the corrent f2py.

The script (or a Makefile) tries to invoke f2py-2.7 instead of f2py. If it is your script or you can easily adjust it, rename all f2py-2.7 to f2py. Otherwise you have to tell us more about the script and show the code of the script.
As foodtooth says, make sure you have f2py 2.7 and not f2py-3, but I don't expect that to be a problem.
You could also make a symlink f2py -> f2py-2.7 for the script.

NS-3 build fails due to g++ compiler error

I am trying to build NS3 on Raspbian (Jessie) and ran into this error:
...
[1507/1924] Compiling src/internet/model/ripng-header.cc
[1508/1924] Compiling src/internet/helper/ripng-helper.cc
[1509/1924] Compiling src/internet/helper/rip-helper.cc
[1510/1924] Compiling build/src/internet/bindings/ns3module.cc
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
Waf: Leaving directory `/home/pi/tarballs/ns-allinone-3.26/ns-3.26/build'
Build failed
-> task in 'ns3module_internet' failed (exit status 4):
{task 3056100368L: cxx ns3module.cc -> ns3module.cc.8.o}
['/usr/bin/g++', '-O0', '-ggdb', '-g3', '-Wall', '-Werror', '-std=c++11', '-Wno-error=deprecated-declarations', '-fstrict-aliasing', '-Wstrict-aliasing', '-fPIC', '-fno-strict-aliasing', '-fstack-protector-strong', '-fwrapv', '-fvisibility=hidden', '-Wno-array-bounds', '-pthread', '-pthread', '-I.', '-I..', '-Isrc/internet/bindings', '-I../src/internet/bindings', '-I/usr/include/python2.7', '-I/usr/include/arm-linux-gnueabihf/python2.7', '-I/usr/include/gtk-2.0', '-I/usr/lib/arm-linux-gnueabihf/gtk-2.0/include', '-I/usr/include/gio-unix-2.0', '-I/usr/include/cairo', '-I/usr/include/pango-1.0', '-I/usr/include/atk-1.0', '-I/usr/include/pixman-1', '-I/usr/include/libpng12', '-I/usr/include/gdk-pixbuf-2.0', '-I/usr/include/harfbuzz', '-I/usr/include/glib-2.0', '-I/usr/lib/arm-linux-gnueabihf/glib-2.0/include', '-I/usr/include/freetype2', '-I/usr/include/libxml2', '-DNS3_BUILD_PROFILE_DEBUG', '-DNS3_ASSERT_ENABLE', '-DNS3_LOG_ENABLE', '-DHAVE_SYS_IOCTL_H=1', '-DHAVE_IF_NETS_H=1', '-DHAVE_NET_ETHERNET_H=1', '-DHAVE_PACKET_H=1', '-DHAVE_IF_TUN_H=1', '-DNS_DEPRECATED=', '-DNS3_DEPRECATED_H', '-DHAVE_PYEXT=1', '-D_FORTIFY_SOURCE=2', '-DNDEBUG', '-DHAVE_GSL=1', '-DHAVE_SQLITE3=1', '-DHAVE_GTK2=1', '-DHAVE_LIBXML2=1', 'src/internet/bindings/ns3module.cc', '-c', '-o', '/home/pi/tarballs/ns-allinone-3.26/ns-3.26/build/src/internet/bindings/ns3module.cc.8.o']
Waf died. Not running tests
I have searched SO and none of the existing questions seem to be relevant. There definitely seems to be something wrong in general because it takes many hours just to get to this point, when I think(?) it should take < 1 hour to build NS-3.

Not sure if you're still looking for a solution, however the problem seems to be related to memory constraints. I solved the issue by enabling/configuring a swapfile.
sudo dd if=/dev/zero of=/swapfile bs=64M count=16
sudo mkswap /swapfile
sudo swapon /swapfile
After you build the project, you have the option to disable the swapfile (only if you want). To do this type this in your terminal:
sudo swapoff /swapfile
sudo rm /swapfile
Hope this helps!

I had a similar problem.
It seems to be either a problem with the ns3 version in combination with python, gcc versions, or, a problem with the RAM or GPU memory.
So, I set up a new Ubuntu VM with double the RAM and did not use the latest ns3 version (ns3 3.33)
Since ns3 version 2.xx are based on python2, I would recommend versions 3.xx.

Python missing or unusable while compiling gdb under Cent OS 5

I checked this question and it is not applicable.
Python missing or unusable error while cross compiling GDB
I'm trying to build gdb 7.5 with python support on Cent OS 5.
configure --with-expat --with-python
Does not log any errors and the Python devel files are installed.
I set LDFLAGS="L/usr/local/expat-2.1 -L/opt/python27/lib
/opt/python27/lib contains:
rwxrwxrwx 1 root root 19 Mar 14 15:35 libpython2.7.so -> libpython2.7.so.1.0
-r-xr-xr-x 1 root root 1893448 Aug 30 2012 libpython2.7.so.1.0
drwxr-xr-x 2 root root 4096 Mar 14 15:35 pkgconfig
drwxr-xr-x 27 root root 20480 Mar 14 21:49 python2.7
Running make produces this error:
checking whether to use python... yes
checking for python... /opt/python27/bin/python
checking for python2.7... no
configure: error: python is missing or unusable
/opt/python27/bin/python exists has the correct permissions and is executable.
Why can't the link process find python?

The next line AFTER the configure: error: line should be something like:
make[1]: [configure-gdb]: ....
telling you that this error comes from running configure in the gdb subdirectory. So you should look in the config.log file there. This should show you something that configure tried to do that failed -- probably a chunk of C code that failed to compile, with an error message from the compiler something like 'can't find include file "Python.h"'. This tells you what the problem is -- it couldn't find the python header file used to embed python in gdb.
Now your problem may be slightly different, but its probably related to not having the python embedding components properly installed.

Why can't the link process find python?
It's not the link process; it's the configure process in one of the subdirectories.
Find out which subdirectory it is (make should say Leaving directory XXXX), and look in config.log in that subdirectory for what went wrong.
The reason I posted is that config.log does not point to a problem, yet when I run make I receive the error from the original post
The make process runs additional configure processes. Your error is clearly from that additional process. You claim to have checked all config.logs in all subdirectories, but that appears unlikely given the symptoms you've described.

Sorry for making you wait 10 years for an answer.
In my case I looked under the gdb/config.log file and found this:
configure:10369: checking for python2.7
configure:10387: gcc -o conftest -g -O2 -I/home/user/stevie-bsp/toolchain-build/builds/destdir/aarch64-host-linux-gnu/include -I/usr/include/python2.7 -I/usr/include/python2.7 -static-libstdc++ -static-libgcc conftest.c -ldl -lncursesw -lm -ldl -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions >&5
conftest.c:54:10: fatal error: Python.h: No such file or directory
54 | #include "Python.h"
| ^~~~~~~~~~
compilation terminated.
Looks like it is a misleading error message when the Python 2 headers aren't installed. I fixed that on Ubuntu with apt install python2-dev; on CentOS the package name would vary.
Secondly, the command python must start the Python 2 interpreter. The build process will not give a clear error about this and silently fails to query the include path for the Python headers. I resolved it by using the python-is-python2 package on Ubuntu. I'm not sure about CentOS but in the worst case you can create a symlink.

Compile Vim with Python support on OS X

I start in the Vim src directory
$ pwd
~/vim/src
$ ./configure \
> --prefix=$HOME/Applications/vim-compiled \
> --enable-rubyinterp=yes \
> --enable-pythoninterp=yes \
> --with-python-config-dir=/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
$ make
At the end of make, I get the following error
ld: warning: in /Library/Frameworks//Python.framework/Python, missing required architecture x86_64 in file
Undefined symbols:
<very long list of Undefined symbols>
...
...
...
<very long list of Undefined symbols>
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [vim] Error 1
What's causing the error? I've found answers in similar questions that advise using MacPorts, Homebrew, or MacVim. I don't want to use any of those. Is there some other way around?
UPDATE: This is the output listing the Undefined symbols http://pastebin.com/ggV87ReF

In my case I've installed MacVim on 10.6.8 using the following command
./configure --prefix=/usr --enable-perlinterp=yes --enable-pythoninterp=yes --with-features=huge --with-python-config-dir=/System/Library/Frameworks/Python.framework/Versions/2.6
sudo make

Have you tried upgrading to the OSX 32-bit/64-bit version of Python 2.7.2? I'm not sure if there was a specific 64-bit version of 2.7, and that might be causing problems.

Take a look at the Homebrew recipe for vim (or macvim) and modify it if necessary. https://github.com/mxcl/homebrew