I have a python project that has a makefile which has
test: clean lint
#pytest app/tests/ --cov ./app -s
I usually type make test in the terminal to trigger this pytest. However, recently it stopped working. Instead, I get an error
make: error: sh -c '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk '' -find make 2> /dev/null' failed with exit code 256: (null) (errno=No such file or directory)
And MacOS pops up a message window saying it needs to download software (without details other than a progress bar). It takes ~10 min to finish installing. But it doesn't change anything. I have installed XCode although I never used it. Pytest still works if I directly type the pytest commands.
My machine is a Macbook Pro with BigSur Intel CPU. I'm not a iOS/MacOS app developer and I don't use Xcode. I don't understand why would make triggers it. How to avoid it?
More info following #MadScientist suggestion:
> type -a make
make is /usr/bin/make
> make --version
OR
> make -d test
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
> /usr/bin/make
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
I am trying to install wxPython but the wheel build fails. The error message is not helpful in indicating what to do or where to look to fix this. Can anyone please help me understand how to build this wheel correctly?
Machine: Linux on Power (this is not x86)
OS: RHEL Server, 7.5 (Maipo)
python version: Python 3.6.4
pip3 version: pip 19.3.1
I noticed this stack overflow post, which is also not helpful because my linux release is not on the list of the ones provided.
Following links above I tried wxPython download page and the following install with pip but in step 5 basically tells you "look at the log and figure it out"....not helpful.
I tried to manually hack the wxPython package using my very limited competence and removed some dependency.....still nothing.
<...>
Finished command: build_wx (1m56.907s)
Running command: build_py
Checking for /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build
Setting top to : /tmp/pip-req-build-dgnp13sp
Setting out to : /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3/config.log)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (0m6.991s)
Finished command: build (2m3.899s)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" -u build.py build' failed with exit code 1.
Building wheel for wxPython (setup.py): finished with status 'error'
ERROR: Failed building wheel for wxPython
<...>
----Update 12/2 (After Robin Dunn's feedback)
Thank you Robin for the directives. Following is the last portion of the config.log from a different run using the build command. Also the error message is slightly different (below, following the config.log), because I had previously used the explicit wheel-build command: pip wheel -v wxPython-4.0.7.post1.tar.gz 2>&1 | tee build.log. Does this log below confirm your theory regarding the Python built with the --enable-shared configure flag?
Testing pyext configuration
==>
#include <Python.h>
#ifdef __cplusplus
extern "C" {
#endif
void Py_Initialize(void);
void Py_Finalize(void);
#ifdef __cplusplus
}
#endif
int main(int argc, char **argv)
{
(void)argc; (void)argv;
Py_Initialize();
Py_Finalize();
return 0;
}
<==
[1/2] Compiling [32mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/test.cpp[0m
['/bin/g++', '-fPIC', '-g', '-fwrapv', '-O3', '-I../../../../../../../../../../../../../../cte/tools/python/vol2/.3.6.4-linux-ppc64le/include/python3.6m', '-I/opt/xsite/cte/tools/python/common2018/include', '-DPYTHONDIR="/usr/local/lib/python3.6/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.6/site-packages"', '-DNDEBUG', '../test.cpp', '-c', '-o/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/test.cpp.1.o']
[2/2] Linking [33mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so[0m
['/bin/g++', '-shared', '-Xlinker', '-export-dynamic', 'test.cpp.1.o', '-o', '/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm']
err: /bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_keyword':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7767:(.text.unlikely+0x608): call to `_Py_keyword' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_comprehension':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7419:(.text.unlikely+0x9f4): call to `_Py_comprehension' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_alias':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7802:(.text.unlikely+0xbec): call to `_Py_alias' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_withitem':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7837:(.text.unlikely+0xdd4): call to `_Py_withitem' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
from /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1: Test does not build: Traceback (most recent call last):
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Configure.py", line 324, in run_build
bld.compile()
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Build.py", line 176, in compile
raise Errors.BuildError(self.producer.error)
waflib.Errors.BuildError: Build failed
-> task in 'testprog' failed with exit status 1 (run with -v to display more information)
Could not build python extensions
from /.....: The configuration failed
and this is the error message that I get this new run, slightly different
msgfmt --verbose -c -o zh_TW.mo zh_TW.po
1710 translated messages, 82 fuzzy translations, 61 untranslated messages.
make[1]: Leaving directory `/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale'
Setting top to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1
Setting out to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/config.log)
Will build using: "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3"
3.6.4 (default, Feb 12 2018, 16:08:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Python's architecture is 64bit
cfg.VERSION: 4.0.7.post1
Running command: build
Running command: build_wx
wxWidgets build options: ['--wxpython', '--unicode', '--gtk3']
Configure options: ['--enable-unicode', '--with-gtk=3', '--enable-sound', '--enable-graphics_ctx', '--enable-display', '--enable-geometry', '--enable-debug_flag', '--enable-optimise', '--disable-debugreport', '--enable-uiactionsim', '--enable-autoidman', '--with-sdl']
/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/configure --enable-unicode --with-gtk=3 --enable-sound --enable-graphics_ctx --enable-display --enable-geometry --enable-debug_flag --enable-optimise --disable-debugreport --enable-uiactionsim --enable-autoidman --with-sdl
make --jobs=128
Building message catalogs in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale
make allmo
Finished command: build_wx (12m36.623s)
Running command: build_py
Checking for /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build
Command '"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (2m7.118s)
Finished command: build (14m43.742s)
It looks like you might be using a custom Python build. Was your Python built with the --enable-shared configure flag? On Linux platforms the waf build tool assumes that Python was built that way, and will try to link with the Python shared library, and will fail if it is not present.
If that doesn't help then you can get more details from waf's configure log. When using pip to do the build then it will have removed the temporary build folders before you can get at them. So if you unpack the source archive and do the build with python3 build.py build you'll be able to find the log in ./build/waf/3.6/gtk3/config.log and hopefully find some useful clues there.
I have set up mpi4py on a new server, and it isn't quite working. When I import mpi4py.MPI, it crashes. However, if I do the same thing under mpiexec, it works. On my other server and on my workstation, both techniques work fine. What am I missing on the new server?
Here's what happens on the new server:
$ python -c 'from mpi4py import MPI; print("OK")'
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
PMI2_Job_GetId failed failed
--> Returned value (null) (14) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
orte_ess_init failed
--> Returned value (null) (14) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
ompi_mpi_init: ompi_rte_init failed
--> Returned "(null)" (14) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[Octomore:45430] Local abort before MPI_INIT completed successfully; not able to aggregate error messages, and not able to guarantee that all other processes were killed!
If I run it with mpiexec, it's fine.
$ mpiexec -np 1 python -c 'from mpi4py import MPI; print("OK")'
OK
I'm running on CentOS 6.7. I've installed Python 2.7 as a software collection, and I've loaded the openmpi/gnu/1.10.2 module. MPICH and MPICH2 are also installed, so they may be conflicting with OpenMPI. I haven't loaded the MPICH modules, though. I'm running Python in a virtualenv:
$ pip list
mpi4py (2.0.0)
pip (8.1.2)
setuptools (18.0.1)
wheel (0.24.0)
It turned out that mpi4py is not compatible with version 1.10.2 of OpenMPI. It works fine with version 1.6.5.
$ module load openmpi/gnu/1.6.5
$ python -c 'from mpi4py import MPI; print("OK")'
OK
While porting a build script to Windows I noticed that I can not call make with the parallel build option -j from within python:
subprocess.call("make -j4 -f Makefile.win32 target".split())
This is on a Windows 7 (in a VM), with Python 3.4.2 (or 2.7.8), GNU Make 3.81 from MinGW. Make itself calls cl.exe to compile about 40 C-files. When killing the offending make.exe in the Taskmanager, this is the output:
make: *** [target] Error 1
make: INTERNAL: Exiting with 1 jobserver tokens available; should be 4!
I tried to install node.js by following instructions here:
https://github.com/joyent/node/wiki/Building-node.js-on-Cygwin-(Windows)
I got these errors after running ./configure:
$ ./configure
Checking for program g++ or c++ : /usr/bin/g++
2 [main] python 6768 C:\cygwin\bin\python.exe: *** fatal error - unable to remap \\?\C:\cygwin\lib\python2.6\lib-dynload\time.dll to same address as parent: 0x3A0000 != 0x3D0000
Stack trace:
Frame Function Args
002891E8 6102796B (002891E8, 00000000, 00000000, 00000000)
002894D8 6102796B (6117EC60, 00008000, 00000000, 61180977)
0028A508 61004F1B (611A7FAC, 61249144, 003A0000, 003D0000)
End of stack trace
3 [main] python 5292 fork: child 6768 - died waiting for dll loading, errno 11
/home/user/node/wscript:228: error: could not configure a cxx compiler!
I did a rebaseall on cygwin and ran configure again and got these errors:
$ ./configure
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for program gcc or cc : /usr/bin/gcc
2 [main] python 6100 C:\cygwin\bin\python.exe: *** fatal error - unable to remap \\?\C:\cygwin\lib\python2.6\lib-dynload\_functools.dll to same address as parent: 0x3A0000 != 0x3D0000
Stack trace:
Frame Function Args
002891E8 6102796B (002891E8, 00000000, 00000000, 00000000)
002894D8 6102796B (6117EC60, 00008000, 00000000, 61180977)
0028A508 61004F1B (611A7FAC, 6124976C, 003A0000, 003D0000)
End of stack trace
2 [main] python 4424 fork: child 6100 - died waiting for dll loading, errno 11
/home/user/node/wscript:230: error: could not configure a c compiler!
What am I doing wrong?
You can follow the steps on this page:
http://www.garethhunt.com/2008/02/11/cygwin-died-waiting-for-dll-loading/
it worked perfectly for me.
You can avoid installing node.js through Cygwin and try to use native Windows executable which would/should be preffered (unless you have some specific reasons to do otherwise).