I have a python project that has a makefile which has
test: clean lint
#pytest app/tests/ --cov ./app -s
I usually type make test in the terminal to trigger this pytest. However, recently it stopped working. Instead, I get an error
make: error: sh -c '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk '' -find make 2> /dev/null' failed with exit code 256: (null) (errno=No such file or directory)
And MacOS pops up a message window saying it needs to download software (without details other than a progress bar). It takes ~10 min to finish installing. But it doesn't change anything. I have installed XCode although I never used it. Pytest still works if I directly type the pytest commands.
My machine is a Macbook Pro with BigSur Intel CPU. I'm not a iOS/MacOS app developer and I don't use Xcode. I don't understand why would make triggers it. How to avoid it?
More info following #MadScientist suggestion:
> type -a make
make is /usr/bin/make
> make --version
OR
> make -d test
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
> /usr/bin/make
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
I try to build wheel for imagecodecs python package. Previous version I build without any problem.
For current revision I got wheels for linux (manylinux2010) and macos, but without openmp support.
I would like to build also with openmp support. For this purpose I use gcc from brew.
From previous version of imagecodecs to dependency there is added snappy, zopflipy and changed version of jxrlib.
I use cibuildweel to build wheel so it is build with python installed from official pkg files.
Build pass, then when test start it fail with (on azure pipelines)
python(53258,0x7fffa20fb380) malloc: *** error for object 0x109c85c00: pointer being freed was not allocated
When I download it on my computer whole test set pass.
On azure pipelines there is no option to login on machine. I try to debug it with
lldb --batch -o "br set -n malloc_error_break" -o "run" -o "bt" -o "c" -- python -m pytest {project}/tests/
Then I got:
+ lldb --batch -o "br set -n malloc_error_break" -o "run" -o "bt" -o "c" -- python -m pytest /Users/runner/runners/2.163.1/work/1/s/tests/
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) settings set -- target.run-args "-m" "pytest" "/Users/runner/runners/2.163.1/work/1/s/tests/"
(lldb) br set -n malloc_error_break
Breakpoint 1: where = libsystem_malloc.dylib`malloc_error_break, address = 0x00000000000119e6
(lldb) run
============================= test session starts ==============================
platform darwin -- Python 2.7.17, pytest-4.6.8, py-1.8.0, pluggy-0.13.1
python(53294,0x7fff97d45380) malloc: *** error for object 0x10fc3ec00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 53294 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x00007fff5f7d89e6 libsystem_malloc.dylib`malloc_error_break
libsystem_malloc.dylib`malloc_error_break:
-> 0x7fff5f7d89e6 <+0>: pushq %rbp
0x7fff5f7d89e7 <+1>: movq %rsp, %rbp
0x7fff5f7d89ea <+4>: nop
0x7fff5f7d89eb <+5>: nopl (%rax)
Target 0: (python) stopped.
Process 53294 launched: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp_g9gpq37/bin/python' (x86_64)
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
* frame #0: 0x00007fff5f7d89e6 libsystem_malloc.dylib`malloc_error_break
frame #1: 0x00007fff5f7ca82d libsystem_malloc.dylib`free + 400
frame #2: 0x000000010face5bd libstdc++.6.dylib`std::locale::_Impl::~_Impl() + 109
frame #3: 0x00007fff97d24998 libsystem_c.dylib`__stdinp + 8
(lldb) c
Whole code is here https://github.com/Czaki/imagecodecs/tree/2019.12.16
Pipelines are here https://dev.azure.com/bokota/open%20source%20contrib/_build?definitionId=3&_a=summary
Any idea how I can debug this? How to find where pointer is created?
EDIT
In this case problem is connected with compiling zopflipy with gcc-9/g++-9. I need to compile test dependencies with clang
You can check Enable system diagnostics to get verbose logs when you Run pipeline.
You can also click add variable to add a variable named system.debug and set its value to true. Please refer to troubleshoot build and release.
If you would like to login on the agent machine. You can configure your self-hosted agent, which will allow you to login and troubleshoot your builds.
I am trying to install wxPython but the wheel build fails. The error message is not helpful in indicating what to do or where to look to fix this. Can anyone please help me understand how to build this wheel correctly?
Machine: Linux on Power (this is not x86)
OS: RHEL Server, 7.5 (Maipo)
python version: Python 3.6.4
pip3 version: pip 19.3.1
I noticed this stack overflow post, which is also not helpful because my linux release is not on the list of the ones provided.
Following links above I tried wxPython download page and the following install with pip but in step 5 basically tells you "look at the log and figure it out"....not helpful.
I tried to manually hack the wxPython package using my very limited competence and removed some dependency.....still nothing.
<...>
Finished command: build_wx (1m56.907s)
Running command: build_py
Checking for /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build
Setting top to : /tmp/pip-req-build-dgnp13sp
Setting out to : /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /tmp/pip-req-build-dgnp13sp/build/waf/3.6/gtk3/config.log)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" /tmp/pip-req-build-dgnp13sp/bin/waf-2.0.8 --wx_config=/tmp/pip-req-build-dgnp13sp/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (0m6.991s)
Finished command: build (2m3.899s)
Command '"/afs/apd.pok.ibm.com/u/mfacchin/wxenvlop/bin/python3" -u build.py build' failed with exit code 1.
Building wheel for wxPython (setup.py): finished with status 'error'
ERROR: Failed building wheel for wxPython
<...>
----Update 12/2 (After Robin Dunn's feedback)
Thank you Robin for the directives. Following is the last portion of the config.log from a different run using the build command. Also the error message is slightly different (below, following the config.log), because I had previously used the explicit wheel-build command: pip wheel -v wxPython-4.0.7.post1.tar.gz 2>&1 | tee build.log. Does this log below confirm your theory regarding the Python built with the --enable-shared configure flag?
Testing pyext configuration
==>
#include <Python.h>
#ifdef __cplusplus
extern "C" {
#endif
void Py_Initialize(void);
void Py_Finalize(void);
#ifdef __cplusplus
}
#endif
int main(int argc, char **argv)
{
(void)argc; (void)argv;
Py_Initialize();
Py_Finalize();
return 0;
}
<==
[1/2] Compiling [32mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/test.cpp[0m
['/bin/g++', '-fPIC', '-g', '-fwrapv', '-O3', '-I../../../../../../../../../../../../../../cte/tools/python/vol2/.3.6.4-linux-ppc64le/include/python3.6m', '-I/opt/xsite/cte/tools/python/common2018/include', '-DPYTHONDIR="/usr/local/lib/python3.6/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.6/site-packages"', '-DNDEBUG', '../test.cpp', '-c', '-o/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/test.cpp.1.o']
[2/2] Linking [33mbuild/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so[0m
['/bin/g++', '-shared', '-Xlinker', '-export-dynamic', 'test.cpp.1.o', '-o', '/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/.conf_check_cfc3ecfbbf37890054f6518ca7961071/testbuild/testprog.cpython-36m-powerpc64le-linux-gnu.so', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu', '-L/afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm', '-lpython3.6m', '-lpthread', '-ldl', '-lutil', '-lm']
err: /bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_keyword':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7767:(.text.unlikely+0x608): call to `_Py_keyword' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_comprehension':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7419:(.text.unlikely+0x9f4): call to `_Py_comprehension' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_alias':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7802:(.text.unlikely+0xbec): call to `_Py_alias' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: /afs/apd.pok.ibm.com/func/vlsi/cte/tools/python/vol2/.3.6.4-linux-ppc64le/lib/python3.6/config-3.6m-powerpc64le-linux-gnu/libpython3.6m.a(Python-ast.o): In function `obj2ast_withitem':
/data/ubrandt/Python-3.6.4/Python/Python-ast.c:7837:(.text.unlikely+0xdd4): call to `_Py_withitem' lacks nop, can't restore toc; recompile with -fPIC
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
from /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1: Test does not build: Traceback (most recent call last):
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Configure.py", line 324, in run_build
bld.compile()
File "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/.waf3-2.0.8-206f2b7a89029e71942a2beb9e1bbbbd/waflib/Build.py", line 176, in compile
raise Errors.BuildError(self.producer.error)
waflib.Errors.BuildError: Build failed
-> task in 'testprog' failed with exit status 1 (run with -v to display more information)
Could not build python extensions
from /.....: The configuration failed
and this is the error message that I get this new run, slightly different
msgfmt --verbose -c -o zh_TW.mo zh_TW.po
1710 translated messages, 82 fuzzy translations, 61 untranslated messages.
make[1]: Leaving directory `/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale'
Setting top to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1
Setting out to : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3
Checking for 'gcc' (C compiler) : /bin/gcc
Checking for 'g++' (C++ compiler) : /bin/g++
Checking for program 'python' : /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3
Checking for python version >= 2.7.0 : 3.6.4
python-config : /opt/xsite/cte/tools/python/3.6/bin/python3.6-config
Asking python-config for pyext '--cflags --libs --ldflags' flags : yes
Testing pyext configuration : Could not build python extensions
The configuration failed
(complete log in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/waf/3.6/gtk3/config.log)
Will build using: "/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3"
3.6.4 (default, Feb 12 2018, 16:08:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Python's architecture is 64bit
cfg.VERSION: 4.0.7.post1
Running command: build
Running command: build_wx
wxWidgets build options: ['--wxpython', '--unicode', '--gtk3']
Configure options: ['--enable-unicode', '--with-gtk=3', '--enable-sound', '--enable-graphics_ctx', '--enable-display', '--enable-geometry', '--enable-debug_flag', '--enable-optimise', '--disable-debugreport', '--enable-uiactionsim', '--enable-autoidman', '--with-sdl']
/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/configure --enable-unicode --with-gtk=3 --enable-sound --enable-graphics_ctx --enable-display --enable-geometry --enable-debug_flag --enable-optimise --disable-debugreport --enable-uiactionsim --enable-autoidman --with-sdl
make --jobs=128
Building message catalogs in /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/ext/wxWidgets/locale
make allmo
Finished command: build_wx (12m36.623s)
Running command: build_py
Checking for /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8...
"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build
Command '"/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" /afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/bin/waf-2.0.8 --wx_config=/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/wxPython-4.0.7.post1_mf1/build/wxbld/gtk3/wx-config --gtk3 --python="/afs/apd.pok.ibm.com/func/vlsi/eclipz/sf5/usr/mfacchin/c01/python_venv/wxenv191202/bin/python3" --out=build/waf/3.6/gtk3 configure build ' failed with exit code 1.
Finished command: build_py (2m7.118s)
Finished command: build (14m43.742s)
It looks like you might be using a custom Python build. Was your Python built with the --enable-shared configure flag? On Linux platforms the waf build tool assumes that Python was built that way, and will try to link with the Python shared library, and will fail if it is not present.
If that doesn't help then you can get more details from waf's configure log. When using pip to do the build then it will have removed the temporary build folders before you can get at them. So if you unpack the source archive and do the build with python3 build.py build you'll be able to find the log in ./build/waf/3.6/gtk3/config.log and hopefully find some useful clues there.
I keep getting this error in by terminal after pip install uwsgi and running uwsgi
I've tried various things thinking it is a gcc path error so added the location to libc.so.6 but it didn't help.
SOMEBODY PLEAAASSSEEE help!
!!! uWSGI process 10506 got Segmentation Fault !!!
*** backtrace of 10506 ***
uwsgi(uwsgi_backtrace+0x2c) [0x46429c]
uwsgi(uwsgi_segfault+0x21) [0x464641]
/lib64/libc.so.6(+0x35670) [0x7fc0fe450670]
uwsgi(uwsgi_setup+0x24b) [0x466a5b]
uwsgi(main+0x9) [0x41ac29]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc0fe43cb15]
uwsgi() [0x41ac59]
*** end of backtrace ***
I got the answer...It's because pip install uwsgi needs to be on version 2.0.14 instead of 2.0.15...There is definitely a bug in the newer version!
While porting a build script to Windows I noticed that I can not call make with the parallel build option -j from within python:
subprocess.call("make -j4 -f Makefile.win32 target".split())
This is on a Windows 7 (in a VM), with Python 3.4.2 (or 2.7.8), GNU Make 3.81 from MinGW. Make itself calls cl.exe to compile about 40 C-files. When killing the offending make.exe in the Taskmanager, this is the output:
make: *** [target] Error 1
make: INTERNAL: Exiting with 1 jobserver tokens available; should be 4!