Develop Reference

How to avoid the make command triggers Xcode

I have a python project that has a makefile which has
test: clean lint
#pytest app/tests/ --cov ./app -s
I usually type make test in the terminal to trigger this pytest. However, recently it stopped working. Instead, I get an error
make: error: sh -c '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk '' -find make 2> /dev/null' failed with exit code 256: (null) (errno=No such file or directory)
And MacOS pops up a message window saying it needs to download software (without details other than a progress bar). It takes ~10 min to finish installing. But it doesn't change anything. I have installed XCode although I never used it. Pytest still works if I directly type the pytest commands.
My machine is a Macbook Pro with BigSur Intel CPU. I'm not a iOS/MacOS app developer and I don't use Xcode. I don't understand why would make triggers it. How to avoid it?
More info following #MadScientist suggestion:
> type -a make
make is /usr/bin/make
> make --version
OR
> make -d test
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun
> /usr/bin/make
xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun

command not found when using makefile to run batch file on Windows 10

When I try to run a makefile (type "make test" in terminal) which runs a batch file I get the equivalent error:
cd directory_path && test.bat
/bin/sh: test.bat: command not found
make: *** [makefile:58: test] Error 127
The makefile is:
.PHONY: test
test:
cd directory_path && test.bat
python path/test.py
(I changed names of the batch file, the directory path, etc. to try and make things more generic)
I can manually type "cd directory_path" and then "test.bat" and those both work. This makefile works on other systems. It only doesn't work on mine. I think it is an issue with how I installed Cygwin and how I run "make.exe".
I can use make to compile C code, but I also get an error trying to use make to run python scripts. The makefile from before also has a command for python path/test.py. This also isn't working on my system. If I delete the batch file line, but keep the python command it throws the error:
python path/test.py
make: python: No such file or directory
make: *** [makefile:59: test] Error 127.
I don't understand why it throws the error of no such file or directory since the path leads to the file, and if i put the python script in the same working directory as the makefile then it still can't find it.
Any ideas/solutions on these problems? Thank you!
EDIT:
Doug Henderson:
I started a cmd prompt at the place with the make file and entered all of these commands in
uname -a
CYGWIN_NT-10.0 james-mobl2 3.1.6(0.340/5/3) 2020-07-09 08:20 x86_64 Cygwin
which make
/usr/bin/make
make -v
GNU Make 4.3
Built for x86_64-pc-cygwin
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
which python
which: no python in
(/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09-hotspot/bin:
/cygdrive/c/windows/system32:
/cygdrive/c/windows:
/cygdrive/c/windows/System32/Wbem:
/cygdrive/c/windows/System32/WindowsPowerShell/v1.0:
/cygdrive/c/windows/System32/OpenSSH:
/cygdrive/c/Program Files/Git/cmd:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps/python:
/cygdrive/c/Program Files/PuTTY:
/usr/bin:
/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09 hotspot/bin:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps)
(I cleaned up the output a little to make it more legible). This and which test.bat are the only commands that failed.
python -V
Python 3.8.5
The change directory changed correctly
which test.bat
which: no test.bat in
(/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09-hotspot/bin:
/cygdrive/c/windows/system32:
/cygdrive/c/windows:
/cygdrive/c/windows/System32/Wbem:
/cygdrive/c/windows/System32/WindowsPowerShell/v1.0:
/cygdrive/c/windows/System32/OpenSSH:
/cygdrive/c/Program Files/Git/cmd:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps/python:
/cygdrive/c/Program Files/PuTTY:
/usr/bin:
/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09 hotspot/bin:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps)
I added cmd /c test.bat to the makefile and it worked, but I've also had the batch file run on different systems without that addition. I also have a lot batch files in the actual makefile so this solution works but isn't ideal. It also didn't fix the python script not running.
I didn't initially have cmd.exe in my path, but I added it before entering those commands.
I'm not sure what you mean by starting my PATH in bash with /usr/bin;/bin;THE_REST
MadScientist:
I was manually entering it into a Windows Terminal. Is it possible to have Cygwin invoke a Windows command.com shell?
What I did was go to directory that contained the makefile. Open a windows terminal by typing 'cmd' in the address bar. I would then type 'make test'. That obviously led to the issues. I then went through and manually entered the commands I thought the makefile was supposed to invoke. So I entered 'cd directory_path && test.bat'. This ran.
Matzeri:
I don't really know. I'm completely new to POSIX, etc. I'm guessing I'm using Windows style, and judging from other responses, I'm trying to use Cygwin to do a non-POSIX style.
Thank you, everyone for responses!

Make sure cygwin is installed with basic packages --> Some basic packages not installed with cygwin try below link to install some basic package, uninstall the cygwin and install it again with following basic package :
https://wiki.usask.ca/display/MESH/Running+Python+from+the+Cygwin+Terminal#RunningPythonfromtheCygwinTerminal-Cygwin
make sure cygwin's bin folder is added to your environment variable (e.g., C:\cygwin64\bin)

If cmd /c test.bat works then . is probably not in your PATH. Always do ./test.bat when you want to run something in the current directory. Never count on the PATH unless you have set it yourself. Also make sure the file is executable. chmod +x test.bat

How to debug python wheel build on macos remote server (azure pipelines)

I try to build wheel for imagecodecs python package. Previous version I build without any problem.
For current revision I got wheels for linux (manylinux2010) and macos, but without openmp support.
I would like to build also with openmp support. For this purpose I use gcc from brew.
From previous version of imagecodecs to dependency there is added snappy, zopflipy and changed version of jxrlib.
I use cibuildweel to build wheel so it is build with python installed from official pkg files.
Build pass, then when test start it fail with (on azure pipelines)
python(53258,0x7fffa20fb380) malloc: *** error for object 0x109c85c00: pointer being freed was not allocated
When I download it on my computer whole test set pass.
On azure pipelines there is no option to login on machine. I try to debug it with
lldb --batch -o "br set -n malloc_error_break" -o "run" -o "bt" -o "c" -- python -m pytest {project}/tests/
Then I got:
+ lldb --batch -o "br set -n malloc_error_break" -o "run" -o "bt" -o "c" -- python -m pytest /Users/runner/runners/2.163.1/work/1/s/tests/
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) settings set -- target.run-args "-m" "pytest" "/Users/runner/runners/2.163.1/work/1/s/tests/"
(lldb) br set -n malloc_error_break
Breakpoint 1: where = libsystem_malloc.dylib`malloc_error_break, address = 0x00000000000119e6
(lldb) run
============================= test session starts ==============================
platform darwin -- Python 2.7.17, pytest-4.6.8, py-1.8.0, pluggy-0.13.1
python(53294,0x7fff97d45380) malloc: *** error for object 0x10fc3ec00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 53294 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x00007fff5f7d89e6 libsystem_malloc.dylib`malloc_error_break
libsystem_malloc.dylib`malloc_error_break:
-> 0x7fff5f7d89e6 <+0>: pushq %rbp
0x7fff5f7d89e7 <+1>: movq %rsp, %rbp
0x7fff5f7d89ea <+4>: nop
0x7fff5f7d89eb <+5>: nopl (%rax)
Target 0: (python) stopped.
Process 53294 launched: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp_g9gpq37/bin/python' (x86_64)
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
* frame #0: 0x00007fff5f7d89e6 libsystem_malloc.dylib`malloc_error_break
frame #1: 0x00007fff5f7ca82d libsystem_malloc.dylib`free + 400
frame #2: 0x000000010face5bd libstdc++.6.dylib`std::locale::_Impl::~_Impl() + 109
frame #3: 0x00007fff97d24998 libsystem_c.dylib`__stdinp + 8
(lldb) c
Whole code is here https://github.com/Czaki/imagecodecs/tree/2019.12.16
Pipelines are here https://dev.azure.com/bokota/open%20source%20contrib/_build?definitionId=3&_a=summary
Any idea how I can debug this? How to find where pointer is created?
EDIT
In this case problem is connected with compiling zopflipy with gcc-9/g++-9. I need to compile test dependencies with clang

You can check Enable system diagnostics to get verbose logs when you Run pipeline.
You can also click add variable to add a variable named system.debug and set its value to true. Please refer to troubleshoot build and release.
If you would like to login on the agent machine. You can configure your self-hosted agent, which will allow you to login and troubleshoot your builds.

Shell expansions are not performed when running a command with `subprocess.call' with `shell=True'

I am trying to build some files using python but it is executing it in a wrong manner.
I have tried to build some files in linux.
when i use "make ./package/feeds/proj/{clean,compile} V=s" command in the terminal, it is working properly, but when i try to run it with a python script, using the command "p = subprocess.call(r'/usr/bin/make package/feeds/proj/{clean,compile} V=s',shell = True))", it is behaving differently.
Logs :
when run in terminal :
make[1]: Entering directory '/local/mnt/workspace/rubaeshk/unused2/qsdk'
make[2]: Entering directory '/local/mnt/workspace/rubaeshk/unused2/qsdk/package/feeds/whc/qca-whc-crash-log'
rm -f /local/mnt/workspace/rubaeshk/unused2/qsdk/bin/ipq/packages/whc/qca-whc-crash-log_*
..(log continued until successfully built)
when run through python script :
WARNING: your configuration is out of sync. Please run make menuconfig, oldconfig or defconfig!
make[1]: Entering directory '/local/mnt/workspace/rubaeshk/unused2/qsdk'
make[1]: *** No rule to make target 'package/feeds/whc/qca-whc-crash-log/{clean,compile}'. Stop.
make[1]: Leaving directory '/local/mnt/workspace/rubaeshk/unused2/qsdk'
/local/mnt/workspace/rubaeshk/unused2/qsdk/include/toplevel.mk:186: recipe for target 'package/feeds/whc/qca-whc-crash-log/{clean,compile}' failed
make: *** [package/feeds/whc/qca-whc-crash-log/{clean,compile}] Error 2
Could someone see where it went wrong..

As explained in subprocess.Popen documentation (subprocess.call delegates to Popen like all other convenience functions), subprocess.call('command', shell=True) in Unix is equivalent to running the argv:
['/bin/sh', '-c', 'command']
And sh does not support brace expansion (this is the official name for the {a,b} syntax).
To run the command with bash instead, you need to override the shell executable used with the executable parameter:
p = subprocess.call('command', shell=True, executable='/bin/bash')
Example:
$ python -c 'import subprocess; subprocess.call("echo /usr/{lib,bin}", shell=True, executable="/bin/bash")'
/usr/lib /usr/bin
Note however that using shell=True is discouraged because it's inherently platform-specific, depends on local shell and its settings, and may be a bug or security vulnerability if you use untrusted input. Better construct the command line by hand and pass the resulting argv.

Brace expansion is not part of the standard shell; it's an add-on feature provided by some shells (like bash) in addition to POSIX. When you run subprocess.call in Python it's probably using /bin/sh, not /bin/bash.
So, write it out: package/feeds/whc/qca-whc-crash-log/clean package/feeds/whc/qca-whc-crash-log/compile

Installing node.js on Windows cygwin error

I tried to install node.js by following instructions here:
https://github.com/joyent/node/wiki/Building-node.js-on-Cygwin-(Windows)
I got these errors after running ./configure:
$ ./configure
Checking for program g++ or c++ : /usr/bin/g++
2 [main] python 6768 C:\cygwin\bin\python.exe: *** fatal error - unable to remap \\?\C:\cygwin\lib\python2.6\lib-dynload\time.dll to same address as parent: 0x3A0000 != 0x3D0000
Stack trace:
Frame Function Args
002891E8 6102796B (002891E8, 00000000, 00000000, 00000000)
002894D8 6102796B (6117EC60, 00008000, 00000000, 61180977)
0028A508 61004F1B (611A7FAC, 61249144, 003A0000, 003D0000)
End of stack trace
3 [main] python 5292 fork: child 6768 - died waiting for dll loading, errno 11
/home/user/node/wscript:228: error: could not configure a cxx compiler!
I did a rebaseall on cygwin and ran configure again and got these errors:
$ ./configure
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for program gcc or cc : /usr/bin/gcc
2 [main] python 6100 C:\cygwin\bin\python.exe: *** fatal error - unable to remap \\?\C:\cygwin\lib\python2.6\lib-dynload\_functools.dll to same address as parent: 0x3A0000 != 0x3D0000
Stack trace:
Frame Function Args
002891E8 6102796B (002891E8, 00000000, 00000000, 00000000)
002894D8 6102796B (6117EC60, 00008000, 00000000, 61180977)
0028A508 61004F1B (611A7FAC, 6124976C, 003A0000, 003D0000)
End of stack trace
2 [main] python 4424 fork: child 6100 - died waiting for dll loading, errno 11
/home/user/node/wscript:230: error: could not configure a c compiler!
What am I doing wrong?

You can follow the steps on this page:
http://www.garethhunt.com/2008/02/11/cygwin-died-waiting-for-dll-loading/
it worked perfectly for me.

You can avoid installing node.js through Cygwin and try to use native Windows executable which would/should be preffered (unless you have some specific reasons to do otherwise).