Develop Reference

command not found when using makefile to run batch file on Windows 10

When I try to run a makefile (type "make test" in terminal) which runs a batch file I get the equivalent error:
cd directory_path && test.bat
/bin/sh: test.bat: command not found
make: *** [makefile:58: test] Error 127
The makefile is:
.PHONY: test
test:
cd directory_path && test.bat
python path/test.py
(I changed names of the batch file, the directory path, etc. to try and make things more generic)
I can manually type "cd directory_path" and then "test.bat" and those both work. This makefile works on other systems. It only doesn't work on mine. I think it is an issue with how I installed Cygwin and how I run "make.exe".
I can use make to compile C code, but I also get an error trying to use make to run python scripts. The makefile from before also has a command for python path/test.py. This also isn't working on my system. If I delete the batch file line, but keep the python command it throws the error:
python path/test.py
make: python: No such file or directory
make: *** [makefile:59: test] Error 127.
I don't understand why it throws the error of no such file or directory since the path leads to the file, and if i put the python script in the same working directory as the makefile then it still can't find it.
Any ideas/solutions on these problems? Thank you!
EDIT:
Doug Henderson:
I started a cmd prompt at the place with the make file and entered all of these commands in
uname -a
CYGWIN_NT-10.0 james-mobl2 3.1.6(0.340/5/3) 2020-07-09 08:20 x86_64 Cygwin
which make
/usr/bin/make
make -v
GNU Make 4.3
Built for x86_64-pc-cygwin
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
which python
which: no python in
(/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09-hotspot/bin:
/cygdrive/c/windows/system32:
/cygdrive/c/windows:
/cygdrive/c/windows/System32/Wbem:
/cygdrive/c/windows/System32/WindowsPowerShell/v1.0:
/cygdrive/c/windows/System32/OpenSSH:
/cygdrive/c/Program Files/Git/cmd:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps/python:
/cygdrive/c/Program Files/PuTTY:
/usr/bin:
/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09 hotspot/bin:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps)
(I cleaned up the output a little to make it more legible). This and which test.bat are the only commands that failed.
python -V
Python 3.8.5
The change directory changed correctly
which test.bat
which: no test.bat in
(/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09-hotspot/bin:
/cygdrive/c/windows/system32:
/cygdrive/c/windows:
/cygdrive/c/windows/System32/Wbem:
/cygdrive/c/windows/System32/WindowsPowerShell/v1.0:
/cygdrive/c/windows/System32/OpenSSH:
/cygdrive/c/Program Files/Git/cmd:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps/python:
/cygdrive/c/Program Files/PuTTY:
/usr/bin:
/cygdrive/c/Program Files/AdoptOpenJDK/jdk-8.0.252.09 hotspot/bin:
/cygdrive/c/Users/james/AppData/Local/Microsoft/WindowsApps)
I added cmd /c test.bat to the makefile and it worked, but I've also had the batch file run on different systems without that addition. I also have a lot batch files in the actual makefile so this solution works but isn't ideal. It also didn't fix the python script not running.
I didn't initially have cmd.exe in my path, but I added it before entering those commands.
I'm not sure what you mean by starting my PATH in bash with /usr/bin;/bin;THE_REST
MadScientist:
I was manually entering it into a Windows Terminal. Is it possible to have Cygwin invoke a Windows command.com shell?
What I did was go to directory that contained the makefile. Open a windows terminal by typing 'cmd' in the address bar. I would then type 'make test'. That obviously led to the issues. I then went through and manually entered the commands I thought the makefile was supposed to invoke. So I entered 'cd directory_path && test.bat'. This ran.
Matzeri:
I don't really know. I'm completely new to POSIX, etc. I'm guessing I'm using Windows style, and judging from other responses, I'm trying to use Cygwin to do a non-POSIX style.
Thank you, everyone for responses!

Make sure cygwin is installed with basic packages --> Some basic packages not installed with cygwin try below link to install some basic package, uninstall the cygwin and install it again with following basic package :
https://wiki.usask.ca/display/MESH/Running+Python+from+the+Cygwin+Terminal#RunningPythonfromtheCygwinTerminal-Cygwin
make sure cygwin's bin folder is added to your environment variable (e.g., C:\cygwin64\bin)

If cmd /c test.bat works then . is probably not in your PATH. Always do ./test.bat when you want to run something in the current directory. Never count on the PATH unless you have set it yourself. Also make sure the file is executable. chmod +x test.bat

sbin/start-stop-daemon not able to start python - ubuntu docker container

I have a simple python script which I want to start a daemon-service in background in docker container
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec 'python /opt/app/uc/monitor/bin/my-application.py'
when I execute this command in a shell I get
/sbin/start-stop-daemon: unable to stat //python /opt/app/uc/monitor/bin/my-application.py (No such file or directory)
However when execute just the below command in shell it works
python /opt/app/uc/monitor/bin/my-application.py
I'm sure the python is installed and all the links have been setup.
Thanks for the help

That error message implies that start-stop-daemon is looking for a file to open (the stat operation is a check before it opens the file) and treating your 'python ... ' argument as if it was a file.
See this example which confirms this. You may need to read the man page for start-stop-daemon, for your Ubuntu version, to check what a valid command would be for your setup.
Simplest solution is probably to create a shell script (say /opt/app/uc/monitor/bin/run-my-application.sh), and put this into it:
#!/bin/bash
python /opt/app/uc/monitor/bin/my-application.py
Be sure to do chmod +x on this file. If python is not found, use which python to find the path to python and use that in the script.
Now try:
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec '/opt/app/uc/monitor/bin/run-my-application.sh'

DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both

I've been usually installed python packages through pip.
For Google App Engine, I need to install packages to another target directory.
I've tried:
pip install -I flask-restful --target ./lib
but it fails with:
must supply either home or prefix/exec-prefix -- not both
How can I get this to work?

Are you using OS X and Homebrew? The Homebrew python page https://github.com/Homebrew/brew/blob/master/docs/Homebrew-and-Python.md calls out a known issue with pip and a work around.
Worked for me.
You can make this "empty prefix" the default by adding a
~/.pydistutils.cfg file with the following contents:
[install]
prefix=
Edit: The Homebrew page was later changed to recommend passing --prefix on the command line, as discussed in the comments below. Here is the last version which contained that text. Unfortunately this only works for sdists, not wheels.
The issue was reported to pip, which later fixed it for --user. That's probably why the section has now been removed from the Homebrew page. However, the problem still occurs when using --target as in the question above.

I believe there is a simpler solution to this problem (Homebrew's Python on macOS) that won't break your normal pip operations.
All you have to do is to create a setup.cfg file at the root directory of your project, usually where your main __init__.py or executable py file is. So if the root folder of your project is: /path/to/my/project/, create a setup.cfg file in there and put the magic words inside:
[install]
prefix=
OK, now you sould be able to run pip's commands for that folder:
pip install package -t /path/to/my/project/
This command will run gracefully for that folder only. Just copy setup.cfg to whatever other projects you might have. No need to write a .pydistutils.cfg on your home directory.
After you are done installing the modules, you may remove setup.cfg.

On OSX(mac), assuming a project folder called /var/myproject
cd /var/myproject
Create a file called setup.cfg and add
[install]
prefix=
Run pip install <packagename> -t .

Another solution* for Homebrew users is simply to use a virtualenv.
Of course, that may remove the need for the target directory anyway - but even if it doesn't, I've found --target works by default (as in, without creating/modifying a config file) when in a virtual environment.
*I say solution; perhaps it's just another motivation to meticulously use venvs...

I hit errors with the other recommendations around --install-option="--prefix=lib". The only thing I found that worked is using PYTHONUSERBASE as described here.
export PYTHONUSERBASE=lib
pip install -I flask-restful --user
this is not exactly the same as --target, but it does the trick for me in any case.

As other mentioned, this is known bug with pip & python installed with homebrew.
If you create ~/.pydistutils.cfg file with "empty prefix" instruction it will fix this problem but it will break normal pip operations.
Until this bug is officially addressed, one of the options would be to create your own bash script that would handle this case:
#!/bin/bash
name=''
target=''
while getopts 'n:t:' flag; do
case "${flag}" in
n) name="${OPTARG}" ;;
t) target="${OPTARG}" ;;
esac
done
if [ -z "$target" ];
then
echo "Target parameter must be provided"
exit 1
fi
if [ -z "$name" ];
then
echo "Name parameter must be provided"
exit 1
fi
# current workaround for homebrew bug
file=$HOME'/.pydistutils.cfg'
touch $file
/bin/cat <<EOM >$file
[install]
prefix=
EOM
# end of current workaround for homebrew bug
pip install -I $name --target $target
# current workaround for homebrew bug
rm -rf $file
# end of current workaround for homebrew bug
This script wraps your command and:
accepts name and target parameters
checks if those parameters are empty
creates ~/.pydistutils.cfg file with "empty prefix" instruction in it
executes your pip command with provided parameters
removes ~/.pydistutils.cfg file
This script can be changed and adapted to address your needs but you get idea. And it allows you to run your command without braking pip. Hope it helps :)

If you're using virtualenv*, it might be a good idea to double check which pip you're using.
If you see something like /usr/local/bin/pip you've broken out of your environment. Reactivating your virtualenv will fix this:
VirtualEnv: $ source bin/activate
VirtualFish: $ vf activate [environ]
*: I use virtualfish, but I assume this tip is relevant to both.

I have a similar issue.
I use the --system flag to avoid the error as I decribe here on other thread where I explain the specific case of my situation.
I post this here expecting that can help anyone facing the same problem.

Shell : workon not found [duplicate]

So, once again, I make a nice python program which makes my life ever the more easier and saves a lot of time. Ofcourse, this involves a virtualenv, made with the mkvirtualenv function of virtualenvwrapper. The project has a requirements.txt file with a few required libraries (requests too :D) and the program won't run without these libraries.
I am trying to add a bin/run-app executable shell script which would be in my path (symlink actually). Now, inside this script, I need to switch to the virtualenv before I can run this program. So I put this in
#!/bin/bash
# cd into the project directory
workon "$(cat .venv)"
python main.py
A file .venv contains the virtualenv name. But when I run this script, I get workon: command not found error.
Of course, I have the virtualenvwrapper.sh sourced in my bashrc but it doesn't seem to be available in this shell script.
So, how can I access those virtualenvwrapper functions here? Or am I doing this the wrong way? How do you launch your python tools, each of which has its own virtualenv!?

Just source the virtualenvwrapper.sh script in your script to import the virtualenvwrapper's functions. You should then be able to use the workon function in your script.
And maybe better, you could create a shell script (you could name it venv-run.sh for example) to run any Python script into a given virtualenv, and place it in /usr/bin, /usr/local/bin, or any directory which is in your PATH.
Such a script could look like this:
#!/bin/sh
# if virtualenvwrapper.sh is in your PATH (i.e. installed with pip)
source `which virtualenvwrapper.sh`
#source /path/to/virtualenvwrapper.sh # if it's not in your PATH
workon $1
python $2
deactivate
And could be used simply like venv-run.sh my_virtualenv /path/to/script.py

I can't find the way to trigger the commands of virtualenvwrapper in shell. But this trick can help: assume your env. name is myenv, then put following lines at the beginning of scripts:
ENV=myenv
source $WORKON_HOME/$ENV/bin/activate

This is a super old thread and I had a similar issue. I started digging for a simpler solution out of curiousity.
gnome-terminal --working-directory='/home/exact/path/here' --tab --title="API" -- bash -ci "workon aaapi && python manage.py runserver 8001; exec bash;"
The --workingdirectory forces the tab to open there by default under the hood and the -ci forces it to work like an interactive interface, which gets around the issues with the venvwrapper not functioning as expected.
You can run as many of these in sequence. It will open tabs, give them an alias, and run the script you want.
Personally I dropped an alias into my bashrc to just do this when I type startdev in my terminal.
I like this because its easy, simple to replicate, flexible, and doesn't require any fiddling with variables and whatnot.

It's a known issue. As a workaround, you can make the content of the script a function and place it in either ~/.bashrc or ~/.profile
function run-app() {
workon "$(cat .venv)"
python main.py
}

If your Python script requires a particular virtualenv then put/install it in virtualenv's bin directory. If you need access to that script outside of the environment then you could make a symlink.
main.py from virtualenv's bin:
#!/path/to/virtualenv/bin/python
import yourmodule
if __name__=="__main__":
yourmodule.main()
Symlink in your PATH:
pymain -> /path/to/virtualenv/bin/main.py
In bin/run-app:
#!/bin/sh
# cd into the project directory
pymain arg1 arg2 ...

Apparently, I was doing this the wrong way. Instead of saving the virtualenv's name in the .venv file, I should be putting the virtualenv's directory path.
(cdvirtualenv && pwd) > .venv
and in the bin/run-app, I put
source "$(cat .venv)/bin/activate"
python main.py
And yay!

add these lines to your .bashrc or .bash_profile
export WORKON_HOME=~/Envs
source /usr/local/bin/virtualenvwrapper.sh
and reopen your terminal and try

You can also call the virtualenv's python executable directly. First find the path to the executable:
$ workon myenv
$ which python
/path/to/virtualenv/myenv/bin/python
Then call from your shell script:
#!/bin/bash
/path/to/virtualenv/myenv/bin/python myscript.py

Python executable not finding libpython shared library

I am installing Python 2.7 on CentOS 5. I built and installed Python as follows
./configure --enable-shared --prefix=/usr/local
make
make install
When I try to run /usr/local/bin/python, I get this error message
/usr/local/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
When I run ldd on /usr/local/bin/python, I get
ldd /usr/local/bin/python
libpython2.7.so.1.0 => not found
libpthread.so.0 => /lib64/libpthread.so.0 (0x00000030e9a00000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000030e9200000)
libutil.so.1 => /lib64/libutil.so.1 (0x00000030fa200000)
libm.so.6 => /lib64/libm.so.6 (0x00000030e9600000)
libc.so.6 => /lib64/libc.so.6 (0x00000030e8e00000)
/lib64/ld-linux-x86-64.so.2 (0x00000030e8a00000)
How do I tell Python where to find libpython?

Try the following:
LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/python
Replace /usr/local/lib with the folder where you have installed libpython2.7.so.1.0 if it is not in /usr/local/lib.
If this works and you want to make the changes permanent, you have two options:
Add export LD_LIBRARY_PATH=/usr/local/lib to your .profile in your home directory (this works only if you are using a shell which loads this file when a new shell instance is started). This setting will affect your user only.
Add /usr/local/lib to /etc/ld.so.conf and run ldconfig. This is a system-wide setting of course.

Putting on my gravedigger hat...
The best way I've found to address this is at compile time. Since you're the one setting prefix anyway might as well tell the executable explicitly where to find its shared libraries. Unlike OpenSSL and other software packages, Python doesn't give you nice configure directives to handle alternate library paths (not everyone is root you know...) In the simplest case all you need is the following:
./configure --enable-shared \
--prefix=/usr/local \
LDFLAGS="-Wl,--rpath=/usr/local/lib"
Or if you prefer the non-linux version:
./configure --enable-shared \
--prefix=/usr/local \
LDFLAGS="-R/usr/local/lib"
The "rpath" flag tells python it has runtime libraries it needs in that particular path. You can take this idea further to handle dependencies installed to a different location than the standard system locations. For example, on my systems since I don't have root access and need to make almost completely self-contained Python installs, my configure line looks like this:
./configure --enable-shared \
--with-system-ffi \
--with-system-expat \
--enable-unicode=ucs4 \
--prefix=/apps/python-${PYTHON_VERSION} \
LDFLAGS="-L/apps/python-${PYTHON_VERSION}/extlib/lib -Wl,--rpath=/apps/python-${PYTHON_VERSION}/lib -Wl,--rpath=/apps/python-${PYTHON_VERSION}/extlib/lib" \
CPPFLAGS="-I/apps/python-${PYTHON_VERSION}/extlib/include"
In this case I am compiling the libraries that python uses (like ffi, readline, etc) into an extlib directory within the python directory tree itself. This way I can tar the python-${PYTHON_VERSION} directory and land it anywhere and it will "work" (provided you don't run into libc or libm conflicts). This also helps when trying to run multiple versions of Python on the same box, as you don't need to keep changing your LD_LIBRARY_PATH or worry about picking up the wrong version of the Python library.
Edit: Forgot to mention, the compile will complain if you don't set the PYTHONPATH environment variable to what you use as your prefix and fail to compile some modules, e.g., to extend the above example, set the PYTHONPATH to the prefix used in the above example with export PYTHONPATH=/apps/python-${PYTHON_VERSION}...

I had the same problem and I solved it this way:
If you know where libpython resides at, I supposed it would be /usr/local/lib/libpython2.7.so.1.0 in your case, you can just create a symbolic link to it:
sudo ln -s /usr/local/lib/libpython2.7.so.1.0 /usr/lib/libpython2.7.so.1.0
Then try running ldd again and see if it worked.

I installed Python 3.5 by Software Collections on CentOS 7 minimal. It all worked fine on its own, but I saw the shared library error mentioned in this question when I tried running a simple CGI script:
tail /var/log/httpd/error_log
AH01215: /opt/rh/rh-python35/root/usr/bin/python: error while loading shared libraries: libpython3.5m.so.rh-python35-1.0: cannot open shared object file: No such file or directory
I wanted a systemwide permanent solution that works for all users, so that excluded adding export statements to .profile or .bashrc files. There is a one-line solution, based on the Red Hat solutions page. Thanks for the comment that points it out:
echo 'source scl_source enable rh-python35' | sudo tee --append /etc/profile.d/python35.sh
After a restart, it's all good on the shell, but sometimes my web server still complains. There's another approach that always worked for both the shell and the server, and is more generic. I saw the solution here and then realized it's actually mentioned in one of the answers here as well! Anyway, on CentOS 7, these are the steps:
vim /etc/ld.so.conf
Which on my machine just had:
include ld.so.conf.d/*.conf
So I created a new file:
vim /etc/ld.so.conf.d/rh-python35.conf
And added:
/opt/rh/rh-python35/root/usr/lib64/
And to manually rebuild the cache:
sudo ldconfig
That's it, scripts work fine!
This was a temporary solution, which didn't work across reboots:
sudo ldconfig /opt/rh/rh-python35/root/usr/lib64/ -v
The -v (verbose) option was just to see what was going on. I saw that it did:
/opt/rh/rh-python35/root/usr/lib64:
libpython3.so.rh-python35 -> libpython3.so.rh-python35
libpython3.5m.so.rh-python35-1.0 -> libpython3.5m.so.rh-python35-1.0
This particular error went away. Incidentally, I had to chown the user to apache to get rid of a permission error after that.
Note that I used find to locate the directory for the library. You could also do:
sudo yum install mlocate
sudo updatedb
locate libpython3.5m.so.rh-python35-1.0
Which on my VM returns:
/opt/rh/rh-python35/root/usr/lib64/libpython3.5m.so.rh-python35-1.0
Which is the path I need to give to ldconfig, as shown above.

This worked for me...
$ sudo apt-get install python2.7-dev

On Solaris 11
Use LD_LIBRARY_PATH_64 to resolve symlink to python libs.
In my case for python3.6 LD_LIBRARY_PATH didn't work but LD_LIBRARY_PATH_64 did.
Hope this helps.
Regards

This answer would be helpful to those who have limited auth access on the server.
I had a similar problem for python3.5 in HostGator's shared hosting. Python3.5 had to be enabled every single damn time after login. Here are my 10 steps for resolution:
Enable the python through scl script python_enable_3.5 or scl enable rh-python35 bash.
Verify that it's enabled by executing python3.5 --version. This should give you your python version.
Execute which python3.5 to get its path. In my case, it was /opt/rh/rh-python35/root/usr/bin/python3.5. You can use this path get the version again (just to verify that this path is working for you.)
Awesome, now please exit out of current shell by scl.
Now, lets get the version again through this complete python3.5 path /opt/rh/rh-python35/root/usr/bin/python3.5 --version.
It won't give you the version but an error. In my case, it was
/opt/rh/rh-python35/root/usr/bin/python3.5: error while loading shared libraries: libpython3.5m.so.rh-python35-1.0: cannot open shared object file: No such file or directory
As mentioned in Tamas' answer, we gotta find that so file. locate doesn't work in shared hosting and you can't install that too.
Use the following command to find where that file is located:
find /opt/rh/rh-python35 -name "libpython3.5m.so.rh-python35-1.0"
Above command would print the complete path (second line) of the file once located. In my case, output was
find: `/opt/rh/rh-python35/root/root': Permission denied
/opt/rh/rh-python35/root/usr/lib64/libpython3.5m.so.rh-python35-1.0
Here is the complete command for the python3.5 to work in such shared hosting which would give the version,
LD_LIBRARY_PATH=/opt/rh/rh-python35/root/usr/lib64 /opt/rh/rh-python35/root/usr/bin/python3.5 --version
Finally, for shorthand, append the following alias in your ~/.bashrc
alias python351='LD_LIBRARY_PATH=/opt/rh/rh-python35/root/usr/lib64 /opt/rh/rh-python35/root/usr/bin/python3.5'
For verification, reload the .bashrc by source ~/.bashrc and execute python351 --version.
Well, there you go, now whenever you login again, you have got python351 to welcome you.
This is not just limited to python3.5, but can be helpful in case of other scl installed softwares.

I installed using the command:
./configure --prefix=/usr \
--enable-shared \
--with-system-expat \
--with-system-ffi \
--enable-unicode=ucs4 &&
make
Now, as the root user:
make install &&
chmod -v 755 /usr/lib/libpython2.7.so.1.0
Then I tried to execute python and got the error:
/usr/local/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
Then, I logged out from root user and again tried to execute the Python and it worked successfully.

All it needs is the installation of libpython [3 or 2] dev files installation.

just install python-lib. (python27-lib). It will install libpython2.7.so1.0. We don't require to manually set anything.