Develop Reference

Can't open file 'demo.py': [Errno 2] No such file or directory when running through a bash script

I have a folder named test-folder with two files.
# file name: demo.py
print('hello world')
# file name: script.sh
python3.7 demo.py
The test-folder is present inside /home/username/Documents/
I have exported the above path to the .bashrc.
Now when I try to execute the script.sh using the following command,
bash script.sh
I get the following error
python3.7: can't open file 'demo.py': [Errno 2] No such file or directory
How can I solve this? I'm on Ubuntu 18.04

Move to the folder containing the file :
$ cd /home/username/Documents/test-folder
after making sure you are in the folder ccontaining the file try:
$ python file_name.py
if it doesn't work try the full path to python and the full path to the file like this:
$ /usr/lib/python3.7 /home/username/Documents/test-folder/file_name.py

If you want to use script.sh from anywhere, your approach won't work.
When executing $ bash script.sh, the file script.sh is expected to be located in the working directory.
You can use absolute paths to solve this:
# filename: script.sh
python3.7 /home/username/Documents/test-folder/demo.py
Try executing this with bash /home/username/Documents/test-folder/script.py.
You can achieve your goal in an arguably simpler way by adding /home/username/Documents/test-folder to your PATH, adding a shebang to demo.py and making demo.sh executable.
$ export PATH=/home/username/Documents/test-folder:$PATH
$ chmod +x /home/username/Documents/test-folder/demo.py
demo.sh will look like this with a shebang:
#!/usr/lib/python3.7
print('hello world')
and can be executed from anywhere with $ demo.sh.

It works for me. Is there anything about your setup that is different than mine?
$ cat demo.py
print("Hello World")
$ cat script.sh
python3.7 demo.py
$ tree
.
└── test-folder
├── demo.py
└── script.sh
1 directory, 2 files
$ cd test-folder/
$ bash script.sh
Hello World
Edit: Try the following.
$ cat script.sh
python3.7 "`dirname "$0"`/demo.py"

If your script.sh file is only there to run the Python script, it’s not needed. Instead make the Python script itself executable.
(1) add this as the first line in your Python file — this tells Unix which interpreter to use when running this file:
#!/usr/bin/env python37
(2) make your Python file executable so it can be run directly:
chmod +x demo.py
(3) put that file on the path, e.g. a path you set via a Bash config file, or one that’s already on the path like /usr/local/bin.
Now any time you give the command demo.py your script will run. Note the .py extension isn’t required — you can name the file anything you like.

Python script with executable permission says "Command Not Found"

I have a python script abc.py. Inside a shell script file, I call it ./abc.py
Inside abc.py, at the top, I have #!/usr/bin/python
test -x abc.py && echo true || echo false return true
On executing the shell script, it says ./abc.py: Command not found.
It works if I call it as python ./abc.py instead of just ./abc.py
The issue is only with this abc.py file that I created today. The older python scripts in the same directory with different purpose and names work without any issue.
I have referred the correct path of the file BTW. What could be the issue?

In tcsh, this happens when the interpreter is invalid:
$ cat foo
#!/invalid
$ tcsh -c './foo'
./foo: Command not found.
$ bash -c './foo'
bash: ./foo: /invalid: bad interpreter: No such file or directory
This could be for several reasons. Perhaps your path is wrong for your system:
% type python
python is /usr/local/bin/python
in this case the script needs to start with #!/usr/local/bin/python instead.
Alternatively, the script could have Windows line endings (carriage returns):
$ cat -v foo
#!/usr/bin/python^M
...^M
In this case, save it with Unix line endings instead.

Trying to replicate your setup as follows:
File: asdf.py
#!/usr/bin/python
print("Hello World")
File: asdf.sh
#!/bin/bash
if [ -x asdf.py ]; then ./asdf.py ; else echo "Arrrrgh. File does not exist or does not have executable permisions"; fi
Now I save the files in the same directory, e.g. the Desktop, make them executable and run the shell script from the terminal.
usr#cmptr $ chmod +x asdf.{py,sh}
usr#cmptr $ ls -la asdf.{py,sh}
-rwxr-xr-x 1 usr usr 613 Mar 15 22:22 asdf.py
-rwxr-xr-x 1 usr usr 56 Mar 15 22:14 asdf.sh
usr#cmptr $ ./asdf.sh
Hello World
Now, maybe I have misunderstood your setup but this should work just fine.

python script doesn't execute when calling it from shell

I have a python 3 program in file foo the file has execution permissions, and the first line of the script is
#!/usr/bin/python3
When I run the file from python directly, i.e:
/usr/bin/python3 foo 3 boo
Everything runs perfectly well, but when I try to run the program by calling the file name I get:
foo 3 boo
foo: Command not found.
Even when specifying the relative or full path to the script I get the same response.
./foo 3 boo
./foo: Command not found.
/full/path/to/the/file/foo 3 boo
/full/path/to/the/file/foo: Command not found.
Some more info that was requested:
head -2 foo
#!/usr/bin/python3
which python3
/usr/bin/python3
Working from tcsh, where is the problem?
I'm working on a remote university computer through ssh, could it be the case that the sysadmins somehow prevent this?

You need to specify the absolute path of the script. Try
./foo 3 boo
while in the same directory as the script. Otherwise, the shell will only look in your PATH for something named foo, which doesn't exist.

You must give the path to the file
./foo
If you just call "foo", the shell will look in the environment variable $PATH the folder that contains the "foo" binary, and he will not find it ...

I'm using python3, just an example illustration..
Just a test script ..
$ cat test.py
#!/python/v3.6.1/bin/python3
import os
print(os.system("ls"))
$ which python3
/python/v3.6.1/bin/python3
Permission to make it executable.. i'm have doubt if the script foo is executable in your case..
$ chmod +x test.py
Test Run..
$ ./test.py
a.py Backup File_Write_Method NWK-old Python_aritsTest Python_Ftp Python_Mail Python_Primer test.py
argpass BASH Network_DeOps Python-3.6.3 Python_Dump Python_FTP Python_Panda readme_python tmp
awk.sh dnspython nslookup Python-3.6.3.tar.xz Python_excep Python_ldap Python_Parsers Regular_Expr tt.csv
0
It might be shell problem as i reproduced the tcsh shell & it fails to run..
$ tcsh
$ ./test
./test: Command not found.
but it runs , when i run it like below..
tcsh -c ./test.py

Assuming your script is executable and you have python3 in /usr/bin/python3. I think you are trying to run your script from a partition mounted with the noexec argument to check if it is the problem you should run:
mount | grep <partition> | grep noexec
If I'm right you can just mount the partition with the exec option or move your script elsewhere.

Why is the way of setting permission by os.mkdir in python different from one of mkdir in bash?

I made directories with file mode by mkdir in bash and os.mkdir in python.
They made directories with different permissions.
My test code in command line is following,
$ mkdir -m 0775 aaa
$ cd aaa
$ mkdir -m 0777 bbb
$ python -c 'import os; os.mkdir("ccc",0o777)'
The permission of directories, aaa, bbb and ccc are following
directory aaa: drwxrwxr-x
directory bbb: drwxrwxrwx
directory ccc: drwxrwxr-x
It seems that mkdir in bash does not care the permission of parent directory but os.mkdir in python does. Is it right?
And why do they have different mechanism?
Thank you very much.

mkdir(1) is temporarily setting the umask to 0 if you specify a mode, as cryptically documented in the manual:
-m, --mode=MODE
set file mode (as in chmod), not a=rwx - umask
Python is just calling the mkdir(2) syscall with the mode given and the usual umask behavior.
The equivalent Python code to what mkdir(1) is doing:
m = os.umask(0)
os.mkdir("ccc")
os.umask(m)

Python CGIHTTPServer crashes with "OSError: [Errno 13] Permission denied"

I am running the following command from my home directory:
python -m CGIHTTPServer
This runs the server, but when I try to access a script in the cgi-bin directory I get:
Traceback (most recent call last):
File "/usr/lib/python2.7/CGIHTTPServer.py", line 251, in run_cgi
os.execve(scriptfile, args, env)
OSError: [Errno 13] Permission denied
Running as root does not make a difference. The files seem to have all the right permissions:
student#bandersnatch:~$ ls -lhR
.:
total 12K
drwxr-xr-x 2 student student 4.0K Jun 13 18:38 cgi-bin
drwxr--r-- 2 student student 4.0K Jun 10 2004 kalpy
-rwxrwxrwx 1 student student 2.0K Jun 13 12:37 test.html
./cgi-bin:
total 8.0K
-rwxr-xr-x 1 student student 31 Jun 13 18:38 test.py
Edit: The content of test.py is:
#!/usr/bin/python
print "test"
The shebang is valid:
~$ which python
/usr/bin/python

Are you, by any chance, running the process as root?
If you use the source, you will see in CGIHTTPServer.py, just before calling execve:
try:
os.setuid(nobody)
except os.error:
pass
That is, it will run the CGI script as nobody, if it is able to change the UID, that is if it is root. If it is not root, this call will most likely fail, and pass on.
So my guess is that you are running the server as root, so the script is run as nobody, but this user doesn't have access to the script. Which is expected, as you say that it is in your home dir.
Two solutions that I can think of:
The recommended: do not run the server as root!
The workaround: copy the script to a directory where nobody can read it (/tmp for example).

Personally, unless there's some reason I'm unaware of, I'd recommend using subprocess.Popen instead of os.execve. I have run into Errno 13 before, trying to start a .app with Popen(['open execName.app']). I had to use Popen(['execName.app/Contents/MacOS/execName', 'arg1', 'arg2'...]) instead. Don't know if that helps, but give it a shot.

I ran into the same problem from ubuntu Linux.
Followed the solution by "Mike", with modification.
Instead doing chmod of the "/usr" which has several folders, change the permissions of the folder containing executable file that was denied. (you can check that server would run fine when loading a static html file in the same location, and shows error only when script is run).
cd /pathto/folder/with/deniedscript
sudo chmod -R 755 ./
Now the script has permission, so should run fine.
Note that -R gives the permission to all files in this folder(and subfolders if any).

When running on Windows the files run right out of the command prompts.
For Linux and Windows users that's not the case!
I get the following error:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/server.py", line 1158, in run_cgi os.execve(scriptfile, args, env) PermissionError: [Errno 13] Permission denied:
You'll need to the following to resolve these issues:
For Linux Users:
1) Ensure shebang is adjusted for Python 3 running on Linux and Mac OSX systems:
#!/usr/bin/env python3
2) Since the original executable files were written on windows they'll have hidden '\r' in the files that must be removed. Here are three possible ways:
a) In terminal command line type: tr -d ‘\r’ < input file name > output file name (just rename the output file a new name --> erase old file --> then rechange output filename back to original)
b) In terminal command line type: cat inputfile | col -b > outputfile (just rename the output file a new name --> erase old file --> then rechange output filename back to original)
c) Download dos2unix, then type in terminal command line: dos2unix input file name
3) Make file executable:
In terminal command line type:
a) chmod 755 filename
or
b) chmod +x filename
or
chmod a+x filename
For Mac OSX users it's almost the same:
Repeat step 1) from Linux
Repeat step 2) from Linux
For step 3 things change:
Based on the apache.org wiki page: https://wiki.apache.org/httpd/13PermissionDenied
It says that you have to make every executable from file location traversing all the away up to the /Users root directory.
You'll have to do the following.
3) In terminal command line:
a) type command: `cd /Users`
b) type command: `sudo chmod -R 755`
Now you can run your server .py file via:
sudo webserver.py
and the input file via normal:
python3 inputfile.py
Now you should be all good to go with no more permission errors! You can make necessary adjustments to shebang and command line if running python 2.