Why running a python file doesn't require the x permission when running it like this:
python script.py
But it does when it's being run as:
./script.py
Because what you are running with python script.py is the python program; then, it loads and runs the script that you specified in parameters, that is script.py (basically a text file). The script file doesn't need to be an executable because what is executed here is the python interpreter (the python binary itself, that should obviously have the x permission).
With .\script.py, you try to run directly your script (still the same text file) as a program. When you do that, you want it to be parsed with the interpreter that you specified in the first line of your script code, the "shebang", e.g. #!/usr/bin/env python. If it's not set with the x permission, the OS doesn't try to "execute" your file (though it might try to open it with the default program, where applicable), so, it will not care about the shebang.
The file itself it interpreted (read) rather than actually executed in your first example. The python application is what needs execute rights.
In the second example the file itself is being executed, so needs those rights in order to proceed.
When we run a script as python script.py we actually invoke the python interpreter which is generally located at /usr/bin/python (The output of which python will tell you where exactly).
The interpreter in turn reads the scripts and executes its code.
It is the interpreter that has the execute permission.
When a script is executed as ./script.py then the script is executed directly and hence the script requires execute permission. The interpreter used is specified by shebang line.
When the kernel finds that the first two bytes are #! then it uses the rest of the line as interpreter and passes the file as argument. Note that to do this the file needs to have execute permission. In the former case we are indirectly doing what the kernel would do had we executed the script as ./script.py
In short for executing by method1 the interpreter needs only read permission but for later it needs to execute it directly
Related
Background
Below, I detail two different ways of running Python files - I am confused about the difference between them.
Running the Python file as an executable.
To run a Python file as an executable, I must first set a shebang in my file (# /usr/bin/env python3), then run $ chmod +x filename.py at the command line, then run $ ./filename.py at the command line to execute the file.
Running the Python file through the python3 command line command.
To run a Python file through the python3 command, I open my command line and run $ python3 filename.py.
My Question
I understand that, when running the Python file as an executable, the shebang directs the computer to launch the (in this case) python3 interpreter which will interpret the subsequent code in the file and therefore run the file. When running the file through the python3 command, I understand that this is just another way of directing the computer to launch python3 to interpret the code in the file. To me, these two techniques therefore seem identical.
Am I missing something? What's the difference, if any, between these two ways of running a Python file?
In practice, they're identical.
The shebang is just a convention that tells the OS what to do with the script file that you've chmod-ed to be executable -- i.e., executable with what. Without it, the OS just treats the file as a text file and will try to execute it as a shell script.
From the point of view of the system the shebang line is necessary when the file is run as an executable. After checking the permission bits the OS sends the file to the program loader which determines how to run the program by parsing the first line as an interpreter directive. Based on this the loader executes python3 based on the specified path to the executable. If instead /usr/bin/env was used in shebang line then the env command finds the executable based on PATH ( env can also be used to make other modifications to the environment using NAME=VALUE pairs). The loader then passes to the program the path that was used when the user tried to run the script as an argument.
In the second case the OS just loads python3 and passes the script as an argument, it doesn't care at all about the file and its permissions.
From the point of view of the user (which IMO is more important) the shebang line is just another level of abstraction that hides the details of implementation (in this case what program is used to run the script). This means the line can be changed (perhaps by modifying the environment or using a different executable) without the user having to change the way they invoke the script. Also, if the user puts the script in a location that is in PATH then they can invoke the script from anywhere without first navigating to the directory or remembering the location.
On the other hand invoking python3 directly allows the user to pass additional flags to python3 for example -i for interactive runs and -m to use additional modules such as pdb for debugging.
Edit: Based on #alaniwi's comment below explained the role of env in finding the path of the python3 executable in more detail.
Nope, you have pretty much captured it.
A practical consequence is that the shebang relieves you from having to remember whether it's python3 frobnicate or python frobnicate or sh frobnicate or bash frobnicate or awk frobnicate or perl frobnicate or...
This also makes it easy down the line to change your mind. Many tools of mine have started life as simple shell scripts, then been rewritten in Python or something else; but the calling interface doesn't change.
Before Unix, there was an unbridgable gap between system utilities (which you invoke simply by name) and user scripts (which before the introduction of the shebang always had to be called with an explicit interpreter).You still see remnants of this division in lesser systems. An important consequence was that users were able to easily and transparently wrap or replace standard commands with their own versions. This in some sense democratized the system, and empowered users to try out and evaluate improvement ideas for the system on their own. (Figuring out why your brilliant theory wasn't so great in practice is also an excellent way to learn and improve.) I don't think the importance of this versatility and flexibility can be overstated; it's one of those things which converted us from mere users to enthusiasts.
I have a script containing a line of the type:
os.system('file.exe <arg>')
file.exe makes some calculations taking <arg> as input and writes an output file. <arg> is a file containing parameter values located in the same folder as file.exe.
The instruction, if executed independently in Python IDLE, works well. However, when executing the script in IDLE it fails. That is, a Windows shell window flashes, but no output file is written. There is a similar behaviour with os.popen() and subprocess.call(). Everything works well in Spyder on another computer.
I was wondering which might be the reason for this behaviour.
In Linux, we usually add a shebang in a script to invoke the respective interpreter. I tried the following example.
I wrote a shell script without a shebang and with executable permission. I was able to execute it using ./. But if I write a similar python program, without shebang, I am not able to execute it.
Why is this so? As far as my understanding, shebang is required to find the interpreter. So how does shell scripts work, but not a python script?
My assumption is that a script without a shebang is executed in the current environment, which at the command line is your default shell, e.g. /bin/bash.
shell scripts will only work if you are in the shell you targeted ... there is not python shell ... as such python will never work without explicity calling python (via shebang or command line)
By default the shell will try to execute the script. The #! notation came later
There’a subtle distinction here. If the target is a binary or begins with a #! shebang line, then the shell calls execv successfully. If the target is a text file without a shebang, then the call to execv will fail, and the shell is free to try launching it under /bin/sh or something else.
http://en.wikipedia.org/wiki/Shebang_(Unix)
Under Unix-like operating systems, when a script with a shebang is run as a program, the program loader parses the rest of the script's initial line as an interpreter directive; the specified interpreter program is run instead, passing to it as an argument the path that was initially used when attempting to run the script.
Now when the #! is not found, every line is interpreted as native shell command. And hence if you write a bash script and run it under bash shell it will work. If you run the same bash script in say a tcsh shell it will not work without the initial #!/usr/bin/tcsh
I am new in python and I have python file name MiscTest.py and now I want to run this file in linux with single name MiscTest only not traditional way like python MiscTest.py.
Can anybody have any idea and also I don't want to right shell script for this ? I know I can do it with shell script
First, you have to make the script executable:
chmod +x MiscTest.py
Next, add this in the first line (the "shebang") of the code:
#!/usr/bin/env python
And make sure that inside the script the execution starts in here (think of this block of code as the main() function in C-like languages):
if __name__ == "__main__":
# call main function
Also, if you need to execute the script from anywhere in the system (without changing directories), don't forget to add the path of the script to the $PATH environment variable of your shell, see this post for details.
By doing all of the above, you won't have to call the script like python MiscTest.py, it'll be enough to say MiscTest.py in the shell, or to double-click it in a file explorer. As a side note, although you can, I believe you shouldn't remove the .py extension of the file, that's how it's recognized as a Python file by the different tools and programs in your system.
To chmod the "MiscTest.py" file so that the owner can read, write, and execute the file, the group can read and execute the file, and the world can execute the file, issue the following command:
chmod 751 ~/MiscTest.py
I've a shell script with two shebangs, the first one tells #!/bin/sh and after a few lines the other one is #!/usr/bin/env python.
When this script is given executable permission and ran as ./script.sh, the script works fine, uses /bin/sh in first part and uses python interpreter in latter part.
But when the script is run as sh script.sh, the second shebang is not recognized and the script fails. Is there anyway I can force to change the interpreter if the script is run explicitly as sh script.sh.
The reason I need this is because I need to run the scripts through a tool which runs as sh script.sh
As far as I know you cannot have two shebang lines in one script. The shebang works only when -
it is on the first line
it starts in column one
If you need to run python code then have it in another script and then call the script by doing
python path/to/the/script.py
A better way to do this would be to use a shell here document. Something like this:
#!/bin/sh
curdir=`pwd`
/usr/bin/env python <<EOF
import os
print os.listdir("$curdir")
EOF
This way, you won't need to distribute the code on two separate files.
As you see, you can even access shell variables from the python code.
have your script.sh script invoke the python interpreter directly:
#!/bin/sh
# include full path if needed
python (your python interpreter arguments)