Develop Reference

Difference Between Running Python File as Executable vs. Running from Command Line?

Background
Below, I detail two different ways of running Python files - I am confused about the difference between them.
Running the Python file as an executable.
To run a Python file as an executable, I must first set a shebang in my file (# /usr/bin/env python3), then run $ chmod +x filename.py at the command line, then run $ ./filename.py at the command line to execute the file.
Running the Python file through the python3 command line command.
To run a Python file through the python3 command, I open my command line and run $ python3 filename.py.
My Question
I understand that, when running the Python file as an executable, the shebang directs the computer to launch the (in this case) python3 interpreter which will interpret the subsequent code in the file and therefore run the file. When running the file through the python3 command, I understand that this is just another way of directing the computer to launch python3 to interpret the code in the file. To me, these two techniques therefore seem identical.
Am I missing something? What's the difference, if any, between these two ways of running a Python file?

In practice, they're identical.
The shebang is just a convention that tells the OS what to do with the script file that you've chmod-ed to be executable -- i.e., executable with what. Without it, the OS just treats the file as a text file and will try to execute it as a shell script.

From the point of view of the system the shebang line is necessary when the file is run as an executable. After checking the permission bits the OS sends the file to the program loader which determines how to run the program by parsing the first line as an interpreter directive. Based on this the loader executes python3 based on the specified path to the executable. If instead /usr/bin/env was used in shebang line then the env command finds the executable based on PATH ( env can also be used to make other modifications to the environment using NAME=VALUE pairs). The loader then passes to the program the path that was used when the user tried to run the script as an argument.
In the second case the OS just loads python3 and passes the script as an argument, it doesn't care at all about the file and its permissions.
From the point of view of the user (which IMO is more important) the shebang line is just another level of abstraction that hides the details of implementation (in this case what program is used to run the script). This means the line can be changed (perhaps by modifying the environment or using a different executable) without the user having to change the way they invoke the script. Also, if the user puts the script in a location that is in PATH then they can invoke the script from anywhere without first navigating to the directory or remembering the location.
On the other hand invoking python3 directly allows the user to pass additional flags to python3 for example -i for interactive runs and -m to use additional modules such as pdb for debugging.
Edit: Based on #alaniwi's comment below explained the role of env in finding the path of the python3 executable in more detail.

Nope, you have pretty much captured it.
A practical consequence is that the shebang relieves you from having to remember whether it's python3 frobnicate or python frobnicate or sh frobnicate or bash frobnicate or awk frobnicate or perl frobnicate or...
This also makes it easy down the line to change your mind. Many tools of mine have started life as simple shell scripts, then been rewritten in Python or something else; but the calling interface doesn't change.
Before Unix, there was an unbridgable gap between system utilities (which you invoke simply by name) and user scripts (which before the introduction of the shebang always had to be called with an explicit interpreter).You still see remnants of this division in lesser systems. An important consequence was that users were able to easily and transparently wrap or replace standard commands with their own versions. This in some sense democratized the system, and empowered users to try out and evaluate improvement ideas for the system on their own. (Figuring out why your brilliant theory wasn't so great in practice is also an excellent way to learn and improve.) I don't think the importance of this versatility and flexibility can be overstated; it's one of those things which converted us from mere users to enthusiasts.

Is there a way to execute .py scripts from path in windows?

I would like to use a python script anywhere within command prompt. This is possible in unix, but I couldn't find anything for windows. Do I have to convert it to .exe?
Edit: not sure why this is being downvoted, maybe it's a silly question but I can't find any similar threads here and I can't be the first person to want to execute .py scripts from their path...
Edit 2: I think my wording was unclear. I would like to know a method to execute python scripts in Windows without needing to specify python path/to/script.py every time. Here is a solution on Linux where the shebang statement invokes the python interpreter, and the script in question can be easily placed in bin: How do I install a script to run anywhere from the command line? . Does there exist a solution like this for Windows?

Here's a solution for running myScript.py:
add to the myScript.py file a first line #!python (or #!python3 if you want to use Python 3)
For instance:
#!python
import sys
sys.stdout.write("hello from Python %s\n" % (sys.version,))
change the "opens with" property of myScript.py to py.exe (to find where it is use where py-- I have it in C:\Windows\py.exe)
put the script myScript.py somewhere in your Windows path
and now you should be able to type myScript.py anywhere in a command prompt and run the Python script with your chosen Python version.
See: https://docs.python.org/3.6/using/windows.html#from-the-command-line

One-word-command from .py (without aliases)

Some programs can be executed from anywhere with a single one-word command. An example of this is youtube-dl, which is a python program that can be executed with the simple command youtube-dl [input]. As far as I have understood, this is simply because there exists a file called /usr/bin/youtube-dl, and /usr/bin is in PATH. However, I do not understand what I have to do to make something like this myself. Right now, I have a python project called testproject that includes a python program like this:
~/testproject/files/myownprogram.py
What do I have to do to make this a binary executable such as youtube-dl?
I know I can make an alias mop="python ~/testproject/files/myownprogram.py", and this is also what I have done, and it works fine. That is, I can write $ mop, and successfully run my program.
But for curiosity's, and learning's, sake, I want to know how to make a file such as the /usr/bin/youtube-dl file, removing the need for aliases.
I find this hard to find information about in search engines...
Any help is greatly appreciated! :-)
Edit:
My question differs from the one marked as duplicate, in that I'm not looking to execute it as a .sh-script. I simply want to execute it as a suffix-less one-word command, similar to all the other executables that are in /usr/bin. :-)
Ex.: $ myown should run my program, without the need for aliases or writing ".sh" or ".py" at the prompt. That is, I want to have a file /usr/bin/myown that somehow runs my testproject at the simple command myownfrom anywhere.

The applications are being executed "from anywhere" because the system has a specific hierarchy of places it looks for these files (the current directory, then the system path). So, it knows to look in /usr/bin because that's in your system path.
As to ensuring it to use python when it's executed, you should add the following to the top of the file (check out some python application source code and you'll see this):
#!/usr/bin/env python
This tells the system to execute the script using the systems "python" command.

Python script gives `: No such file or directory`

I have several python scripts which work just fine but one script has (as of this morning) started giving me this error if I try to run it from the bash:
: No such file or directory
I am able to run the 'broken' script by doing python script_name.py and after looking around a bit the general idea that I picked up was that maybe my line ending of the hashbang got changed (silently) so I looked at the line ending of a working script and a broken script via the :set list option in VI as indicated in this question -> View line-endings in a text file
Both files appear to end using the same character (a $) so I am kind of at a loss on how to proceed from here. Specifically, how to actually 'see' the line ending in case the set list was not the right method.
PS: The script is executable and the shebang is in there, I stated that it's just this 1 script that was working fine before the weekend but it started giving me this error as of this morning.
-- edit: --
Running the script through dos2unix does get it working again but I would like to know of any way to visualize the line ending somehow in VI(M) or why Geany somehow converted the line endings in the first place (as I never work on a dos/windows system anyhow).

From the comments above it looks like you have dos line endings, and so the hashbang line is not properly processed.
Line ending style are not shown with :set list in Vim because that option is only used when reading/writing the file. In memory line endings are always that, line-endings. The line ending style used for a file is kept in a Vim per-file option, weirdly called fileformat.
To see/change the line ending style from Vim, you can use the following commands:
:set fileformat
:set ff
It will show dos or unix. You want unix, of course ;-).
To change it quickly you can save the file with:
:w ++ff=unix
Or if you prefer:
:set ff=unix
And then save the file normally.
So see all the gory details just do :help fileformat, :help file-formats and :help fileformats

You can also use the dos2unix command to convert the file format
dos2unix
This helped me to run the python scripts
This normally happens when we open files in windows do changes and save it.
if you open the file locate the ^M characters at the end of every line
Thanks

Personally, I find it kinda wrong using direct path to python interpreter. As you dont use windows platform, you should have program env, usually in /usr/bin (/usr/bin/env). Try using following shebang:
#!/usr/bin/env python
Different distros store python binary in /bin or /usr/bin (or some weird locations), and this one makes your script config-independent (as far as possible, here we have possibility that env is stored elsewhere; still - it is less possible that env is not in /usr/bin than that python is mislocated).
I had similiar problem (if not exactly the same) and that worked for me.
Also, I have both python interpreters (2.7.x and 3.x) installed, so I need to use "python3" argument for env. AFAIR usually distros link different names to different binaries, so "env python" will run python2.7 on my system, "env python3" (also python33, or smth like that) will run p3k, and "env python2" (also python27, etc) will run python 2.7.x. Declaring which version of interpreter should be used seems like a good idea too.

I came across this problem editing my code on Windows, checking it in with git, and checking out and running it on Linux.
My solution was: tell git to Do The Right Thing. I issued this command on the Windows box:
git config --global core.autocrlf true
Modified the files and checked them in; voila, no such problem any more.
As discussed on the Git documentation.

What does it mean in linux scripts? #!/usr/bin/python -tt

I know that in the begining of .sh bash scripts is
#!/bin/bash
which points to the command interpeter executable.
But during watching Google Python Class http://www.youtube.com/watch?v=tKTZoB2Vjuk I noticed that for python they use
#!/usr/bin/python -tt
. Surfing the Internet I also have found such styles of this notation:
#!/usr/local/bin/python
and even
#!/usr/bin/env python
.
So, I'm new with Python and I'm ordinary Linux user and I have a few questions about this "magic" line:
First of all, what is the right form of this line? and why?
What does -tt key means in #!/usr/bin/python -tt ?
What program is parsing this line in Linux?
What syntax of this line for any script?
Why this line is so necessary if each file have it's extension?
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
It's really interesting to me.
What's this line? Why this line? How to write this line? Why in such a way?...

Question #1) The line is called a shebang, and there's no right form that works universally. e.g.
#!python
#!/usr/bin/python
#!/usr/local/bin/python
#!/usr/bin/python -t
are all valid/acceptable forms, but may not work on all systems:
#!python will work only if the python executable is somewhere in your shell's PATH
#!/usr/bin/python only works if the python binary is actually in /usr/bin
#!/usr/local/bin/python also only works if python is in /usr/local/bin
Question #2)
#!/usr/bin/python -tt is passing the -tt option to python, as if you'd done:
$ python -t somescript.py
at the shell prompt. You can pass arbitary command line arguments to the interpreter on the shebang line.
Question #3)
The line is interpreted by the OS kernel and the shell you're currently using. The stuff after the #! simply tells the OS which program should be fired up to "execute" the rest of the script.
Question #4)
The script syntax depends on the language you're using. E.g. a PHP shell script must take the form of
#!/usr/bin/php
<?php
... php code here ...
A #!/usr/bin/perl perl script must use Perl syntax, etc... If you put PHP code with a Perl shebang, you'll just have Perl barf up the script with syntax errors, as PHP code is NOT perl code
Question #5)
Shebangs are for Unix systems, where file extensions were never really used to identify file types to the OS. A .c file was understood to be a C language source code file, but that's merely a convention. You could put a Bash shell script into a .c file, make it executable, and with the #!/bin/bash shebang, it would execute as a Bash script.
Determining executable types by file extension is more of a Windows thing.
Question #6)
That goes back the stuff in question #1 - if the shebang claims the interpreter is at some OTHER path than where it is, this particular script can't be executed until the shebang is fixed, or the interpreter is moved. Shebangs are very handy, but not infallible.
Thankfully, most interpreters are installed in fairly standard locations these days, so it'd be somewhat unusual to find (say) Perl installed at /some/wonky/weird/path instead of /usr/bin

From the manpage:
-t Issue a warning when a source file mixes tabs and spaces
for indentation in a way that makes it depend on the worth of a tab
expressed in spaces. Issue an
error when the option is given twice.
The right form of the line is the one you want to use.
It's the interpreter that reads this line known as shebang. If you write a python script with first line as "#!/usr/bin/python" & invoke it using bash, it's the /bin/sh interpreter that reads first line and starts the proper interpreter.
It's a shebang. The syntax of feature consists of the character sequence #!, i.e. the number sign and an exclamation point character
File extensions are not relevant in linux generally. You can have a python script that doesn't have a .py extension.
For ex.
shadyabhi#archlinux ~ $ cat a
print "Hello World"
shadyabhi#archlinux ~ $ python2 a
Hello World
shadyabhi#archlinux ~ $
Even the shebangs are only necessary if you want to start a script using $./script as in this case you didn't mention the interpreter you want to use.

#!/usr/bin/env python
issue errors about inconsistent tab usage
Kernel
#!/path_to_the_interpreter or /usr/bin/env
*nix does not check extensinon at all(except some DE could do that)
This is why you should use #!/usr/bin/env
More info at wiki

The different paths are to where the python interpreter has been installed. Different flavours of Linux install it in different places.
Linux doesn't care for extensions its a Windows thing.
The bash session uses the line to call the correct interpreter for the script your running.
The different places to where the files are stored, called and used are all based on defined places where files should be and located by software. Dev for devices, home for user stored area, bin for programs. But as time has gone by, different systems require different locations.
I would suggest getting a book on Linux/Unix and learning the basics of the file system. It does help a lot.

This is called a shebang. It tells the system that it should pass the file as an argument to the specified program instead of trying to execute it per se.
First of all, what is the right form of this line? and why?
The correct path is wherever your python interpreter is installed. The arguments (-tt) will depend on what you want. Some people insist on #!/usr/bin/env in case the interpreter happens to be elewhere.
What does -tt key means in #!/usr/bin/python -tt ?
I don't use python, so someone else will have to answer this.
When I launch any script in Linux (not exact Python script) what program parses and uses this line? I consider that it's not bash because even for bash scripts this line is needed.
I've heard (and am pretty sure) it's the kernel. Even if it were bash, it would need the line to tell bash it is supposed to be a script it should interpret instead of passing to another program. /usr/bin/env is a command that searches the PATH for the specified argument and passes the script through the program it finds.
What syntax of this line for any script? And what's the name of interpeter that parses it?
The syntax is the same as a command line, #!command arguments, but command has to be an absolute path, the PATH doesn't get searched.
Why this line is so necessary if each file have it's extension?
Extensions mean nothing in *nix. I could name a bash script script.pl, script.exe, or even script without an extension. If the script has the right shebang line, it gets passed through the right interpreter, otherwise the kernel tries to execute it as an executable and fails. The system doesn't know about extensions. They're a convention for users, nothing more.
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
If I understand this correctly, you're saying different systems / distributions keep the interpreters in different places (e.g. /usr/bin/python and /usr/local/bin/python), and asking how the system knows which to use?
The answer is, it uses the one which is at the absolute path you gave it. This is actually a slight problem with executable scripts, and the reason why /usr/bin/env has come into vogue. As I said, env searches the PATH for the correct interpreter, so as long as your system has a /usr/bin/env, you're set, you don't need to look up or guarantee the location of the interpreter.