The typical header should be
#!/usr/bin/env python
But I found below also works when executing the script like $python ./my_script.py
#!/usr/bin/python
#!python
What's difference between these 2 headers? What could be the problem for 2nd one? Please also discussing the case for python interpreter is in PATH or not. Thanks.
First, any time you run a script using the interpreter explicitly, as in
$ python ./my_script.py
$ ksh ~/bin/redouble.sh
$ lua5.1 /usr/local/bin/osbf3
the #! line is always ignored. The #! line is a Unix feature of executable scripts only, and you can see it documented in full on the man page for execve(2). There you will find that the word following #! must be the pathname of a valid executable. So
#!/usr/bin/env python
executes whatever python is on the users $PATH. This form is resilient to the Python interpreter being moved around, which makes it somewhat more portable, but it also means that the user can override the standard Python interpreter by putting something ahead of it in $PATH. Depending on your goals, this behavior may or may not be OK.
Next,
#!/usr/bin/python
deals with the common case that a Python interpreter is installed in /usr/bin. If it's installed somewhere else, you lose. But this is a good way to ensure you get exactly the version you want or else nothing at all ("fail-stop" behavior), as in
#!/usr/bin/python2.5
Finally,
#!python
works only if there is a python executable in the current directory when the script is run. Not recommended.
I'd suggest 3 things in the beginning of your script:
First, as already being said use environment:
#!/usr/bin/env python
Second, set your encoding:
# -*- coding: utf-8 -*-
Third, set some doc string:
"""This is a awesome
python script!"""
And for sure I would use " " (4 spaces) for ident.
Final header will look like:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""This is a awesome
python script!"""
Best wishes and happy coding.
The Python executable might be installed at a location other than /usr/bin, but env is nearly always present in that location so using /usr/bin/envis more portable.
From the manpage for env (GNU coreutils 6.10):
env - run a program in a modified environment
In theory you could use env to reset the environment (removing many of the existing environment variables) or add additional environment variables in the script header. Practically speaking, the two versions you mentioned are identical. (Though others have mentioned a good point: specifying python through env lets you abstractly specify python without knowing its path.)
Yes, there is - python may not be in /usr/bin, but for example in /usr/local/bin (BSD).
When using virtualenv, it may even be something like ~/projects/env/bin/python
The /usr/bin/env python becomes very useful when your scripts depend on environment settings for example using scripts which rely on python virtualenv. Each virtualenv has its own version of python binary which is required for adding packages installed in virtualenv to python path (without touching PYTHONPATH env).
As more and more people have started to used virtualenv for python development prefer to use /usr/bin/env python unless you don't want people to use their custom python binary.
Note: You should also understand that there are potential security issues (in multiuser environments) when you let people run your scripts in their custom environments. You can get some ideas from here.
Related
I have a script which uses the shebang #!/usr/bin/env python. It works great on machines where Python 3 is the only version available, but on the machines which have both Python 2 and Python 3, it runs the script with Python 2.
If I modify the shebang to be #!/usr/bin/env python3, it would work on the machines with Python 2 and Python 3, but on the machines which have only Python 3, it would fail with “No such file or directory” error.
One solution is to create an alias alias python=python3.
Are there other solutions to have a same shebang working uniformly on every machine?
Unfortunately, there is no universally working way of doing this that would work across any and all up front unknown Linux hosts and you are largely left at the mercy of distro maintainers and local host configuration.
alias won't help, because interpreter specified by #! handled by the kernel and /usr/bin/env it will exec in this case does not know about aliases of your shell.
When using env, you could make sure that the name following env is first found and means what you want it to mean by:
making sure all hosts are setup the same way in that respect (expected packaged are installed or at least symlinks are created)
having user specific construct for execution of your script, such as:
mkdir /tmp/bin
ln -s /usr/bin/python /tmp/bin/python3
PATH="/tmp/bin:${PATH}" ./myscript.py
But none of this is really great and ultimately what you've asked for.
Your interpreter (this is harder than it may sound though, interpreter resolution code is quite simple; where to put it, how to call it for kernel to find and use it) could also be a simple shell script that tries to figure it out that you pack with your python code, but any option you look at isn't really great I am afraid.
There is a PEP-394 for that which suggested / expected on U*X-like system:
you get python for python2
and python3 for python3
But it recognizes this has never been entirely consistently applied... and also not as useful in 2020:
However, these recommendations implicitly assumed that Python 2 would always be available. As Python 2 is nearing its end of life in 2020 (PEP 373, PEP 404), distributions are making Python 2 optional or removing it entirely. This means either removing the python command or switching it to invoke Python 3. Some distributors also decided that their users were better served by ignoring the PEP's original recommendations, and provided system administrators with the freedom to configure their systems based on the needs of their particular environment.
TL;DR unfortunately there is no way that universally works and compensates for decisions of various distro and even individual host maintainers. :(
I would most likely opt to stick with #!/usr/bin/env python3 (which has so far been the recommended naming) and add a README that explains the prerequisites and how to setup the host just to be sure.
For the sake of completeness I should add, the PEP does make a recommendation in this regard: setup and use virtual environment or use a (third party) environment manager. However the way I read the question: "Portable interpreter specification that does not make any assumptions about nor poses any (additional) requirements to target host configuration", this would then not fit the bill and would not mean a substantial improvement over saying: make sure you have python3 executable in the search path and create a symlink if not.
#!/usr/bin/env python3 is the most correct general solution. On systems where Python 3 is installed, this should work, regardless of whether Python 2 is installed or not.
That said, there’s still no guarantee that the right version of Python 3 is installed, and hard-coding, say, python3.7 isn’t viable since most users won’t have several versions installed.
Luckily there’s a better solution: If you distribute your application as a Python package with Setuptools and specify entry_points for your binary/binaries, the installation process will pick the correct interpreter and hard-code its path into your application scripts. This will work regardless of environment, as long as the installer (e.g. pip) can find Python at all.
As a very simple example, say your project example has the following folder structure:
example
├── example
│ ╰── __init__.py
├── pyproject.toml
╰── setup.cfg
And let’s assume your __init__.py has a function main that we want to use as an entry point for an executable script.
Then pyproject.toml would look like this:1
[build-system]
requires = ["setuptools", "wheel"]
And setup.cfg looks as follows:1,2
[metadata]
name = example
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
example-bin = example:main
Now you can build your project using e.g. pip3 bdist path-to-project, which will generate the Wheel installation bundle. When you install that (either after uploading it to PyPI or locally (via pip3 install wheel-filename.whl), pip3 will install the binary example-bin.
example-bin will launch your package’s entry point function, and will work regardless of what the Python 3 binary is called on your system, because pip3 install … creates this file with an absolute path in the shebang line. For example, on my system the first line of that file is
#!/usr/local/opt/python/bin/python3.7
1 Rather than use pyproject.toml and setup.cfg, the same works using the legacy setup.py file — but using the above is simpler.
2 Note that, at the time of writing, there’s a typo in the Setuptools quickstart: instead of [entry_point], it should read [entry_points].
Here's what I've settled on for now, though I'm still trying to write my code such that it can work with Python 2 in the cases where that's necessary. You can modify this to your needs if you have a few known paths, for instance (it doesn't all need to be on one line):
#!/bin/bash
_='''' # Prefer python3 if it exists
command -v python3 &> /dev/null && exec python3 $0 "$#" || exec python $0 "$#"
'''
# Python code here
print ("Hello")
That's based on an example from this page about hybrid scripts: https://gist.github.com/andyneff/fafba17b748bba6d7cd5
The script is initially loaded in bash, which sees the first line beginning with _='''', which is interpreted by bash as setting the variable _ to two null strings ('') in a row.
The second line uses command -v python3 to try and determine if python3 is in the path or defined as an alias, with stdout and stderr redirected to the null device. If it has a successful return code, then exec python3 $0 "$#" is run, where $0 is the name of the script and "$#" expands to the list of arguments. If it fails, the equivalent python command is run.
Since the command is exec'd, the python3/python command replaces the current bash process, and bash never sees the following lines.
When Python reads the file, it sees _='''' and interprets it as the beginning of a '''...''' multi-line triple-quoted string, even though it begins with four apostrophes rather than just three, so the script ends up with a harmless extra _ variable set.
I'm trying to run a Python script from the command line as a command on Windows -- so no usage of "Python" or ".py". If my script is named "testing.py", I am attempting to make this name into a command and call "testing" from the command line.
Going through the docs it seems I need to use this shebang #!/usr/bin/env python as long as I have Python in my PATH.
https://docs.python.org/3/using/windows.html#shebang-lines
I also have the script folder in my PATH, so something like
"testing.py" is currently working from the command line.
According to the docs and this tutorial,
https://dbader.org/blog/how-to-make-command-line-commands-with-python
I should be able to evoke my Python script just by "testing" if I have the proper paths within PATH and the above shebang. However, I can't seem to get the script running withouth adding ".py".
The accepted answer by #AKX is incorrect for Windows 10 standard Python 3, certainly in the latest Windows 10 (1903) if not earlier.
(Note: I cannot speak to how this may or may not work under WSL.)
I have several versions of Python installed (2.7, 3.6, 3.7, and most recently Python 3.8b1). I've been using the #!/usr/bin/env shebang for years in my scripts for cross-platform compatibility (usually to distinguish Py2 vs Py3 scripts).
I've created a little script in a folder (C:\so_test\awtest.py):
#!/usr/bin/env python3.6
import sys
print(sys.version)
If I run this with awtest.py or just awtest I get 3.6.x reported (showing it's running with Python 3.6). If I change the shebang to refer to 3.7, I get 3.7.x reported. If I change the shebang to just #!/usr/bin/env python3 I get the latest version of Python installed (3.8).
Now, if I add that folder to my path (path=%PATH%;C:\so_test in the command window you're testing in, or in the main env vars (you will need to restart the command window if you do the latter though)), I can change to a different directory and run awtest or awtest.py and they still work and refer to the folder in the path. If I remove the script folder from the path these files are no longer found.
While I wouldn't necessarily expect this to work on Windows prior to 10 or Python 2.7, this functionality appears to be the way of things going forward.
No, Windows does not support shebang lines.
The documentation you've linked relates to the py launcher installed by Python, which can interpret various shebang lines to choose a Python version to run a script with.
setuptools is able to generate wrapper .exes for your Python scripts, but it gets a little involved and already assumes you have a package with a setup.py and so on.
Locally, if you really, really need this, you probably could add .py to the PATHEXT environment variable, so the Windows command line looks up .pys like it looks up .exes (and various others; the current modern default is .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC). However, this will naturally not scale for distributing apps, as all of your users would need to set that too.
My recommendation is to stick with just that boring old python testing.py, really.
You can use shebang in windows by setting the path of your interpreter as the first line in the file(you will see a marker on VSCode that says 'set as interpreter ' on that line).
Using windows 10,Python version 3.9 see example:
#!C:/Users/waithira/AppData/Local/Programs/Python/Python39/python.exe
print('hello world')
if you're not going to do this often. You can still add a batch file to your path testing.bat containing the command that you need to execute your code.
#echo off
python testing.py
It's a borring workaround but it works without needing to mention the extention since windows interpret batch files the same way it interpret executables.
I just came across shebangs (#!) for the first time while learning Python and I'm still trying to understand how exactly they work. After doing some research, I landed on this page where it listed four different types of shebangs. However, I'm not really sure what the difference is in Windows.
#! /usr/bin/env python
#! /usr/bin/python
#! /usr/local/bin/python
#! python
Now, I'm betting that the first one has something to do with virtual environments, but I'm not quite sure what. And according to this StackOverflow thread, the paths listed above actually are for POSIX systems, not for Windows... which confounds me even more because they are somehow being translated into Windows directories through some magic. Here's a quote:
A bin directory is created on POSIX systems only . . . Some paths
within the virtualenv are slightly different on Windows: scripts and
executables on Windows go in ENV\Scripts\ instead of ENV/bin/ and
libraries go in ENV\Lib\ rather than ENV/lib/.
Can anyone give a beginner a little more information on how exactly shebangs like these work on Windows?
The documentation is not totally explicit, but on my reading of it, there is no difference between those shebang lines. The shebang handling on Windows is purely "virtual" --- that is, the paths in the shebang lines are not actually mapped onto any paths on the Windows file system. Rather, the use of any of these "virtual paths" simply means "use the default system Python when running this file via the py Python launcher". The purpose of allowing the shebang line on Windows is to let the Python script specify arguments to Python, or Python versions (e.g., python3). You can find more info about how the default system Python is determined, how to override it, etc., in the documentation linked above.
Incidentally, note that on Windows these shebangs are only used when you run the Python script using the py launcher.
I have a problem which is caused by our encapsulated design. Up till now lots of our scripts were written in bash and as a result the #!/bin/bash was always simple.
However now that we are rewriting our scripts in python that is a bit more difficult. We deliver a specific version of python (to avoid version differences in client installed environments from breaking our implementation). Because the specific version of python lives in a installed directory structure I need to route to it.
However I don't think the #! statement can accept environment variables from the shell that executes the file(tried and got a bad interpreter).
eg:
in foo.py I have #!$dirloc/wherepythonlives/python
In the bash shell I executed the file and got bad interpreter.
Is there a way of sneaking an environment variable into that #! line?
Or will I have to depend on an explicit path? We want to support multiple versions of our software (which may mean multiple python versions) on one environment so I was hoping to somehow keep Python's !# statement inside the directory level we install into.
A common way to do this is to use the env program:
#!/usr/bin/env python
This will cause env to look along the PATH environment for a binary called python.
I'm not aware of being able to use environment variable in the shebang. You can use relative paths though
#! ../../usr/bin/python
edit:
You could always use env to specify to use a specific version. Then if that version can be found in $PATH it will be used otherwise the script will fail
#! /usr/bin/env python2.7
Or you could make the entry point a generic script instead.
eg
#! /usr/bin/env bash
if [[ $MYPYTHON ]]
then
$MYPYTHON main.py
else
echo error message
fi
The optimal solution to this dilemma is using distutils (setup.py, which creates correct stubs for you automatically, for a number of given "console entry points") and virtualenv (handling the "isolated multiple installations" part).
I suppose it all depends on how and in what environment your scripts will be invoked. You could call you scripts using #!/usr/bin/env python, which would allow you to control which python is used by manipulating the environment's PATH.
You could always specify a wrapper script as the interpreter, which runs a python executable relative to the script's location:
foo.py:
#!/bin/pyselector
import sys
sys.exit(0)
pyselector:
#!/bin/sh
SCRIPT_PATH="$(readlink -f $1)"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
"${SCRIPT_DIR}/my/local/python" "$#"
I'm working on a new server for a new workplace, and I'm trying to reuse a CGI script I wrote in Python earlier this year. My CGI script starts off with
#!/local/usr/bin/python
But when I run this on the new server, it complains that there's no such folder. Obviously Python's kept in a different place on this box, but I've got no idea where.
I haven't done much unix before, just enough to get around, so if there's some neat trick I should know here I'd appreciate it :)
Thanks!
Try:
which python
in a terminal.
For this very reason it is recommend that you change your shebang line to be more path agnostic:
#!/usr/bin/env python
See this mailing list message for more information:
Consider the possiblities that in a different machine, python may be installed at /usr/bin/python or /bin/python in those cases, #!/usr/local/bin/python will fail.
For those cases, we get to call the env executable with argument which will determine the arguments path by searching in the $PATH and use it correctly.
(env is almost always located in /usr/bin/ so one need not worry that env is not present at /usr/bin.)
# which python
/usr/local/bin/python
Update:
I misread. Replace your header with
#!/usr/bin/env python
This will pull in the python location from the user that runs the script's environmental settings
Try: which python or whereis python
It is a good idea to use backticks for header Python script:
`which python`
The proper way to solve this problem is with
#!/usr/bin/env python
which allows for the use of a binary in the PATH in a shebang.