In many python scripts do I read the shebang directive #!/usr/bin/env python
I understand it tells which interpreter to use, like in a bash script: #!/bin/bash , but I fail to understand how the python interpreter is specified. If I simply run
$ /usr/bin/env
I get a list of variable path such as SHELL=/bin/bash or JAVA_HOME==/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home.
The thing is there is no information about python when running this command. So I would like to better understand what does /usr/bin/env do exactly, and in which way #!/usr/bin/env python tells where my current python interpreter is.
Try running /usr/bin/env python and see what happens.
When given an argument, env runs the executable it finds.
BTW I also use it with bash scripts: /usr/bin/env bash because distros don't agree on its location (/bin/bash vs /usr/bin/bash)
See also: https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my
EDIT- extra explanations:
When given an argument, e.g. python, env behaves exactly as any shell would do when trying to find an executable: look at the PATH environment variable, split it at :, and for each directory, try to find an executable named python. The first matching executable is launched.
Typical content of the PATH variable: /bin:/usr/bin:/usr/local/bin
Related
I'm developing a set of script in python3, as shebang I use this:
#!/usr/bin/env python3
Everything goes ok, but in some virtual machines where are executed the name of interpreter is python3.5. I will like to be able to execute my scripts in both enviroment but I can't change the filesystem of virtual machine (so I discard solutions like make a link from python3.5 to python3 )
I look at man of env but I don't find any way to specify a searching pattern or something like that.
I try to set an alias at begining of my sessions pointing to right python interpreter but env don't use it.
My unique solution is call my scripts saying which interpreter must use but is very anoying:
python3.5 myscript.py
Any idea is welcome!, thanks!
No need to bring in separate shell and python scripts, a single file can be both!
Replace your shebang line with this sequence:
#!/bin/sh
# Shell commands follow
# Next line is bilingual: it starts a comment in Python, and is a no-op in shell
""":"
# Find a suitable python interpreter (adapt for your specific needs)
for cmd in python3.5 python3 /opt/myspecialpython/bin/python3.5.99 ; do
command -v > /dev/null $cmd && exec $cmd $0 "$#"
done
echo "OMG Python not found, exiting!!!!!11!!eleven" >2
exit 2
":"""
# Previous line is bilingual: it ends a comment in Python, and is a no-op in shell
# Shell commands end here
# Python script follows (example commands shown)
import sys
print ("running Python!")
print (sys.argv)
If you can install scripts, you can also install a wrapper called python3.5 which simply dispatches python3.
#!/bin/sh
exec env python3 "$#"
You'll obviously need to chmod a+x this script just like the others you install.
You'll have to add the script's directory to your PATH after the system python3.5 directory to avoid having this go into an endless loop, and only use this script as a fallback when the system doesn't already provide python3.5.
As you noted, env doesn't know or care about your personal shell aliases or functions, and doesn't provide for any dynamic calculation of the binary to run by itself; but you have the shell at your disposal (and Python of course, once you find it!) -- it simply uses the PATH so if you can install your other scripts in a directory which is in your PATH (which must be the case for the #!/usr/bin/env shebang to make sense in the first place) you can store this script there, too.
As noted in comments, it's weird and user-hostile to only install python3.5 and not at least optionally make python3 a symlink to it, so perhaps you could eventually persuade whoever maintains the image you are installing into to provide this.
You could create a shell script that uses python 3.5 if it is installed, otherwise uses python 3 and executes your script with the correct version.
No need for python shebang.
In your shell script you may test if which python3.5 returns something; if it does, then python3.5 is installed, otherwise you'd have to use python3
When I try to run a python script with python3 it does not work but it works when I just use python. Why is this?
I have a simple hello.py file:
__author__ = 'A'
print("hellow")
When I use python ~/path/hello.py with geektool it works, but not with python3 ~/path/hello.py, the same works from terminal.
Also, where can I see geektool's log file?
From the comments, it looks like you have Python 3 installed at /usr/local/bin/python3. It could be that that's not part of the default PATH, but you've configured your login shell to add it to the PATH. Since your other program either executes the program directly or does it through a non-login shell, it won't read that configuration, and the PATH will remain at its default, excluding that directory. If that's the case, you might have to instead change your command to have an absolute path to Python:
/usr/local/bin/python3 /path/to/hello.py
This should work from the Terminal and any other environments.
I have a problem which is caused by our encapsulated design. Up till now lots of our scripts were written in bash and as a result the #!/bin/bash was always simple.
However now that we are rewriting our scripts in python that is a bit more difficult. We deliver a specific version of python (to avoid version differences in client installed environments from breaking our implementation). Because the specific version of python lives in a installed directory structure I need to route to it.
However I don't think the #! statement can accept environment variables from the shell that executes the file(tried and got a bad interpreter).
eg:
in foo.py I have #!$dirloc/wherepythonlives/python
In the bash shell I executed the file and got bad interpreter.
Is there a way of sneaking an environment variable into that #! line?
Or will I have to depend on an explicit path? We want to support multiple versions of our software (which may mean multiple python versions) on one environment so I was hoping to somehow keep Python's !# statement inside the directory level we install into.
A common way to do this is to use the env program:
#!/usr/bin/env python
This will cause env to look along the PATH environment for a binary called python.
I'm not aware of being able to use environment variable in the shebang. You can use relative paths though
#! ../../usr/bin/python
edit:
You could always use env to specify to use a specific version. Then if that version can be found in $PATH it will be used otherwise the script will fail
#! /usr/bin/env python2.7
Or you could make the entry point a generic script instead.
eg
#! /usr/bin/env bash
if [[ $MYPYTHON ]]
then
$MYPYTHON main.py
else
echo error message
fi
The optimal solution to this dilemma is using distutils (setup.py, which creates correct stubs for you automatically, for a number of given "console entry points") and virtualenv (handling the "isolated multiple installations" part).
I suppose it all depends on how and in what environment your scripts will be invoked. You could call you scripts using #!/usr/bin/env python, which would allow you to control which python is used by manipulating the environment's PATH.
You could always specify a wrapper script as the interpreter, which runs a python executable relative to the script's location:
foo.py:
#!/bin/pyselector
import sys
sys.exit(0)
pyselector:
#!/bin/sh
SCRIPT_PATH="$(readlink -f $1)"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
"${SCRIPT_DIR}/my/local/python" "$#"
I'm working on a new server for a new workplace, and I'm trying to reuse a CGI script I wrote in Python earlier this year. My CGI script starts off with
#!/local/usr/bin/python
But when I run this on the new server, it complains that there's no such folder. Obviously Python's kept in a different place on this box, but I've got no idea where.
I haven't done much unix before, just enough to get around, so if there's some neat trick I should know here I'd appreciate it :)
Thanks!
Try:
which python
in a terminal.
For this very reason it is recommend that you change your shebang line to be more path agnostic:
#!/usr/bin/env python
See this mailing list message for more information:
Consider the possiblities that in a different machine, python may be installed at /usr/bin/python or /bin/python in those cases, #!/usr/local/bin/python will fail.
For those cases, we get to call the env executable with argument which will determine the arguments path by searching in the $PATH and use it correctly.
(env is almost always located in /usr/bin/ so one need not worry that env is not present at /usr/bin.)
# which python
/usr/local/bin/python
Update:
I misread. Replace your header with
#!/usr/bin/env python
This will pull in the python location from the user that runs the script's environmental settings
Try: which python or whereis python
It is a good idea to use backticks for header Python script:
`which python`
The proper way to solve this problem is with
#!/usr/bin/env python
which allows for the use of a binary in the PATH in a shebang.
The typical header should be
#!/usr/bin/env python
But I found below also works when executing the script like $python ./my_script.py
#!/usr/bin/python
#!python
What's difference between these 2 headers? What could be the problem for 2nd one? Please also discussing the case for python interpreter is in PATH or not. Thanks.
First, any time you run a script using the interpreter explicitly, as in
$ python ./my_script.py
$ ksh ~/bin/redouble.sh
$ lua5.1 /usr/local/bin/osbf3
the #! line is always ignored. The #! line is a Unix feature of executable scripts only, and you can see it documented in full on the man page for execve(2). There you will find that the word following #! must be the pathname of a valid executable. So
#!/usr/bin/env python
executes whatever python is on the users $PATH. This form is resilient to the Python interpreter being moved around, which makes it somewhat more portable, but it also means that the user can override the standard Python interpreter by putting something ahead of it in $PATH. Depending on your goals, this behavior may or may not be OK.
Next,
#!/usr/bin/python
deals with the common case that a Python interpreter is installed in /usr/bin. If it's installed somewhere else, you lose. But this is a good way to ensure you get exactly the version you want or else nothing at all ("fail-stop" behavior), as in
#!/usr/bin/python2.5
Finally,
#!python
works only if there is a python executable in the current directory when the script is run. Not recommended.
I'd suggest 3 things in the beginning of your script:
First, as already being said use environment:
#!/usr/bin/env python
Second, set your encoding:
# -*- coding: utf-8 -*-
Third, set some doc string:
"""This is a awesome
python script!"""
And for sure I would use " " (4 spaces) for ident.
Final header will look like:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""This is a awesome
python script!"""
Best wishes and happy coding.
The Python executable might be installed at a location other than /usr/bin, but env is nearly always present in that location so using /usr/bin/envis more portable.
From the manpage for env (GNU coreutils 6.10):
env - run a program in a modified environment
In theory you could use env to reset the environment (removing many of the existing environment variables) or add additional environment variables in the script header. Practically speaking, the two versions you mentioned are identical. (Though others have mentioned a good point: specifying python through env lets you abstractly specify python without knowing its path.)
Yes, there is - python may not be in /usr/bin, but for example in /usr/local/bin (BSD).
When using virtualenv, it may even be something like ~/projects/env/bin/python
The /usr/bin/env python becomes very useful when your scripts depend on environment settings for example using scripts which rely on python virtualenv. Each virtualenv has its own version of python binary which is required for adding packages installed in virtualenv to python path (without touching PYTHONPATH env).
As more and more people have started to used virtualenv for python development prefer to use /usr/bin/env python unless you don't want people to use their custom python binary.
Note: You should also understand that there are potential security issues (in multiuser environments) when you let people run your scripts in their custom environments. You can get some ideas from here.