With the following command in bash:
python myscript.py filename_pattern*
I got two different sys.argv in two Linux machines:
Machine A: sys.argv[1] = filename_pattern*
Machine B: sys.argv[1] = filename_pattern-2013-06-30
Note: filename_pattern-2013-06-30 is a file in my current directory.
One of my colleague tell me that's the evil of bash. But I check the bash in two machines are of the same version, and I checked ~/.bashrc, /etc/bashrc, /etc/profile.d/*.sh too.
Can anyone point out how come the two same version bash act differently ?
It is because in one of your machine, in the folder, there in no file that could match the pattern. So when this happens, the * remains. You can test with one computer, with and without a file match the pattern. There is another reason, the shell option nullglob is disabled. You can read the GNU bash reference for this.
Why you see the different behavior on the two machines is either down to the files present in the directory on both machines or based on the shell options, as pointed out in the comments.
Either way, the way to avoid the problem is to simply surround the argument in quotes so that Bash treats it literally.
python myscript.py "filename_pattern*"
You could handle it in your python program like this example that I found in an introductory book on python. Then it won't matter if bash expands it or not.
import glob
import sys
sys.argv = [item for arg in sys.argv for item in glob.glob(arg)]
Related
In the pip program, the She-bang is
#!/usr/local/bin/python
if __name__ == "__main__":
# Python program body
while in the Install Certificates.command that Python Launcher offers:
#!/bin/sh
/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 << "EOF"
# python program body
EOF
Are there any differences between those two approaches? And is there any reason to prefer one to another?
It seems to me they are all the same, except for the second one has one more bash subroutine. Is this right?
In the general case, you simply want to specify the interpreter you actually want.
Outside of this, you sometimes see workarounds like this as portability hacks. On POSIX systems, /usr/bin/env covers the majority of scenarios quite nicely; but if you need portability to older or otherwise peculiar systems, falling back to the lowest common denominator and then working your way back up to a place where you can reliably run e.g. Python on a variety of systems may require all kinds of unobvious constructs. (The previous - upvoted! - answer by Dan D. is a good example.)
There are also cases where you want sh to set something up (fetch some environment variables which are specified in a file which uses sh syntax, for example) and then hand over execution to Python;
#!/bin/sh
# source some variables
. /etc/defaults/myenv.sh
# Then run Python
exec env python -c '
# ... Your Python script here
' "$#"
There is a line length limit on the #! line. Perhaps they did that to get around that.
The options are the path to the program but only if it is short enough. Use of env python which uses the path. Or chain loading like this.
This specific code for the Install Certificates.command script was introduced in Python Issue #17128. As far as I can tell, the author hasn't explained why he wrote the code this way.
Note that .command files are Shell scripts on Mac OS X that can be executed by double-clicking on them in Finder.
I believe the likely explanation is that the author simply wanted to honour Mac OS X's expectation that .command files should be Shell scripts.
You could test this by placing the following content in a file ~/Desktop/test.command:
#!/usr/bin/env python
print "Hello world"
Then view the Desktop folder in Finder, and note that it is reported as a "shell" file:
(Although it is reported incorrectly as a Shell file, this Python script can still be executed by double-clicking on it. It doesn't break Finder or anything.)
To answer the specific question, one reason for preferring this pattern might be, as Dan D. said, to avoid a Shebang line limit.
In general, you would prefer to use #!/usr/bin/env python as your Shebang line. Creating a Bash Heredoc (i.e. the python3.6 << EOF pattern) would create all sorts of problems, such as your syntax highlighting won't work, you have to watch out for Bash variable interpolation inside the Heredoc, etc.
I'm trying to make a simple command that will let me run bash fully in Python, including output strings.
This function worked great on systems I use at my job:
import subprocess
def run(command):
output = subprocess.check_output(command, shell=True)
return output
However, now I'm using it at home and the PATH variable doesn't match the one in my Terminal.
So when I execute
run('ls')
I get this:
/bin/sh: ls: command not found
Which makes sense because, nonsensically, the PATH I get from running
print run('/usr/bin/env')
is
PATH=/Library/Frameworks/Python.framework/Versions/Current/bin/
Now, I could remedy all this by using:
run('/bin/ls')
But that defeats the entire purpose of using this command, which is to faithfully emulate the bash shell.
How do I make run() use the system's PATH or create an equivalent function that just works?
(No platitudes about the dangers of using 'shell=True', please. This is all personal use with innocuous commands like ls and ps axw.)
I'm leaving this question up because I think the run() command above is useful and I couldn't find anything similar on SO.
However, my solution was very system specific. The problem was, I am running this script within Sublime Text 2 and had manually replaced the path in Python.sublime-settings with
"path": "/Library/Frameworks/Python.framework/Versions/Current/bin/",
I did this because I'd had problems getting ST2 to find the right version of python. Well, this also overwrites the system-wide PATH variable within ST2, thus blocking my access to simple shell programs like ls.
Erasing the "path": ... line from Python.sublime-settings fixed my problem.
EDIT
As suggested by mklement0, changing the line to append the path works as well:
"path": "$PATH:/Library/Frameworks/Python.framework/Versions/Current/bin/",
I know that in the begining of .sh bash scripts is
#!/bin/bash
which points to the command interpeter executable.
But during watching Google Python Class http://www.youtube.com/watch?v=tKTZoB2Vjuk I noticed that for python they use
#!/usr/bin/python -tt
. Surfing the Internet I also have found such styles of this notation:
#!/usr/local/bin/python
and even
#!/usr/bin/env python
.
So, I'm new with Python and I'm ordinary Linux user and I have a few questions about this "magic" line:
First of all, what is the right form of this line? and why?
What does -tt key means in #!/usr/bin/python -tt ?
What program is parsing this line in Linux?
What syntax of this line for any script?
Why this line is so necessary if each file have it's extension?
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
It's really interesting to me.
What's this line? Why this line? How to write this line? Why in such a way?...
Question #1) The line is called a shebang, and there's no right form that works universally. e.g.
#!python
#!/usr/bin/python
#!/usr/local/bin/python
#!/usr/bin/python -t
are all valid/acceptable forms, but may not work on all systems:
#!python will work only if the python executable is somewhere in your shell's PATH
#!/usr/bin/python only works if the python binary is actually in /usr/bin
#!/usr/local/bin/python also only works if python is in /usr/local/bin
Question #2)
#!/usr/bin/python -tt is passing the -tt option to python, as if you'd done:
$ python -t somescript.py
at the shell prompt. You can pass arbitary command line arguments to the interpreter on the shebang line.
Question #3)
The line is interpreted by the OS kernel and the shell you're currently using. The stuff after the #! simply tells the OS which program should be fired up to "execute" the rest of the script.
Question #4)
The script syntax depends on the language you're using. E.g. a PHP shell script must take the form of
#!/usr/bin/php
<?php
... php code here ...
A #!/usr/bin/perl perl script must use Perl syntax, etc... If you put PHP code with a Perl shebang, you'll just have Perl barf up the script with syntax errors, as PHP code is NOT perl code
Question #5)
Shebangs are for Unix systems, where file extensions were never really used to identify file types to the OS. A .c file was understood to be a C language source code file, but that's merely a convention. You could put a Bash shell script into a .c file, make it executable, and with the #!/bin/bash shebang, it would execute as a Bash script.
Determining executable types by file extension is more of a Windows thing.
Question #6)
That goes back the stuff in question #1 - if the shebang claims the interpreter is at some OTHER path than where it is, this particular script can't be executed until the shebang is fixed, or the interpreter is moved. Shebangs are very handy, but not infallible.
Thankfully, most interpreters are installed in fairly standard locations these days, so it'd be somewhat unusual to find (say) Perl installed at /some/wonky/weird/path instead of /usr/bin
From the manpage:
-t Issue a warning when a source file mixes tabs and spaces
for indentation in a way that makes it depend on the worth of a tab
expressed in spaces. Issue an
error when the option is given twice.
The right form of the line is the one you want to use.
It's the interpreter that reads this line known as shebang. If you write a python script with first line as "#!/usr/bin/python" & invoke it using bash, it's the /bin/sh interpreter that reads first line and starts the proper interpreter.
It's a shebang. The syntax of feature consists of the character sequence #!, i.e. the number sign and an exclamation point character
File extensions are not relevant in linux generally. You can have a python script that doesn't have a .py extension.
For ex.
shadyabhi#archlinux ~ $ cat a
print "Hello World"
shadyabhi#archlinux ~ $ python2 a
Hello World
shadyabhi#archlinux ~ $
Even the shebangs are only necessary if you want to start a script using $./script as in this case you didn't mention the interpreter you want to use.
#!/usr/bin/env python
issue errors about inconsistent tab usage
Kernel
#!/path_to_the_interpreter or /usr/bin/env
*nix does not check extensinon at all(except some DE could do that)
This is why you should use #!/usr/bin/env
More info at wiki
The different paths are to where the python interpreter has been installed. Different flavours of Linux install it in different places.
Linux doesn't care for extensions its a Windows thing.
The bash session uses the line to call the correct interpreter for the script your running.
The different places to where the files are stored, called and used are all based on defined places where files should be and located by software. Dev for devices, home for user stored area, bin for programs. But as time has gone by, different systems require different locations.
I would suggest getting a book on Linux/Unix and learning the basics of the file system. It does help a lot.
This is called a shebang. It tells the system that it should pass the file as an argument to the specified program instead of trying to execute it per se.
First of all, what is the right form of this line? and why?
The correct path is wherever your python interpreter is installed. The arguments (-tt) will depend on what you want. Some people insist on #!/usr/bin/env in case the interpreter happens to be elewhere.
What does -tt key means in #!/usr/bin/python -tt ?
I don't use python, so someone else will have to answer this.
When I launch any script in Linux (not exact Python script) what program parses and uses this line? I consider that it's not bash because even for bash scripts this line is needed.
I've heard (and am pretty sure) it's the kernel. Even if it were bash, it would need the line to tell bash it is supposed to be a script it should interpret instead of passing to another program. /usr/bin/env is a command that searches the PATH for the specified argument and passes the script through the program it finds.
What syntax of this line for any script? And what's the name of interpeter that parses it?
The syntax is the same as a command line, #!command arguments, but command has to be an absolute path, the PATH doesn't get searched.
Why this line is so necessary if each file have it's extension?
Extensions mean nothing in *nix. I could name a bash script script.pl, script.exe, or even script without an extension. If the script has the right shebang line, it gets passed through the right interpreter, otherwise the kernel tries to execute it as an executable and fails. The system doesn't know about extensions. They're a convention for users, nothing more.
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
If I understand this correctly, you're saying different systems / distributions keep the interpreters in different places (e.g. /usr/bin/python and /usr/local/bin/python), and asking how the system knows which to use?
The answer is, it uses the one which is at the absolute path you gave it. This is actually a slight problem with executable scripts, and the reason why /usr/bin/env has come into vogue. As I said, env searches the PATH for the correct interpreter, so as long as your system has a /usr/bin/env, you're set, you don't need to look up or guarantee the location of the interpreter.
What does this line of code mean? Without it, my python3 http server can't understand and let the browser download an empty .py file (depend on the link to the .py file)
#! /usr/local/bin/python3
It's not a Python thing, it a hashbang (or shebang) line which indicates which interpreter should process the file.
The rules vary but, in its simplest form, a file with the name xyz (containing that as the first line), when run from the command line with xyz, will run it using that interpreter, similar to:
/usr/local/bin/python3 xyz
This is not a python specific notion, see http://en.wikipedia.org/wiki/Shebang_(Unix)
It's the shebang/hashbang line and a Linux/UNIX thing, not Python-related at all.
When executing the file, the kernel will see the #! magic and use whatever comes after it to execute the script. The actual program that gets launched by the kernel will be program-from-shebang script-file-path [script-args]
Note that it's usually not a good thing to include a .../local/... path but rather use e.g. #!/usr/bin/env python3 which will result in python3 being looked up in the current PATH which is much more portable.
That is not python-specific but is called Shebang and tells the operating system with which program to run this script.
UNIX Shebang? See http://en.wikipedia.org/wiki/Shebang_(Unix). The space between ! and the first / probably shouldn't be there.
Okay, I'm having one of those moments that makes me question my ability to use a computer. This is not the sort of question I imagined asking as my first SO post, but here goes.
Started on Zed's new "Learn Python the Hard Way" since I've been looking to get back into programming after a 10 year hiatus and python was always what I wanted. This book has really spoken to me. That being said, I'm having a serious issue with pydoc from the command. I've got all the directories in c:/python26 in my system path and I can execute pydoc from the command line just fine regardless of pwd - but it accepts no arguments. Doesn't matter what I type, I just get the standard pydoc output telling me the acceptable arguments.
Any ideas? For what it's worth, I installed ActivePython as per Zed's suggestion.
C:\Users\Chevee>pydoc file
pydoc - the Python documentation tool
pydoc.py <name> ...
Show text documentation on something. <name> may be the name of a
Python keyword, topic, function, module, or package, or a dotted
reference to a class or function within a module or module in a
package. If <name> contains a '\', it is used as the path to a
Python source file to document. If name is 'keywords', 'topics',
or 'modules', a listing of these things is displayed.
pydoc.py -k <keyword>
Search for a keyword in the synopsis lines of all available modules.
pydoc.py -p <port>
Start an HTTP server on the given port on the local machine.
pydoc.py -g
Pop up a graphical interface for finding and serving documentation.
pydoc.py -w <name> ...
Write out the HTML documentation for a module to a file in the current
directory. If <name> contains a '\', it is treated as a filename; if
it names a directory, documentation is written for all the contents.
C:\Users\Chevee>
EDIT: New information, pydoc works just fine in PowerShell. As a linux user, I have no idea why I'm trying to use cmd anyways--but I'd still love to figure out what's up with pydoc and cmd.
EDIT 2: More new information. In cmd...
c:\>python c:/python26/lib/pydoc.py file
...works just fine. Everything works just fine with just pydoc in PowerShell without me worrying about pwd, or extensions or paths.
In Windows Powershell use: python -m pydoc
Examples:
python -m pydoc open
python -m pydoc raw_input
python -m pydoc argv
When you type the name of a file at the windows command prompt, cmd can check the windows registry for the default file association, and use that program to open it. So if the Inkscape installer associated .py files with its own version of python, cmd might preferentially run that and ignore the PATH entirely. See this question.
Based on your second edit, you may have more than one copy of pydoc.py in your path, with the 'wrong' one first such that when it starts up it doesn't have the correct environment in which to execute.
python -m pydoc -k/p/g/w <name>
Syntax for pydoc on windows:
alt1:
C:\path\PythonXX\python.exe C:\path\PythonXX\Lib\pydoc.py -k/p/g/w X:\path\file_to_doc.py
alt2:
python -m pydoc -k/p/g/w X:\path\file_to_doc.py
Of which the latter is the one to prefer, duh. However it requires your windows installation to have registered python to the environment variable "Path".
Setup windows environment variables:
Look at this site for a guide on where to find them. The one you'll be looking for is "Path". If you select Path and click Edit you will see a long row of paths pointing to different folders. The Path's you see here is what allows you to basically reach a veriety of programs in the command line by just entering the name of the program, instead of the whole path to it. So what you want to do here is to locate your Python installation and copy its full path like this: X:\subfolders\PythonXX\ Then you add it to the very end of the long row of Path's like this:
X:\earlier\path\to\something;X:\subfolders\PythonXX\
Notice the ";" that seperates the different paths, make sure not to forget it. When done, click to confirm/Ok, then you would need to restart any cmd.exe that's already open.
The pydoc.py
The thing is that pydoc is a module of the standard python lib, and it's also powered by python. The windows environment, of what I understand, requires you to specify with which program you want to run a certain file with. So to run the pydoc.py-file we would use:
Open file in windows cmd.exe:
X:\subfolders\Python27\python.exe X:\subfolders\Python27\Lib\pydoc.py
pydoc.py's arguments:
pydoc.py comes with a veriety of command line-based features that allows you to enter certain arguments:
-k/p/g/w of which will trigger different behaviours of the pydoc.py-program.
Send arguments to a program through command line:
The syntax to enter these arguments is of what I know always after the x:\pathtofile\filename.suffix, seperated by a simple space. Which gives us the final:
alt1:
X:\subfolders\Python27\python.exe X:\subfolders\Python27\Lib\pydoc.py -w X:\path\file_to_doc.py
alt2 (with python registered to path):
python -m pydoc -w X:\path\file_to_doc.py
The "w"-option will give you a HTML-documentation for the file you want to run documentation on. Notice that pydoc.py will (according to my tests) create the documentation-file in the current working directory. Meaning that you will need to place yourself in a folder of choice before you actually run the command.
The function of -m
Of what I can find, the -m seem to handle registry entries, atleast in the msiexec.exe. I guess it might be used for programs in general this way. So my speculative idea of it is that if "-m" is applied, the pursuing arguments paths will be rewritten so that the .exe-file will be used as a path-reference. But as said, rather speculative.
-m Rewrites all required computer-specific registry entries. (in msiexec.exe) According to Microsoft