I apologize if this is a duplicate, I couldn't find any other examples of this question.
I'm trying to write a program for natural language recognition, and I was reading a blog post where someone had attempted to do something similar and it recommended using these two lines to capture the output.
#capture output of script
./get-language.py | tee preptxt
I can't figure out what this code is attempting to do. I assume it's running the get-language file, but that syntax doesn't look correct to me. Could someone point me in the right direction?
On Unix systems you can run executable files with /path/to/my/executable.
This is nothing python specific.
As . is the current working directory, you are executing the get-language.py script located in the current directory.
However, there are two things you need to do, to get this working for your scripts:
1. add a shebang
This is the first line of your script, it tells the shell which program to use.
To use the python interpreter that is first in the PATH use:
#!/usr/bin/env python
2. Add the permissions to make your script executable:
You need to permit execution of your script. This can be done with chmod:
chmod +x myscript.py
The last part is a so called piping operation.
If you call two programs like this:
$ program_a | program_b
The output (stdout) of program_a is fed into program_b.
That's a shell thing, not Python. . is the current directory; you're running get-language.py from there.
This is not a line of python, but rather a shell script.
It is running a script called get-language.py (which I assume was described elsewhere on that blog and does the actual language processing), and then it is "piping" that output through the unix command line program tee, which stores the output in a file.
Piping takes the output of one program, and uses it as the input of another program.
Related
I'm trying to change the terminal directory through a python script. I've seen this post and others like it so I know about os.chdir, but it's not working the way I'd like. os.chdir appears to change the directory, but only for the python script. For instance I have this code.
#! /usr/bin/env python
import os
os.chdir("/home/chekid/work2/")
print os.getcwd()
Unfortunately after running I'm still in the directory of the python script (e.g. /home/chekid) rather than the directory I want to be in. See below.
gandalf(pts/42):~> pwd
/home/chekid
gandalf(pts/42):~> ./changedirectory.py
/home/chekid/work2
gandalf(pts/42):~> pwd
/home/chekid
Any thoughts on what I should do?
Edit: Looks like what I'm trying to do doesn't exist in 'normal' python. I did find a work around, although it doesn't look so elegant to me.
cd `./changedirectory.py`
You can't. The shell's current directory belongs to the shell, not to you.
(OK, you could ptrace(2) the shell and make it call chdir(2), but that's probably not a great design, won't work on Windows, and I would not begin to know how to do it in pure Python except that you'd probably have to mess around with ctypes or something similar.)
You could launch a subshell with your current working directory. That might be close enough to what you need:
os.chdir('/path/to/somewhere')
shell = os.environ.get('SHELL', '/bin/sh')
os.execl(shell, shell)
# execl() does not return; it replaces the Python process with a new shell process
The original shell will still be there, so make sure you don't leave it hanging around. If you initially call Python with the exec builtin (e.g. exec python /path/to/script.py), then the original shell will be replaced with the Python process and you won't have to worry about this. But if Python exits without launching the shell, you'll be left with no shell open at all.
You can if you cheat: Make a bash script that calls your python script. The python script returns the path you want to change directory to. Then the bash script does the acctual chdir. Of course you would have to run the bash script in your bash shell using "source".
The current working directory is an attribute of a process. It cannot be changed by another program, such as changing the current working directory in your shell by running a separate Python program. This is why cd is always a shell built-in command.
You can make your python print the directory you want to move to, and then call your script with cd "$(./python-script.py)". In condition your script actually does not print anything else.
Some programs can be executed from anywhere with a single one-word command. An example of this is youtube-dl, which is a python program that can be executed with the simple command youtube-dl [input]. As far as I have understood, this is simply because there exists a file called /usr/bin/youtube-dl, and /usr/bin is in PATH. However, I do not understand what I have to do to make something like this myself. Right now, I have a python project called testproject that includes a python program like this:
~/testproject/files/myownprogram.py
What do I have to do to make this a binary executable such as youtube-dl?
I know I can make an alias mop="python ~/testproject/files/myownprogram.py", and this is also what I have done, and it works fine. That is, I can write $ mop, and successfully run my program.
But for curiosity's, and learning's, sake, I want to know how to make a file such as the /usr/bin/youtube-dl file, removing the need for aliases.
I find this hard to find information about in search engines...
Any help is greatly appreciated! :-)
Edit:
My question differs from the one marked as duplicate, in that I'm not looking to execute it as a .sh-script. I simply want to execute it as a suffix-less one-word command, similar to all the other executables that are in /usr/bin. :-)
Ex.: $ myown should run my program, without the need for aliases or writing ".sh" or ".py" at the prompt. That is, I want to have a file /usr/bin/myown that somehow runs my testproject at the simple command myownfrom anywhere.
The applications are being executed "from anywhere" because the system has a specific hierarchy of places it looks for these files (the current directory, then the system path). So, it knows to look in /usr/bin because that's in your system path.
As to ensuring it to use python when it's executed, you should add the following to the top of the file (check out some python application source code and you'll see this):
#!/usr/bin/env python
This tells the system to execute the script using the systems "python" command.
I find that when I start the python shell I have a bunch of commands I always type to get into the state I want. It is tiresome to keep re-typing these commands, so I have bundled them into a script. Now I just type:
execfile('script.py')
as soon as I enter the shell, and it goes through all the steps to get me to the state I need to be in.
Now I'd like to take it one step further. How can I get the python shell to automatically run script.py every time I start the shell, so I don't have to keep re-typing even that one line?
Here's a way without having to mess with environment variables:
For example, if I had a script with the following in it called script.py:
#!/usr/bin/env python
print("example")
I could tell python to run this before bringing me to the interpreter with the -i flag.
$ python -i script.py
example
>>>
I think you're looking for the PYTHONSTARTUP environment variable
I'd suggest you to use IPython, if possible. It gives tones of great features, and autoexec is only one of them. But of course, correct answer is mentioned by #mgilston
Create a file called usercustomize.py, and place it in your USER_SITE directory (which you can find as import site; site._script(). This file will be executed every time you start an interpreter.
There's a similar file called sitecustomize.py which is executed anytime anyone starts Python.
This is a Windows solution, but I'm sure a Unix equivalent could be done. I created a file \MyPythonCode\autoexec.py (the last line of which is a print("Autoexec Complete")) and a BAT file MyPython.Bat which has:
cd \myPythonCode
python -i autoexec.py
I just use the command: mypython
to start the Python environment every time, now.
I know that in the begining of .sh bash scripts is
#!/bin/bash
which points to the command interpeter executable.
But during watching Google Python Class http://www.youtube.com/watch?v=tKTZoB2Vjuk I noticed that for python they use
#!/usr/bin/python -tt
. Surfing the Internet I also have found such styles of this notation:
#!/usr/local/bin/python
and even
#!/usr/bin/env python
.
So, I'm new with Python and I'm ordinary Linux user and I have a few questions about this "magic" line:
First of all, what is the right form of this line? and why?
What does -tt key means in #!/usr/bin/python -tt ?
What program is parsing this line in Linux?
What syntax of this line for any script?
Why this line is so necessary if each file have it's extension?
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
It's really interesting to me.
What's this line? Why this line? How to write this line? Why in such a way?...
Question #1) The line is called a shebang, and there's no right form that works universally. e.g.
#!python
#!/usr/bin/python
#!/usr/local/bin/python
#!/usr/bin/python -t
are all valid/acceptable forms, but may not work on all systems:
#!python will work only if the python executable is somewhere in your shell's PATH
#!/usr/bin/python only works if the python binary is actually in /usr/bin
#!/usr/local/bin/python also only works if python is in /usr/local/bin
Question #2)
#!/usr/bin/python -tt is passing the -tt option to python, as if you'd done:
$ python -t somescript.py
at the shell prompt. You can pass arbitary command line arguments to the interpreter on the shebang line.
Question #3)
The line is interpreted by the OS kernel and the shell you're currently using. The stuff after the #! simply tells the OS which program should be fired up to "execute" the rest of the script.
Question #4)
The script syntax depends on the language you're using. E.g. a PHP shell script must take the form of
#!/usr/bin/php
<?php
... php code here ...
A #!/usr/bin/perl perl script must use Perl syntax, etc... If you put PHP code with a Perl shebang, you'll just have Perl barf up the script with syntax errors, as PHP code is NOT perl code
Question #5)
Shebangs are for Unix systems, where file extensions were never really used to identify file types to the OS. A .c file was understood to be a C language source code file, but that's merely a convention. You could put a Bash shell script into a .c file, make it executable, and with the #!/bin/bash shebang, it would execute as a Bash script.
Determining executable types by file extension is more of a Windows thing.
Question #6)
That goes back the stuff in question #1 - if the shebang claims the interpreter is at some OTHER path than where it is, this particular script can't be executed until the shebang is fixed, or the interpreter is moved. Shebangs are very handy, but not infallible.
Thankfully, most interpreters are installed in fairly standard locations these days, so it'd be somewhat unusual to find (say) Perl installed at /some/wonky/weird/path instead of /usr/bin
From the manpage:
-t Issue a warning when a source file mixes tabs and spaces
for indentation in a way that makes it depend on the worth of a tab
expressed in spaces. Issue an
error when the option is given twice.
The right form of the line is the one you want to use.
It's the interpreter that reads this line known as shebang. If you write a python script with first line as "#!/usr/bin/python" & invoke it using bash, it's the /bin/sh interpreter that reads first line and starts the proper interpreter.
It's a shebang. The syntax of feature consists of the character sequence #!, i.e. the number sign and an exclamation point character
File extensions are not relevant in linux generally. You can have a python script that doesn't have a .py extension.
For ex.
shadyabhi#archlinux ~ $ cat a
print "Hello World"
shadyabhi#archlinux ~ $ python2 a
Hello World
shadyabhi#archlinux ~ $
Even the shebangs are only necessary if you want to start a script using $./script as in this case you didn't mention the interpreter you want to use.
#!/usr/bin/env python
issue errors about inconsistent tab usage
Kernel
#!/path_to_the_interpreter or /usr/bin/env
*nix does not check extensinon at all(except some DE could do that)
This is why you should use #!/usr/bin/env
More info at wiki
The different paths are to where the python interpreter has been installed. Different flavours of Linux install it in different places.
Linux doesn't care for extensions its a Windows thing.
The bash session uses the line to call the correct interpreter for the script your running.
The different places to where the files are stored, called and used are all based on defined places where files should be and located by software. Dev for devices, home for user stored area, bin for programs. But as time has gone by, different systems require different locations.
I would suggest getting a book on Linux/Unix and learning the basics of the file system. It does help a lot.
This is called a shebang. It tells the system that it should pass the file as an argument to the specified program instead of trying to execute it per se.
First of all, what is the right form of this line? and why?
The correct path is wherever your python interpreter is installed. The arguments (-tt) will depend on what you want. Some people insist on #!/usr/bin/env in case the interpreter happens to be elewhere.
What does -tt key means in #!/usr/bin/python -tt ?
I don't use python, so someone else will have to answer this.
When I launch any script in Linux (not exact Python script) what program parses and uses this line? I consider that it's not bash because even for bash scripts this line is needed.
I've heard (and am pretty sure) it's the kernel. Even if it were bash, it would need the line to tell bash it is supposed to be a script it should interpret instead of passing to another program. /usr/bin/env is a command that searches the PATH for the specified argument and passes the script through the program it finds.
What syntax of this line for any script? And what's the name of interpeter that parses it?
The syntax is the same as a command line, #!command arguments, but command has to be an absolute path, the PATH doesn't get searched.
Why this line is so necessary if each file have it's extension?
Extensions mean nothing in *nix. I could name a bash script script.pl, script.exe, or even script without an extension. If the script has the right shebang line, it gets passed through the right interpreter, otherwise the kernel tries to execute it as an executable and fails. The system doesn't know about extensions. They're a convention for users, nothing more.
And what about that in each computer interpreter for some kind of scripts will be stored in different place than in another? And script couldn't be run.
If I understand this correctly, you're saying different systems / distributions keep the interpreters in different places (e.g. /usr/bin/python and /usr/local/bin/python), and asking how the system knows which to use?
The answer is, it uses the one which is at the absolute path you gave it. This is actually a slight problem with executable scripts, and the reason why /usr/bin/env has come into vogue. As I said, env searches the PATH for the correct interpreter, so as long as your system has a /usr/bin/env, you're set, you don't need to look up or guarantee the location of the interpreter.
What does this line of code mean? Without it, my python3 http server can't understand and let the browser download an empty .py file (depend on the link to the .py file)
#! /usr/local/bin/python3
It's not a Python thing, it a hashbang (or shebang) line which indicates which interpreter should process the file.
The rules vary but, in its simplest form, a file with the name xyz (containing that as the first line), when run from the command line with xyz, will run it using that interpreter, similar to:
/usr/local/bin/python3 xyz
This is not a python specific notion, see http://en.wikipedia.org/wiki/Shebang_(Unix)
It's the shebang/hashbang line and a Linux/UNIX thing, not Python-related at all.
When executing the file, the kernel will see the #! magic and use whatever comes after it to execute the script. The actual program that gets launched by the kernel will be program-from-shebang script-file-path [script-args]
Note that it's usually not a good thing to include a .../local/... path but rather use e.g. #!/usr/bin/env python3 which will result in python3 being looked up in the current PATH which is much more portable.
That is not python-specific but is called Shebang and tells the operating system with which program to run this script.
UNIX Shebang? See http://en.wikipedia.org/wiki/Shebang_(Unix). The space between ! and the first / probably shouldn't be there.