Develop Reference

Python print redirect as stdin command line argument

I have a binary foo which requires two command line arguments: username and password.
I have written script.py to generate the username and password. Currently, I am using print to print them to stdout and then manually copy and paste them in the shell when I call foo in shell, i.e.,
$python script.py
username password
(i copied and paste the output below)
$./foo username password
However, I need to generate special bytes which are not printable in stdout and therefore if I copy and paste from stdout, these special byte values are gone. How can I redirect my python output as the argument for foo?
BTW: I have tried using call in subprocess to directly call foo in python, this is not ideal because if I trigger a seg fault in the foo, it does not reflected in bash.

Run:
./foo $(python script.py)
To demonstrates that this works and provides foo with two arguments, let's use this script.py:
$ cat script.py
#!/usr/bin/python
print("name1 pass1")
And let's use this foo so that we can see what arguments were provided to it:
$ cat foo
#!/bin/sh
echo "1=$1 2=$2"
Here is the result of execution:
$ ./foo $(python script.py)
1=name1 2=pass1
As you can see, foo received the name as its first argument and the password as its second argument.
Security Note: The OP has stated that this is not relevant for his application but, for others who may read this with other applications in mind, be aware that passing a password on a command line is not secure: full command lines are readily available to all users on a system.

So subprocess worked but you didn't used because you than didn't got the output from 'foo' (the binary)?
You can use the communicate() function to get output from the binary back.

Access previous bash command in python

I would like to be able to log the command used to run the current python script within the script itself. For instance this is something I tried:
#test.py
import sys,subprocess
with open('~/.bash_history','r') as f:
for line in f.readlines():
continue
with open('logfile','r') as f:
f.write('the command you ran: %s'%line.strip('\n'))
However the .bash_history does not seem to be ordered in chronological order. What's the best recommended way to achieve the above for easy logging? Thanks.
Update: unfortunately sys.argv doesn't quite solve my problem because I need to use process subtitution as input variables sometimes.
e.g. python test.py <( cat file | head -3)

What you want to do is not universally possible. As devnull says, the history file in bash is not written for every command typed. In some cases it's not written at all (user sets HISTFILESIZE=0, or uses a different shell).
The command as typed is parsed and processed long before your python script is invoked. Your question is therefore not related to python at all. Wether what you want to do is possible or not is entirely up to the invoking shell. bash does not provide what you want.
If your can control the caller's shell, you could try using zsh instead. There, if you setopt INC_APPEND_HISTORY, zsh will append to its history file for each command typed, so you can do the parse history file hack.

One option is to use sys.argv. It will contain a list of arguments you passed to the script.
import sys
print 'Number of arguments:', len(sys.argv), 'arguments.'
print 'Argument List:', str(sys.argv)
Example output:
>python test.py
Number of arguments: 1 arguments.
Argument List: ['test.py']
>python test.py -l ten
Number of arguments: 3 arguments.
Argument List: ['test.py', '-l', 'ten']
As you can see, the sys.argv variable contains the name of the script and then each individual parameter passed. It does miss the python portion of the command, though.

Multiple arguments with stdin in Python

I have a burning question that concerns passing multiple stdin arguments when running a Python script from a Unix terminal.
Consider the following command:
$ cat file.txt | python3.1 pythonfile.py
Then the contents of file.txt (accessed through the "cat" command) will be passed to the python script as standard input. That works fine (although a more elegant way would be nice). But now I have to pass another argument, which is simply a word which will be used as a query (and later two words). But I cannot find out how to do that properly, as the cat pipe will yield errors. And you can't use the standard input() in Python because it will result in an EOF-error (you cannot combine stdin and input() in Python).

I am reasonably sure that the stdin marker with do the trick:
cat file.txt | python3.1 prearg - postarg
The more elegant way is probably to pass file.txt as an argument then open and read it.

The argparse module would give you a lot more flexibility to play with command line arguments.
import argparse
parser = argparse.ArgumentParser(prog='uppercase')
parser.add_argument('-f','--filename',
help='Any text file will do.') # filename arg
parser.add_argument('-u','--uppercase', action='store_true',
help='If set, all letters become uppercase.') # boolean arg
args = parser.parse_args()
if args.filename: # if a filename is supplied...
print 'reading file...'
f = open(args.filename).read()
if args.uppercase: # and if boolean argument is given...
print f.upper() # do your thing
else:
print f # or do nothing
else:
parser.print_help() # or print help
So when you run without arguments you get:
/home/myuser$ python test.py
usage: uppercase [-h] [-f FILENAME] [-u]
optional arguments:
-h, --help show this help message and exit
-f FILENAME, --filename FILENAME
Any text file will do.
-u, --uppercase If set, all letters become uppercase.

Let's say there is an absolute need for one to pass content as stdin, not filepath because your script resides in a docker container or something, but you also have other arguments that you are required to pass...so do something like this
import sys
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-dothis', '--DoThis', help='True or False', required=True)
# add as many such arguments as u want
args = vars(parser.parse_args())
if args['DoThis']=="True":
content = ""
for line in sys.stdin:
content = content + line
print "stdin - " + content
To run this script do
$ cat abc.txt | script.py -dothis True
$ echo "hello" | script.py -dothis True
The variable content would store in it whatever was printed out on the left side of the pipe, '|', and you would also be able to provide script arguments.

While Steve Barnes answer will work, it isn't really the most "pythonic" way of doing things. A more elegant way is to use sys arguments and open and read the file in the script itself. That way you don't have to pipe the output of the file and figure out a workaround, you can just pass the file name as another parameter.
Something like (in the python script):
import sys
with open(sys.argv[1].strip) as f:
file_contents = f.readlines()
# Do basic transformations on file contents here
transformed_file_contents = format(file_contents)
# Do the rest of your actions outside the with block,
# this will allow the file to close and is the idiomatic
# way to do this in python
So (in the command line):
python3.1 pythonfile.py file.txt postarg1 postarg2

what is a commandline argument?

I am a newcomer to python (also very sorry if this is a newb question but) i have no idea what a command line argument is. when sys.argv is called, what exactly are the arguments? Any help with understanding this would be a great service.

Try running this program:
import sys
print(sys.argv)
You should see results similar to this:
% test.py
['/home/unutbu/pybin/test.py']
% test.py foo
['/home/unutbu/pybin/test.py', 'foo']
% test.py foo bar
['/home/unutbu/pybin/test.py', 'foo', 'bar']
% python test.py foo
['test.py', 'foo']
So, you see sys.argv is a list. The first item is the path to (or filename of) the script being run, followed by command-line arguments.

Given the command myscript.py arg1 arg2 arg3, the arguments are arg1, arg2 and arg3. sys.argv will also include the script name (i.e. myscript.py) in the first position.
Command line arguments are not specific to python.

Command line arguments are parameters you type after the script name. Eg. if you type: python test.py arg1, the first argument is arg1.
For examples, take a look at jhu.edu.

The command line arguments are the strings you type after a command in the command line, for instance:
python --help
Here --help is the argument for the python command, that shows a help page with the valid command line arguments for the python command.
In a python program, you have access to the arguments in sys.argv, so let's say you started a python script like this:
python myscript.py -x -y
When myscript.py starts, sys.argv[1] will have as value the string '-x' and sys.argv[2] will have as value the string '-y'. What you do with those arguments is up to you, and there are modules to help you easily define command line arguments, for instance argparse.

The arguments are usually used as a way of telling the program what it should do when it is run.
If I had a program named writefile.py, and I wanted the user to tell it which file to write, then I would run it with python writefile.py targetfile.txt. My sample writefile.py:
import sys
file = open(sys.argv[1], 'w') # sys.argv[0] = 'writefile.py' (unutbu's answer)
file.write('ABCDE')
file.close
After running this, I'll have a file named targetfile.txt with the contents "ABCDE". If I ran it with python writefile.py abcde.txt, I'd have abcde.txt with the contents "ABCDE".

What does "sys.argv[1]" mean? (What is sys.argv, and where does it come from?)

I'm currently teaching myself Python and was just wondering (In reference to my example below) in simplified terms what the sys.argv[1] represents. Is it simply asking for an input?
#!/usr/bin/python3.1
# import modules used here -- sys is a very standard one
import sys
# Gather our code in a main() function
def main():
print ('Hello there', sys.argv[1])
# Command line args are in sys.argv[1], sys.argv[2] ..
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
You may have been directed here because you were asking about an IndexError in your code that uses sys.argv. The problem is not in your code; the problem is that you need to run the program in a way that makes sys.argv contain the right values. Please read the answers to understand how sys.argv works.
If you have read and understood the answers, and are still having problems on Windows, check if Python Script does not take sys.argv in Windows fixes the issue. If you are trying to run the program from inside an IDE, you may need IDE-specific help - please search, but first check if you can run the program successfully from the command line.

I would like to note that previous answers made many assumptions about the user's knowledge. This answer attempts to answer the question at a more tutorial level.
For every invocation of Python, sys.argv is automatically a list of strings representing the arguments (as separated by spaces) on the command-line. The name comes from the C programming convention in which argv and argc represent the command line arguments.
You'll want to learn more about lists and strings as you're familiarizing yourself with Python, but in the meantime, here are a few things to know.
You can simply create a script that prints the arguments as they're represented. It also prints the number of arguments, using the len function on the list.
from __future__ import print_function
import sys
print(sys.argv, len(sys.argv))
The script requires Python 2.6 or later. If you call this script print_args.py, you can invoke it with different arguments to see what happens.
> python print_args.py
['print_args.py'] 1
> python print_args.py foo and bar
['print_args.py', 'foo', 'and', 'bar'] 4
> python print_args.py "foo and bar"
['print_args.py', 'foo and bar'] 2
> python print_args.py "foo and bar" and baz
['print_args.py', 'foo and bar', 'and', 'baz'] 4
As you can see, the command-line arguments include the script name but not the interpreter name. In this sense, Python treats the script as the executable. If you need to know the name of the executable (python in this case), you can use sys.executable.
You can see from the examples that it is possible to receive arguments that do contain spaces if the user invoked the script with arguments encapsulated in quotes, so what you get is the list of arguments as supplied by the user.
Now in your Python code, you can use this list of strings as input to your program. Since lists are indexed by zero-based integers, you can get the individual items using the list[0] syntax. For example, to get the script name:
script_name = sys.argv[0] # this will always work.
Although interesting, you rarely need to know your script name. To get the first argument after the script for a filename, you could do the following:
filename = sys.argv[1]
This is a very common usage, but note that it will fail with an IndexError if no argument was supplied.
Also, Python lets you reference a slice of a list, so to get another list of just the user-supplied arguments (but without the script name), you can do
user_args = sys.argv[1:] # get everything after the script name
Additionally, Python allows you to assign a sequence of items (including lists) to variable names. So if you expect the user to always supply two arguments, you can assign those arguments (as strings) to two variables:
user_args = sys.argv[1:]
fun, games = user_args # len(user_args) had better be 2
So, to answer your specific question, sys.argv[1] represents the first command-line argument (as a string) supplied to the script in question. It will not prompt for input, but it will fail with an IndexError if no arguments are supplied on the command-line following the script name.

sys.argv[1] contains the first command line argument passed to your script.
For example, if your script is named hello.py and you issue:
$ python3.1 hello.py foo
or:
$ chmod +x hello.py # make script executable
$ ./hello.py foo
Your script will print:
Hello there foo

sys.argv is a list.
This list is created by your command line, it's a list of your command line arguments.
For example:
in your command line you input something like this,
python3.2 file.py something
sys.argv will become a list ['file.py', 'something']
In this case sys.argv[1] = 'something'

Just adding to Frederic's answer, for example if you call your script as follows:
./myscript.py foo bar
sys.argv[0] would be "./myscript.py"
sys.argv[1] would be "foo" and
sys.argv[2] would be "bar" ... and so forth.
In your example code, if you call the script as follows ./myscript.py foo , the script's output will be "Hello there foo".

Adding a few more points to Jason's Answer :
For taking all user provided arguments: user_args = sys.argv[1:]
Consider the sys.argv as a list of strings as (mentioned by Jason). So all the list manipulations will apply here. This is called "List Slicing". For more info visit here.
The syntax is like this: list[start:end:step]. If you omit start, it will default to 0, and if you omit end, it will default to length of list.
Suppose you only want to take all the arguments after 3rd argument, then:
user_args = sys.argv[3:]
Suppose you only want the first two arguments, then:
user_args = sys.argv[0:2] or user_args = sys.argv[:2]
Suppose you want arguments 2 to 4:
user_args = sys.argv[2:4]
Suppose you want the last argument (last argument is always -1, so what is happening here is we start the count from back. So start is last, no end, no step):
user_args = sys.argv[-1]
Suppose you want the second last argument:
user_args = sys.argv[-2]
Suppose you want the last two arguments:
user_args = sys.argv[-2:]
Suppose you want the last two arguments. Here, start is -2, that is second last item and then to the end (denoted by :):
user_args = sys.argv[-2:]
Suppose you want the everything except last two arguments. Here, start is 0 (by default), and end is second last item:
user_args = sys.argv[:-2]
Suppose you want the arguments in reverse order:
user_args = sys.argv[::-1]

sys.argv is a list containing the script path and command line arguments; i.e. sys.argv[0] is the path of the script you're running and all following members are arguments.

To pass arguments to your python script
while running a script via command line
> python create_thumbnail.py test1.jpg test2.jpg
here,
script name - create_thumbnail.py,
argument 1 - test1.jpg,
argument 2 - test2.jpg
With in the create_thumbnail.py script i use
sys.argv[1:]
which give me the list of arguments i passed in command line as
['test1.jpg', 'test2.jpg']

sys.argv is a attribute of the sys module. It says the arguments passed into the file in the command line. sys.argv[0] catches the directory where the file is located. sys.argv[1] returns the first argument passed in the command line. Think like we have a example.py file.
example.py
import sys # Importing the main sys module to catch the arguments
print(sys.argv[1]) # Printing the first argument
Now here in the command prompt when we do this:
python example.py
It will throw a index error at line 2. Cause there is no argument passed yet. You can see the length of the arguments passed by user using if len(sys.argv) >= 1: # Code.
If we run the example.py with passing a argument
python example.py args
It prints:
args
Because it was the first arguement! Let's say we have made it a executable file using PyInstaller. We would do this:
example argumentpassed
It prints:
argumentpassed
It's really helpful when you are making a command in the terminal. First check the length of the arguments. If no arguments passed, do the help text.

sys.argv will display the command line args passed when running a script or you can say sys.argv will store the command line arguments passed in python while running from terminal.
Just try this:
import sys
print sys.argv
argv stores all the arguments passed in a python list. The above will print all arguments passed will running the script.
Now try this running your filename.py like this:
python filename.py example example1
this will print 3 arguments in a list.
sys.argv[0] #is the first argument passed, which is basically the filename.
Similarly, argv[1] is the first argument passed, in this case 'example'.