Develop Reference

Confused how to use argparse

I am trying to write a python script that can be run from the command line with
python script.py --input file.txt
or
python script.py -i file.txt
script.py will take in the file file.txt and open/read.
This is what I have so far:
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser(description="I'm not sure what I should write here.")
parser.add_argument('-i','--input', help='Input file name',required=True)
args = parser.parse_args()
Now...How do I actually access the input file?
What I want is to write to an output file, e.g. python script.py --outputfile file.csv? I do not understand how to interact with writing the file. Sorry if this is too easy.

It is pretty simple!
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser(description="Something like -- File reader: reads file line by line.")
parser.add_argument('-i','--input', help='Input file name',required=True)
args = parser.parse_args()
with open(args.input) as fp:
for line in fp:
print line
If you want to write some content to the input file. Open the file in write mode and write to it whatever you want.
with open(args.input, 'w') as fp:
fp.write("Hello World!")
If you want a separate file to write, add an argument to your argparser in write mode and then write to it.
parser = argparse.ArgumentParser(description="Something like -- File reader/writer: reads/writes files line by line.")
parser.add_argument('-i','--input', help='Input file name',required=True)
parser.add_argument('-o','--output', help='Output file name',type=argparse.FileType('w'),required=True)
args = parser.parse_args()
with open(args.input) as fp:
for line in fp:
print line
with open(args.output) as fp:
fp.write("Hello World!")

You can access the string the user enter after -i with:
args.input
For example:
argp_test.py -i my_file_name.txt
Now:
print(args.input)
prints:
my_file_name.txt
Next step is to read the file content:
with open(args.input) as fobj:
for line in fobj:
# do something with this line
BTW, you got a syntax error in this line:
parser = argparse.ArgumentParser(description='I'm not sure what I should write here.')
It should look like this:
parser = argparse.ArgumentParser(description="I'm not sure what I should write here.")
When you use a ' in your string, you need to use " at the begging and end of your string. So use:
"I'm not"
instead of:
'I'm not'

Cannot retrieve filenames using Python docopt library

I am reading the book Violent Python and one example is a zip file cracker, which tests a dictionary file of potential passwords (a text file) against a zip file.
I am trying to use the docopt library to parse the command line and give me the filenames for these two files. Here is my code.
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Basic zip bruteforcer
Usage:
your_script.py (-f <file>) (-z <zip> )
your_script.py -h | --help
Options:
-h --help Show this screen.
-f --file specify dictionary file. (Required argument)
-z --zip specify zip file.(Required argument)
-t --thread Thread count. (Optional)
-v --verbose Turn debug on. (Optional)
"""
from docopt import docopt
import zipfile
from threading import Thread
def extractzip(zfile, password):
try:
zfile.extractall(pwd = password)
print 'password found: ', password
except:
return
def main():
zfile = zipfile.ZipFile(zip)
with open(file, 'r') as pass_file:
for line in pass_file.readlines():
password = line.strip('\n')
t = Thread(target = extractzip, args = (zfile, password))
t.start()
if __name__ == "__main__":
arguments = docopt(__doc__, version='0.1a')
extractzip(arguments['<file>'], ['<zip>'])
Here is my code.

You can do something like
print arguments['<file>']
to get the file name, and similarly the zip file. I personally haven't used extractzip and am not sure how that works. But since arguments is simply a list, you can easily get the values by accessing the required index directly.

how to integrate optparse options with the variables names in python

I am very newbie to python and to optparse module in general. I have figured out how to add options in python script using optparse but having trouble linking the options with my variable names in python.
import sys
from optparse import OptionParser
def main ():
parser = OptionParser()
parser.add_option("-f", "--file", dest="in_filename",
help="Input fasta file", metavar="FILE")
parser.add_option("-o", "--out", dest="out_filename",
help="Output fasta file", metavar="FILE")
parser.add_option("-i", "--id", dest="id",
help="Id name to change", metavar="ID")
(options,args) = parser.parse_args()
with open(f, 'r') as fh_in:
with open(o, 'w') as fh_out:
id = i
result = {}
count = 1
for line in fh_in:
line = line.strip()
if line.startswith(">"):
line = line[1:]
result[line] = id + str(count)
count = count + 1
header = ">" + str(result[line])
fh_out.write(header)
fh_out.write("\n")
else:
fh_out.write(line)
fh_out.write("\n")
main()
When i run this i get this below traceback and error:
python header_change.py -f consensus_seq.txt -o consensus_seq_out.fa -i "my_test"
Traceback (most recent call last):
File "/Users/upendrakumardevisetty/Documents/git_repos/scripts/header_change.py", line 36, in <module>
main()
File "/Users/upendrakumardevisetty/Documents/git_repos/scripts/header_change.py", line 18, in main
with open(f, 'r') as fh_in:
NameError: global name 'f' is not defined
Can someone point to me what i am doing wrong.

You've got two problems here.
First, as the optparse tutorial shows, optparse doesn't create global variables, it creates attributes in the options namespace that it returns:
parse_args() returns two values:
options, an object containing values for all of your options—e.g. if --file takes a single string argument, then options.file will be the filename supplied by the user, or None if the user did not supply that option
args, the list of positional arguments leftover after parsing options
So, if the user typed -f, you're not going to have f, you're going to have options.f.
Second, f isn't the right name anyway. You explicitly specified a different destination, instead of the default:
parser.add_option("-f", "--file", dest="in_filename",
help="Input fasta file", metavar="FILE")
So it's going to do what you asked and store the file in in_filename.
And likewise for the other options. So, your code should start off like this:
with open(options.in_filename, 'r') as fh_in:
with open(options.out_filename, 'w') as fh_out:

Python: pass arguments to a script

I have a working code to print random lines from a csv column.
#!/usr/bin/python
import csv
from random import shuffle
filename = 'example.csv'
col = 2
sample = 100
with open(filename, 'r') as f:
reader = csv.reader(f)
data = [row[col] for row in reader]
shuffle(data)
print '\n'.join(data[:sample])
How can I parameterize this script by passing filename, col & sample (e.g. 100 values)?

Use argparse module:
The argparse module makes it easy to write user-friendly command-line
interfaces. The program defines what arguments it requires, and
argparse will figure out how to parse those out of sys.argv. The
argparse module also automatically generates help and usage messages
and issues errors when users give the program invalid arguments.
It's pretty powerful: you can specify help messages, make validations, provide defaults..whatever you can imagine about working with command-line arguments.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--position", type=int)
parser.add_argument("-s", "--sample", type=int)
args = parser.parse_args()
col = args.position
sample = args.sample
print col
print sample
Here's what on the command-line:
$ python test.py --help
usage: test.py [-h] [-p POSITION] [-s SAMPLE]
optional arguments:
-h, --help show this help message and exit
-p POSITION, --position POSITION
-s SAMPLE, --sample SAMPLE
$ python test.py -p 10 -s 100
10
100
$ python test.py --position 10 --sample 100
10
100
Speaking about the code you've provided:
unused import random statement
move from random import shuffle to the top of the script
no need to call f.close() (especially with ;) - with handles closing the file automagically
Here's how the code would look like after the fixes:
#!/usr/bin/python
import argparse
import csv
from random import shuffle
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--position", type=int)
parser.add_argument("-s", "--sample", type=int)
args = parser.parse_args()
with open('<filename>', 'r') as f:
reader = csv.reader(f)
data = [row[args.position] for row in reader]
shuffle(data)
print '\n'.join(data[:args.sample])

You can use the sys module like this to pass command line arguments to your Python script.
import sys
name_of_script = sys.argv[0]
position = sys.argv[1]
sample = sys.argv[2]
and then your command line would be...
./myscript.py 10 100

I could not install argparse due to lack of permission - meanwhile.
Here's the solution:
#!/usr/bin/python
import csv # This will help us reading csv formated files.
import sys
import optparse #import optparse
from random import shuffle
parser=optparse.OptionParser()
# import options
parser.add_option('-f','--filename',help='Pass the csv filename')
parser.add_option('-p','--position',help='column position in the file',type=int)
parser.add_option('-s','--sample',help='sample size',type=int)
(opts,args) = parser.parse_args() # instantiate parser
# Program to select random values
with open('<filepath>'+opts.filename,'r') as f:
reader=csv.reader(f)
data=[row[opts.position] for row in reader]
shuffle(data)
#print '\n'.join(data[:opts.sample])
# create o/p file
file=open("<opfilename>.txt","w")
file.write('\n'.join(data[:opts.sample]))
file.close()

Python Type Error (probably simple)

I'm trying open a txt file and read it, but I'm getting a type error and I'm unsure why. If you you could please provide a reasoning along with the correct syntax, I'm trying to get a better grasp of what's going on underneath. Here's the code, it's pretty simple I think:
from sys import argv
script = argv
filename = argv
txt = open(filename)
print "Here's your file %r" %filename
print txt.read()
Muchas Gracias

argv is a list, not a string. You want
script = argv[0]
filename = argv[1]
Consider using argparse instead of handing sys.argv directly:
>>> import argparse
>>> parser = argparse.ArgumentParser(description="Print a file.")
>>> parser.add_argument("path", type=str, nargs=1, help="Path to the file to be printed.")
_StoreAction(option_strings=[], dest='path', nargs=1, const=None, default=None, type=<type 'str'>, choices=None, help='Path to the file to be printed.', metavar=None)
>>> args = parser.parse_args()
>>> print args
Namespace(path=[<...>])
It looks much more complicated, but it will make your command-line utility much more flexible, easier to expand, and it will ensure you get proper documentation on the command line.

First of all, argv is a list of arguments. Open doesn't take a list. That's why you're getting a type error.
Second, open (should) take 2 parameters, the filename and the mode (yes mode is optional, but get in the habit of putting it there. Replace with
import sys
script = sys.argv[0]
filename = sys.argv[1]
txt = open(filename, 'r')
print "Here's your file %r" %filename
print txt.read()

argv will be a list, while filename should be a string.
You should probably be using filename = argv[1]