I would like my script to receive these mutually exclusive input options:
an input file containing a JSON (script.py -i input.json);
a string containing a JSON (script.py '{"a":1}');
a JSON from stdin (echo '{"a":1}' | script.py or cat input.json | script.py).
and these mutually exclusive output options:
an output file containing a JSON;
a JSON in stdout.
So I tried with this code
import json,sys,argparse
parser = argparse.ArgumentParser(description='Template for python script managing JSON as input/output format')
group = parser.add_mutually_exclusive_group()
group.add_argument('--input-file', '-i', type=str, help='Input file name containing a valid JSON.', default=sys.stdin)
group.add_argument('json', nargs='?', type=str, help='Input string containing a valid JSON.' , default=sys.stdin)
parser.add_argument('--output-file', '-o',type=str, help='Output file name.')
args = parser.parse_args()
if not sys.stdin.isatty():
data = sys.stdin.read()
else:
# args = parser.parse_args()
if args.input_file :
data=open(args.input_file).read()
elif args.json :
data=args.json
datain=json.loads(data)
dataout=json.dumps(datain, indent=2)
if args.output_file :
output_file=open(args.output_file, 'w')
output_file.write(dataout+'\n')
output_file.close()
else:
print (dataout)
But it does not work with stdin as it requires at least one of the two group options.
How can I add stdin in the list of input options?
Adding the default=sys.stdin argument works if I call it like that
echo '{}' | ./script.py -
but not like that:
echo '{}' | ./script.py
I would take advantage of argparse.FileType with a default value of sys.stdin.
import json,sys,argparse
parser = argparse.ArgumentParser(description='Template for python script managing JSON as input/output format')
group = parser.add_mutually_exclusive_group()
group.add_argument(
'--input-file', '-i',
type=argparse.FileType('r'),
default=sys.stdin,
help='Input file name containing a valid JSON.')
group.add_argument(
'json',
nargs='?',
type=str,
help='Input string containing a valid JSON.')
parser.add_argument(
'--output-file', '-o',
type=argparse.FileType('w'),
help='Output file name.',
default=sys.stdout)
args = parser.parse_args()
data = args.json or args.input_file.read()
datain=json.loads(data)
dataout=json.dumps(datain, indent=2)
args.output_file.write(dataout)
With:
group.add_argument('--input-file', '-i')
You could test
if args.input_file is None:
<-i wasn't supplied>
else:
if args.input_file == '-':
f = sys.stdin
else:
f = open(args.input_file)
data = f.read() # etc
Or may be better:
if args.input_file == '-':
data = sys.stdin.read()
else
with open(args.input_file) as f:
f.read()
A tricky thing with stdin is that you don't want to close it after use like you would with a regular file name. And you can't use it in a with.
Similarly with stdout.
Some code sets a flag when it opens a file, as opposed to receiving an already open one, so it can remember to close the file at the end.
group.add_argument('--input-file','-i',nargs='?', default=None, const=sys.stdin)
would set arg.input_file to stdin when given -i without an argument. But I think looking for a plain - string is a better idea.
Related
I want to have some options in argparse module such as --pm-export however when I try to use it like args.pm-export I get the error that there is not attribute pm. How can I get around this issue? Is it possible to have - in command line options?
As indicated in the argparse docs:
For optional argument actions, the value of dest is normally inferred from the option strings. ArgumentParser generates the value of dest by taking the first long option string and stripping away the initial -- string. Any internal - characters will be converted to _ characters to make sure the string is a valid attribute name
So you should be using args.pm_export.
Unfortunately, dash-to-underscore replacement doesn't work for positional arguments (not prefixed by --).
E.g:
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('logs-dir',
help='Directory with .log and .log.gz files')
parser.add_argument('results-csv', type=argparse.FileType('w'),
default=sys.stdout,
help='Output .csv filename')
args = parser.parse_args()
print args
# gives
# Namespace(logs-dir='./', results-csv=<open file 'lool.csv', mode 'w' at 0x9020650>)
So, you should use 1'st argument to add_argument() as attribute name and metavar kwarg to set how it should look in help:
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('logs_dir', metavar='logs-dir',
nargs=1,
help='Directory with .log and .log.gz files')
parser.add_argument('results_csv', metavar='results-csv',
nargs=1,
type=argparse.FileType('w'),
default=sys.stdout,
help='Output .csv filename')
args = parser.parse_args()
print args
# gives
# Namespace(logs_dir=['./'], results_csv=[<open file 'lool.csv', mode 'w' at 0xb71385f8>])
Dashes are converted to underscores:
import argparse
pa = argparse.ArgumentParser()
pa.add_argument('--foo-bar')
args = pa.parse_args(['--foo-bar', '24'])
print args # Namespace(foo_bar='24')
Concise and explicit but probably not always acceptable way would be to use vars():
#!/usr/bin/env python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('a-b')
args = vars(parser.parse_args())
print(args['a-b'])
getattr(args, 'positional-arg')
This is another OK workaround for positional arguments:
#!/usr/bin/env python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('a-b')
args = parser.parse_args(['123'])
assert getattr(args, 'a-b') == '123'
Tested on Python 3.8.2.
I guess the last option is to change shorten option -a to --a
import argparse
parser = argparse.ArgumentParser(description="Help")
parser.add_argument("--a", "--argument-option", metavar="", help="") # change here
args = parser.parse_args()
option = args.a # And here
print(option)
I have a python3 script that requires three arguments: -orig, -corr, -out, where "orig" and "corr" are used to generate "out". "-orig" allows for only one file as argument, while "-corr" allows for multiple files as input. The script uses ExitStack for handling multiple files at the same time.
When I run the script in command line specifying the files for each argument:
python3 myscript.py -orig <orig_file> -cor <cor_file1> [<cor_file2> ...] -out <outputfile>
it works without problem, but I would like to run the script over several "orig" files and the corresponding "corr" files. Any ideas?
This is the start of the script:
with ExitStack() as stack:
in_files = [stack.enter_context(open(i)) for i in [args.orig]+args.cor]
# Process each line of all input files.
for line_id, line in enumerate(zip(*in_files)):
orig_sent = line[0].strip()
cor_sents = line[1:]
And the section corresponding to the arguments:
if __name__ == "__main__":
# Define and parse program input
parser = argparse.ArgumentParser(description="blabla",formatter_class=argparse.RawTextHelpFormatter, usage="%(prog)s [-h] [options] -orig ORIG -cor COR [COR ...] -out OUT")
parser.add_argument("-orig", help="The path to the original text file.", required=True)
parser.add_argument("-cor", help="The paths to >= 1 corrected text files.", nargs="+", default=[], required=True)
parser.add_argument("-out", help="The output filepath.", required=True)
args = parser.parse_args()
# Run the program.
main(args)
I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script
If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()
You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?
You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2
Here is the current code.
import time
import collections
from modules import outputs
from modules import scrub
from modules import lookups
parser = argparse.ArgumentParser(description='AppMap Converter to Generate Asset Files from AppMapp Data')
parser.add_argument("operation", nargs='?', default="empty", help='The operation to perform')
parser.add_argument("input", nargs='?', default="empty", help='The input AppMapp File Path')
parser.add_argument("output", nargs='?', default="empty", help='The output Asset File Path')
args = parser.parse_args()
start = time.time()
if(args.operation == "Convert"):
input_file_path = args.input
output_file_path = args.output
#DO LOTS OF STUFF
else:
exit()
The script is called sacsproc, so I run it from the command line as follows:
./sacsproc Convert input.csv output.csv
This all works nicely, the problem is that I need more sacsproc commands which may have a totally different set of secondary parameters. i.e. one command might be:
./sacsproc Clean -rts input.csv output.csv err.csv
Thus, I am trying to determine how one defines arguments that are conditional on the first argument? In my mind, I'm thinking about the zfs command line utilities that do what I am trying to do (e.g. zpool create mirror sdb sdc vs. zpool remove sda).
use subparsers
subparsers = parser.add_subparsers(help="sub-command help")
group1 = subparsers.add_parser("something",help="do something")
group1.set_defaults(which="g1") # some default value (so you know which group was activated)
group1.add_argument("ARG",help='do something on ARG')
group2 = subparsers.add_parser("other",help="do something else")
group2.set_defaults(which="g2") # give some default value
group2.add_argument("ARG",help='do something else on ARG')
ok ...
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help="sub-command help")
g1 = subparsers.add_parser("thing1",help="bind to a port and just echo back anything it gets ... with a prompt")
g1.set_defaults(which="g1")
g1.add_argument("input",help='the input file')
g1.add_argument("output",help='the output file')
g2 = subparsers.add_parser("thing2",help="create a bridge between two ports, this is useful for generating a logfile")
g2.set_defaults(which="g2")
g2.add_argument("input",help='thie input file')
g2.add_argument("output",help='the output file')
g2.add_argument("error",help="the err file")
def print_help(args):
print "ARGS:",args
try:
parser.parse_args(args)
except:
print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n"
print_help(["-h"])
print_help(["thing1","-h"])
print_help(["thing2","-h"])
Hi I'm trying to use argparse for filename input from command line but I'm struggling to get it working.
I want to take a string passed from the command line (-d) which corresponds to a filename (datbase.csv) and store it in the variable inputargs.snp_database_location.
This gets taken as input to my load_search_snaps function as shown in my code below which opens the file and does stuff (pseudocode) to it.
import csv, sys, argparse
parser = argparse.ArgumentParser(description='Search a list of variants against the in house database')
parser.add_argument('-d', '--database',
action='store',
dest='snp_database_location',
type=str,
nargs=1,
help='File location for the in house variant database',
default='Error: Database location must be specified')
inputargs = parser.parse_args()
def load_search_snps(input_file):
with open(input_file, 'r+') as varin:
id_store_dictgroup = csv.DictReader(varin)
#do things with id_store_dictgroup
return result
load_search_snps(inputargs.snp_database_location)
using the command in bash:
python3 snp_freq_V1-0_export.py -d snpstocheck.csv
I get the following error when I try and pass it a regular csv file from the same directory using command line:
File "snp_freq_V1-0_export.py", line 33, in load_search_snps
with open(input_file, 'r+') as varin: TypeError: invalid file: ['snpstocheck.csv']
If I feed the filepath in from within the script it works perfectly. As far as I can tell I get a string for snp_database_location which matches the filename string, but then I get the error. What am I missing that's giving the type error?
nargs=1 makes inputargs.snp_database_location a list (with one element), not a string.
In [49]: import argparse
In [50]: parser = argparse.ArgumentParser()
In [51]: parser.add_argument('-d', nargs=1)
Out[51]: _StoreAction(option_strings=['-d'], dest='d', nargs=1, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [52]: args = parser.parse_args(['-d', 'snpstocheck.csv'])
In [53]: args.d
Out[53]: ['snpstocheck.csv']
To fix, remove nargs=1.