Develop Reference

Using the '--help' command in argparse

I am using the argparse library but for whatever reason I'm having difficult printing the -h argument. Here is the entire source I have:
# df.py
import argparse
parser = argparse.ArgumentParser(description='Dedupe assets in our library.')
parser.add_argument('--masters', nargs='?', default=None, type=int, help='Enter one or more ids.')
if __name__ == '__main__':
print ('hi')
I was under the impression that entering in the --h flag via:
$ python df.py --help
Would automatically print the help stuff for the file using argparse but I seem to be making false assumptions. It seems like I also have to add in something like this into my code?
if '--help' in sys.argv: print (parser.parse_args(['-h']))
What is the 'proper' way to print out the help args when using the argparse library?

You forgot to actually parse the arguments; if you put parser.parse_args() in after defining the parser, it would respond to -h/--help. Typically, you'd do something like:
args = parser.parse_args()
so that the args object can be used to access the parsed argument data.
I'll also note that the argument parsing should almost certainly be controlled by the if __name__ == '__main__': guard; if you're not being invoked as the main script, parsing the command line is unusual, to say the least. Idiomatic code would look something like:
# df.py
def main():
import argparse # Could be moved to top level, but given it's only used
# in main, it's not a terrible idea to import in main
parser = argparse.ArgumentParser(description='Dedupe assets in our library.')
parser.add_argument('--masters', nargs='?', type=int, help='Enter one or more ids.')
args = parser.parse_args()
print ('hi')
# Do something with args.masters or whatever
if __name__ == '__main__':
main()

Use argparse to send arguments to function within Python script

I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script

If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()

You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?

You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2

Python argparse mutually exclusive with stdin being one of the options

I would like my script to receive these mutually exclusive input options:
an input file containing a JSON (script.py -i input.json);
a string containing a JSON (script.py '{"a":1}');
a JSON from stdin (echo '{"a":1}' | script.py or cat input.json | script.py).
and these mutually exclusive output options:
an output file containing a JSON;
a JSON in stdout.
So I tried with this code
import json,sys,argparse
parser = argparse.ArgumentParser(description='Template for python script managing JSON as input/output format')
group = parser.add_mutually_exclusive_group()
group.add_argument('--input-file', '-i', type=str, help='Input file name containing a valid JSON.', default=sys.stdin)
group.add_argument('json', nargs='?', type=str, help='Input string containing a valid JSON.' , default=sys.stdin)
parser.add_argument('--output-file', '-o',type=str, help='Output file name.')
args = parser.parse_args()
if not sys.stdin.isatty():
data = sys.stdin.read()
else:
# args = parser.parse_args()
if args.input_file :
data=open(args.input_file).read()
elif args.json :
data=args.json
datain=json.loads(data)
dataout=json.dumps(datain, indent=2)
if args.output_file :
output_file=open(args.output_file, 'w')
output_file.write(dataout+'\n')
output_file.close()
else:
print (dataout)
But it does not work with stdin as it requires at least one of the two group options.
How can I add stdin in the list of input options?
Adding the default=sys.stdin argument works if I call it like that
echo '{}' | ./script.py -
but not like that:
echo '{}' | ./script.py

I would take advantage of argparse.FileType with a default value of sys.stdin.
import json,sys,argparse
parser = argparse.ArgumentParser(description='Template for python script managing JSON as input/output format')
group = parser.add_mutually_exclusive_group()
group.add_argument(
'--input-file', '-i',
type=argparse.FileType('r'),
default=sys.stdin,
help='Input file name containing a valid JSON.')
group.add_argument(
'json',
nargs='?',
type=str,
help='Input string containing a valid JSON.')
parser.add_argument(
'--output-file', '-o',
type=argparse.FileType('w'),
help='Output file name.',
default=sys.stdout)
args = parser.parse_args()
data = args.json or args.input_file.read()
datain=json.loads(data)
dataout=json.dumps(datain, indent=2)
args.output_file.write(dataout)

With:
group.add_argument('--input-file', '-i')
You could test
if args.input_file is None:
<-i wasn't supplied>
else:
if args.input_file == '-':
f = sys.stdin
else:
f = open(args.input_file)
data = f.read() # etc
Or may be better:
if args.input_file == '-':
data = sys.stdin.read()
else
with open(args.input_file) as f:
f.read()
A tricky thing with stdin is that you don't want to close it after use like you would with a regular file name. And you can't use it in a with.
Similarly with stdout.
Some code sets a flag when it opens a file, as opposed to receiving an already open one, so it can remember to close the file at the end.
group.add_argument('--input-file','-i',nargs='?', default=None, const=sys.stdin)
would set arg.input_file to stdin when given -i without an argument. But I think looking for a plain - string is a better idea.

Using Argparse to create required argument with multiple options?

If I understand Argparse correctly, the positional arguments are the required arguments that the user can specify. I need to create a positional argument with argparse where the user can specify a certain type of argument that is displayed if he/she brings up the -h option. I've tried using add_argument_group but it simply only displays a header with a description of the other arguments when you bring up the -h option.
def Main():
parser = argparse.ArgumentParser(description = __doc__, formatter_class = argparse.RawDescriptionHelpFormatter)
parser.add_argument("input_directory",help = "The input directory where all of the files reside in")
sub_parser = parser.add_argument_group('File Type')
sub_parser.add_argument(".txt",help = "The input file is a .txt file")
sub_parser.add_argument(".n12",help = "The input file is a .n12 file")
sub_parser.add_argument(".csv",help = "The input file is a .csv file")
parser.parse_args()
if __name__ == "__main__":
Main()
So when I run the script, I should specify in order to run the script. If I choose either .txt, .n12, or .csv as my argument, then the script should run. However, if the I don't specify the file type from those 3 options listed, then the script wouldn't run.
Is there an argparse function that I'm missing that can specify multiple options for a positional argument?

Use the choices= parameter to force the user to choose from a restricted set of values.
import argparse
def Main():
parser = argparse.ArgumentParser()
parser.add_argument("input_directory",help = "The input directory where all of the files reside in")
parser.add_argument("file_type", help = "File Type", choices=['.txt', '.n12', '.csv'])
ns = parser.parse_args()
print(ns)
if __name__ == "__main__":
Main()

I think you're making this too complicated. If I understand your problem correctly, you want the user to enter two arguments: a directory name and a file type. You application will accept only three values for file type. How about simply doing this:
import argparse
def Main():
parser = argparse.ArgumentParser(description = __doc__, formatter_class = argparse.RawDescriptionHelpFormatter)
parser.add_argument("input_directory", help = "The input directory where all of the files reside in")
parser.add_argument("file_type", help="One of: .txt, .n12, .csv")
args = parser.parse_args()
print(args)
if __name__ == "__main__":
Main()
... and adding application logic to reject invalid values for file type.
You access the user-entered values through the object returned by parse_args().

Use option grouping feature use add_mutually_exclusive_group() instead of add_argument_group()
import argparse
def Main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("input_directory", help="The input directory where all of the files reside in")
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("-txt", action='store_true', help="The input file is a .txt file")
group.add_argument("-n12", action='store_true', help="The input file is a .n12 file")
group.add_argument("-csv", action='store_true', help="The input file is a .csv file")
print parser.parse_args()
if __name__ == "__main__":
Main()

Call function based on argparse

I'm new to python and currently playing with it.
I have a script which does some API Calls to an appliance. I would like to extend the functionality and call different functions based on the arguments given when calling the script.
Currently I have the following:
parser = argparse.ArgumentParser()
parser.add_argument("--showtop20", help="list top 20 by app",
action="store_true")
parser.add_argument("--listapps", help="list all available apps",
action="store_true")
args = parser.parse_args()
I also have a
def showtop20():
.....
and
def listapps():
....
How can I call the function (and only this) based on the argument given?
I don't want to run
if args.showtop20:
#code here
if args.listapps:
#code here
as I want to move the different functions to a module later on keeping the main executable file clean and tidy.

Since it seems like you want to run one, and only one, function depending on the arguments given, I would suggest you use a mandatory positional argument ./prog command, instead of optional arguments (./prog --command1 or ./prog --command2).
so, something like this should do it:
FUNCTION_MAP = {'top20' : my_top20_func,
'listapps' : my_listapps_func }
parser.add_argument('command', choices=FUNCTION_MAP.keys())
args = parser.parse_args()
func = FUNCTION_MAP[args.command]
func()

At least from what you have described, --showtop20 and --listapps sound more like sub-commands than options. Assuming this is the case, we can use subparsers to achieve your desired result. Here is a proof of concept:
import argparse
import sys
def showtop20():
print('running showtop20')
def listapps():
print('running listapps')
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Create a showtop20 subcommand
parser_showtop20 = subparsers.add_parser('showtop20', help='list top 20 by app')
parser_showtop20.set_defaults(func=showtop20)
# Create a listapps subcommand
parser_listapps = subparsers.add_parser('listapps', help='list all available apps')
parser_listapps.set_defaults(func=listapps)
# Print usage message if no args are supplied.
# NOTE: Python 2 will error 'too few arguments' if no subcommand is supplied.
# No such error occurs in Python 3, which makes it feasible to check
# whether a subcommand was provided (displaying a help message if not).
# argparse internals vary significantly over the major versions, so it's
# much easier to just override the args passed to it.
if len(sys.argv) <= 1:
sys.argv.append('--help')
options = parser.parse_args()
# Run the appropriate function (in this case showtop20 or listapps)
options.func()
# If you add command-line options, consider passing them to the function,
# e.g. `options.func(options)`

There are lots of ways of skinning this cat. Here's one using action='store_const' (inspired by the documented subparser example):
p=argparse.ArgumentParser()
p.add_argument('--cmd1', action='store_const', const=lambda:'cmd1', dest='cmd')
p.add_argument('--cmd2', action='store_const', const=lambda:'cmd2', dest='cmd')
args = p.parse_args(['--cmd1'])
# Out[21]: Namespace(cmd=<function <lambda> at 0x9abf994>)
p.parse_args(['--cmd2']).cmd()
# Out[19]: 'cmd2'
p.parse_args(['--cmd1']).cmd()
# Out[20]: 'cmd1'
With a shared dest, each action puts its function (const) in the same Namespace attribute. The function is invoked by args.cmd().
And as in the documented subparsers example, those functions could be written so as to use other values from Namespace.
args = parse_args()
args.cmd(args)
For sake of comparison, here's the equivalent subparsers case:
p = argparse.ArgumentParser()
sp = p.add_subparsers(dest='cmdstr')
sp1 = sp.add_parser('cmd1')
sp1.set_defaults(cmd=lambda:'cmd1')
sp2 = sp.add_parser('cmd2')
sp2.set_defaults(cmd=lambda:'cmd2')
p.parse_args(['cmd1']).cmd()
# Out[25]: 'cmd1'
As illustrated in the documentation, subparsers lets you define different parameter arguments for each of the commands.
And of course all of these add argument or parser statements could be created in a loop over some list or dictionary that pairs a key with a function.
Another important consideration - what kind of usage and help do you want? The different approaches generate very different help messages.

If your functions are "simple enough" take adventage of type parameter https://docs.python.org/2.7/library/argparse.html#type
type= can take any callable that takes a single string argument and
returns the converted value:
In your example (even if you don't need a converted value):
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
This simple script:
import argparse
def showtop20(dummy):
print "{0}\n".format(dummy) * 5
parser = argparse.ArgumentParser()
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
args = parser.parse_args()
Will give:
# ./test.py --listapps test
test
test
test
test
test
test

Instead of using your code as your_script --showtop20, make it into a sub-command your_script showtop20 and use the click library instead of argparse. You define functions that are the name of your subcommand and use decorators to specify the arguments:
import click
#click.group()
#click.option('--debug/--no-debug', default=False)
def cli(debug):
print(f'Debug mode is {"on" if debug else "off"}')
#cli.command() # #cli, not #click!
def showtop20():
# ...
#cli.command()
def listapps():
# ...
See https://click.palletsprojects.com/en/master/commands/

# based on parser input to invoke either regression/classification plus other params
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--target", type=str)
parser.add_argument("--type", type=str)
parser.add_argument("--deviceType", type=str)
args = parser.parse_args()
df = pd.read_csv(args.path)
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]
if args.type == "classification":
classify = AutoML(df, args.target, args.type, args.deviceType)
classify.class_dist()
classify.classification()
elif args.type == "regression":
reg = AutoML(df, args.target, args.type, args.deviceType)
reg.regression()
else:
ValueError("Invalid argument passed")
# Values passed as : python app.py --path C:\Users\Abhishek\Downloads\adult.csv --target income --type classification --deviceType GPU

You can evaluate using evalwhether your argument value is callable:
import argparse
def list_showtop20():
print("Calling from showtop20")
def list_apps():
print("Calling from listapps")
my_funcs = [x for x in dir() if x.startswith('list_')]
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--function", required=True,
choices=my_funcs,
help="function to call", metavar="")
args = parser.parse_args()
eval(args.function)()