I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script
If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()
You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?
You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2
Related
I want to have some options in argparse module such as --pm-export however when I try to use it like args.pm-export I get the error that there is not attribute pm. How can I get around this issue? Is it possible to have - in command line options?
As indicated in the argparse docs:
For optional argument actions, the value of dest is normally inferred from the option strings. ArgumentParser generates the value of dest by taking the first long option string and stripping away the initial -- string. Any internal - characters will be converted to _ characters to make sure the string is a valid attribute name
So you should be using args.pm_export.
Unfortunately, dash-to-underscore replacement doesn't work for positional arguments (not prefixed by --).
E.g:
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('logs-dir',
help='Directory with .log and .log.gz files')
parser.add_argument('results-csv', type=argparse.FileType('w'),
default=sys.stdout,
help='Output .csv filename')
args = parser.parse_args()
print args
# gives
# Namespace(logs-dir='./', results-csv=<open file 'lool.csv', mode 'w' at 0x9020650>)
So, you should use 1'st argument to add_argument() as attribute name and metavar kwarg to set how it should look in help:
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('logs_dir', metavar='logs-dir',
nargs=1,
help='Directory with .log and .log.gz files')
parser.add_argument('results_csv', metavar='results-csv',
nargs=1,
type=argparse.FileType('w'),
default=sys.stdout,
help='Output .csv filename')
args = parser.parse_args()
print args
# gives
# Namespace(logs_dir=['./'], results_csv=[<open file 'lool.csv', mode 'w' at 0xb71385f8>])
Dashes are converted to underscores:
import argparse
pa = argparse.ArgumentParser()
pa.add_argument('--foo-bar')
args = pa.parse_args(['--foo-bar', '24'])
print args # Namespace(foo_bar='24')
Concise and explicit but probably not always acceptable way would be to use vars():
#!/usr/bin/env python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('a-b')
args = vars(parser.parse_args())
print(args['a-b'])
getattr(args, 'positional-arg')
This is another OK workaround for positional arguments:
#!/usr/bin/env python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('a-b')
args = parser.parse_args(['123'])
assert getattr(args, 'a-b') == '123'
Tested on Python 3.8.2.
I guess the last option is to change shorten option -a to --a
import argparse
parser = argparse.ArgumentParser(description="Help")
parser.add_argument("--a", "--argument-option", metavar="", help="") # change here
args = parser.parse_args()
option = args.a # And here
print(option)
I am using the argparse library but for whatever reason I'm having difficult printing the -h argument. Here is the entire source I have:
# df.py
import argparse
parser = argparse.ArgumentParser(description='Dedupe assets in our library.')
parser.add_argument('--masters', nargs='?', default=None, type=int, help='Enter one or more ids.')
if __name__ == '__main__':
print ('hi')
I was under the impression that entering in the --h flag via:
$ python df.py --help
Would automatically print the help stuff for the file using argparse but I seem to be making false assumptions. It seems like I also have to add in something like this into my code?
if '--help' in sys.argv: print (parser.parse_args(['-h']))
What is the 'proper' way to print out the help args when using the argparse library?
You forgot to actually parse the arguments; if you put parser.parse_args() in after defining the parser, it would respond to -h/--help. Typically, you'd do something like:
args = parser.parse_args()
so that the args object can be used to access the parsed argument data.
I'll also note that the argument parsing should almost certainly be controlled by the if __name__ == '__main__': guard; if you're not being invoked as the main script, parsing the command line is unusual, to say the least. Idiomatic code would look something like:
# df.py
def main():
import argparse # Could be moved to top level, but given it's only used
# in main, it's not a terrible idea to import in main
parser = argparse.ArgumentParser(description='Dedupe assets in our library.')
parser.add_argument('--masters', nargs='?', type=int, help='Enter one or more ids.')
args = parser.parse_args()
print ('hi')
# Do something with args.masters or whatever
if __name__ == '__main__':
main()
Here is the current code.
import time
import collections
from modules import outputs
from modules import scrub
from modules import lookups
parser = argparse.ArgumentParser(description='AppMap Converter to Generate Asset Files from AppMapp Data')
parser.add_argument("operation", nargs='?', default="empty", help='The operation to perform')
parser.add_argument("input", nargs='?', default="empty", help='The input AppMapp File Path')
parser.add_argument("output", nargs='?', default="empty", help='The output Asset File Path')
args = parser.parse_args()
start = time.time()
if(args.operation == "Convert"):
input_file_path = args.input
output_file_path = args.output
#DO LOTS OF STUFF
else:
exit()
The script is called sacsproc, so I run it from the command line as follows:
./sacsproc Convert input.csv output.csv
This all works nicely, the problem is that I need more sacsproc commands which may have a totally different set of secondary parameters. i.e. one command might be:
./sacsproc Clean -rts input.csv output.csv err.csv
Thus, I am trying to determine how one defines arguments that are conditional on the first argument? In my mind, I'm thinking about the zfs command line utilities that do what I am trying to do (e.g. zpool create mirror sdb sdc vs. zpool remove sda).
use subparsers
subparsers = parser.add_subparsers(help="sub-command help")
group1 = subparsers.add_parser("something",help="do something")
group1.set_defaults(which="g1") # some default value (so you know which group was activated)
group1.add_argument("ARG",help='do something on ARG')
group2 = subparsers.add_parser("other",help="do something else")
group2.set_defaults(which="g2") # give some default value
group2.add_argument("ARG",help='do something else on ARG')
ok ...
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help="sub-command help")
g1 = subparsers.add_parser("thing1",help="bind to a port and just echo back anything it gets ... with a prompt")
g1.set_defaults(which="g1")
g1.add_argument("input",help='the input file')
g1.add_argument("output",help='the output file')
g2 = subparsers.add_parser("thing2",help="create a bridge between two ports, this is useful for generating a logfile")
g2.set_defaults(which="g2")
g2.add_argument("input",help='thie input file')
g2.add_argument("output",help='the output file')
g2.add_argument("error",help="the err file")
def print_help(args):
print "ARGS:",args
try:
parser.parse_args(args)
except:
print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n"
print_help(["-h"])
print_help(["thing1","-h"])
print_help(["thing2","-h"])
I cant see m to figure out how to iterate over the accepted args of argparse. I get I can iterate over the parsed_args result, but what I want is to iterate over the arguments the parser is configured with ( ie with optparse you can iterate over the args ).
for example:
parser = argparse.ArgumentParser( prog = 'myapp' )
parser.add_argument( '--a', .. )
parser.add_argument( '--b', ...)
parser.add_argument( '--c', ... )
for arg in parser.args():
print arg
would result in
--a
--b
--c
You'll probably want to getattr from the args:
args = parser.parse_args()
for arg in vars(args):
print arg, getattr(args, arg)
Result:
a None
c None
b None
If you want to list the optionals you can do it this way:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar')
parser.add_argument('--baz')
for option in parser._optionals._actions:
print(option.option_strings)
I don't see a practical reason to iterate over them however. You can always see the options via --help.
A bit late to the game here, but I found a way to do this without reading from private variables by using a custom help formatter that collects the arguments it is asked to format.
The following program will print ['-h', '--help', '--a', '--b', '--c']
import argparse
class ArgCollector(argparse.HelpFormatter):
# Will store the arguments in a class variable since argparse uses a class
# name, not an instance of a class
args = []
def add_argument(self, action):
# Just remember the options
self.args.extend(action.option_strings)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--a')
parser.add_argument('--b')
parser.add_argument('--c')
# Install our new help formatter, use it, then restore the original
# formatter
original_formatter_class = parser.formatter_class
parser.formatter_class = ArgCollector
parser.format_help()
parser.formatter_class = original_formatter_class
# Print the args that argparse would accept
print(ArgCollector.args)
if __name__ == '__main__':
main()
I'm new to python and currently playing with it.
I have a script which does some API Calls to an appliance. I would like to extend the functionality and call different functions based on the arguments given when calling the script.
Currently I have the following:
parser = argparse.ArgumentParser()
parser.add_argument("--showtop20", help="list top 20 by app",
action="store_true")
parser.add_argument("--listapps", help="list all available apps",
action="store_true")
args = parser.parse_args()
I also have a
def showtop20():
.....
and
def listapps():
....
How can I call the function (and only this) based on the argument given?
I don't want to run
if args.showtop20:
#code here
if args.listapps:
#code here
as I want to move the different functions to a module later on keeping the main executable file clean and tidy.
Since it seems like you want to run one, and only one, function depending on the arguments given, I would suggest you use a mandatory positional argument ./prog command, instead of optional arguments (./prog --command1 or ./prog --command2).
so, something like this should do it:
FUNCTION_MAP = {'top20' : my_top20_func,
'listapps' : my_listapps_func }
parser.add_argument('command', choices=FUNCTION_MAP.keys())
args = parser.parse_args()
func = FUNCTION_MAP[args.command]
func()
At least from what you have described, --showtop20 and --listapps sound more like sub-commands than options. Assuming this is the case, we can use subparsers to achieve your desired result. Here is a proof of concept:
import argparse
import sys
def showtop20():
print('running showtop20')
def listapps():
print('running listapps')
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Create a showtop20 subcommand
parser_showtop20 = subparsers.add_parser('showtop20', help='list top 20 by app')
parser_showtop20.set_defaults(func=showtop20)
# Create a listapps subcommand
parser_listapps = subparsers.add_parser('listapps', help='list all available apps')
parser_listapps.set_defaults(func=listapps)
# Print usage message if no args are supplied.
# NOTE: Python 2 will error 'too few arguments' if no subcommand is supplied.
# No such error occurs in Python 3, which makes it feasible to check
# whether a subcommand was provided (displaying a help message if not).
# argparse internals vary significantly over the major versions, so it's
# much easier to just override the args passed to it.
if len(sys.argv) <= 1:
sys.argv.append('--help')
options = parser.parse_args()
# Run the appropriate function (in this case showtop20 or listapps)
options.func()
# If you add command-line options, consider passing them to the function,
# e.g. `options.func(options)`
There are lots of ways of skinning this cat. Here's one using action='store_const' (inspired by the documented subparser example):
p=argparse.ArgumentParser()
p.add_argument('--cmd1', action='store_const', const=lambda:'cmd1', dest='cmd')
p.add_argument('--cmd2', action='store_const', const=lambda:'cmd2', dest='cmd')
args = p.parse_args(['--cmd1'])
# Out[21]: Namespace(cmd=<function <lambda> at 0x9abf994>)
p.parse_args(['--cmd2']).cmd()
# Out[19]: 'cmd2'
p.parse_args(['--cmd1']).cmd()
# Out[20]: 'cmd1'
With a shared dest, each action puts its function (const) in the same Namespace attribute. The function is invoked by args.cmd().
And as in the documented subparsers example, those functions could be written so as to use other values from Namespace.
args = parse_args()
args.cmd(args)
For sake of comparison, here's the equivalent subparsers case:
p = argparse.ArgumentParser()
sp = p.add_subparsers(dest='cmdstr')
sp1 = sp.add_parser('cmd1')
sp1.set_defaults(cmd=lambda:'cmd1')
sp2 = sp.add_parser('cmd2')
sp2.set_defaults(cmd=lambda:'cmd2')
p.parse_args(['cmd1']).cmd()
# Out[25]: 'cmd1'
As illustrated in the documentation, subparsers lets you define different parameter arguments for each of the commands.
And of course all of these add argument or parser statements could be created in a loop over some list or dictionary that pairs a key with a function.
Another important consideration - what kind of usage and help do you want? The different approaches generate very different help messages.
If your functions are "simple enough" take adventage of type parameter https://docs.python.org/2.7/library/argparse.html#type
type= can take any callable that takes a single string argument and
returns the converted value:
In your example (even if you don't need a converted value):
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
This simple script:
import argparse
def showtop20(dummy):
print "{0}\n".format(dummy) * 5
parser = argparse.ArgumentParser()
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
args = parser.parse_args()
Will give:
# ./test.py --listapps test
test
test
test
test
test
test
Instead of using your code as your_script --showtop20, make it into a sub-command your_script showtop20 and use the click library instead of argparse. You define functions that are the name of your subcommand and use decorators to specify the arguments:
import click
#click.group()
#click.option('--debug/--no-debug', default=False)
def cli(debug):
print(f'Debug mode is {"on" if debug else "off"}')
#cli.command() # #cli, not #click!
def showtop20():
# ...
#cli.command()
def listapps():
# ...
See https://click.palletsprojects.com/en/master/commands/
# based on parser input to invoke either regression/classification plus other params
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--target", type=str)
parser.add_argument("--type", type=str)
parser.add_argument("--deviceType", type=str)
args = parser.parse_args()
df = pd.read_csv(args.path)
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]
if args.type == "classification":
classify = AutoML(df, args.target, args.type, args.deviceType)
classify.class_dist()
classify.classification()
elif args.type == "regression":
reg = AutoML(df, args.target, args.type, args.deviceType)
reg.regression()
else:
ValueError("Invalid argument passed")
# Values passed as : python app.py --path C:\Users\Abhishek\Downloads\adult.csv --target income --type classification --deviceType GPU
You can evaluate using evalwhether your argument value is callable:
import argparse
def list_showtop20():
print("Calling from showtop20")
def list_apps():
print("Calling from listapps")
my_funcs = [x for x in dir() if x.startswith('list_')]
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--function", required=True,
choices=my_funcs,
help="function to call", metavar="")
args = parser.parse_args()
eval(args.function)()