When I run the following code:
def read_args():
parser = default_parser()
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
I have the following error output:
The following arguments are required: --name
However when I add the --name argument:
def read_args():
parser = default_parser()
parser.add_argument('--name', type=str, default='cifar10test')
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
is also creating problem.
Any ideas?
It appears that default_parser contains a --name argument which is required. What you're doing in your second example is defining the argument twice - once in default_parser and once in your program. Instead, you should be passing a --name argument when calling your program from the command line.
Example:
python cifar.py -N=1200 --tensorboard-dir=file.txt --name=cool_name
Alternatively, you could remove default_parser and construct your own ArgumentParser:
`parser = argparse.ArgumentParser()`
Full working demo:
import argparse
def read_args():
parser = argparse.ArgumentParser()
parser.add_argument('--tensorboard-dir', type=str,
default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000,
help="Use N training examples.")
return parser.parse_args()
def main():
flags = vars(read_args())
# You can access your args as a dictionary
for key in flags:
print("{} is {}".format(key, flags[key]))
main()
The parser returns a Namespace object, but we can access its (much simpler) internal dictionary using vars(Namespace). You can then get your arguments by accessing the dictionary, for example, flags['N']. Note that tensorboard-dir becomes tensorboard_dir inside your python program to avoid issues with the subtraction operator.
Call it from the command line (I'm using Bash):
python cifar.py -N=1200 --tensorboard-dir=file.txt
Output:
tensorboard_dir is file.txt
N is 1200
Related
I have a Python script that can deploy some software in three different environments, let's call them development, testing and production. In order to select which environment the script should work with, I want to use mutually exclusive flags, like so:
./deploy.py --development
./deploy.py --testing
./deploy.py --production
So far I have this:
parser = argparse.ArgumentParser(description="Manage deployment")
group = parser.add_mutually_exclusive_group()
group.add_argument("-d", "--development", action='store_true', required=False)
group.add_argument("-t", "--testing", action='store_true', required=False)
group.add_argument("-p", "--production", action='store_true', required=False)
args = parser.parse_args()
Thing is, I want to have the environment in a single variable, so I can easily check it, instead of having to manually check args.development, args.testing and args.production.
Is there a way of having a common variable that the three arguments could write to so I could do something like args.environment?
Instead of using action='store_true', you can use action='store_const', assign an individual const value for each argument and the dest option of add_argument, so that all the arguments in the mutually exclusive group have the same destination in the object returned by parse_args.
parser = argparse.ArgumentParser(description="Manage deployment")
group = parser.add_mutually_exclusive_group()
group.add_argument("-d", "--development", action='store_const', dest='environment', const='development', required=False)
group.add_argument("-t", "--testing", action='store_const', dest='environment', const='testing', required=False)
group.add_argument("-p", "--production", action='store_const', dest='environment', const='production', required=False)
args = parser.parse_args()
print(args)
The output is:
$ ./test_args.py --production
Namespace(environment='production')
Instead of 3 arguments, you can just use one with choices:
parser = argparse.ArgumentParser(description="Manage deployment")
parser.add_argument("environment", choices=['development', 'testing', 'production'])
args = parser.parse_args()
print(args)
Examples:
>>> test_args.py
usage: concept.py [-h] {development,testing,production}
test_args.py: error: the following arguments are required: environment
>>> test_args.py testing
Namespace(environment='testing')
I'm trying to build a script that uses subparsers arguments. However, I can not pass any of the sub-arguments as a parameter. Resulting in "invalid choice:" for any input combination.
Example input:
python3 preprocess.py -d ../data/acm/ tf -l en
Complete Output:
usage: preprocess.py [-h] [-k FOLDS] -d DATASETDIR [DATASETDIR ...] {tf} ...
preprocess.py: error: invalid choice: 'en' (choose from 'tf')
The code is
parser = argparse.ArgumentParser(description='Split input dataset into k folds of cross-validation.')
parser.add_argument('-k', '--folds', default=10, help='Number of folds for K fold cross-validation.', type=int)
required_args = parser.add_argument_group('required arguments')
required_args.add_argument('-d','--datasetdir', type=str, nargs='+', help='Dataset path (For more info: readme.txt)', required=True)
parser_subparsers = parser.add_subparsers(title="Representations", description="Choose the representations")
parser_tf = parser_subparsers.add_parser('tf', help='TF helper')
parser_tf.add_argument('-l', '--language', type=str, help='Language', default='en', choices=['en'])
parser_tf.add_argument('-s', '--stopword', type=bool, help='Skip stopwords', default=True)
args = parser.parse_args()
Since --datasetdir has nargs="+" the other argument(s) are being slurped up as additional dataset dir, rather than invoking the subparser.
CLI suggestion: change datasetdir into a plain old positional argument, with ability to separate paths using os.pathsep. It will be difficult to wrangle argparse into what you wanted to do, and using optional arguments with required=True is a code smell in the first place.
New interface will look something like this:
python3 preprocess.py ../data/acm/:/dir2:/dir3 tf -l en
I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script
If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()
You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?
You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2
I'm new to python and currently playing with it.
I have a script which does some API Calls to an appliance. I would like to extend the functionality and call different functions based on the arguments given when calling the script.
Currently I have the following:
parser = argparse.ArgumentParser()
parser.add_argument("--showtop20", help="list top 20 by app",
action="store_true")
parser.add_argument("--listapps", help="list all available apps",
action="store_true")
args = parser.parse_args()
I also have a
def showtop20():
.....
and
def listapps():
....
How can I call the function (and only this) based on the argument given?
I don't want to run
if args.showtop20:
#code here
if args.listapps:
#code here
as I want to move the different functions to a module later on keeping the main executable file clean and tidy.
Since it seems like you want to run one, and only one, function depending on the arguments given, I would suggest you use a mandatory positional argument ./prog command, instead of optional arguments (./prog --command1 or ./prog --command2).
so, something like this should do it:
FUNCTION_MAP = {'top20' : my_top20_func,
'listapps' : my_listapps_func }
parser.add_argument('command', choices=FUNCTION_MAP.keys())
args = parser.parse_args()
func = FUNCTION_MAP[args.command]
func()
At least from what you have described, --showtop20 and --listapps sound more like sub-commands than options. Assuming this is the case, we can use subparsers to achieve your desired result. Here is a proof of concept:
import argparse
import sys
def showtop20():
print('running showtop20')
def listapps():
print('running listapps')
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Create a showtop20 subcommand
parser_showtop20 = subparsers.add_parser('showtop20', help='list top 20 by app')
parser_showtop20.set_defaults(func=showtop20)
# Create a listapps subcommand
parser_listapps = subparsers.add_parser('listapps', help='list all available apps')
parser_listapps.set_defaults(func=listapps)
# Print usage message if no args are supplied.
# NOTE: Python 2 will error 'too few arguments' if no subcommand is supplied.
# No such error occurs in Python 3, which makes it feasible to check
# whether a subcommand was provided (displaying a help message if not).
# argparse internals vary significantly over the major versions, so it's
# much easier to just override the args passed to it.
if len(sys.argv) <= 1:
sys.argv.append('--help')
options = parser.parse_args()
# Run the appropriate function (in this case showtop20 or listapps)
options.func()
# If you add command-line options, consider passing them to the function,
# e.g. `options.func(options)`
There are lots of ways of skinning this cat. Here's one using action='store_const' (inspired by the documented subparser example):
p=argparse.ArgumentParser()
p.add_argument('--cmd1', action='store_const', const=lambda:'cmd1', dest='cmd')
p.add_argument('--cmd2', action='store_const', const=lambda:'cmd2', dest='cmd')
args = p.parse_args(['--cmd1'])
# Out[21]: Namespace(cmd=<function <lambda> at 0x9abf994>)
p.parse_args(['--cmd2']).cmd()
# Out[19]: 'cmd2'
p.parse_args(['--cmd1']).cmd()
# Out[20]: 'cmd1'
With a shared dest, each action puts its function (const) in the same Namespace attribute. The function is invoked by args.cmd().
And as in the documented subparsers example, those functions could be written so as to use other values from Namespace.
args = parse_args()
args.cmd(args)
For sake of comparison, here's the equivalent subparsers case:
p = argparse.ArgumentParser()
sp = p.add_subparsers(dest='cmdstr')
sp1 = sp.add_parser('cmd1')
sp1.set_defaults(cmd=lambda:'cmd1')
sp2 = sp.add_parser('cmd2')
sp2.set_defaults(cmd=lambda:'cmd2')
p.parse_args(['cmd1']).cmd()
# Out[25]: 'cmd1'
As illustrated in the documentation, subparsers lets you define different parameter arguments for each of the commands.
And of course all of these add argument or parser statements could be created in a loop over some list or dictionary that pairs a key with a function.
Another important consideration - what kind of usage and help do you want? The different approaches generate very different help messages.
If your functions are "simple enough" take adventage of type parameter https://docs.python.org/2.7/library/argparse.html#type
type= can take any callable that takes a single string argument and
returns the converted value:
In your example (even if you don't need a converted value):
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
This simple script:
import argparse
def showtop20(dummy):
print "{0}\n".format(dummy) * 5
parser = argparse.ArgumentParser()
parser.add_argument("--listapps", help="list all available apps",
type=showtop20,
action="store")
args = parser.parse_args()
Will give:
# ./test.py --listapps test
test
test
test
test
test
test
Instead of using your code as your_script --showtop20, make it into a sub-command your_script showtop20 and use the click library instead of argparse. You define functions that are the name of your subcommand and use decorators to specify the arguments:
import click
#click.group()
#click.option('--debug/--no-debug', default=False)
def cli(debug):
print(f'Debug mode is {"on" if debug else "off"}')
#cli.command() # #cli, not #click!
def showtop20():
# ...
#cli.command()
def listapps():
# ...
See https://click.palletsprojects.com/en/master/commands/
# based on parser input to invoke either regression/classification plus other params
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--target", type=str)
parser.add_argument("--type", type=str)
parser.add_argument("--deviceType", type=str)
args = parser.parse_args()
df = pd.read_csv(args.path)
df = df.loc[:, ~df.columns.str.contains('^Unnamed')]
if args.type == "classification":
classify = AutoML(df, args.target, args.type, args.deviceType)
classify.class_dist()
classify.classification()
elif args.type == "regression":
reg = AutoML(df, args.target, args.type, args.deviceType)
reg.regression()
else:
ValueError("Invalid argument passed")
# Values passed as : python app.py --path C:\Users\Abhishek\Downloads\adult.csv --target income --type classification --deviceType GPU
You can evaluate using evalwhether your argument value is callable:
import argparse
def list_showtop20():
print("Calling from showtop20")
def list_apps():
print("Calling from listapps")
my_funcs = [x for x in dir() if x.startswith('list_')]
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--function", required=True,
choices=my_funcs,
help="function to call", metavar="")
args = parser.parse_args()
eval(args.function)()
write a script that takes two optional boolean arguments,"--verbose‚" and ‚"--live", and two required string arguments, "base"and "pattern". Please set up the command line processing using argparse.
This is the code I have so far for the question, I know I am getting close but something is not quite right. Any help is much appreciated.Thanks for all the quick useful feedback.
def main():
import argparse
parser = argparse.ArgumentParser(description='')
parser.add_argument('base', type=str)
parser.add_arguemnt('--verbose', action='store_true')
parser.add_argument('pattern', type=str)
parser.add_arguemnt('--live', action='store_true')
args = parser.parse_args()
print(args.base(args.pattern))
The string arguments are not required by default, so you need to state that. Also the print statement that uses the arguments is incorrect.
#!/usr/bin/python
import argparse
if __name__=="__main__":
parser = argparse.ArgumentParser(description='eg $python myargs.py --base arg1 --pattern arg2 [--verbose] [--live]')
parser.add_argument('--base', required=True, type=str)
parser.add_argument('--pattern', required=True, type=str)
parser.add_argument('--verbose', action='store_true')
parser.add_argument('--live', action='store_true')
args = parser.parse_args()
print "args.base=" + str(args.base)
print "args.pattern=" + str(args.pattern)
print "args.verbose=" + str(args.verbose)
print "args.live=" + str(args.live)
the #!/usr/bin/python at the top enables the script to be called directly, though python must be located there (to confirm, type $which python), and you must set the file to have execute permission ($chmod +x myargs.py)