I'm trying to build a script that uses subparsers arguments. However, I can not pass any of the sub-arguments as a parameter. Resulting in "invalid choice:" for any input combination.
Example input:
python3 preprocess.py -d ../data/acm/ tf -l en
Complete Output:
usage: preprocess.py [-h] [-k FOLDS] -d DATASETDIR [DATASETDIR ...] {tf} ...
preprocess.py: error: invalid choice: 'en' (choose from 'tf')
The code is
parser = argparse.ArgumentParser(description='Split input dataset into k folds of cross-validation.')
parser.add_argument('-k', '--folds', default=10, help='Number of folds for K fold cross-validation.', type=int)
required_args = parser.add_argument_group('required arguments')
required_args.add_argument('-d','--datasetdir', type=str, nargs='+', help='Dataset path (For more info: readme.txt)', required=True)
parser_subparsers = parser.add_subparsers(title="Representations", description="Choose the representations")
parser_tf = parser_subparsers.add_parser('tf', help='TF helper')
parser_tf.add_argument('-l', '--language', type=str, help='Language', default='en', choices=['en'])
parser_tf.add_argument('-s', '--stopword', type=bool, help='Skip stopwords', default=True)
args = parser.parse_args()
Since --datasetdir has nargs="+" the other argument(s) are being slurped up as additional dataset dir, rather than invoking the subparser.
CLI suggestion: change datasetdir into a plain old positional argument, with ability to separate paths using os.pathsep. It will be difficult to wrangle argparse into what you wanted to do, and using optional arguments with required=True is a code smell in the first place.
New interface will look something like this:
python3 preprocess.py ../data/acm/:/dir2:/dir3 tf -l en
Related
I am trying to code a module for my deep learning model. I wish to store these arguments using argeparse. There is some problem occuring in args = parser.parse_args()! Also What are the benefits of using the argparse library?
import numpy as np
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--dataset', type=str,
help='Dataset')
parser.add_argument('--epoch', type=int, default=40,
help='Training Epochs')
parser.add_argument('--node_dim', type=int, default=64,
help='Node dimension')
parser.add_argument('--num_channels', type=int, default=2,
help='number of channels')
parser.add_argument('--lr', type=float, default=0.005,
help='learning rate')
parser.add_argument('--weight_decay', type=float, default=0.001,
help='l2 reg')
parser.add_argument('--num_layers', type=int, default=2,
help='number of layer')
parser.add_argument('--norm', type=str, default='true',
help='normalization')
parser.add_argument('--adaptive_lr', type=str, default='false',
help='adaptive learning rate')
args = parser.parse_args()
print(args)
The above code is a part of full code, as given in the link below
Error:
usage: ipykernel_launcher.py [-h] [--dataset DATASET] [--epoch EPOCH]
[--node_dim NODE_DIM]
[--num_channels NUM_CHANNELS] [--lr LR]
[--weight_decay WEIGHT_DECAY]
[--num_layers NUM_LAYERS] [--norm NORM]
[--adaptive_lr ADAPTIVE_LR]
ipykernel_launcher.py: error: unrecognized arguments: -f /Users/anshumansinha/Library/Jupyter/runtime/kernel-1e4c0f41-a4d7-4388-be24-640dd3411f56.json
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
The above code is a part of a code, the full code can be seen on the following github link : link
You called ipykernel_launcher.py with the argument -f. But -f is not in your list of arguments.
Is -f the argument, which you want to use for the input file? Then you should add something like this to your code:
parser.add_argument('--file', '-f', type=str)
The benefit of argparse is that you don't have to write by hand the code for parsing the arguments. You can try this. It is more code than one normally thinks. Especially if you have positional arguments.
I have a code that needs to build its command line arguments dynamically, based on configuration file. Schematically, what I end up doing is
parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", type=str, default="",
help="Path to config file.")
args, unknown = parser.parse_known_args()
#here do stuff with args.config to extend parser list of arguments, and then:
parser.parse_args()
The argument management seems to work perfectly well but the trouble I have is that --help will exit at the first call parse_known_args, instead of the second parser.parse_args() which would have shown all the dynamically added arguments.... Is there a way to solve this?
I had the same problem some time ago. It can be solved by using two parsers.
A first pre-parser, with add_help=False to determine the configuration file;
The second full parser with help.
# test.py
import argparse
# First pre-parser to determine the configuration file
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument("-c", "--config", default="", help="Path to config file.")
args = parser.parse_known_args()
# Do something with args.config
# Full and real parser
parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", default="", help="Path to config file.")
parser.add_argument("-o", "--option", help="Option with default value from config file")
parser.parse_args()
This result in:
$ python3 test.py --help
usage: test.py [-h] [-c CONFIG] [-o OPTION]
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Path to config file.
-o OPTION, --option OPTION
Option with default value from config file
I have a Python script that can deploy some software in three different environments, let's call them development, testing and production. In order to select which environment the script should work with, I want to use mutually exclusive flags, like so:
./deploy.py --development
./deploy.py --testing
./deploy.py --production
So far I have this:
parser = argparse.ArgumentParser(description="Manage deployment")
group = parser.add_mutually_exclusive_group()
group.add_argument("-d", "--development", action='store_true', required=False)
group.add_argument("-t", "--testing", action='store_true', required=False)
group.add_argument("-p", "--production", action='store_true', required=False)
args = parser.parse_args()
Thing is, I want to have the environment in a single variable, so I can easily check it, instead of having to manually check args.development, args.testing and args.production.
Is there a way of having a common variable that the three arguments could write to so I could do something like args.environment?
Instead of using action='store_true', you can use action='store_const', assign an individual const value for each argument and the dest option of add_argument, so that all the arguments in the mutually exclusive group have the same destination in the object returned by parse_args.
parser = argparse.ArgumentParser(description="Manage deployment")
group = parser.add_mutually_exclusive_group()
group.add_argument("-d", "--development", action='store_const', dest='environment', const='development', required=False)
group.add_argument("-t", "--testing", action='store_const', dest='environment', const='testing', required=False)
group.add_argument("-p", "--production", action='store_const', dest='environment', const='production', required=False)
args = parser.parse_args()
print(args)
The output is:
$ ./test_args.py --production
Namespace(environment='production')
Instead of 3 arguments, you can just use one with choices:
parser = argparse.ArgumentParser(description="Manage deployment")
parser.add_argument("environment", choices=['development', 'testing', 'production'])
args = parser.parse_args()
print(args)
Examples:
>>> test_args.py
usage: concept.py [-h] {development,testing,production}
test_args.py: error: the following arguments are required: environment
>>> test_args.py testing
Namespace(environment='testing')
When I run the following code:
def read_args():
parser = default_parser()
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
I have the following error output:
The following arguments are required: --name
However when I add the --name argument:
def read_args():
parser = default_parser()
parser.add_argument('--name', type=str, default='cifar10test')
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
is also creating problem.
Any ideas?
It appears that default_parser contains a --name argument which is required. What you're doing in your second example is defining the argument twice - once in default_parser and once in your program. Instead, you should be passing a --name argument when calling your program from the command line.
Example:
python cifar.py -N=1200 --tensorboard-dir=file.txt --name=cool_name
Alternatively, you could remove default_parser and construct your own ArgumentParser:
`parser = argparse.ArgumentParser()`
Full working demo:
import argparse
def read_args():
parser = argparse.ArgumentParser()
parser.add_argument('--tensorboard-dir', type=str,
default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000,
help="Use N training examples.")
return parser.parse_args()
def main():
flags = vars(read_args())
# You can access your args as a dictionary
for key in flags:
print("{} is {}".format(key, flags[key]))
main()
The parser returns a Namespace object, but we can access its (much simpler) internal dictionary using vars(Namespace). You can then get your arguments by accessing the dictionary, for example, flags['N']. Note that tensorboard-dir becomes tensorboard_dir inside your python program to avoid issues with the subtraction operator.
Call it from the command line (I'm using Bash):
python cifar.py -N=1200 --tensorboard-dir=file.txt
Output:
tensorboard_dir is file.txt
N is 1200
I am using argparse to parse the Python command line which is supposed to look like this:
python script_name.py --sdks=first, second
My script looks like this:
sdk_choises = ['aio','sw']
parser = argparse.ArgumentParser(description='Blah blah')
parser.add_argument('--sdks', action='append', nargs='+', required=True, help='specifies target SDK(s)')
args = parser.parse_args()
if 'aio' in args.sdks:
# do something with aio
if 'sw' in args.sdks:
# do something with sw
When I execute:
python script_name.py --sdks=aio, sw I get error:
"usage: script.py [-h] --sdks SDKS [SDKS ...]
build.py: error: unrecognized arguments: sw"
I'd like to be able to choose one or all choices:
python script_name.py --sdks=first
python script_name.py --sdks=second
python script_name.py --sdks=first, second
Where did I go wrong?
The following works nice:
import argparse
parser = argparse.ArgumentParser(description='Blah blah')
parser.add_argument('--sdks', nargs='+', required=True, help='specifies target SDK(s)')
args = parser.parse_args()
print(args.sdks)
You don't need the = when passing options, just use:
$ python test.py --sdks ai pw
['ai', 'pw']
If you prefer your original form of the comma-separated list, plus check if the argument is valid, then I recommend:
parser.add_argument('--sdks', nargs=1, type=lambda s: [sdk_choises[sdk_choises.index(f)] for f in s.split(',')], ...
Even cleaner way is to define it in a separate function similar to lambda above:
parser.add_argument('--sdks', nargs=1, type=my_parse_function, ...
argparse docs has examples for the parsing function with proper error reporting.
With nargs=1 you'd need to remove one extra list layer.