Develop Reference

`argparse` multiple choice argument?

I am using argparse to parse the Python command line which is supposed to look like this:
python script_name.py --sdks=first, second
My script looks like this:
sdk_choises = ['aio','sw']
parser = argparse.ArgumentParser(description='Blah blah')
parser.add_argument('--sdks', action='append', nargs='+', required=True, help='specifies target SDK(s)')
args = parser.parse_args()
if 'aio' in args.sdks:
# do something with aio
if 'sw' in args.sdks:
# do something with sw
When I execute:
python script_name.py --sdks=aio, sw I get error:
"usage: script.py [-h] --sdks SDKS [SDKS ...]
build.py: error: unrecognized arguments: sw"
I'd like to be able to choose one or all choices:
python script_name.py --sdks=first
python script_name.py --sdks=second
python script_name.py --sdks=first, second
Where did I go wrong?

The following works nice:
import argparse
parser = argparse.ArgumentParser(description='Blah blah')
parser.add_argument('--sdks', nargs='+', required=True, help='specifies target SDK(s)')
args = parser.parse_args()
print(args.sdks)
You don't need the = when passing options, just use:
$ python test.py --sdks ai pw
['ai', 'pw']

If you prefer your original form of the comma-separated list, plus check if the argument is valid, then I recommend:
parser.add_argument('--sdks', nargs=1, type=lambda s: [sdk_choises[sdk_choises.index(f)] for f in s.split(',')], ...
Even cleaner way is to define it in a separate function similar to lambda above:
parser.add_argument('--sdks', nargs=1, type=my_parse_function, ...
argparse docs has examples for the parsing function with proper error reporting.
With nargs=1 you'd need to remove one extra list layer.

Is it possible to name subparser (from Python argparse) like '--name' or even '-n'?

I have started work with argparse because I have to write Unix-style argument parsing for mailer script.
With 'auto' option, which provides filename mailer, it will execute script received as filename, grabbing prepared mail and just sending it. In second mode --manual, it should be able to build any mail from scratch - one file as content, few attachments, few options to provide mailing list. I really wanted to use --long_opt convention (and maybe -short_opt too) there, but if I understand I should use subparsers.
Is it possible to name subparser like '--name' or even '-n'? For now, I guess not - I have an error.
EXAMPLE USECASES:
(1) python script.py --auto 'anotherScriptName.py'
(2) python script.py --manual --content 'htmlcontent.txt' --attach 'file1.jpg' 'file2.xlsx' --mail-All
(3) python script.py --manual --content 'htmlcontent.txt' --mail-to 'mail1#gmail.com' 'mail2#yahoo.com'
(4) python script.py --manual --content 'htmlcontent.txt' --attach 'file1.txt' --mail-g ALL_DEVELOPERS
ad. 4) ALL_DEVELOPERS could be string or int (key for group in database)
Code below:
parser = argparse.ArgumentParser(description='Standard UNIX-like option parser for customizing script')
subparsers = parser.add_subparsers()
p = subparsers.add_parser('--manual') ### HERE '--manual'
That would be great, but i want to have many arguments to parse here (and there all will be required) and in second option I want just one path to file with script which will do all work:
parser.add_argument('--manual', action='store_true', dest = 'mode', default = False)
What I have:
p.add_argument('--content', action = 'store', dest = 'content', require=True, help = "no help for you, it's just path to html content")
p.add_argument('--attach', action = 'store', dest = 'attachments', nargs='*', help = "attachments")
p.add_argument('--mail', '-m', choices=['-All', '-fromFile', '-group', '-to'], dest = 'mailOpt', require=True)
p.add_argument( dest = 'mails', nargs='*') # 0 args if -All selected, raise error in other case
p2 = subparsers.add_parser('--auto') # Like '--manual'
p2.add_argument('--filename', action='store')
If it is not possbile, should I use _parse_known_args_ function or it's bad idea and just leave names 'manual' and 'auto' because they seems clear enough?

Multiple invocation of the same subcommand in a single command line

I'm trying to figure out how to use argparser to do the following:
$ python test.py executeBuild --name foobar1 executeBuild --name foobar2 ....
getBuild itself is a sub-command. My goal is to have the script have the capability to chain a series of sub-command (executeBuild being one of them) and execute them in order. In the example above, it would execute a build, then setup the environment, then execute build again. How can I accomplish this with argparse? I've tried the following:
main_parser = argparse.ArgumentParser(description='main commands')
subparsers = main_parser.add_subparsers(help='SubCommands', dest='command')
build_parser = subparsers.add_parser('executeBuild')
build_parser.add_argument('--name', action='store', nargs=1, dest='build_name')
check_parser = subparsers.add_parser('setupEnv')
args, extra=main_parser.parse_known_args()
However, it appears that whenever I do this, it goes into the subcommand of executeBuild and report it doesn't know what executeBuild is. I've tried parsing out the extra so I can do a repeat call / chain, however, the first view property appears to have been overwritten, so I can't even just save the extra options and iterate thru.

You are asking argparse something it was not written for : it is good at parsing one command line (but only one) and you want to parse multiple commands in one single line. IMHO, you have to do an initial splitting on your arguments array, and then use argparse on each subcommand. Following function takes a list of arguments (could be sys.argv), skips the first and split remaining in arrays beginning on each known subcommand. You can then use argparse on each sublist :
def parse(args, subcommands):
cmds = []
cmd = None
for arg in args[1:]:
if arg in (subcommands):
if cmd is not None:
cmds.append(cmd)
cmd = [arg]
else:
cmd.append(arg)
cmds.append(cmd)
return cmds
In your example :
parse(['test.py', 'executeBuild', '--name', 'foobar1', 'executeBuild', '--name', 'foobar2'],
('executeBuild',))
=>
[['executeBuild', '--name', 'foobar1'], ['executeBuild', '--name', 'foobar2']]
Limits : subcommands are used as reserved words and cannot be used as option arguments.

Splitting sys.argv before hand is a good solution. But it can also be done while parsing using an argument with nargs=argparse.REMAINDER. This type of argument gets the rest of the strings, regardless of whether they look like flags or not.
Replacing the parse_known_args with this code:
...
build_parser.add_argument('rest', nargs=argparse.REMAINDER)
check_parser.add_argument('rest', nargs=argparse.REMAINDER)
extras = 'executeBuild --name foobar1 setupEnv executeBuild --name foobar2'.split()
# or extras = sys.argv[1:]
while extras:
args = main_parser.parse_args(extras)
extras = args.rest
delattr(args,'rest')
print args
# collect args as needed
prints:
Namespace(build_name=['foobar1'], command='executeBuild')
Namespace(command='setupEnv')
Namespace(build_name=['foobar2'], command='executeBuild')
In the documentation:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
A problem with REMAINDER is that can be too greedy. http://bugs.python.org/issue14174. As a result build_parser and check_parser can't have other positional arguments.
A way around the greedy REMAINDER is to use argparse.PARSER. This is the nargs value that subparsers uses (undocumented). It's like REMAINDER, except that the first string must look like an 'argument' (no '-'), and is matched against choices (if given). PARSER isn't as greedy as REMAINDER, so the subparsers can have other positional arguments.
There's some extra code involving an 'exit' string and dummy parser. This is to get around the fact that the PARSER argument is 'required' (somewhat like nargs='+')
from argparse import ArgumentParser, PARSER, SUPPRESS
main_parser = ArgumentParser(prog='MAIN')
parsers = {'exit': None}
main_parser.add_argument('rest',nargs=PARSER, choices=parsers)
build_parser = ArgumentParser(prog='BUILD')
parsers['executeBuild'] = build_parser
build_parser.add_argument('cmd')
build_parser.add_argument('--name', action='store', nargs=1, dest='build_name')
build_parser.add_argument('rest',nargs=PARSER, choices=parsers, help=SUPPRESS)
check_parser = ArgumentParser(prog='CHECK')
parsers['setupEnv'] = check_parser
check_parser.add_argument('cmd')
check_parser.add_argument('foo')
check_parser.add_argument('rest',nargs=PARSER, choices=parsers, help=SUPPRESS)
argv = sys.argv[1:]
if len(argv)==0:
argv = 'executeBuild --name foobar1 setupEnv foo executeBuild --name foobar2'.split()
argv.append('exit') # extra string to properly exit the loop
parser = main_parser
while parser:
args = parser.parse_args(argv)
argv = args.rest
delattr(args,'rest')
print(parser.prog, args)
parser = parsers.get(argv[0], None)
sample output:
('MAIN', Namespace())
('BUILD', Namespace(build_name=['foobar1'], cmd='executeBuild'))
('CHECK', Namespace(cmd='setupEnv', foo='foo'))
('BUILD', Namespace(build_name=['foobar2'], cmd='executeBuild'))
Another possibility is to use '--' to separate command blocks:
'executeBuild --name foobar1 -- setupEnv -- executeBuild --name foobar2'
However there is problem when there are several '--': http://bugs.python.org/issue13922

Pass a directory with argparse (no type checking needed)

I've written a file crawler and I'm trying to expand it. I want to use argparse to handle settings for the script including passing the starting directory in at the command line.
Example: /var/some/directory/
I have gotten several other arguments to work but I'm unable to pass this directory in correctly. I don't care if it's preceded by a flag or not, (e.g -d /path/to/start/) but I need to make sure that at the very least, this is argument is used as it will be the only mandatory option for the script to run.
Code Sample:
parser = argparse.ArgumentParser(description='py pub crawler...')
parser.add_argument('-v', '--verbose', help='verbose output from crawler', action="store_true")
parser.add_argument('-d', '--dump', help='dumps and replaces existing dictionaries', action="store_true")
parser.add_argument('-f', '--fake', help='crawl only, nothing stored to DB', action="store_true")
args = parser.parse_args()
if args.verbose:
verbose = True
if args.dump:
dump = True
if args.fake:
fake = True

Simply add:
parser.add_argument('directory',help='directory to use',action='store')
before your args = parser.parse_args() line. A simple test from the commandline shows that it does the right thing (printing args at the end of the script):
$ python test.py /foo/bar/baz
Namespace(directory='/foo/bar/baz', dump=False, fake=False, verbose=False)
$ python test.py
usage: test.py [-h] [-v] [-d] [-f] directory
test.py: error: too few arguments

String index out of range when converting to argparse

I changed from optparse to argparse but when I try to run it I get the following error:
if not option_string[0] in self.prefix_chars:
IndexError: string index out of range
My code is:
usage = "%prog -f <fasta TFs> -a <database all> -s <database small> -d <pfam database> [options]"
version = "1.0.1"
description = " "
epilog = " "\
" "
parser = argparse.ArgumentParser(usage=usage, description=description,
version="%prog "+version, epilog=epilog)
# options for running the program
# TF file
parser.add_argument("-f", "", dest="TF", metavar="<file>",
help="input file with transcription factors")
parser.set_defaults(fasta=None)
I can't find where this error comes from, how can fix this?

In argparse you can't pass empty argument strings to add_argument. argparse is trying to find a valid prefix_char (e.g. "-" or "--") in the empty string you pass (""), causing the error. Try this instead:
parser.add_argument("-f", dest="TF", metavar="<file>",
help="input file with transcription factors")

The only way to get this error is by asking for an index that doesn't exist -- in this case, option_string must be empty.