Develop Reference

Python convert dictionary to argparse

Right now, I have a script that can accept command line arguments using argparse. For example, like this:
#foo.py
def function_with_args(optional_args=None):
parser = argparse.ArgumentParser()
# add some arguments
args = parser.parse_args(optional_args)
# do something with args
However, I'd like to be able to use this function with a dictionary instead, for example with something like this:
def function_using_dict(**kwargs):
# define parser and add some arguments
args = parser.parse_dict_args(kwargs)
# everything else is the same
Note that I have a lot of arguments with default values in argparse which I'd like to use, so the following wouldn't work:
def function_no_default_args(**kwargs):
args = kwargs # not using default values that we add to the parser!

argparse.Namespace is a relatively simple object subclass, with most of its code devoted to displaying the attributes (as print(args) shows). Internally parse_args uses get_attr and set_attr to access the namespace, minimizing the assumptions about attributes names.
When using subparsers, the subparser starts with a 'blank' namespace, and uses the following code to copy its values to the main namespace.
# In case this subparser defines new defaults, we parse them
# in a new namespace object and then update the original
# namespace for the relevant parts.
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
for key, value in vars(subnamespace).items():
setattr(namespace, key, value)
Originally the main namespace was passed to the subparser, eg. parser.parse_known_args(arg_strings, namespace), but the current version lets the subparser defaults take priority.
Handling defaults is a bit complicated. If you don't have any required arguments then
args = parser.parse_args([])
will set all the defaults. Or you could look at the start of parse.parse_known_args to see how defaults are inserted into the namespace at the start of parsing. Just beware that there's an added step at the end of parsing that runs remaining defaults through their respective type functions.

If you are trying to convert the result of parse_args into a dict, you can probably just do this:
kwargs = vars(args)
After your comment, I thought about it. Going to go with your existing function.
#foo.py
def function_with_args_and_default_kwargs(optional_args=None, **kwargs):
parser = argparse.ArgumentParser()
# add some arguments
# add the other arguments
for k, v in kwargs.items():
parser.add_argument('--' + k, default=v)
args = parser.parse_args(optional_args)
# do something with args

Share dest between positional arguments and options

I have this very simple ArgumentParser instance, with an optional positional argument and an option, which writes a constant to the same destination:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', action='store_const', dest='path', const='<all>')
parser.add_argument('path', nargs='?')
# Prints None instead of '<all>'
print(parser.parse_args(['-a']).path)
But no matter what, parsing the command line ['-a'] does not yield a Namespace instance with path set to that constant. Instead, the default from the positional argument is used.
What am I doing wrong?
My use case is that the user should be able to specify a path (actually a list of paths). This list of paths defaults to the current working directory. But instead of using that default, -a can be passed, which should result in some configured root directory to be used. The full code for this part of the argument parser is this:
all_sentinel = object()
parser = argparse.ArgumentParser()
paths_group = parser.add_mutually_exclusive_group()
paths_group.add_argument('-a', action='store_const', dest='paths', const=all_sentinel)
paths_group.add_argument('paths', nargs='*', default=['.'])

A positional with nargs='?' has some special handling of its default (here None).
Normally defaults are assigned to the Namespace at the start of parsing, and overwritten by the actions, such as the optional.
Because an empty list of values satisfies the nargs, that positional is always 'seen'. But rather than assign [] or some other 'blank' to it, the parser assigns the default. So the positional's default overwrites the value set by '-a'.
nargs='*' gets the same kind of special handling.
I suspect that if you had another positional argument before the '-a', that you wouldn't see this effect. The '?*' positional would be processed before the '-a', and not overwrite its value.
Optionals are only processed if the flag occurs. Positionals are always processed, regardless of the nargs. The 'optional' positionals are processed, but with some extra handling of the defaults. But when they are processed relative to flagged arguments can vary.
That's some tricky behavior that I'm aware of simply because I've studied the code in detail, and answered a lot questions here and on the Python bug/issues.
Sharing the dest often does work, but that's more by default than design. It's the result of other design choices. argparse makes no promises in that regard. So if it isn't reliable, don't use it.

How do I get back the option string using argparse?

parser = argparse.ArgumentParser()
parser.add_argument("-p", "--pattern", help="Pattern file")
args = parser.parse_args()
Now is it possible to get back the string "--pattern" from args?
I need the string so that I can construct a cmd list to pass to Popen like Popen(['some_other_program', args.pattern.option_string, args.pattern], ...) without repeating it (and having to maintain it in two places) (Popen(['some_other_prog', '--pattern', args.pattern], ...)).
I need to create a wrapper for another program. Some of the args need to be passed to the wrapped program (via Popen) and some are required by the wrapper.
Is there a better method than the following example?
pass_1 = '--to-be-passed'
parser = argparse.ArgumentParser()
parser.add_argument("-p", pass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
...
process = Popen(['wrapped_program', pass_1, args.pass_1], ...)
...
This method of keeping the args in variables is not very good as:
Maintaining short options along with long options becomes difficult.
Popen if called in another function requires passing these variables(or a dict of them) to the function. This seems redundant as args passed to it should be sufficient.

Add a dest to your add_argument call.
parser.add_argmument("p", "--pattern", dest="pattern", help="your help text")
args = parser.parse_args()
args = vars(args)
The you can reference the pattern with args["pattern"] .

There doesn't seem to be an easy way to get the original option strings from the result of a parser.parse_args(), but you can get them from the parser object. You just need to peek into its __dict__, in order to retrieve the parser settings after it's created. In your case you want the _option_string_actions field. Unfortunately this doesn't seem officially supported, as I couldn't find a ArgumentParser method dedicated to this, so YMMV. On Python 3:
Demo:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=int, default=1000, help='intensity of bar')
parser.add_argument('--bar', '-b', type=str, default='bla', help='whatever')
store_action_dict=vars(parser)['_option_string_actions']
print(store_action_dict.keys()) # dict_keys(['--help', '-b', '-f', '-h', '--foo', '--bar'])

The deleted answers and comments indicate there is some confusion as to what you want. So I'll add to that confusion.
Normally the parser does not record the option string. However it is provided to the Action __call__ method. So a custom Action class could save it. The FooAction custom class example in the argparse docs illustrates this.
If I define this action subclass:
In [324]: class PassThru(argparse._StoreAction):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, [values, option_string])
In [324]: p.add_argument('-o','--other',action=PassThru)
The option string is recorded along with the value ('-o' or '--other'):
In [322]: p.parse_args('-p test -o teseting'.split())
Out[322]: Namespace(other=['teseting', '-o'], pass_me_on='test')
In [323]: p.parse_args('-p test --other teseting'.split())
Out[323]: Namespace(other=['teseting', '--other'], pass_me_on='test')
Obviously the option_string and value could be recorded in a different order, in a dictionary, as seperate attributes in the Namespace, etc.
There are other ways of passing options to another program, particularly if the wrapping parser does not need to handle them itself.
argparse gets the arguments from sys.argv[1:], and does not change it. So even if your parser uses some of the arguments, you could pass that list on to popen (all or in part).
The argparse docs has an example, under nargs=REMAINDER, of parsing some arguments for itself, and collecting the rest to pass to another program. This is their example:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
So you could call popen with something like
plist = ['wrapped_program']
plist.extend(args.args)
popen(plist, ...)
Using parse.parse_known_args can also be used to collect unparsed words into an 'extras' list. That section of the docs talks about passing those strings on to another program (just as you are doing). In contrast with the REMAINDER case, the extra stuff does not have to be last.
These work, of course, only if this parser doesn't need --pattern for itself. If it parses it, then it won't appear appear in the REMAINDER or extras. In that case you will have to add it back to the list that you give popen
I would tweak your parser thus:
pass_1 = 'passed' # without the -- or internal -
dpass_` = '--'+pass_
parser = argparse.ArgumentParser()
parser.add_argument("-p", dpass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
process = Popen(['wrapped_program', dpass_1, getattr(args, pass_1)], ...)
another option:
parser = argparse.ArgumentParser()
pass_action = parser.add_argument("-p", '--pass-me-on', help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
If you print pass_action (in a shell) you'll get something like:
_StoreAction(option_strings=['-p', '--pass-me-on'], dest='pass_me_on', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
So you could pull the --name and dest from that object, thus:
process = Popen(['wrapped_program', pass_action.option_strings[-1], getattr(args, pass_action.dest), ...], ...)
You have to look in sys.argv to see which option_string was used (the long, short or other). The parser does not record that anywhere.
Note '--pass-me-on' produced dest='pass_me_on'. The conversion of - to _ can complicate deriving one string from the other.
If you have a dest string, you have to use getattr to pull it from the args namespace, or use vars(args)[dest] (dictionary access).
Another issue. If --patten has nargs='+', its value will be a list, as opposed to a string. You'd have to careful when merging that into thepopen` argument list.

Python argparse: single-valued argument but allow specified multiple times on command line

In python argparse, is it possible to declare an argument which is just single-valued, instead a list, but allows being specified multiple times, the latest one overrides the earlier ones?
The use case is, I am writing a Python program that reads command line argument from a ~/.xxxrc file, where .xxxrc file has an command line argument per line. I want to allow user override the value in ~/.xxxrc file through command line. My plan is to implicitly adds an #~/.xxxrc to the beginning of argv before passing it to argparse, so that argparse will reads in the file. Now the problem turns into my question above.
Is it possible to achieve this effect? If not, is there any alternative solution to my use case.

The argparse module does that by default. Take a look at this example, where we specify the same flag twice, and the last one wins:
import argparse
parser = argparse.ArgumentParser(description='example')
parser.add_argument('-a', '--add')
options = parser.parse_args(['-a', 'foo', '-a', 'bar'])
print 'add = {}'.format(options.add) # output: add = bar

Yes, you can just create a custom action with an nargs='*' or nargs='+'.
Something like this should work:
class GetLastAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if values:
setattr(namespace, self.dest, values[-1])
parser = argparse.ArgumentParser()
parser.add_argument('-a', nargs='+', action=GetLastAction)

Using argparse to create custom command line formats

I realize that the question is rather general but I didn't know exactly how to ask it for what I am doing, but here goes.
I want to create a tool that allows option in the following format which also uses custom actions:
tool.py {start|stop|restart|configure}
Each of the above commands are mutually exclusive and some can have separate unique options. All will call a custom action (subclassed argparse.Action).
tool.py start
The above will do nothing because no arguments (via "add_argument()") was defined.
I though about making a subparser, but doing so doesn't work initially unless you set default arguments, via "set_defaults()". However, doing this and setting:
class CustomAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
print('Args: %r %r %r' % (namespace, values, option_string))
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser(help="Basic daemon.")
subparsers = parser.add_subparsers()
start_parser = subparsers.add_parser("start")
start_parser.set_defaults(start=True, action=CustomAction)
doesn't seem to kick off the custom action as expected. Below is the output I get:
$ custom_parser.py start
Namespace(action=<class '__main__.BasicAction'>, start=True)
I can see that the values are being assigned, but NOT called.
I basically want to have exclusive parent options that can be specified without child argument but still allow exclusive sub-arguments like so, if desired:
tool.py configure {interval|recipients}
Any ideas?

You can use subparsers coupled with default functions
def start_something():
do_starting_actions()
def stop_something():
do_terminal_actions()
def parse_args():
parser = ArgumentParser()
subparsers = parser.add_subparsers()
start = subparsers.add_parser("start")
start.set_defaults(func=start_something)
stop = subparsers.add_parser("stop")
stop.set_defaults(func=stop_something)
# ...
return parser.parse_args()
def main():
args = parse_args()
args.func()
Then you can call the parser from the command line
mymodule.py start
If you wanted to extend the subparser you could do it like:
start = subparsers.add_parser("start")
start.add_argument("--foo")