Develop Reference

python argparse different value if arg is provided in command line or absent

I want to use python's argparse to config the parameters input from command line. The behaviours for parsing the arguments can be:
None if the name is absent from command line.
Defaults to a value if the name is provided but left empty.
To the provided value.
Example code can be:
parser = argparse.ArgumentParser()
parser.add_argument("-generate-config", metavar="-g", nargs="?")
Parsing it:
>>> parser.parse_args(["-g"]) # leave it empty/didn't provide value
Namespace(generate_config=None)
>>> parser.parse_args(["-g", "config.txt"])
Namespace(generate_config='config.txt')
>>> parser.parse_args([]) # absent in the command line
Namespace(generate_config=None)
So leave it empty or don't provide the argument in the command line, both values are None. How can I set different behaviours/values for these two situations?

After searching stackflow, setting default to argparse.SUPPRESS in add_argument is of great help, as inspired by this answer.
parser = argparse.ArgumentParser()
parser.add_argument("-generate-config", metavar="-g", nargs="?", default=argparse.SUPPRESS)
So:
>>> args = parser.parse_args(["-g"]) # leave it empty/didn't provide value
>>> args
Namespace(generate_config=None)
>>> args = parser.parse_args([]) # the argument is empty
>>> args
Namespace()
>>> args = parser.parse_args(["-g", "parameters.txt"])
>>> args
Namespace(generate_config='parameters.txt')
Though not exactly as what I want, it still can be achieved by checking "generate_config" in args and args.generate_config is None.

Why is there a difference when calling argparse.parse_args() or .parse_args(sys.argv)

I have created the following argument parser in my python code.
parser = argparse.ArgumentParser()
parser.add_argument('projectPath')
parser.add_argument('-project')
parser.add_argument('-release')
parser.add_argument('--test', default=False, action='store_true')
args = parser.parse_args()
and I'm executing my program the following way.
myProgram.py /path/to/file -project super --test
it works fine if I use the sysntax above with
args = parser.parse_args()
However if I take and use the sys.argv as input
args = parser.parse_args(sys.argv)
The parser is suddenly picky about the order of the arguments and I get the unrecognized argument error.
usage: fbu.py [-h] [-project PROJECT] [-release RELEASE] [--test] projectPath
fbu.py: error: unrecognized arguments: /path/to/file
As I can see from the error and also using the -h argument. The path argument must be last and the error makes sense in the last example.
But why does it not care about the order in the first example ?
EDIT: I'm using python version 3.4.3

sys.argv contains the script name as the first item, i.e. myProgram.py. That argument takes the spot of projectPath. Now there's one additional positional argument /path/to/file, which can't be matched to any arguments, hence the error.
Calling parse_args without arguments ArgumentParser is clever enough to omit the script name from being parsed. But when explicitly passing an array of arguments, it can't do that and will parse everything.

As you can see from looking at the source code for parse_known_args (which is called by parse_args):
if args is None:
# args default to the system args
args = _sys.argv[1:]
When you don't provide the arguments explicitly, Python removes the first item from .argv (which is the name of the script). If you pass the arguments manually, you must do this yourself:
parser.parse_args(sys.argv[1:])
This isn't explicitly covered in the documentation, but note that this section doesn't include a script name when calling parse_args manually:
Beyond sys.argv
Sometimes it may be useful to have an ArgumentParser parse arguments
other than those of sys.argv. This can be accomplished by passing a
list of strings to parse_args(). This is useful for testing at the
interactive prompt:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument(
... 'integers', metavar='int', type=int, choices=xrange(10),
... nargs='+', help='an integer in the range 0..9')
>>> parser.add_argument(
... '--sum', dest='accumulate', action='store_const', const=sum,
... default=max, help='sum the integers (default: find the max)')
>>> parser.parse_args(['1', '2', '3', '4'])
Namespace(accumulate=<built-in function max>, integers=[1, 2, 3, 4])
>>> parser.parse_args('1 2 3 4 --sum'.split())
Namespace(accumulate=<built-in function sum>, integers=[1, 2, 3, 4])
The advantage of passing the arguments manually is that it makes it easier to test the parsing functionality, as you can pass in a list of appropriate arguments rather than trying to patch sys.argv.

How do I get back the option string using argparse?

parser = argparse.ArgumentParser()
parser.add_argument("-p", "--pattern", help="Pattern file")
args = parser.parse_args()
Now is it possible to get back the string "--pattern" from args?
I need the string so that I can construct a cmd list to pass to Popen like Popen(['some_other_program', args.pattern.option_string, args.pattern], ...) without repeating it (and having to maintain it in two places) (Popen(['some_other_prog', '--pattern', args.pattern], ...)).
I need to create a wrapper for another program. Some of the args need to be passed to the wrapped program (via Popen) and some are required by the wrapper.
Is there a better method than the following example?
pass_1 = '--to-be-passed'
parser = argparse.ArgumentParser()
parser.add_argument("-p", pass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
...
process = Popen(['wrapped_program', pass_1, args.pass_1], ...)
...
This method of keeping the args in variables is not very good as:
Maintaining short options along with long options becomes difficult.
Popen if called in another function requires passing these variables(or a dict of them) to the function. This seems redundant as args passed to it should be sufficient.

Add a dest to your add_argument call.
parser.add_argmument("p", "--pattern", dest="pattern", help="your help text")
args = parser.parse_args()
args = vars(args)
The you can reference the pattern with args["pattern"] .

There doesn't seem to be an easy way to get the original option strings from the result of a parser.parse_args(), but you can get them from the parser object. You just need to peek into its __dict__, in order to retrieve the parser settings after it's created. In your case you want the _option_string_actions field. Unfortunately this doesn't seem officially supported, as I couldn't find a ArgumentParser method dedicated to this, so YMMV. On Python 3:
Demo:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=int, default=1000, help='intensity of bar')
parser.add_argument('--bar', '-b', type=str, default='bla', help='whatever')
store_action_dict=vars(parser)['_option_string_actions']
print(store_action_dict.keys()) # dict_keys(['--help', '-b', '-f', '-h', '--foo', '--bar'])

The deleted answers and comments indicate there is some confusion as to what you want. So I'll add to that confusion.
Normally the parser does not record the option string. However it is provided to the Action __call__ method. So a custom Action class could save it. The FooAction custom class example in the argparse docs illustrates this.
If I define this action subclass:
In [324]: class PassThru(argparse._StoreAction):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, [values, option_string])
In [324]: p.add_argument('-o','--other',action=PassThru)
The option string is recorded along with the value ('-o' or '--other'):
In [322]: p.parse_args('-p test -o teseting'.split())
Out[322]: Namespace(other=['teseting', '-o'], pass_me_on='test')
In [323]: p.parse_args('-p test --other teseting'.split())
Out[323]: Namespace(other=['teseting', '--other'], pass_me_on='test')
Obviously the option_string and value could be recorded in a different order, in a dictionary, as seperate attributes in the Namespace, etc.
There are other ways of passing options to another program, particularly if the wrapping parser does not need to handle them itself.
argparse gets the arguments from sys.argv[1:], and does not change it. So even if your parser uses some of the arguments, you could pass that list on to popen (all or in part).
The argparse docs has an example, under nargs=REMAINDER, of parsing some arguments for itself, and collecting the rest to pass to another program. This is their example:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
So you could call popen with something like
plist = ['wrapped_program']
plist.extend(args.args)
popen(plist, ...)
Using parse.parse_known_args can also be used to collect unparsed words into an 'extras' list. That section of the docs talks about passing those strings on to another program (just as you are doing). In contrast with the REMAINDER case, the extra stuff does not have to be last.
These work, of course, only if this parser doesn't need --pattern for itself. If it parses it, then it won't appear appear in the REMAINDER or extras. In that case you will have to add it back to the list that you give popen
I would tweak your parser thus:
pass_1 = 'passed' # without the -- or internal -
dpass_` = '--'+pass_
parser = argparse.ArgumentParser()
parser.add_argument("-p", dpass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
process = Popen(['wrapped_program', dpass_1, getattr(args, pass_1)], ...)
another option:
parser = argparse.ArgumentParser()
pass_action = parser.add_argument("-p", '--pass-me-on', help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
If you print pass_action (in a shell) you'll get something like:
_StoreAction(option_strings=['-p', '--pass-me-on'], dest='pass_me_on', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
So you could pull the --name and dest from that object, thus:
process = Popen(['wrapped_program', pass_action.option_strings[-1], getattr(args, pass_action.dest), ...], ...)
You have to look in sys.argv to see which option_string was used (the long, short or other). The parser does not record that anywhere.
Note '--pass-me-on' produced dest='pass_me_on'. The conversion of - to _ can complicate deriving one string from the other.
If you have a dest string, you have to use getattr to pull it from the args namespace, or use vars(args)[dest] (dictionary access).
Another issue. If --patten has nargs='+', its value will be a list, as opposed to a string. You'd have to careful when merging that into thepopen` argument list.

How to use a Python keyword as an argument name for argparse?

lambda has a keyword function in Python:
f = lambda x: x**2 + 2*x - 5
What if I want to use it as a variable name? Is there an escape sequence or another way?
You may ask why I don't use another name. This is because I'd like to use argparse:
parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
parser.add_argument("-l","--lambda",help="Defines the quantity called lambda", type=float)
args = parser.parse_args()
print args.lambda # syntax error!
Script called with --help option gives:
...
optional arguments
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
Because of that, I would like to stay with lambda as the variable name. Solutions may be argparse-related as well.

You can use dynamic attribute access to access that specific attribute still:
print getattr(args, 'lambda')
Better still, tell argparse to use a different attribute name:
parser.add_argument("-l", "--lambda",
help="Defines the quantity called lambda",
type=float, dest='lambda_', metavar='LAMBDA')
Here the dest argument tells argparse to use lambda_ as the attribute name:
print args.lambda_
The help text still will show the argument as --lambda, of course; I set metavar explicitly as it otherwise would use dest in uppercase (so with the underscore):
>>> import argparse
>>> parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
>>> parser.add_argument("-l", "--lambda",
... help="Defines the quantity called lambda",
... type=float, dest='lambda_', metavar='LAMBDA')
_StoreAction(option_strings=['-l', '--lambda'], dest='lambda_', nargs=None, const=None, default=None, type=<type 'float'>, choices=None, help='Defines the quantity called lambda', metavar='LAMBDA')
>>> parser.print_help()
usage: Calculate something with a quantity commonly called lambda.
[-h] [-l LAMBDA]
optional arguments:
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
>>> args = parser.parse_args(['--lambda', '4.2'])
>>> args.lambda_
4.2

There is an argparse-specific way of dealing with this. From the documentation:
If you prefer to have dict-like view of the attributes, you can use
the standard Python idiom, vars().
Therefore, you should be able to write:
print vars(args)["lambda"] # No keyword used, no syntax error.

argparse provides destination functionality for arguments if the long option name is not the desired attribute name for the argument.
For instance:
parser = argparse.ArgumentParser()
parser.add_argument("--lambda", dest="function")
args = parser.parse_args()
print(args.function)

Reserved word as argument in argparse [duplicate]

lambda has a keyword function in Python:
f = lambda x: x**2 + 2*x - 5
What if I want to use it as a variable name? Is there an escape sequence or another way?
You may ask why I don't use another name. This is because I'd like to use argparse:
parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
parser.add_argument("-l","--lambda",help="Defines the quantity called lambda", type=float)
args = parser.parse_args()
print args.lambda # syntax error!
Script called with --help option gives:
...
optional arguments
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
Because of that, I would like to stay with lambda as the variable name. Solutions may be argparse-related as well.

You can use dynamic attribute access to access that specific attribute still:
print getattr(args, 'lambda')
Better still, tell argparse to use a different attribute name:
parser.add_argument("-l", "--lambda",
help="Defines the quantity called lambda",
type=float, dest='lambda_', metavar='LAMBDA')
Here the dest argument tells argparse to use lambda_ as the attribute name:
print args.lambda_
The help text still will show the argument as --lambda, of course; I set metavar explicitly as it otherwise would use dest in uppercase (so with the underscore):
>>> import argparse
>>> parser = argparse.ArgumentParser("Calculate something with a quantity commonly called lambda.")
>>> parser.add_argument("-l", "--lambda",
... help="Defines the quantity called lambda",
... type=float, dest='lambda_', metavar='LAMBDA')
_StoreAction(option_strings=['-l', '--lambda'], dest='lambda_', nargs=None, const=None, default=None, type=<type 'float'>, choices=None, help='Defines the quantity called lambda', metavar='LAMBDA')
>>> parser.print_help()
usage: Calculate something with a quantity commonly called lambda.
[-h] [-l LAMBDA]
optional arguments:
-h, --help show this help message and exit
-l LAMBDA, --lambda LAMBDA
Defines the quantity called lambda
>>> args = parser.parse_args(['--lambda', '4.2'])
>>> args.lambda_
4.2

There is an argparse-specific way of dealing with this. From the documentation:
If you prefer to have dict-like view of the attributes, you can use
the standard Python idiom, vars().
Therefore, you should be able to write:
print vars(args)["lambda"] # No keyword used, no syntax error.

argparse provides destination functionality for arguments if the long option name is not the desired attribute name for the argument.
For instance:
parser = argparse.ArgumentParser()
parser.add_argument("--lambda", dest="function")
args = parser.parse_args()
print(args.function)