I have a python program that runs depending on some parameters. Let's say, one of the parameters is C with a default value of 3. So when I run it without any arguments, it does
$ python parsing.py
C=3
When I load a file for my initial data, it can get some parameter values from that file. For example if my file MakeC5 says that the program should run with C=5, I will get
$ python parsing.py --file=MakeC5
C=5
On the other hand, if I specify a different value for C as optional argument, that value is taken, and has priority over the value in the file.
$ python parsing.py --C=4
C=4
$ python parsing.py --file=MakeC5 --C=4
C=4
Up to here, I can just check if the value specified on the command line is different from the default value and otherwise take the one from the file, as with
if parameters.C == parser.get_default('C'):
parameters.C = load_file(parameters.file)["C"]
But this approach does not work if I give the default value for C on the command line as in
$ python parsing.py --file=MakeC5 --C=3
C=3
How do I get that case right? Is there a way that does not need to parse the command line twice as in
parameters = parser.parse_args()
parameters_from_file = load_file(parameters.file)
parser.set_defaults(**parameters_from_file)
parameters = parser.parse_args()
which does not look like “the obvious pythonic way to do this” to me? This is quasi reading a configuration file that can be specified as argument, so I expect there to be a good way.
Assuming you use the argparse module, simply don't set a default argument when you add the argument. Then the attribute will be None if the argument was not present, and you can check for that instead of parsing arguments twice:
parser.add_argument('--C' , action='store', dest='C')
parser.add_argument('--file', action='store', dest='file')
# ...
args = parser.parse_args()
# ...
if args.C is None:
# Argument `--C` not given on command line
if args.file is not None:
# Load the file, set `args.C` if the file contains the `C` option
...
if args.C is None:
# `C` was neither specified on the command line argument given,
# nor in the configuration file, set default value
args.C = 3
print 'C =', args.C
Within argsparse there is fromfile-prefix-chars and it would seem to do some of what you wish if you can live with it's limitations.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', default="ABC")
args = parser.parse_args()
print '#1', args.foo
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', default="ABC")
args = parser.parse_args(['--foo', 'BAR' ])
print '#2', args.foo
import argparse
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('--foo', default="ABC")
args = parser.parse_args(['--foo', 'BAR', '#myargs' ])
print '#3', args.foo
import argparse
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('--foo', default="ABC")
args = parser.parse_args(['#myargs', '--foo', 'BAR' ])
print '#4', args.foo
myargs
--foo
FISH
Output
#1 ABC
#2 BAR
#3 FISH
#4 BAR
Notice how the position of #myargs on the command line affects the value of foo, when it's at the front it's valued if overriden by --foo and vice versa.
This would seem to do what you want where:
default-value < config-file < cli-args
Although it relies on you remembering to put the config file in the right order, unless of course you take the args and reorder them so that they first in the args.
Hopefully food for thought.
Related
Optparse, the old version just ignores all unrecognised arguments and carries on. In most situations, this isn't ideal and was changed in argparse. But there are a few situations where you want to ignore any unrecognised arguments and parse the ones you've specified.
For example:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', dest="foo")
parser.parse_args()
$python myscript.py --foo 1 --bar 2
error: unrecognized arguments: --bar
Is there anyway to overwrite this?
Replace
args = parser.parse_args()
with
args, unknown = parser.parse_known_args()
For example,
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
args, unknown = parser.parse_known_args(['--foo', 'BAR', 'spam'])
print(args)
# Namespace(foo='BAR')
print(unknown)
# ['spam']
You can puts the remaining parts into a new argument with parser.add_argument('args', nargs=argparse.REMAINDER) if you want to use them.
Actually argparse does still "ignore" _unrecognized_args. As long as these "unrecognized" arguments don't use the default prefix you will hear no complaints from the parser.
Using #anutbu's configuration but with the standard parse.parse_args(), if we were to execute our program with the following arguments.
$ program --foo BAR a b +cd e
We will have this Namespaced data collection to work with.
Namespace(_unrecognized_args=['a', 'b', '+cd', 'e'], foo='BAR')
If we wanted the default prefix - ignored we could change the ArgumentParser and decide we are going to use a + for our "recognized" arguments instead.
parser = argparse.ArgumentParser(prefix_chars='+')
parser.add_argument('+cd')
The same command will produce
Namespace(_unrecognized_args=['--foo', 'BAR', 'a', 'b'], cd='e')
Put that in your pipe and smoke it =)
nJoy!
I have a script that consumes command line arguments and I would like to implement two argument-passing schemes, namely:
Typing the arguments out at the command line.
Storing the argument list in a file, and passing the name of this file to the program via the command line.
To that end I am passing the argument fromfile_prefix_chars to the ArgumentParser constructor.
script.py
from argparse import ArgumentParser
parser = ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('filename', nargs='?')
parser.add_argument('--foo', nargs='?', default=1)
parser.add_argument('--bar', nargs='?', default=1)
args = parser.parse_args()
print(args)
args.txt
--foo
2
--bar
2
Sample use cases
$ python script.py --foo 3
Namespace(bar=1, filename=None, foo='3')
$ python script.py #args.txt --foo 3
Namespace(bar='2', filename=None, foo='3')
I was expecting that args.filename would retain the name of the file, but surprinsingly enough it has the value None instead. I am aware that I could get the file name from sys.argv through a bit of processing. Is there a cleaner way (ideally an argparse-based approach) to elicit the name of the arguments file?
Your script.py, plus the file. I have added the file name to the file itself.
args.txt
args.txt
--foo
2
--bar
2
testing:
1803:~/mypy$ python3 stack56811067.py --foo 3
Namespace(bar=1, filename=None, foo='3')
1553:~/mypy$ python3 stack56811067.py #args.txt --foo 3
Namespace(bar='2', filename='args.txt', foo='3')
From my testing, using fromfile_prefix_chars means that argparse will not actually pass the argument to your program. Instead, argparse sees the #args.txt, intercepts it, reads from it, and passes the arguments without #args.txt to your program. This is presumably because most people don't really need the filename, just need the arguments within, so argparse saves you the trouble of creating another argument to store something you don't need.
Unfortunately, all of the arguments are stored as local variables in argparse.py, so we cannot access them. I suppose that you could override some of argparse's functions. Keep in mind that this is a horrible, disgusting, hacky solution and I feel that parsing sys.argv is 100% better.
from argparse import ArgumentParser
# Most of the following is copied from argparse.py
def customReadArgs(self, arg_strings):
# expand arguments referencing files
new_arg_strings = []
for arg_string in arg_strings:
# for regular arguments, just add them back into the list
if not arg_string or arg_string[0] not in self.fromfile_prefix_chars:
new_arg_strings.append(arg_string)
# replace arguments referencing files with the file content
else:
try:
fn = arg_string[1:]
with open(fn) as args_file:
# What was changed: before was []
arg_strings = [fn]
for arg_line in args_file.read().splitlines():
for arg in self.convert_arg_line_to_args(arg_line):
arg_strings.append(arg)
arg_strings = self._read_args_from_files(arg_strings)
new_arg_strings.extend(arg_strings)
except OSError:
err = _sys.exc_info()[1]
self.error(str(err))
# return the modified argument list
return new_arg_strings
ArgumentParser._read_args_from_files = customReadArgs
parser = ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('filename', nargs='?')
parser.add_argument('--foo', nargs='?', default=1)
parser.add_argument('--bar', nargs='?', default=1)
args = parser.parse_args()
print(args)
Just for the record, here's a quick and dirty solution I came up with. It basically consists in creating a copy of parser and set its from_file_prefix_chars attribute to None:
import copy
parser_dupe = copy.copy(parser)
parser_dupe.fromfile_prefix_chars = None
args_raw = parser_dupe.parse_args()
if args_raw.filename:
args.filename = args_raw.filename[1:]
I have read quite a few questions and answers on how to define, parse and run multiple subparsers to run sth like
tool.py func_a -a 12 func_b -b 15 input.txt output.txt
^-- main parser args
^--------------- subparser b
^---------------------------- subparser a
They usually suggest something like:
def func_a(args):
pass
def func_b(args):
pass
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
# Define subparsers
subp_a = subparsers.add_parser(func_a.__name__)
subp_a.set_defaults(func=func_a)
subp_a.add_argument('-a')
subp_b = subparsers.add_parser(func_b.__name__)
subp_b.set_defaults(func=func_b)
subp_b.add_argument('-b')
# Define global parameters
parser.add_argument('input', type=argparse.FileType('r'))
parser.add_argument('output', type=argparse.FileType('wb', 0))
# Parse and run through all arguments
rest = sys.argv[1:]
while rest:
args, rest = parser.parse_known_args(rest)
args.func(args)
However this implementation has one problem: The arguments input and output are defined for the main parser and should only be used once. However, each call of parser.parse_known_args(rest) expects the values to be set and consequently removes it from rest.
This means the first call to parse_known_args retrieves the values and each subsequent call fails due to missing arguments.
Is there a solution to overcome this without manually copying the values into the rest list?
Yes, the first parse_known_args consumes the filenames along with the 1st subparser level.
You could define 2 parsers, one with the 'global' positionals, one without. First parse with the 'with' parser, hanging onto its args (for the filenames). Then work your way down the subparser stack with the 'without' parser.
Here's a simpler example:
parser = argparse.ArgumentParser()
parser.add_argument('foo')
subp = parser.add_subparsers(dest='cmd')
cmd1 = subp.add_parser('cmd1')
cmd1.add_argument('-a')
cmd2 = subp.add_parser('cmd2')
# parser.add_argument('bar') # confusing location
parser1 = argparse.ArgumentParser()
subp = parser1.add_subparsers(dest='cmd')
cmd1 = subp.add_parser('cmd1', parents=[cmd1], add_help=False)
cmd2 = subp.add_parser('cmd2')
# Parse and run through all arguments
args, rest = parser.parse_known_args()
print args, rest
while rest:
args, rest = parser1.parse_known_args(rest)
print args, rest
I put the parser foo before subp because it creates less confusion there. A positional at the end (bar) may be confused with arguments for one of the subparsers, or one of the nested subparsers - especially when generating error messages.
To get an idea of how parse_args will allocate argument strings to the various positionals, try a script like:
parser = argparse.ArgumentParser()
parser.add_argument('foo')
parser.add_argument('cmd', nargs=argparse.PARSER, choices=['cmd1','cmd2'])
parser.add_argument('bar')
print parser.parse_args()
To parser, the subparsers argument looks just a like a positional that takes at least one string (kind of like '+'), and the 1st must match choices. It will receive all the 'remaining' strings, consistent with the demands of the other positionals.
I am trying to make a Python program that uses the argparse module to parse command-line options.
I want to make an optional argument that can either be named or positional. For example, I want myScript --username=batman to do the same thing as myScript batman. I also want myScript without a username to be valid. Is this possible? If so, how can it be done?
I tried various things similar to the code below without any success.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin")
group.add_argument("user-name", default="admin")
args = parser.parse_args()
EDIT: The above code throws an exception saying ValueError: mutually exclusive arguments must be optional.
I am using Python 2.7.2 on OS X 10.8.4.
EDIT: I tried Gabriel Jacobsohn's suggestion but I couldn't get it working correctly in all cases.
I tried this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin", nargs="?")
group.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)
and running myScript batman would print Namespace(user_name='batman'), but myScript -u batman and myScript --user-name=batman would print Namespace(user_name='admin').
I tried changing the name user-name to user_name in the 1st add_argument line and that sometimes resulted in both batman and admin in the namespace or an error, depending on how I ran the program.
I tried changing the name user_name to user-name in the 2nd add_argument line but that would print either Namespace(user-name='batman', user_name='admin') or Namespace(user-name='admin', user_name='batman'), depending on how I ran the program.
The way the ArgumentParser works, it always checks for any trailing positional arguments after it has parsed the optional arguments. So if you have a positional argument with the same name as an optional argument, and it doesn't appear anywhere on the command line, it's guaranteed to override the optional argument (either with its default value or None).
Frankly this seems like a bug to me, at least when used in a mutually exclusive group, since if you had specified the parameter explicitly it would have been an error.
That said, my suggested solution, is to give the positional argument a different name.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-u','--username')
group.add_argument('static_username',nargs='?',default='admin')
Then when parsing, you use the optional username if present, otherwise fallback to the positional static_username.
results = parser.parse_args()
username = results.username or results.static_username
I realise this isn't a particularly neat solution, but I don't think any of the answers will be.
Here is a solution that I think does everything you want:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
# Gather all extra args into a list named "args.extra"
parser.add_argument("extra", nargs='*')
args = parser.parse_args()
# Set args.user_name to first extra arg if it is not given and len(args.extra) > 0
if args.user_name == parser.get_default("user_name") and args.extra:
args.user_name = args.extra.pop(0)
print args
If you run myScript -u batman or myScript --user-name=batman, args.user_name is set to 'batman'. If you do myScript batman, args.user_name is again set to 'batman'. And finally, if you just do myScript, args.user_name is set to 'admin'.
Also, as an added bonus, you now have all of the extra arguments that were passed to the script stored in args.extra. Hope this helps.
Try to use the "nargs" parameter of the add_argument method.
This way it works for me.
Now you can add the username twice,
so you have to check it and raise an error, if you want.
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
parser.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)
Optparse, the old version just ignores all unrecognised arguments and carries on. In most situations, this isn't ideal and was changed in argparse. But there are a few situations where you want to ignore any unrecognised arguments and parse the ones you've specified.
For example:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', dest="foo")
parser.parse_args()
$python myscript.py --foo 1 --bar 2
error: unrecognized arguments: --bar
Is there anyway to overwrite this?
Replace
args = parser.parse_args()
with
args, unknown = parser.parse_known_args()
For example,
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
args, unknown = parser.parse_known_args(['--foo', 'BAR', 'spam'])
print(args)
# Namespace(foo='BAR')
print(unknown)
# ['spam']
You can puts the remaining parts into a new argument with parser.add_argument('args', nargs=argparse.REMAINDER) if you want to use them.
Actually argparse does still "ignore" _unrecognized_args. As long as these "unrecognized" arguments don't use the default prefix you will hear no complaints from the parser.
Using #anutbu's configuration but with the standard parse.parse_args(), if we were to execute our program with the following arguments.
$ program --foo BAR a b +cd e
We will have this Namespaced data collection to work with.
Namespace(_unrecognized_args=['a', 'b', '+cd', 'e'], foo='BAR')
If we wanted the default prefix - ignored we could change the ArgumentParser and decide we are going to use a + for our "recognized" arguments instead.
parser = argparse.ArgumentParser(prefix_chars='+')
parser.add_argument('+cd')
The same command will produce
Namespace(_unrecognized_args=['--foo', 'BAR', 'a', 'b'], cd='e')
Put that in your pipe and smoke it =)
nJoy!