I would like to have the following syntax:
python utility.py file1 FILE1 file2 FILE2
where file1 and file2 are optional arguments.
It is simple to make it working with this syntax:
python utility.py --file1 FILE1 --file2 FILE2
using
parser.add_argument('--file1',type=file)
parser.add_argument('--file2',type=file)
however, if I remove the dashes, argparse starts to interprete it as a positional rather than optional argument...
In other words, is it possible to specifically tell argparse whether an argument is optional or positional so that I can have optional parameters without the dashes?
There is no way to get argparse to do this for you. However, you can make argparse accept any number of positional arguments:
parser.add_argument('FILES',nargs='*')
options=parser.parse_args()
file1,optional_files=options.FILES[0],options.FILES[1:]
Of course, you may want to add some checks to make sure that at least 1 file was given, etc.
EDIT
I'm still not 100% sure what you want here, but if file1 and file2 are literal strings, you can work around that a little bit by preprocessing sys.argv. Of course, this will still format your help message strangely, but you can always add an epilog explaining that either form is OK:
import argparse
import sys
mangle_args=('file1','file2')
arguments=['--'+arg if arg in mangle_args else arg for arg in sys.argv[1:]]
parser=argparse.ArgumentParser()
parser.add_argument('--file1')
parser.add_argument('--file2')
options=parser.parse_args(arguments)
Another Example would be:
train.py
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Arguments for wake_word")
parser.add_argument('data', type=str, help="path to data")
parser.add_argument('output', type=str, help="model save path")
parser.add_argument('batch_size', type=int, help="batch size")
parser.add_argument('epochs', type=int, help="no.s of epochs")
args = parser.parse_args()
print(args.data + args.output + args.batch_size + args.epochs)
then you can just run this code with arguments without dash
train.py /path/to/data/ /path/to/output_files/ 128 100
And, in ascending order
Had same problem. My workaround is:
lastarg = sys.argv[-1]
if len(sys.argv) > 1 and lastarg[0] != '-':
sys.argv[-1] = '-file'
sys.argv.append(lastarg)
argparser = argparse.ArgumentParser()
argparser.add_argument('-d', action='store_true')
argparser.add_argument('-nrv', action='store_true')
argparser.add_argument('-file', type=str, default='')
args = argparser.parse_args()
Related
In this example script
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--modes', help="test", nargs='+', type=str)
args = parser.parse_args()
write_mode = args.modes
print(write_mode)
There is only one argument modes.
However, python test.py --modes sdfsf and python test.py --mode sdfsf give me the same output (['sdfsf']) which means the parser treats mode as modes.
Is this a bug of argparse?
This is an example of prefix matching, which is allowed by argparse by default.
Turn it off by using argparse.ArgumentParser(..., allow_abbrev=False).
I have a script that consumes command line arguments and I would like to implement two argument-passing schemes, namely:
Typing the arguments out at the command line.
Storing the argument list in a file, and passing the name of this file to the program via the command line.
To that end I am passing the argument fromfile_prefix_chars to the ArgumentParser constructor.
script.py
from argparse import ArgumentParser
parser = ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('filename', nargs='?')
parser.add_argument('--foo', nargs='?', default=1)
parser.add_argument('--bar', nargs='?', default=1)
args = parser.parse_args()
print(args)
args.txt
--foo
2
--bar
2
Sample use cases
$ python script.py --foo 3
Namespace(bar=1, filename=None, foo='3')
$ python script.py #args.txt --foo 3
Namespace(bar='2', filename=None, foo='3')
I was expecting that args.filename would retain the name of the file, but surprinsingly enough it has the value None instead. I am aware that I could get the file name from sys.argv through a bit of processing. Is there a cleaner way (ideally an argparse-based approach) to elicit the name of the arguments file?
Your script.py, plus the file. I have added the file name to the file itself.
args.txt
args.txt
--foo
2
--bar
2
testing:
1803:~/mypy$ python3 stack56811067.py --foo 3
Namespace(bar=1, filename=None, foo='3')
1553:~/mypy$ python3 stack56811067.py #args.txt --foo 3
Namespace(bar='2', filename='args.txt', foo='3')
From my testing, using fromfile_prefix_chars means that argparse will not actually pass the argument to your program. Instead, argparse sees the #args.txt, intercepts it, reads from it, and passes the arguments without #args.txt to your program. This is presumably because most people don't really need the filename, just need the arguments within, so argparse saves you the trouble of creating another argument to store something you don't need.
Unfortunately, all of the arguments are stored as local variables in argparse.py, so we cannot access them. I suppose that you could override some of argparse's functions. Keep in mind that this is a horrible, disgusting, hacky solution and I feel that parsing sys.argv is 100% better.
from argparse import ArgumentParser
# Most of the following is copied from argparse.py
def customReadArgs(self, arg_strings):
# expand arguments referencing files
new_arg_strings = []
for arg_string in arg_strings:
# for regular arguments, just add them back into the list
if not arg_string or arg_string[0] not in self.fromfile_prefix_chars:
new_arg_strings.append(arg_string)
# replace arguments referencing files with the file content
else:
try:
fn = arg_string[1:]
with open(fn) as args_file:
# What was changed: before was []
arg_strings = [fn]
for arg_line in args_file.read().splitlines():
for arg in self.convert_arg_line_to_args(arg_line):
arg_strings.append(arg)
arg_strings = self._read_args_from_files(arg_strings)
new_arg_strings.extend(arg_strings)
except OSError:
err = _sys.exc_info()[1]
self.error(str(err))
# return the modified argument list
return new_arg_strings
ArgumentParser._read_args_from_files = customReadArgs
parser = ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('filename', nargs='?')
parser.add_argument('--foo', nargs='?', default=1)
parser.add_argument('--bar', nargs='?', default=1)
args = parser.parse_args()
print(args)
Just for the record, here's a quick and dirty solution I came up with. It basically consists in creating a copy of parser and set its from_file_prefix_chars attribute to None:
import copy
parser_dupe = copy.copy(parser)
parser_dupe.fromfile_prefix_chars = None
args_raw = parser_dupe.parse_args()
if args_raw.filename:
args.filename = args_raw.filename[1:]
I need to implement a command line interface in which the program accepts subcommands.
For example, if the program is called “foo”, the CLI would look like
foo cmd1 <cmd1-options>
foo cmd2
foo cmd3 <cmd3-options>
cmd1 and cmd3 must be used with at least one of their options and the three cmd* arguments are always exclusive.
I am trying to use subparsers in argparse, but with no success for the moment. The problem is with cmd2, that has no arguments:
if I try to add the subparser entry with no arguments, the namespace returned by parse_args will not contain any information telling me that this option was selected (see the example below).
if I try to add cmd2 as an argument to the parser (not the subparser), then argparse will expect that the cmd2 argument will be followed by any of the subparsers arguments.
Is there a simple way to achieve this with argparse? The use case should be quite common…
Here follows what I have attempted so far that is closer to what I need:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='Functions')
parser_1 = subparsers.add_parser('cmd1', help='...')
parser_1.add_argument('cmd1_option1', type=str, help='...')
parser_2 = subparsers.add_parser(cmd2, help='...')
parser_3 = subparsers.add_parser('cmd3', help='...')
parser_3.add_argument('cmd3_options', type=int, help='...')
args = parser.parse_args()
First of all subparsers are never inserted in the namespace. In the example you posted if you try to run the script as:
$python3 test_args.py cmd1 1
Namespace(cmd1_option1='1')
where test_args.py contain the code you provided (with the import argparse at the beginning and print(args) at the end).
Note that there is no mention to cmd1 only to its argument. This is by design.
As pointed out in the comments you can add that information passing the dest argument to the add_subparsers call.
The usual way to handle these circumstances is to use the set_defaults method of the subparsers:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='Functions')
parser_1 = subparsers.add_parser('cmd1', help='...')
parser_1.add_argument('cmd1_option1', type=str, help='...')
parser_1.set_defaults(parser1=True)
parser_2 = subparsers.add_parser('cmd2', help='...')
parser_2.set_defaults(parser2=True)
parser_3 = subparsers.add_parser('cmd3', help='...')
parser_3.add_argument('cmd3_options', type=int, help='...')
parser_3.set_defaults(parser_3=True)
args = parser.parse_args()
print(args)
Which results in:
$python3 test_args.py cmd1 1
Namespace(cmd1_option1='1', parser1=True)
$python3 test_args.py cmd2
Namespace(parser2=True)
In general different subparser will, most of the time, handle the arguments in completely different ways. The usual pattern is to have different functions to run the different commands and use set_defaults to set a func attribute. When you parse the arguments you simply call that callable:
subparsers = parser.add_subparsers()
parser_1 = subparsers.add_parser(...)
parser_1.set_defaults(func=do_command_one)
parser_k = subparsers.add_parser(...)
parser_k.set_defaults(func=do_command_k)
args = parser.parse_args()
if args.func:
args.func(args)
The subparser identity can be added to the main Namespace if the add_subparsers command is given a dest.
From the documentation:
However, if it is necessary to check the name of the subparser that was invoked, the dest keyword argument to the add_subparsers() call will work:
>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers(dest='subparser_name')
>>> subparser1 = subparsers.add_parser('1')
>>> subparser1.add_argument('-x')
>>> subparser2 = subparsers.add_parser('2')
>>> subparser2.add_argument('y')
>>> parser.parse_args(['2', 'frobble'])
Namespace(subparser_name='2', y='frobble')
By default the dest is argparse.SUPPRESS, which keeps subparsers from adding the name to the namespace.
I am now starting exploring Python, and was testing how arguments can be passed to a script with "argparse".
The way I wrote a sample script was as following, where arguments passed through flags -i and -o are compulsory and flag -u is optional:
#!/usr/bin/python
import sys
import argparse
## set usage options and define arguments
usage = "usage: %prog [options]"
parser = argparse.ArgumentParser(usage)
parser.add_argument("-i", action="store", dest="input", help="input file")
parser.add_argument("-o", action="store", dest="output", help="output file")
parser.add_argument("-u", action="store_true", dest="isunfolded", default=False, help="optional flag")
args = parser.parse_args()
print len(sys.argv)
if len(sys.argv) < 2:
# parser.print_help()
print 'Incorrect number of params'
exit()
else:
print "Correct number of params: ", len(sys.argv)
Running this script:
> ./test-args.py -i a -o b
prints:
5
Correct number of params: 5
I understand the printing statement in the if conditional (5 is higher than 2), however, after reading the argparse documentation (https://docs.python.org/3/library/argparse.html) I still don't quite understand why -i and -o flags are counted as arguments. This behaviour seems to be quite different from e.g. perl Getopt::Std, which I'm more used to.
So, the question is what is the best way of parsing arguments in Python and to evaluate the presence of mandatory arguments (without using required=True)
It gives you 5 because sys.argv contains the raw input passed to python as arguments (the script name and 4 arguments).
You can see argparse as an abstraction for this, so once you use it, you can forget about sys.argv. In most cases it is better not to mix these two methods.
argparse its a nice way to handle arguments, I don't quite get why you don't want to use the required option when that's exactly the way to go. Another alternative is to parse the sys.argv yourself (regex maybe?) and drop argparse altogether.
There's a Python getopt which probably is similar to the Perl one (assuming both are modelled after the C/Unix version).
https://docs.python.org/2/library/getopt.html
In your code, sys.argv is a list of strings from the command line (as interpreted by the shell and interpreter). It is the raw input for any of the parsers ('getopt', 'optparse', 'argparse'). And it is available for your parsing as well. When learning it is a good idea to include a
print sys.argv
line. parser.parse_args() uses this list. sys.argv[0] is used as prog attribute (in the default usage), while sys.argv[1:] (the rest) is parsed according to the rules you define with add_argument. For testing I often like to use parse_args with a custom list of strings, e.g.
print parser.parse_args(['-i', 'input', '-o', 'output', '-u'])
With your definition I'd expect to see something like:
Namespace(input='input', output='output', isunfolded=True)
The parser returns an object (type argparse.Namespace), which has attributes defined by your arguments. Values are usually accessed with expressions like args.input, args.isunfolded. The docs also show how you easily express this as a dictionary.
By long standing UNIX conventions, arguments flagged by strings like '-i' are options, that is they are optional. argparse generalizes this concept by letting you specify a required=True parameter.
Other arguments are positionals. They are interpreted according to their order. And as such they are normally required. What argparse adds is the ability to define those positionals, such as type, nargs, etc. With nargs='?' they are optional. Many of the nargs values are similar to regular expression characters (e.g. +?*). In fact argparse uses a form a regular expression parsing to allocate strings among arguments.
I'd refine your arguments thus (taking advantage of various defaults)
a1 = parser.add_argument("-i", "--input", help="input file") # 'store' is the default
a2 = parser.add_argument("-o", "--output",help="output file") # use the --output as dest
a3 = parser.add_argument("-u", "--isunfolded", action="store_true", help="optional flag")
If input and output were required, I could change them to:
parser.add_argument("input", help="input file") # 'store' is the default
parser.add_argument("output",help="output file") # use the --output as dest
parser.add_argument("-u", "--isunfolded", action="store_true", help="optional flag")
Now input and output are positional arguments, as in test.py -u inputfile outputfile
By using a1 = parser... I can look at the object produced by this statement.
print a1
produces
_StoreAction(option_strings=['-i', '--input'], dest='input', nargs=None, const=None,
default=None, type=None, choices=None, help='input file', metavar=None)
This tells me that a1 is a _StoreAction object (a subclass of argparse.Action). It also displays a number (not all) of its attributes, ones that define its action. A positional, on the other hand, has values like these:
a2 = p.add_argument("output", help="output file")
_StoreAction(option_strings=[], dest='output', nargs=None, const=None,
default=None, type=None, choices=None, help='output file', metavar=None)
It may also be instructive to look at a1.required and a2.required, which are respectively False and True. required is an Action attribute that is not routinely displayed, but is, never the less accessible.
I've pulled all these test values from a parser defined in an interactive shell (Ipython). It's a great way to explore Python and modules like argparse.
After reading other related posts it seems that the best way to do this is as was suggested by #Rufflewind and inspect the args itself:
if not args.input or not args.output:
print 'Incorrect number of params'
exit()
else:
print "Correct number of params"
I am trying to make a Python program that uses the argparse module to parse command-line options.
I want to make an optional argument that can either be named or positional. For example, I want myScript --username=batman to do the same thing as myScript batman. I also want myScript without a username to be valid. Is this possible? If so, how can it be done?
I tried various things similar to the code below without any success.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin")
group.add_argument("user-name", default="admin")
args = parser.parse_args()
EDIT: The above code throws an exception saying ValueError: mutually exclusive arguments must be optional.
I am using Python 2.7.2 on OS X 10.8.4.
EDIT: I tried Gabriel Jacobsohn's suggestion but I couldn't get it working correctly in all cases.
I tried this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-u", "--user-name", default="admin", nargs="?")
group.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)
and running myScript batman would print Namespace(user_name='batman'), but myScript -u batman and myScript --user-name=batman would print Namespace(user_name='admin').
I tried changing the name user-name to user_name in the 1st add_argument line and that sometimes resulted in both batman and admin in the namespace or an error, depending on how I ran the program.
I tried changing the name user_name to user-name in the 2nd add_argument line but that would print either Namespace(user-name='batman', user_name='admin') or Namespace(user-name='admin', user_name='batman'), depending on how I ran the program.
The way the ArgumentParser works, it always checks for any trailing positional arguments after it has parsed the optional arguments. So if you have a positional argument with the same name as an optional argument, and it doesn't appear anywhere on the command line, it's guaranteed to override the optional argument (either with its default value or None).
Frankly this seems like a bug to me, at least when used in a mutually exclusive group, since if you had specified the parameter explicitly it would have been an error.
That said, my suggested solution, is to give the positional argument a different name.
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-u','--username')
group.add_argument('static_username',nargs='?',default='admin')
Then when parsing, you use the optional username if present, otherwise fallback to the positional static_username.
results = parser.parse_args()
username = results.username or results.static_username
I realise this isn't a particularly neat solution, but I don't think any of the answers will be.
Here is a solution that I think does everything you want:
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
# Gather all extra args into a list named "args.extra"
parser.add_argument("extra", nargs='*')
args = parser.parse_args()
# Set args.user_name to first extra arg if it is not given and len(args.extra) > 0
if args.user_name == parser.get_default("user_name") and args.extra:
args.user_name = args.extra.pop(0)
print args
If you run myScript -u batman or myScript --user-name=batman, args.user_name is set to 'batman'. If you do myScript batman, args.user_name is again set to 'batman'. And finally, if you just do myScript, args.user_name is set to 'admin'.
Also, as an added bonus, you now have all of the extra arguments that were passed to the script stored in args.extra. Hope this helps.
Try to use the "nargs" parameter of the add_argument method.
This way it works for me.
Now you can add the username twice,
so you have to check it and raise an error, if you want.
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--user-name", default="admin")
parser.add_argument("user_name", default="admin", nargs="?")
args = parser.parse_args()
print(args)