Original post:
If one has an executable mini_program.py that uses argparse with the following structure:
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
opts = parser.parse_args()
if __name__ == "__main__":
main()
How can one create a controller program parent_program.py that uses argparse (I think with subparser?) to have a similar usage to below:
python parent_program.py --help
blah-blah list of programs that can be used
then using the subprogram:
python parent_program.py mini_program --help
-X description
-y description
etc...
How could all of the parameters propagate up from mini_program.py to the parent_program.py?
EDIT (More specific with error message):
The program
import argparse
def main():
parser = argparse.ArgumentParser()
# Subprograms
subprograms = parser.add_subparsers(title="subprograms")
# ============
# mini-program
# ============
parser_miniprogram = subprograms.add_parser("miniprogram")
# Input
parser_miniprogram.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
parser_miniprogram.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
opts = parser.parse_args()
opts_miniprogram = parser_miniprogram.parse_args()
print(opts_miniprogram.__dict__)
if __name__ == "__main__":
main()
Checking to make sure the docs work
# parent program
python parent_program.py --help
usage: parent_program.py [-h] {miniprogram} ...
optional arguments:
-h, --help show this help message and exit
subprograms:
{miniprogram}
# miniprogram
python parent_program.py miniprogram --help
usage: parent_program.py miniprogram [-h] [-X ATTRIBUTE_MATRIX]
[-y TARGET_VECTOR]
optional arguments:
-h, --help show this help message and exit
-X ATTRIBUTE_MATRIX, --attribute_matrix ATTRIBUTE_MATRIX
Input: Path/to/Tab-separated-value.tsv
-y TARGET_VECTOR, --target_vector TARGET_VECTOR
Input: Path/to/Tab-separated-value.tsv
Trying to run it:
python parent_program.py miniprogram -X ../../Data/X_iris.noise_100.tsv.gz -y ../../Data/y_iris.tsv
usage: parent_program.py miniprogram [-h] [-X ATTRIBUTE_MATRIX]
[-y TARGET_VECTOR]
parent_program.py miniprogram: error: unrecognized arguments: miniprogram
The parent program could have code like
import mini_program
import sys
<do its own parsing>
if 'use_mini':
<modify sys.argv>
mini_program.main()
As written, importing mini_program doesn't run its parser. But calling its main will, but using the list it finds in sys.argv.
The parent parser should be written in a way that it accepts arguments that it needs, and doesn't choke on inputs the mini wants, '-X' and '-y'. It would then puts those 'extra' values in a modified sys.argv, which the mini parser can handle.
parse_known_args is one way of accepting unknown arguments,
https://docs.python.org/3/library/argparse.html#partial-parsing
nargs=argparse.REMAINDER, https://docs.python.org/3/library/argparse.html#nargs, is another way of collecting remaining arguments for passing on.
If mini main was written as:
def main(argv=None):
parser = argparse.ArgumentParser()
parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
opts = parser.parse_args(argv)
it could be called with
mini_program.main(['-X', 'astring','-y','another'])
that is, with an explicit argv list, instead of working through sys.argv.
Keeping the main parser from responding to a '-h' help could be tricky. subparsers is probably the cleanest way of doing that.
You could combine subparsers with the invocation of a the mini main. I won't try to work out those details now.
Another way to define the main is:
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
return parser
And use it as
opts = main().parse_args()
opts = mini_program.main().parse_args()
in other words, use main to define the parser, but delay the parsing.
My actual solution was an adaptation to the above:
# Controller
def main(argv=None):
parser = argparse.ArgumentParser(prog="parent_program", add_help=True)
parser.add_argument("subprogram")
opts = parser.parse_args(argv)
return opts.subprogram
# Initialize
if __name__ == "__main__":
# Get the subprogram
subprogram = main([sys.argv[1]])
module = importlib.import_module(subprogram)
module.main(sys.argv[2:])
Related
I went through a "little" issue using python argument parser, I've created a parser like shown below:
parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument('-l', '--log-level', help="log level")
parser.add_argument('-o', '--out-dir', help="output directory")
args, remaining = parser.parse_known_args()
outpu_dir = args.out_dir
parser.add_argument('-f', '--log-file', help = "log file directory")
args = parser.parse_args()
and the problem is that when calling the program with --help option, arguments added after parse_known_args() are not listed in the menu, I checked other topics related to this, but my case is a bit different! could any one give a help here please ?
Any code which is below the call to parse_known_args will not execute at all.
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('-a')
parser.parse_known_args()
1/0
When running with --help this generates
usage: main.py [-h] [-a A]
optional arguments:
-h, --help show this help message and exit
-a A
Process finished with exit code 0
and not a ZeroDivisionError exception.
You can work around it by adding the help option after all the other args have been added & you've parsed the known args.
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter, SUPPRESS
# don't add help right now
parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter, add_help=False)
parser.add_argument('-l', '--log-level', help="log level")
parser.add_argument('-o', '--out-dir', help="output directory")
args, remaining = parser.parse_known_args()
outpu_dir = args.out_dir
parser.add_argument('-f', '--log-file', help="log file directory")
# add help later, when all other args have been added
parser.add_argument('-h', '--help', action='help', default=SUPPRESS,
help='Show this help message and exit.')
args = parser.parse_args()
Which results in:
(venv) ➜ python main.py --help
usage: main.py [-l LOG_LEVEL] [-o OUT_DIR] [-f LOG_FILE] [-h]
optional arguments:
-l LOG_LEVEL, --log-level LOG_LEVEL
log level (default: None)
-o OUT_DIR, --out-dir OUT_DIR
output directory (default: None)
-f LOG_FILE, --log-file LOG_FILE
log file directory (default: None)
-h, --help Show this help message and exit.
But it's much better if you can restructure your code to add all the argusments first before doing any parsing (i.e. avoid the call to parse_known_aergs before any other add_argument calls)
I have a code that needs to build its command line arguments dynamically, based on configuration file. Schematically, what I end up doing is
parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", type=str, default="",
help="Path to config file.")
args, unknown = parser.parse_known_args()
#here do stuff with args.config to extend parser list of arguments, and then:
parser.parse_args()
The argument management seems to work perfectly well but the trouble I have is that --help will exit at the first call parse_known_args, instead of the second parser.parse_args() which would have shown all the dynamically added arguments.... Is there a way to solve this?
I had the same problem some time ago. It can be solved by using two parsers.
A first pre-parser, with add_help=False to determine the configuration file;
The second full parser with help.
# test.py
import argparse
# First pre-parser to determine the configuration file
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument("-c", "--config", default="", help="Path to config file.")
args = parser.parse_known_args()
# Do something with args.config
# Full and real parser
parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config", default="", help="Path to config file.")
parser.add_argument("-o", "--option", help="Option with default value from config file")
parser.parse_args()
This result in:
$ python3 test.py --help
usage: test.py [-h] [-c CONFIG] [-o OPTION]
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Path to config file.
-o OPTION, --option OPTION
Option with default value from config file
I have this code as the test
import argparse
def main():
parser = argparse.ArgumentParser("test", description="subparser help test")
commands = parser.add_subparsers(dest="command", title="Commands")
subparser_a = commands.add_parser("parser_a", description="description_of_parser_a")
subparser_a.add_argument("--foo")
subparser_a.add_argument("--bar")
subparser_b = commands.add_parser("parser_b", description="description_of_parser_b")
subparser_b.add_argument("--foo-b")
subparser_b.add_argument("--bar-b")
args = parser.parse_args()
if __name__ == '__main__':
main()
but when i run python test.py --help, it shows something like this
usage: test [-h] {parser_a,parser_b} ...
subparser help test
optional arguments:
-h, --help show this help message and exit
Commands:
{parser_a,parser_b}
but what i expect it to print is something like this
usage: test [-h] {parser_a,parser_b} ...
subparser help test
optional arguments:
-h, --help show this help message and exit
Commands:
parser_a description_of_parser_a
parser_b description_of_parser_b
is their any workaround or way to do that?
You have to change your code like below. description= -> help= in add_parser
import argparse
def main():
...
subparser_a = commands.add_parser("parser_a", help="description_of_parser_a")
...
subparser_b = commands.add_parser("parser_b", help="description_of_parser_b")
...
args = parser.parse_args()
if __name__ == '__main__':
main()
I'm trying to write a script that would take some flags and files as arguments and then execute other scripts, depend on the flag that the user chooses. For example, the command line should look like that:
main_script.py -flag1 -file_for_flag_1 another_file_for_flag_1
and
main_script.py -flag2 -file_for_flag_2
I tried to use the argparse library, but I don't know how to take the input files as arguments for the next steps and manipulate them as I want. I started with:
parser = argparse.ArgumentParser(description="Processing inputs")
parser.add_argument(
"-flat_map",
type=str,
nargs="+",
help="generates a flat addressmap from the given files",
)
parser.add_argument(
"-json_convert",
type=str,
nargs="+",
help="generates a flat addressmap from the given files",
)
args = parser.parse_args(args=["-flat_map"])
print(args)
I printed args in the end to see what I get from it but I got nothing I can work with.
Would like to have some guidance. Thanks.
You can convert the args to a dict (where the key is the arg option and the value is the arg value) if it's more convenient for you:
args_dict = {key: value for key, value in vars(parser.parse_args()).items() if value}
Using argparse you can use sub-commands to select sub-modules:
import argparse
def run_command(parser, args):
if args.command == 'command1':
# add code for command1 here
print(args)
elif args.command == 'command2':
# add code for command2 here
print(args)
parser = argparse.ArgumentParser(
prog='PROG',
epilog="See '<command> --help' to read about a specific sub-command."
)
subparsers = parser.add_subparsers(dest='command', help='Sub-commands')
A_parser = subparsers.add_parser('command1', help='Command 1')
A_parser.add_argument("--foo")
A_parser.add_argument('--bar')
A_parser.set_defaults(func=run_command)
B_parser = subparsers.add_parser('command2', help='Command 2')
B_parser.add_argument('--bar')
B_parser.add_argument('--baz')
B_parser.set_defaults(func=run_command)
args = parser.parse_args()
if args.command is not None:
args.func(parser, args)
else:
parser.print_help()
This generates a help page like so:
~ python args.py -h
usage: PROG [-h] {command1,command2} ...
positional arguments:
{command1,command2} Sub-commands
command1 Command 1
command2 Command 2
optional arguments:
-h, --help show this help message and exit
See '<command> --help' to read about a specific sub-command.
and help text for each sub-command:
~ python args.py B -h
arg.py command2 -h
usage: PROG command2 [-h] [--bar BAR] [--baz BAZ]
optional arguments:
-h, --help show this help message and exit
--bar BAR
--baz BAZ
I'm writing a program that use argparse, for parsing some arguments that I need.
for now I have this:
parser.add_argument('--rename', type=str, nargs=2, help='some help')
when I run this script I see this:
optional arguments:
-h, --help show this help message and exit
--rename RENAME RENAME
some help
How can I change my code in that way that the help "page" will show me:
--rename OLDFILE NEWFILE
Can I then use OLDFILE and NEWFILE value in this way?
args.rename.oldfile
args.rename.newfile
If you set metavar=('OLDFILE', 'NEWFILE'):
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--rename', type=str, nargs=2, help='some help',
metavar=('OLDFILE', 'NEWFILE'))
args = parser.parse_args()
print(args)
Then test.py -h yields
usage: test.py [-h] [--rename OLDFILE NEWFILE]
optional arguments:
-h, --help show this help message and exit
--rename OLDFILE NEWFILE
some help
You can then access the arguments with
oldfile, newfile = args.rename
If you really want to access the oldfile with args.rename.oldfile
you could set up a custom action:
import argparse
class RenameAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest,
argparse.Namespace(
**dict(zip(('oldfile', 'newfile'),
values))))
parser = argparse.ArgumentParser()
parser.add_argument('--rename', type=str, nargs=2, help='some help',
metavar=('OLDFILE', 'NEWFILE'),
action=RenameAction)
args = parser.parse_args()
print(args.rename.oldfile)
but it extra code does not really seem worth it to me.
Read the argparse documentation (http://docs.python.org/2.7/library/argparse.html#metavar):
Different values of nargs may cause the metavar to be used multiple times. Providing a tuple to metavar specifies a different display for each of the arguments:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('-x', nargs=2)
>>> parser.add_argument('--foo', nargs=2, metavar=('bar', 'baz'))
>>> parser.print_help()
usage: PROG [-h] [-x X X] [--foo bar baz]
optional arguments:
-h, --help show this help message and exit
-x X X
--foo bar baz