Develop Reference

python argparse corss sub-command option

using argparse, i am trying to create an optional argument which is available globally (in all commands and sub-commands).
for instance, setting a --verbose optional argument. in the same way that --help is available by default.
the following snippet only works for me in the non-subcommand
parser.add_argument(
'-v', '--verbose',
help='verbose',
type=bool,
default=False,
action=argparse.BooleanOptionalAction
)
how can it be done?

Each parser, main and sub, gets an automatic help argument (unless you specify add_help=False). Further more a '-h' exits right away after displaying its message. So if the '-h' is before the subcommand string, you see the main help. If after you see that subcommand's help.
To make a command like '-v' available both in the main and the sub parsers, you have to define it in all parsers. The parents can streamline that. But this has problems, as #Alex points out. The default value for the subcommand overrides any value set in the main (default or user).
You can get around this by specifying a different dest for the main and the subs. You can still use the same '-v', but the values will be in different attributes in args.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-v','--verbose', action='store_true')
sp = parser.add_subparsers(dest='cmd')
sp1 = sp.add_parser('cmd1')
sp1.add_argument('-v', '--verbose', action='store_true', dest='subverbose')
args = parser.parse_args()
print(args)
sample runs:
0925:~/mypy$ python3 stack65773318.py -h
usage: stack65773318.py [-h] [-v] {cmd1} ...
positional arguments:
{cmd1}
optional arguments:
-h, --help show this help message and exit
-v, --verbose
0928:~/mypy$ python3 stack65773318.py cmd1 -h
usage: stack65773318.py cmd1 [-h] [-v]
optional arguments:
-h, --help show this help message and exit
-v, --verbose
0928:~/mypy$ python3 stack65773318.py -v
Namespace(cmd=None, verbose=True)
0928:~/mypy$ python3 stack65773318.py -v cmd1
Namespace(cmd='cmd1', subverbose=False, verbose=True)
0928:~/mypy$ python3 stack65773318.py cmd1 -v
Namespace(cmd='cmd1', subverbose=True, verbose=False)
0928:~/mypy$ python3 stack65773318.py -v cmd1 -v
Namespace(cmd='cmd1', subverbose=True, verbose=True)

You probably want to use parents like so:
import argparse
base_parser = argparse.ArgumentParser(add_help=False)
base_parser.add_argument("-v", "--verbose", help="verbose", action="store_true")
parser = argparse.ArgumentParser(parents=[base_parser])
subparsers = parser.add_subparsers()
parser_1 = subparsers.add_parser("sub", parents=[base_parser])
parser_1.add_argument("--foo", help="Some opt")
args = parser.parse_args()
print(args)
What this generates:
~ python args.py -h
usage: args.py [-h] [-v] {sub} ...
positional arguments:
{sub}
optional arguments:
-h, --help show this help message and exit
-v, --verbose verbose
~ python args.py sub -h
usage: args.py sub [-h] [-v] [--foo FOO]
optional arguments:
-h, --help show this help message and exit
-v, --verbose verbose
--foo FOO Some opt
This method requires the -v flag be used after the sub-command, though.
~ python args.py sub -v --foo 9
Namespace(verbose=True, foo='9') # -v is recognised
~ python args.py -v sub --foo 9
Namespace(verbose=False, foo='9') # -v is not recognised

How would I run command lines in Python?

I am playing around with this Instagram ID tracker because my friends are constantly changing usernames so I can never keep up. The code is from here. The only way I am aware of, is that you have to go to any sort of terminal and type a command in this format
$ usage: t.py [-h] -u USERNAME [-p] [-s] [-t]
I've already tried to do some on my own and I recently saw other people running command lines in the Python text editor but it doesn't seem to work for me. This is what I did
import os
import main
import api
os.system('python3 main.py -u <USERNAME>')
There are 2 files, main.py and api.py. I imported both but when I run it, it still says
usage: t.py [-h] -u USERNAME [-p] [-s] [-t]
t.py: error: the following arguments are required: -u/--username
Is there an easier way to go about this?

Behaviour of the docopt module (argument parser)

I currently use the docopt lib for the first time so I surely do something wrong
My script is :
"""prog
Usage:
prog.py (-h | --help)
prog.py (--version)
prog.py -s TAG [-t NB_NUC]
Options:
-h, --help help
--version version
-s TAG Some TAG I want.
-t NB_NUC A number of nuc.
"""
If I write:
python prog.py -s SMT
I get:
{'--help': False,
'--version': False,
'-h': False,
'-s': True,
'-t': True,
'NB_NUC': None,
'TAG': 'SMT'}
And it seems to be correct, but if I write :
python prog.py -s -t 10 -> TAG contain 10 (instead of None)
python prog.py -t 10 -s SMT -> TAG contain always 10 (instead of SMT) and NB_NUC contain SMT (instead of 10)
python prog.py -s SMT -t -> TAG contain SMT and NB_NUC contain None (and its what I expected on this way)
So, I tried lot a combination, but I don't understand how this is supposed to word...
What I want is TAG always contains the values which correspond with the -s argument, with None or an error if nothing is given after -s, and I don't understand why it's not the case..
Thanks for your help !

Your are almost there, just need the "<...>" around the arguments:
"""prog
Usage:
prog.py (-h | --help)
prog.py (--version)
prog.py -s TAG [-t NB_NUC]
Options:
-h, --help help
--version version
-s TAG Some TAG I want.
-t NB_NUC A number of nuc.
"""

The problem came from the fact that previous version of docopt didn't work with tabulated indentation.
Actual version does, and the PEP8 recommend usage of spaces anyway.
And for the formating the easiest way is to only write
Usage:
prog.py (-h | --help)
prog.py (-v | --version)
prog.py [options] <mandatory_file>
And to put the differents options and their descriptions in the Options part.

Formatting Usage with Docopt

I am having a problem with my usage statements in docopt.
This is how I'd expect usage to work in the script. The optional parameters (defined with []), I would like to be able to use them together or individually. So -t -o or -o or -t should be valid. At the moments I cant use -o without -t.
If i use pipe | to separate them I can't use both at the same time. I've tried various combinations. I cant seem to get it work as id like. Can anyone point out where I am going wrong?
"""
Description:
Script does stuff
Usage:
script.py (-d <ditem>) (-u <uitem>) (-p <pitem>) (-s <sfile>) [-t <tfile>] [-o <ofile>] [-v]
script.py (-d <ditem>) (-l) [-t <tfile>] [-o <ofile>] [-v]
script.py -h | --help
script.py --version
Options:
-v --verbose Does stuff
-t --tfile Does stuff
-o --output Does stuff
-l --litem Does stuff
-u --uitem Does stuff
-p --pitem Does stuff
-d --ditem Does stuff
-s --sitem Does stuff
-h --help Show this screen.
--version Show version.
"""

I was able to resolve this by using the following:
By adding the usage strings script.py (-d <ditem>) (-l) ([-t <tfile>] | [-o <ofile>]) [-v] and another script.py (-d <ditem>) (-l) [-t <tfile>] [-o <ofile>] [-v] means I can use -t and -o independently or -t -o together. However Im not able to use them in this order -o -t.
Description:
Script does stuff
Usage:
script.py (-d <ditem>) (-u <uitem>) (-p <pitem>) (-s <sfile>) [-t <tfile>] [-o <ofile>] [-v]
script.py (-d <ditem>) (-l) ([-t <tfile>] | [-o <ofile>]) [-v]
script.py (-d <ditem>) (-l) [-t <tfile>] [-o <ofile>] [-v]
script.py -h | --help
script.py --version

To allow -t along, -o along, -t and -o together:
Script does stuff.
Usage:
script.py [-t] [-o]
Options:
-t --tfile Does stuff
-o --output Does stuff
If it is an error when both -t and -o are absent:
Script does stuff.
Usage:
script.py -t
script.py -o
script.py -t -o
Options:
-t --tfile Does stuff
-o --output Does stuff

Error when trying to use memory_profiler module

I am trying to use the memory_profiler module to profile the memory usage of a large Python program. There seems to be a memory leak somewhere in my program, so I'm hoping this module will help me find the leak.
I installed memory_profiler using pip and tested it with the sample code provided here. This works perfectly.
When I try to use it with my program, I add the #profile decorator to my main() function and run the profiler from the command line the same way:
$ python -m memory_profiler engine.py
I get the following error, and my program fails to run (everything hangs):
usage: memory_profiler.py [-h] [-c CHARSET] [-i] [-o] [-l LOG_LEVEL] [-g] [-k]
[-w] [-s STOP] [-x TEXTS] [-z SIZE] [-t TIMEOUT]
[-p] [-d DEVICE]
memory_profiler.py: error: unrecognized arguments: engine.py
Any ideas what I might be doing wrong?