Python multiple user arguments to a list - python

I've got not words to thank you all of you for such great advice. Now everything started to make sense. I apologize for for my bad variable naming. It was just because I wanted to quickly learn and I wont carry out such practices when I write the final script with my own enhancements which will be posted here.
I want to go an another step further by passing the values we've isolated (ip,port,and name) to a template. I tried but couldn't get it right even though I feel close. The text I want to construct looks like this. (
Host Address:<IP>:PORT:<1>
mode tcp
bind <IP>:<PORT> name <NAME>
I have tried this within the working script provided by rahul.(I've edited my original code abiding stackexchange's regulations. Please help out just this once as well. Many thanks in advance.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input
def func_three(help):
for i in help:
print(i)
for kkk in kkk_list:
bb = re.split(":|,", kkk)
XXX=func_three(bb)
for n in XXX:
ip, port, name = n
template ="""HOST Address:{0}:PORT:{1}
mode tcp
bind {0}:{1} name {2}"""
sh = template.format(ip,port,name)
print sh
orignial post:--
Beginner here. I wrote the below code and it doesn't get me anywhere.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("INPUT")
args = p.parse_args()
KKK= args.INPUT
bb=re.split(":|,", KKK)
def func_three(help):
for i in help:
#print help
return help
#func_three(bb[0:3])
YY = var1, var2, var3 = func_three(bb[0:3])
print YY
The way to run this script should be "script.py :". i.e: script.py 192.168.1.10:80:string 172.25.16.2:100:string
As you can see if one argument is passed I have no problems. But when there are more arguments I cant determine how to workout the regexes and get this done via a loop.
So to recap, this is how i want the output to look like to proceed further.
192.168.1.10
80
name1
172.25.16.2
100
name2
If there are better other ways to achieve this please feel free to suggest.

I would say what you are doing could be done more simply. If you want to split the input whenever a colon appears you could use:
#!/usr/bin/python
import sys
# sys.argv is the list of arguments you pass when you run the program
# but sys.argv[0] is the actual program name
# so you want to start at sys.argv[1]
for arg in sys.argv[1:]:
listVar = arg.split(':')
for i in listVar:
print i
# Optionally print a new line
print

Please name your variable with respect to context. You will need to use nargs=* for accepting multiple arguments. I have added the updated code below which prints as you wanted.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input # ['192.168.1.10:80:name1', '172.25.16.2:100:name3']
def func_three(help):
for i in help:
print(i)
for kkk in kkk_list:
bb = re.split(":|,", kkk)
func_three(bb)
print('\n')
# This prints
# 192.168.1.10
# 80
# name1
# 172.25.16.2
# 100
# name3
Updated Code for new requirement
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input # ['192.168.1.10:80:name1', '172.25.16.2:100:name3']
def printInFormat(ip, port, name):
formattedText = '''HOST Address:{ip}:PORT:{port}
mode tcp
bind {ip}:{port} name {name}'''.format(ip=ip,
port=port,
name=name)
textWithoutExtraWhitespaces = '\n'.join([line.strip() for line in formattedText.splitlines()])
# you can break above thing
# text = ""
# for line in formattedText.splitlines():
# text += line.strip()
# text += "\n"
print(formattedText)
for kkk in kkk_list:
ip, port, name = re.split(":|,", kkk)
printInFormat(ip, port, name)
# HOST Address:192.168.1.10:PORT:80
# mode tcp
# bind 192.168.1.10:80 name name1
# HOST Address:172.25.16.2:PORT:100
# mode tcp
# bind 172.25.16.2:100 name name3

Bad variable names aside, if you want to use argparse (which I think is a good habit, even if it is somewhat more complex initially) you should use the nargs='+' option:
#!/usr/bin/env python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("INPUT", nargs='+')
args = p.parse_args()
KKK= args.INPUT
def func_three(help):
for i in help:
#print help
return help
for kkk in KKK:
bb=re.split(":|,", kkk)
#func_three(bb[0:3])
YY = var1, var2, var3 = func_three(bb[0:3])
print YY

If you look at the documentation for argparse, you'll notice that there's an nargs argument you can pass to add_argument, which allows you to group more than one input.
For example:
p.add_argument('INPUT', nargs='+')
Would make it so that there is a minimum of one argument, but all arguments will be gathered into a list.
Then you can go through each of your inputs like this:
args = p.parse_args()
for address in args.INPUT:
ip, port = address.split(':')

Related

Resolve argparse alias back to the original command

I'm using a subparser/subcommand that has an alias.
I'm using the dest option for the subparser to store the name of the subcommand so I can get it later.
Currently if the subcommand's name is reallyLongName and the alias is r (say) then the dest option stores either reallyLongName or r exactly - whatever I typed in gets stored. This is annoying because I now have to check for the name of the command or any of its aliases in order to identify the command.
Is there a way to get argparse to store the subcommand's name in the dest field in some sort of single, canonical text string?
For example, given the following code:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='command', help='sub-command help')
parser_ag = subparsers.add_parser( 'mySubcommand',
aliases=['m'],
help='Subcommand help')
print(parser.parse_args('mySubcommand'.split()))
print(parser.parse_args('m'.split()))
the following output is produced:
Namespace(command='mySubcommand')
Namespace(command='m')
Desired result: command has a single, canonical value for both, for example:
Namespace(command='mySubcommand')
Namespace(command='mySubcommand')
There was a Python bug/issue requesting this - saving the 'base' name, rather than the alias. You can't change that without changing argparse.py code. I think the change would limited to the Action subclass that handles subparsers. https://bugs.python.org/issue36664
But I point out that there's simpler way of handling this. Just use set_defaults as documented near the end of the https://docs.python.org/3/library/argparse.html#sub-commands section. There
parser_foo.set_defaults(func=foo)
is used to set a subparser specific function, but it could just as well be used to set the 'base' name.
parser_foo.set_defaults(name='theIncrediblyLongAlias')
This was surprisingly difficult to dig out. When you add a subparser, it gets stored in the parents ._actions attribute. From there it is just digging through attributes to get what you need. Below I create dictionaries to reference the subparser arguments by the dest name, and then added a function that lets us remap the inputted arguments to the primary argument name.
from collections import defaultdict
def get_subparser_aliases(parser, dest):
out = defaultdict(list)
prog_str = parser.prog
dest_dict = {a.dest: a for a in parser._actions}
try:
choices = dest_dict.get(dest).choices
except AttributeError:
raise AttributeError(f'The parser "{parser}" has no subparser with a `dest` of "{dest}"')
for k, v in choices.items():
clean_v = v.prog.replace(prog_str, '', 1).strip()
out[clean_v].append(k)
return dict(out)
def remap_args(args, mapping, dest):
setattr(args, dest, mapping.get(getattr(args, dest)))
return args
Using your example, we can remap the parse args using:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='command', help='sub-command help')
parser_ag = subparsers.add_parser('mySubcommand',
aliases=['m'],
help='Subcommand help')
args = parser.parse_args('m'.split())
mapping = get_subparser_aliases(parser, 'command')
remap_args(args, mapping, 'command')
print(args)
# prints:
Namespace(command='mySubcommand')
Here is an example of it at work with multiple subparser levels.. We have a parser with an optional argument and a subparser. The subparser has 3 possible arguments, the last of which invoke another subparser (a sub-subparser), with 2 possible arguments.
You can examine either the top level parser or the first level subparser to see alias mappings.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--someoption', '-s', action='store_true')
subparser1 = parser.add_subparsers(help='sub-command help', dest='sub1')
parser_r = subparser1.add_parser('reallyLongName', aliases=['r'])
parser_r.add_argument('foo', type=int, help='foo help')
parser_s = subparser1.add_parser('otherReallyLong', aliases=['L'])
parser_s.add_argument('bar', choices='abc', help='bar help')
parser_z = subparser1.add_parser('otherOptions', aliases=['oo'])
subparser2 = parser_z.add_subparsers(help='sub-sub-command help', dest='sub2')
parser_x = subparser2.add_parser('xxx', aliases=['x'])
parser_x.add_argument('fizz', type=float, help='fizz help')
parser_y = subparser2.add_parser('yyy', aliases=['y'])
parser_y.add_argument('blip', help='blip help')
get_subparser_aliases(parser, 'sub1')
# returns:
{'reallyLongName': ['reallyLongName', 'r'],
'otherReallyLong': ['otherReallyLong', 'L'],
'otherOptions': ['otherOptions', 'oo']}
get_subparser_aliases(parser_z, 'sub2')
# returns:
{'xxx': ['xxx', 'x'], 'yyy': ['yyy', 'y']}
Using this with the function above, we can remap the collected args to their longer names.
args = parser.parse_args('-s oo x 1.23'.split())
print(args)
# prints:
Namespace(fizz=1.23, someoption=True, sub1='oo', sub2='x')
for p, dest in zip((parser, parser_z), ('sub1', 'sub2')):
mapping = get_subparser_aliases(p, dest)
remap_args(args, mapping, dest)
print(args)
# prints:
Namespace(fizz=1.23, someoption=True, sub1='otherOptions', sub2='xxx')

Command-line argument overwritten by default value when using subparser

I have a program with sub commands, but they all have common arguments (e.g. they all require input and output directories) which I included in the parent parser to avoid redundancies. However, I want each subcommand to have a different default value, but this causes the value provided in the command-line to be ignored.
MWE:
import argparse
top_parser = argparse.ArgumentParser()
top_parser.add_argument("--input-dir", type=str)
subparsers = top_parser.add_subparsers()
generate_parser = subparsers.add_parser("generate")
generate_parser.set_defaults(input_dir=".")
process_parser = subparsers.add_parser("process")
process_parser.set_defaults(input_dir="SOME_OTHER_DIR")
generate_args = top_parser.parse_args("--input-dir USE_THIS_DIR generate".split())
print("generate_args = ", generate_args)
process_args = top_parser.parse_args("--input-dir USE_THIS_DIR process".split())
print("process_args = ", process_args)
This gives:
generate_args = Namespace(input_dir='.')
process_args = Namespace(input_dir='SOME_OTHER_DIR')
but I want:
generate_args = Namespace(input_dir='USE_THIS_DIR')
process_args = Namespace(input_dir='USE_THIS_DIR')
I can circumvent this by separately adding the argument to each subparser, but I would like to avoid this redundancy if possible.
One workaround would be to check the value of input_dir after parsing, and substitute a subparser-specific default at that time.
import argparse
top_parser = argparse.ArgumentParser()
top_parser.add_argument("--input-dir", type=str)
subparsers = top_parser.add_subparsers()
generate_parser = subparsers.add_parser("generate")
generate_parser.set_defaults(alt_input_dir=".")
process_parser = subparsers.add_parser("process")
process_parser.set_defaults(alt_input_dir="SOME_OTHER_DIR")
args = top_parser.parse_args()
if args.input_dir is None:
args.input_dir = args.alt_input_dir
del args.alt_input_dir

Parsing a variable from command line to a url

I have a python script called dlimage. I want to type a variable in terminal like this $ python dlimage.py 1 2 and have 1 and 2 correspond to the the url in download_web_image to become http://www.example.com/1/2.jpg and download the image. How do I go about doing this?
import urllib.request
import argparse
def download_web_image(url):
urllib.request.urlretrieve(url)
parser = argparse.ArgumentParser()
parser.add_argument("num1", "num2")
args = parser.parse_args()
download_web_image("http://www.example.com/"num1"/"num2".jpg")
EDIT 2:
I finally got it to work. Thanks everyone for your help!
Code that worked:
import urllib
import argparse
def download_web_image(url):
IMAGE = url.rsplit('/',1)[1]
urllib.urlretrieve(url, IMAGE)
parser = argparse.ArgumentParser()
parser.add_argument("num1")
parser.add_argument("num2")
args = parser.parse_args()
download_web_image("https://www.example.com/{num1}/{num2}.jpg".format(num1=args.num1, num2=args.num2))
When I try part of your code, I get an error:
In [1663]: parser = argparse.ArgumentParser()
In [1664]: parser.add_argument("num1", "num2")
....
ValueError: invalid option string 'num1': must start with a character '-'
The arguments to the add_argument method are wrong.
What you should be using is:
parser = argparse.ArgumentParser()
parser.add_argument("num1")
parser.add_argument("num2")
In which case the help will look like:
In [1668]: parser.print_help()
usage: ipython3 [-h] num1 num2
positional arguments:
num1
num2
optional arguments:
-h, --help show this help message and exit
and testing an input equivalent to myprog 1 2
In [1669]: args = parser.parse_args(['1','2'])
In [1670]: args
Out[1670]: Namespace(num1='1', num2='2')
In [1671]: args.num1
Out[1671]: '1'
In [1672]: args.num2
Out[1672]: '2'
Now I can format a URL with:
In [1675]: "https://www.example.com/{}/{}.jpg".format(args.num1, args.num2)
Out[1675]: 'https://www.example.com/1/2.jpg'
So there are 2 problems with your code:
Each argument, num1 and num2 has to be defined in a separate add_argument statement. Read the docs to see what else you can add to that statement, such as the help. You are trying to define 2 arguments in one statement, and getting an error.
Secondly you need to use a correct format. I added the {} ({0} and {num1} styles also work). OR in the older Py2 style:
"https://www.example.com/%s/%s.jpg"%(args.num1, args.num2)
Change this
parser.add_argument("num1, "num2")
to
parser.add_argument("num1", "num2")
for this I would use format:
download_web_image("http://www.example.com/{num1}/{num2}.jpg".format(num1=args.num1, num2 = args.num2))
Here is an example:
num1 = 5
num2 = 6
"http://www.example.com/{num1}/{num2}.jpg".format(num1 = num1, num2 = num2)
output:
'http://www.example.com/5/6.jpg'
format makes it easy to insert defined parameters into a string.
Assuming you are using python3, I tried to make something simple and as close to the approach you were trying originally. Hope this helps.
import urllib.request
import sys
def download_web_image(url):
urllib.request.urlretrieve(url)
download_web_image("http://www.example.com/{0}/{1}.jpg".format(sys.argv[1], sys.argv[2]))
I imagine you want to pass N arguments so to generate an URL I would do this :
import sys
args = sys.argv[1:]
baseURL = "http://www.example.com"
url = baseURL + "/" + "/".join(args)
print url
INPUT:
$ python dlimage.py 1 2 3.jpg
OUTPUT:
http://www.example.com/1/2/3.jpg
As for the download, you could try this answer, or read data at the url and write it as a file named as the last argument :
import urllib2
urlResponse = urllib2.urlopen(url)
urlData = urlResponse.read()
outFile = open(args[-1], 'w')
outFile.write(urlData)
outFile.close()
I'm behind a work proxy and I get an error when I download the file, might update my answer later this day at home.

Python counting the unique occurences of a string in a file

I’m trying to count the unique IP addresses in a Apache log-file using python 3.3.1
The thing is I don’t think that it is counting everything correctly.
Here is my code:
import argparse
import os
import sys
from collections import Counter
#
# This function counts the unique IP adresses in the logfile
#
def print_unique_ip(logfile):
IPset = set()
for line in logfile:
head, sep, tail = line.partition(" ")
if(len(head) > 1):
IPset.update(head)
print(len(IPset))
return
#
# This is the main function of the program
#
def main():
parser = argparse.ArgumentParser(description="An appache log file processor")
parser.add_argument('-l', '--log-file', help='This is the log file to work on', required=True)
parser.add_argument('-n', help='Displays the number of unique IP adresses', action='store_true')
parser.add_argument('-t', help='Displays top T IP adresses', type=int)
parser.add_argument('-v', help='Displays the number of visits of a IP adress')
arguments = parser.parse_args()
if(os.path.isfile(arguments.log_file)):
logfile = open(arguments.log_file)
else:
print('The file <', arguments.log_file, '> does not exist')
sys.exit
if(arguments.n == True):
print_unique_ip(logfile)
if(arguments.t):
print_top_n_ip(arguments.t, logfile)
if(arguments.v):
number_of_ocurrences(arguments.v, logfile)
return
if __name__ == '__main__':
main()
I have left put everything else.
When I run it I get
$ python3 assig4.py -l apache_short.log -n
12
But I know that there are more than 12 unique IPs in the file
It doesn’t seem to be giving me the right result. What I am trying to do is to read the file line by line, then when I find an IP address I put it into a set as it only saves unique elements and then I print out the length of said set.
IPset.update(head)
Bug. This will not do what you're expecting. You want to add each IP to your set instead. Examples make it clearest:
>>> s1 = set()
>>> s2 = set()
>>> s1.add('11.22.33.44')
>>> s2.update('11.22.33.44')
>>> s1
set(['11.22.33.44'])
>>> s2
set(['1', '3', '2', '4', '.'])

How to parse multiple nested sub-commands using python argparse?

I am implementing a command line program which has interface like this:
cmd [GLOBAL_OPTIONS] {command [COMMAND_OPTS]} [{command [COMMAND_OPTS]} ...]
I have gone through the argparse documentation. I can implement GLOBAL_OPTIONS as optional argument using add_argument in argparse. And the {command [COMMAND_OPTS]} using Sub-commands.
From the documentation it seems I can have only one sub-command. But as you can see I have to implement one or more sub-commands. What is the best way to parse such command line arguments useing argparse?
I came up with the same qustion, and it seems i have got a better answer.
The solution is we shall not simply nest subparser with another subparser, but we can add subparser following with a parser following another subparser.
Code tell you how:
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument('--user', '-u',
default=getpass.getuser(),
help='username')
parent_parser.add_argument('--debug', default=False, required=False,
action='store_true', dest="debug", help='debug flag')
main_parser = argparse.ArgumentParser()
service_subparsers = main_parser.add_subparsers(title="service",
dest="service_command")
service_parser = service_subparsers.add_parser("first", help="first",
parents=[parent_parser])
action_subparser = service_parser.add_subparsers(title="action",
dest="action_command")
action_parser = action_subparser.add_parser("second", help="second",
parents=[parent_parser])
args = main_parser.parse_args()
#mgilson has a nice answer to this question. But problem with splitting sys.argv myself is that i lose all the nice help message Argparse generates for the user. So i ended up doing this:
import argparse
## This function takes the 'extra' attribute from global namespace and re-parses it to create separate namespaces for all other chained commands.
def parse_extra (parser, namespace):
namespaces = []
extra = namespace.extra
while extra:
n = parser.parse_args(extra)
extra = n.extra
namespaces.append(n)
return namespaces
argparser=argparse.ArgumentParser()
subparsers = argparser.add_subparsers(help='sub-command help', dest='subparser_name')
parser_a = subparsers.add_parser('command_a', help = "command_a help")
## Setup options for parser_a
## Add nargs="*" for zero or more other commands
argparser.add_argument('extra', nargs = "*", help = 'Other commands')
## Do similar stuff for other sub-parsers
Now after first parse all chained commands are stored in extra. I reparse it while it is not empty to get all the chained commands and create separate namespaces for them. And i get nicer usage string that argparse generates.
parse_known_args returns a Namespace and a list of unknown strings. This is similar to the extra in the checked answer.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
sub = parser.add_subparsers()
for i in range(1,4):
sp = sub.add_parser('cmd%i'%i)
sp.add_argument('--foo%i'%i) # optionals have to be distinct
rest = '--foo 0 cmd2 --foo2 2 cmd3 --foo3 3 cmd1 --foo1 1'.split() # or sys.argv
args = argparse.Namespace()
while rest:
args,rest = parser.parse_known_args(rest,namespace=args)
print args, rest
produces:
Namespace(foo='0', foo2='2') ['cmd3', '--foo3', '3', 'cmd1', '--foo1', '1']
Namespace(foo='0', foo2='2', foo3='3') ['cmd1', '--foo1', '1']
Namespace(foo='0', foo1='1', foo2='2', foo3='3') []
An alternative loop would give each subparser its own namespace. This allows overlap in positionals names.
argslist = []
while rest:
args,rest = parser.parse_known_args(rest)
argslist.append(args)
The solution provide by #Vikas fails for subcommand-specific optional arguments, but the approach is valid. Here is an improved version:
import argparse
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help', dest='subparser_name')
# create the parser for the "command_a" command
parser_a = subparsers.add_parser('command_a', help='command_a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "command_b" command
parser_b = subparsers.add_parser('command_b', help='command_b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# parse some argument lists
argv = ['--foo', 'command_a', '12', 'command_b', '--baz', 'Z']
while argv:
print(argv)
options, argv = parser.parse_known_args(argv)
print(options)
if not options.subparser_name:
break
This uses parse_known_args instead of parse_args. parse_args aborts as soon as a argument unknown to the current subparser is encountered, parse_known_args returns them as a second value in the returned tuple. In this approach, the remaining arguments are fed again to the parser. So for each command, a new Namespace is created.
Note that in this basic example, all global options are added to the first options Namespace only, not to the subsequent Namespaces.
This approach works fine for most situations, but has three important limitations:
It is not possible to use the same optional argument for different subcommands, like myprog.py command_a --foo=bar command_b --foo=bar.
It is not possible to use any variable length positional arguments with subcommands (nargs='?' or nargs='+' or nargs='*').
Any known argument is parsed, without 'breaking' at the new command. E.g. in PROG --foo command_b command_a --baz Z 12 with the above code, --baz Z will be consumed by command_b, not by command_a.
These limitations are a direct limitation of argparse. Here is a simple example that shows the limitations of argparse -even when using a single subcommand-:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('spam', nargs='?')
subparsers = parser.add_subparsers(help='sub-command help', dest='subparser_name')
# create the parser for the "command_a" command
parser_a = subparsers.add_parser('command_a', help='command_a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "command_b" command
parser_b = subparsers.add_parser('command_b', help='command_b help')
options = parser.parse_args('command_a 42'.split())
print(options)
This will raise the error: argument subparser_name: invalid choice: '42' (choose from 'command_a', 'command_b').
The cause is that the internal method argparse.ArgParser._parse_known_args() it is too greedy and assumes that command_a is the value of the optional spam argument. In particular, when 'splitting' up optional and positional arguments, _parse_known_args() does not look at the names of the arugments (like command_a or command_b), but merely where they occur in the argument list. It also assumes that any subcommand will consume all remaining arguments.
This limitation of argparse also prevents a proper implementation of multi-command subparsers. This unfortunately means that a proper implementation requires a full rewrite of the argparse.ArgParser._parse_known_args() method, which is 200+ lines of code.
Given these limitation, it may be an options to simply revert to a single multiple-choice argument instead of subcommands:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--bar', type=int, help='bar help')
parser.add_argument('commands', nargs='*', metavar='COMMAND',
choices=['command_a', 'command_b'])
options = parser.parse_args('--bar 2 command_a command_b'.split())
print(options)
#options = parser.parse_args(['--help'])
It is even possible to list the different commands in the usage information, see my answer https://stackoverflow.com/a/49999185/428542
You can always split up the command-line yourself (split sys.argv on your command names), and then only pass the portion corresponding to the particular command to parse_args -- You can even use the same Namespace using the namespace keyword if you want.
Grouping the commandline is easy with itertools.groupby:
import sys
import itertools
import argparse
mycommands=['cmd1','cmd2','cmd3']
def groupargs(arg,currentarg=[None]):
if(arg in mycommands):currentarg[0]=arg
return currentarg[0]
commandlines=[list(args) for cmd,args in intertools.groupby(sys.argv,groupargs)]
#setup parser here...
parser=argparse.ArgumentParser()
#...
namespace=argparse.Namespace()
for cmdline in commandlines:
parser.parse_args(cmdline,namespace=namespace)
#Now do something with namespace...
untested
Improving on the answer by #mgilson, I wrote a small parsing method which splits argv into parts and puts values of arguments of commands into hierarchy of namespaces:
import sys
import argparse
def parse_args(parser, commands):
# Divide argv by commands
split_argv = [[]]
for c in sys.argv[1:]:
if c in commands.choices:
split_argv.append([c])
else:
split_argv[-1].append(c)
# Initialize namespace
args = argparse.Namespace()
for c in commands.choices:
setattr(args, c, None)
# Parse each command
parser.parse_args(split_argv[0], namespace=args) # Without command
for argv in split_argv[1:]: # Commands
n = argparse.Namespace()
setattr(args, argv[0], n)
parser.parse_args(argv, namespace=n)
return args
parser = argparse.ArgumentParser()
commands = parser.add_subparsers(title='sub-commands')
cmd1_parser = commands.add_parser('cmd1')
cmd1_parser.add_argument('--foo')
cmd2_parser = commands.add_parser('cmd2')
cmd2_parser.add_argument('--foo')
cmd2_parser = commands.add_parser('cmd3')
cmd2_parser.add_argument('--foo')
args = parse_args(parser, commands)
print(args)
It behaves properly, providing nice argparse help:
For ./test.py --help:
usage: test.py [-h] {cmd1,cmd2,cmd3} ...
optional arguments:
-h, --help show this help message and exit
sub-commands:
{cmd1,cmd2,cmd3}
For ./test.py cmd1 --help:
usage: test.py cmd1 [-h] [--foo FOO]
optional arguments:
-h, --help show this help message and exit
--foo FOO
And creates a hierarchy of namespaces containing the argument values:
./test.py cmd1 --foo 3 cmd3 --foo 4
Namespace(cmd1=Namespace(foo='3'), cmd2=None, cmd3=Namespace(foo='4'))
You could try arghandler. This is an extension to argparse with explicit support for subcommands.
Built a full Python 2/3 example with subparsers, parse_known_args and parse_args (running on IDEone):
from __future__ import print_function
from argparse import ArgumentParser
from random import randint
def main():
parser = get_parser()
input_sum_cmd = ['sum_cmd', '--sum']
input_min_cmd = ['min_cmd', '--min']
args, rest = parser.parse_known_args(
# `sum`
input_sum_cmd +
['-a', str(randint(21, 30)),
'-b', str(randint(51, 80))] +
# `min`
input_min_cmd +
['-y', str(float(randint(64, 79))),
'-z', str(float(randint(91, 120)) + .5)]
)
print('args:\t ', args,
'\nrest:\t ', rest, '\n', sep='')
sum_cmd_result = args.sm((args.a, args.b))
print(
'a:\t\t {:02d}\n'.format(args.a),
'b:\t\t {:02d}\n'.format(args.b),
'sum_cmd: {:02d}\n'.format(sum_cmd_result), sep='')
assert rest[0] == 'min_cmd'
args = parser.parse_args(rest)
min_cmd_result = args.mn((args.y, args.z))
print(
'y:\t\t {:05.2f}\n'.format(args.y),
'z:\t\t {:05.2f}\n'.format(args.z),
'min_cmd: {:05.2f}'.format(min_cmd_result), sep='')
def get_parser():
# create the top-level parser
parser = ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "sum" command
parser_a = subparsers.add_parser('sum_cmd', help='sum some integers')
parser_a.add_argument('-a', type=int,
help='an integer for the accumulator')
parser_a.add_argument('-b', type=int,
help='an integer for the accumulator')
parser_a.add_argument('--sum', dest='sm', action='store_const',
const=sum, default=max,
help='sum the integers (default: find the max)')
# create the parser for the "min" command
parser_b = subparsers.add_parser('min_cmd', help='min some integers')
parser_b.add_argument('-y', type=float,
help='an float for the accumulator')
parser_b.add_argument('-z', type=float,
help='an float for the accumulator')
parser_b.add_argument('--min', dest='mn', action='store_const',
const=min, default=0,
help='smallest integer (default: 0)')
return parser
if __name__ == '__main__':
main()
I had more or less the same requirements: Being able to set global arguments and being able to chain commands and execute them in order of command line.
I ended up with the following code. I did use some parts of the code from this and other threads.
# argtest.py
import sys
import argparse
def init_args():
def parse_args_into_namespaces(parser, commands):
'''
Split all command arguments (without prefix, like --) in
own namespaces. Each command accepts extra options for
configuration.
Example: `add 2 mul 5 --repeat 3` could be used to a sequencial
addition of 2, then multiply with 5 repeated 3 times.
'''
class OrderNamespace(argparse.Namespace):
'''
Add `command_order` attribute - a list of command
in order on the command line. This allows sequencial
processing of arguments.
'''
globals = None
def __init__(self, **kwargs):
self.command_order = []
super(OrderNamespace, self).__init__(**kwargs)
def __setattr__(self, attr, value):
attr = attr.replace('-', '_')
if value and attr not in self.command_order:
self.command_order.append(attr)
super(OrderNamespace, self).__setattr__(attr, value)
# Divide argv by commands
split_argv = [[]]
for c in sys.argv[1:]:
if c in commands.choices:
split_argv.append([c])
else:
split_argv[-1].append(c)
# Globals arguments without commands
args = OrderNamespace()
cmd, args_raw = 'globals', split_argv.pop(0)
args_parsed = parser.parse_args(args_raw, namespace=OrderNamespace())
setattr(args, cmd, args_parsed)
# Split all commands to separate namespace
pos = 0
while len(split_argv):
pos += 1
cmd, *args_raw = split_argv.pop(0)
assert cmd[0].isalpha(), 'Command must start with a letter.'
args_parsed = commands.choices[cmd].parse_args(args_raw, namespace=OrderNamespace())
setattr(args, f'{cmd}~{pos}', args_parsed)
return args
#
# Supported commands and options
#
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--print', action='store_true')
commands = parser.add_subparsers(title='Operation chain')
cmd1_parser = commands.add_parser('add', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
cmd1_parser.add_argument('add', help='Add this number.', type=float)
cmd1_parser.add_argument('-r', '--repeat', help='Repeat this operation N times.',
default=1, type=int)
cmd2_parser = commands.add_parser('mult', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
cmd2_parser.add_argument('mult', help='Multiply with this number.', type=float)
cmd2_parser.add_argument('-r', '--repeat', help='Repeat this operation N times.',
default=1, type=int)
args = parse_args_into_namespaces(parser, commands)
return args
#
# DEMO
#
args = init_args()
# print('Parsed arguments:')
# for cmd in args.command_order:
# namespace = getattr(args, cmd)
# for option_name in namespace.command_order:
# option_value = getattr(namespace, option_name)
# print((cmd, option_name, option_value))
print('Execution:')
result = 0
for cmd in args.command_order:
namespace = getattr(args, cmd)
cmd_name, cmd_position = cmd.split('~') if cmd.find('~') > -1 else (cmd, 0)
if cmd_name == 'globals':
pass
elif cmd_name == 'add':
for r in range(namespace.repeat):
if args.globals.print:
print(f'+ {namespace.add}')
result = result + namespace.add
elif cmd_name == 'mult':
for r in range(namespace.repeat):
if args.globals.print:
print(f'* {namespace.mult}')
result = result * namespace.mult
else:
raise NotImplementedError(f'Namespace `{cmd}` is not implemented.')
print(10*'-')
print(result)
Below an example:
$ python argstest.py --print add 1 -r 2 mult 5 add 3 mult -r 5 5
Execution:
+ 1.0
+ 1.0
* 5.0
+ 3.0
* 5.0
* 5.0
* 5.0
* 5.0
* 5.0
----------
40625.0
Another package which supports parallel parsers is "declarative_parser".
import argparse
from declarative_parser import Parser, Argument
supported_formats = ['png', 'jpeg', 'gif']
class InputParser(Parser):
path = Argument(type=argparse.FileType('rb'), optional=False)
format = Argument(default='png', choices=supported_formats)
class OutputParser(Parser):
format = Argument(default='jpeg', choices=supported_formats)
class ImageConverter(Parser):
description = 'This app converts images'
verbose = Argument(action='store_true')
input = InputParser()
output = OutputParser()
parser = ImageConverter()
commands = '--verbose input image.jpeg --format jpeg output --format gif'.split()
namespace = parser.parse_args(commands)
and namespace becomes:
Namespace(
input=Namespace(format='jpeg', path=<_io.BufferedReader name='image.jpeg'>),
output=Namespace(format='gif'),
verbose=True
)
Disclaimer: I am the author. Requires Python 3.6. To install use:
pip3 install declarative_parser
Here is the documentation and here is the repo on GitHub.
In order to parse the sub commands, I used the following (referred from argparse.py code). It parses the sub parser arguments and retains the help for both. Nothing additional passed there.
args, _ = parser.parse_known_args()
you can use the package optparse
import optparse
parser = optparse.OptionParser()
parser.add_option("-f", dest="filename", help="corpus filename")
parser.add_option("--alpha", dest="alpha", type="float", help="parameter alpha", default=0.5)
(options, args) = parser.parse_args()
fname = options.filename
alpha = options.alpha

Categories

Resources