Develop Reference

access "implicit" metavar value in argument help string

When you call add_argument on an argparse.ArgumentParser() without an explicit action, you get the "store" action. In the auto-generated --help output you get the uppercase of the long option, unless you set metavar:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--version', metavar='X.Y.Z')
parser.add_argument('--date'),
parser.parse_args(['--help'])
displays:
usage: try.py [-h] [--version X.Y.Z] [--date DATE]
optional arguments:
-h, --help show this help message and exit
--version X.Y.Z
--date DATE
In this I would call X.Y.Z an explicit metavar, and DATE an implicit metavar.
If you want to have more useful help you can do:
parser.add_argument('--version', metavar='X.Y.Z',
help = "set version to % (metavar)s")
which gives (only changed lines shown):
--version X.Y.Z set version to X.Y.Z
and being able to use that %(metavar)s in the help string is nice because when you change metavar='MAJOR.MINOR', the help doesn't need to be updated (which you are bound to forget).
But if you add the help for the --date argument, with the implicit metavar:
parser.add_argument('--date',
help="use %(metavar)s instead of today's date")
you get:
--date DATE use None instead of today
And that None is not what I expected, nor what I want.
Of course I can always hard-code 'DATE' in the help, or explicitly provide the metavar (especially when it is used in the help string). But when I do that, I am bound to forget to update the metavar when I change the name of the long option.
Is there an "automatic" way to get DATE in the help string instead of None ?
Or am I using %(metavar)s where I should be using something else (and if so, what)?

One thing you can do before calling parser.parse_args() is update those actions added to the parser that have a metavar attribute that is None:
for action in parser._actions:
if not hasattr(action, 'metavar') or not hasattr(action, 'dest'):
continue
metavar = getattr(action, 'metavar')
if metavar is None:
action.metavar = action.dest.upper()
That generates output like:
--version X.Y.Z set version to X.Y.Z
--date DATE use DATE instead of today
But as we say in the BDFL's native language: "mooi is anders" ¹
¹ beautiful looks different

It's not going to be easy, at least not within argparse.
When you add_argument it creates an Action class object, and assigns attributes:
a1 = parser.add_argument('--version', metavar='X.Y.Z')
a2 = parser.add_argument('--date')
a1.metavar will be 'X.Y.Z', a2.metavar will be the default None.
That's the value that is used in the help line, something like:
`'help %(metavar)`%{'metavar':action.metavar}'
That action.metavar attribute can be modified after creating the Action, as demonstrated in the other answer.
But for the usage and first part of help it does something like:
def _metavar_formatter(self, action, default_metavar):
if action.metavar is not None:
result = action.metavar
elif action.choices is not None:
choice_strs = [str(choice) for choice in action.choices]
result = '{%s}' % ','.join(choice_strs)
else:
result = default_metavar
...
default_metavar is different for positionals and optionals, but basically is derived from action.dest. So the displayed metavar is generated on the fly, and not stored anywhere.
The %(metavar)s is handled in:
def _expand_help(self, action):
params = dict(vars(action), prog=self._prog)
for name in list(params):
if params[name] is SUPPRESS:
del params[name]
for name in list(params):
if hasattr(params[name], '__name__'):
params[name] = params[name].__name__
if params.get('choices') is not None:
choices_str = ', '.join([str(c) for c in params['choices']])
params['choices'] = choices_str
return self._get_help_string(action) % params
vars(action) makes a dictionary from all the attributes of the action.
I can imagine creating a Formatter subclass that modifies one or more of the methods. The existing subclasses work by modifying just one or two low level methods. But to do that requires studying the code.
In [329]: p = argparse.ArgumentParser()
In [330]: a1 = p.add_argument('--version', metavar='X.Y.Z')
In [331]: a1
Out[331]: _StoreAction(option_strings=['--version'], dest='version', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar='X.Y.Z')
In [332]: vars(a1)
Out[332]:
{'option_strings': ['--version'],
'dest': 'version',
'nargs': None,
'const': None,
'default': None,
'type': None,
'choices': None,
'required': False,
'help': None,
'metavar': 'X.Y.Z',
'container': <argparse._ArgumentGroup at 0x7f72ecc4b4a8>}
A help with several parameters:
In [333]: a1.help='help %(metavar)s, %(dest)s, %(required)s'
In [334]: p.print_help()
usage: ipython3 [-h] [--version X.Y.Z]
optional arguments:
-h, --help show this help message and exit
--version X.Y.Z help X.Y.Z, version, False

Python multiple user arguments to a list

I've got not words to thank you all of you for such great advice. Now everything started to make sense. I apologize for for my bad variable naming. It was just because I wanted to quickly learn and I wont carry out such practices when I write the final script with my own enhancements which will be posted here.
I want to go an another step further by passing the values we've isolated (ip,port,and name) to a template. I tried but couldn't get it right even though I feel close. The text I want to construct looks like this. (
Host Address:<IP>:PORT:<1>
mode tcp
bind <IP>:<PORT> name <NAME>
I have tried this within the working script provided by rahul.(I've edited my original code abiding stackexchange's regulations. Please help out just this once as well. Many thanks in advance.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input
def func_three(help):
for i in help:
print(i)
for kkk in kkk_list:
bb = re.split(":|,", kkk)
XXX=func_three(bb)
for n in XXX:
ip, port, name = n
template ="""HOST Address:{0}:PORT:{1}
mode tcp
bind {0}:{1} name {2}"""
sh = template.format(ip,port,name)
print sh
orignial post:--
Beginner here. I wrote the below code and it doesn't get me anywhere.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("INPUT")
args = p.parse_args()
KKK= args.INPUT
bb=re.split(":|,", KKK)
def func_three(help):
for i in help:
#print help
return help
#func_three(bb[0:3])
YY = var1, var2, var3 = func_three(bb[0:3])
print YY
The way to run this script should be "script.py :". i.e: script.py 192.168.1.10:80:string 172.25.16.2:100:string
As you can see if one argument is passed I have no problems. But when there are more arguments I cant determine how to workout the regexes and get this done via a loop.
So to recap, this is how i want the output to look like to proceed further.
192.168.1.10
80
name1
172.25.16.2
100
name2
If there are better other ways to achieve this please feel free to suggest.

I would say what you are doing could be done more simply. If you want to split the input whenever a colon appears you could use:
#!/usr/bin/python
import sys
# sys.argv is the list of arguments you pass when you run the program
# but sys.argv[0] is the actual program name
# so you want to start at sys.argv[1]
for arg in sys.argv[1:]:
listVar = arg.split(':')
for i in listVar:
print i
# Optionally print a new line
print

Please name your variable with respect to context. You will need to use nargs=* for accepting multiple arguments. I have added the updated code below which prints as you wanted.
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input # ['192.168.1.10:80:name1', '172.25.16.2:100:name3']
def func_three(help):
for i in help:
print(i)
for kkk in kkk_list:
bb = re.split(":|,", kkk)
func_three(bb)
print('\n')
# This prints
# 192.168.1.10
# 80
# name1
# 172.25.16.2
# 100
# name3
Updated Code for new requirement
#!/usr/bin/python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("input", help="input the data in format ip:port:name", nargs='*')
args = p.parse_args()
kkk_list = args.input # ['192.168.1.10:80:name1', '172.25.16.2:100:name3']
def printInFormat(ip, port, name):
formattedText = '''HOST Address:{ip}:PORT:{port}
mode tcp
bind {ip}:{port} name {name}'''.format(ip=ip,
port=port,
name=name)
textWithoutExtraWhitespaces = '\n'.join([line.strip() for line in formattedText.splitlines()])
# you can break above thing
# text = ""
# for line in formattedText.splitlines():
# text += line.strip()
# text += "\n"
print(formattedText)
for kkk in kkk_list:
ip, port, name = re.split(":|,", kkk)
printInFormat(ip, port, name)
# HOST Address:192.168.1.10:PORT:80
# mode tcp
# bind 192.168.1.10:80 name name1
# HOST Address:172.25.16.2:PORT:100
# mode tcp
# bind 172.25.16.2:100 name name3

Bad variable names aside, if you want to use argparse (which I think is a good habit, even if it is somewhat more complex initially) you should use the nargs='+' option:
#!/usr/bin/env python
import argparse
import re
import string
p = argparse.ArgumentParser()
p.add_argument("INPUT", nargs='+')
args = p.parse_args()
KKK= args.INPUT
def func_three(help):
for i in help:
#print help
return help
for kkk in KKK:
bb=re.split(":|,", kkk)
#func_three(bb[0:3])
YY = var1, var2, var3 = func_three(bb[0:3])
print YY

If you look at the documentation for argparse, you'll notice that there's an nargs argument you can pass to add_argument, which allows you to group more than one input.
For example:
p.add_argument('INPUT', nargs='+')
Would make it so that there is a minimum of one argument, but all arguments will be gathered into a list.
Then you can go through each of your inputs like this:
args = p.parse_args()
for address in args.INPUT:
ip, port = address.split(':')

Python argparse: Get name of subparser program in help string

I'm writing an argument parser for a python module with various subparsers. My objective is to have an argument that's shared whose Argument constructor is passed to multiple children:
from argparse import ArgumentParser
parser = ArgumentParser(prog = 'master')
parser1 = ArgumentParser(help = None)
parser1.add_argument('foo', type = int, help = 'Number of times to process %(prog)s') # Line of interest
parser2 = ArgumentParser(help = None)
parser2.add_argument('--bar', type = int, default = 0, help = 'Start at this number')
parser3 = ArgumentParser(help = None)
parser3.add_argument('--baz', type = str, default = 'DEFAULT', help = 'Init file with this text')
subparsers = parser.add_subparsers()
sp1 = subparsers.add_parser('prog1', parents = [parser1, parser2])
sp2 = subparsers.add_parser('prog2', parents = [parser1, parser3])
parser.parse_args('prog1 -h'.split())
The desired output would be something like
usage: master prog1 [-h] [--bar BAR] foo
positional arguments:
foo Number of times to process prog1
optional arguments:
-h, --help show this message and exit
--bar Start at this number
When I use this exact setup, i get master prog1 instead of prog1 in the help string for foo. What should i change in the line marked #Line of interest to get the desired result?

This is not a direct answer to your question, but I would use Click_ for what you are trying to do.
Click_ in three points:
arbitrary nesting of commands
automatic help page generation
supports lazy loading of subcommands at runtime

I can explain what is going on, but may not be able offer a solution.
The short answer is that the sp1.prog is used both in its usage format and as the %(prog)s value in the help lines. And it is constructed with that usage line in mind.
sp1 = subparsers.add_parser('prog1', parents = [parser1, parser2])
creates a parser, and adds arguments from the parents. add_parser is a method of the class _SubParsersAction (the subparser Action class). And the prog attribute for that parser is created with:
if kwargs.get('prog') is None:
kwargs['prog'] = '%s %s' % (self._prog_prefix, name)
You should be able to see this attribute with print(sp1.prog) (I expect 'master prog1'). That's the value that is used in the usage line, and in any of the help lines with %(prog)s.
subparsers._prog_prefix is derived from parser.prog (see the add_subparsers code for details). But you can also specify a prog parameter:
sp1 = subparsers.add_parser('prog1', parents = [parser1, parser2], prog='prog1')
That should correct the string in the help line. But it will also change the string the usage.
You could also give the subparser an explicit usage:
sp1 = subparsers.add_parser('prog1', parents = [parser1, parser2], prog='prog1', usage='master prog1 ...')
Without doing surgery to the HelpFormatter I don't think you can change prog in the help lines without also changing it in the usage.
And given the way parents works you can't change the help line for prog1 foo without also changing it for prog2 foo. parents copies Action objects by reference, so the two subparsers share the foo Action object.
You many have to abandon the parents approach, at least for this argument, and hardcode the name. If you need to add the argument to multiple subparsers, write a little utility function to facilitate that. The parents mechanism is just (usually) a convenience, something that saves some typing/editing.
This modified script will illustrate my points
parser = ArgumentParser(prog = 'master')
parser1 = ArgumentParser(add_help = False)
fooarg=parser1.add_argument('foo', type = int, help = 'foo prog: %(prog)s') # Line of interest
parser2 = ArgumentParser(add_help = False)
parser2.add_argument('--bar', type = int, default = 0, help = 'Start at this number')
parser3 = ArgumentParser(add_help = False)
parser3.add_argument('--baz', type = str, default = 'DEFAULT', help = 'Init file with this text')
subparsers = parser.add_subparsers(prog='subparsers')
sp1 = subparsers.add_parser('prog1', parents = [parser1, parser2], prog='name1')
sp2 = subparsers.add_parser('prog2', parents = [parser1, parser3])
#parser.print_help()
# fooarg is an Action for both subparsers
# print(fooarg.help)
# fooarg.help = 'FOO HELP'
print('==>sp1 prog:', sp1.prog)
sp1.print_help()
print('==>sp2 prog:', sp2.prog)
sp2.print_help()
sp1.prog = 'custom'
sp1.print_help()
# addition
fooarg.default = 'default'
fooarg.metavar = 'META'
fooarg.help = 'prog: %(prog)s, dest=%(dest)s, nargs=%(nargs)s, type=%(type)s, default=%(default)s'
sp1.print_help()
This last bit adds a bunch of Action attributes to the help. But prog is the only one that comes from the parser:
positional arguments:
META prog: custom, dest=foo, nargs=None, type=int, default=default

Argparse File Output in Brackets and Single Quotes

Objective: To add a prefix or suffix to a string.
Problem: The output to the file is modified to include a [''] around the input. (ex: ['prefix']word, or word['suffix'])
Question: How do I remove the bracket and single quote around the input string?
Code:
parser.add_argument('-p', dest='prefix', metavar='[Prefix]', nargs=1, help="Add a user defined prefix")
parser.add_argument('-s', dest='suffix', metavar='[Suffix]', nargs=1, help="Add a user defined suffix")
#Adding a prefix to the string
elif args.prefix:
s = str(args.prefix)
print s
def addprefix(n):
p = s + n
args.outfile.write(p)
myline = args.infile.readline()
while myline:
addprefix(myline)
myline = args.infile.readline()
args.infile.close
#Adding a suffix to the string
elif args.suffix:
s = str(args.suffix)
def addsuf(n):
p = str(n.strip()+s+"\n")
args.outfile.write(p)
myline = args.infile.readline()
while myline:
addsuf(myline)
myline = args.infile.readline()
args.infile.close
Side Note: This is a snippet of code from the much larger program.
Thanks in advance.

I'm guessing the problem is nargs=1. This tells argparse that you want to make a list. Instead, remove the nargs=1 bit and put in action='store'. You may also want to specify a default.
parser.add_argument('-p', dest='prefix',
metavar='[Prefix]',action='store',default='',
help="Add a user defined prefix")

How can I talk to UniProt over HTTP in Python?

I'm trying to get some results from UniProt, which is a protein database (details are not important). I'm trying to use some script that translates from one kind of ID to another. I was able to do this manually on the browser, but could not do it in Python.
In http://www.uniprot.org/faq/28 there are some sample scripts. I tried the Perl one and it seems to work, so the problem is my Python attempts. The (working) script is:
## tool_example.pl ##
use strict;
use warnings;
use LWP::UserAgent;
my $base = 'http://www.uniprot.org';
my $tool = 'mapping';
my $params = {
from => 'ACC', to => 'P_REFSEQ_AC', format => 'tab',
query => 'P13368 P20806 Q9UM73 P97793 Q17192'
};
my $agent = LWP::UserAgent->new;
push #{$agent->requests_redirectable}, 'POST';
print STDERR "Submitting...\n";
my $response = $agent->post("$base/$tool/", $params);
while (my $wait = $response->header('Retry-After')) {
print STDERR "Waiting ($wait)...\n";
sleep $wait;
print STDERR "Checking...\n";
$response = $agent->get($response->base);
}
$response->is_success ?
print $response->content :
die 'Failed, got ' . $response->status_line .
' for ' . $response->request->uri . "\n";
My questions are:
1) How would you do that in Python?
2) Will I be able to massively "scale" that (i.e., use a lot of entries in the query field)?

question #1:
This can be done using python's urllibs:
import urllib, urllib2
import time
import sys
query = ' '.join(sys.argv)
# encode params as a list of 2-tuples
params = ( ('from','ACC'), ('to', 'P_REFSEQ_AC'), ('format','tab'), ('query', query))
# url encode them
data = urllib.urlencode(params)
url = 'http://www.uniprot.org/mapping/'
# fetch the data
try:
foo = urllib2.urlopen(url, data)
except urllib2.HttpError, e:
if e.code == 503:
# blah blah get the value of the header...
wait_time = int(e.hdrs.get('Retry-after', 0))
print 'Sleeping %i seconds...' % (wait_time,)
time.sleep(wait_time)
foo = urllib2.urlopen(url, data)
# foo is a file-like object, do with it what you will.
foo.read()

You're probably better off using the Protein Identifier Cross Reference service from the EBI to convert one set of IDs to another. It has a very good REST interface.
http://www.ebi.ac.uk/Tools/picr/
I should also mention that UniProt has very good webservices available. Though if you are tied to using simple http requests for some reason then its probably not useful.

Let's assume that you are using Python 2.5.
We can use httplib to directly call the web site:
import httplib, urllib
querystring = {}
#Build the query string here from the following keys (query, format, columns, compress, limit, offset)
querystring["query"] = ""
querystring["format"] = "" # one of html | tab | fasta | gff | txt | xml | rdf | rss | list
querystring["columns"] = "" # the columns you want comma seperated
querystring["compress"] = "" # yes or no
## These may be optional
querystring["limit"] = "" # I guess if you only want a few rows
querystring["offset"] = "" # bring on paging
##From the examples - query=organism:9606+AND+antigen&format=xml&compress=no
##Delete the following and replace with your query
querystring = {}
querystring["query"] = "organism:9606 AND antigen"
querystring["format"] = "xml" #make it human readable
querystring["compress"] = "no" #I don't want to have to unzip
conn = httplib.HTTPConnection("www.uniprot.org")
conn.request("GET", "/uniprot/?"+ urllib.urlencode(querystring))
r1 = conn.getresponse()
if r1.status == 200:
data1 = r1.read()
print data1 #or do something with it
You could then make a function around creating the query string and you should be away.

check this out bioservices. they interface a lot of databases through Python.
https://pythonhosted.org/bioservices/_modules/bioservices/uniprot.html
conda install bioservices --yes

in complement to O.rka answer:
Question 1:
from bioservices import UniProt
u = UniProt()
res = u.get_df("P13368 P20806 Q9UM73 P97793 Q17192".split())
This returns a dataframe with all information about each entry.
Question 2: same answer. This should scale up.
Disclaimer: I'm the author of bioservices

There is a python package in pip which does exactly what you want
pip install uniprot-mapper