I have such script:
import argparse
parser = argparse.ArgumentParser(
description='Text file conversion.'
)
parser.add_argument("inputfile", help="file to process", type=str)
parser.add_argument("-o", "--out", default="output.txt",
help="output name")
parser.add_argument("-t", "--type", default="detailed",
help="Type of processing")
args = parser.parse_args()
for arg in args:
print(arg)
But it doesnt work. I get error:
TypeError: 'Namespace' object is not iterable
How to iterate over arguments and their value?
Add vars if you want to iterate over a Namespace object:
for arg in vars(args):
print arg, getattr(args, arg)
Namespace objects aren't iterable, the standard docs suggest doing the following if you want a dictionary:
>>> vars(args)
{'foo': 'BAR'}
So
for key, value in vars(args).items():
# do stuff
To be honest, I am not sure why you want to iterate over the arguments. That somewhat defeats the purpose of having an argument parser.
After
args = parser.parse_args()
to display the arguments, use:
print args # or print(args) in python3
The args object (of type argparse.Namespace) isn't iterable (i.e. not a list), but it has a .__str__ method, which displays the values nicely.
args.out and args.type give the values of the 2 arguments you defined. This works most of the time. getattr(args, key) the most general way of accessing the values, but usually isn't needed.
vars(args)
turns the namespace into a dictionary, which you can access with all the dictionary methods. This is spelled out in the docs.
ref: the Namespace paragraph of the docs - https://docs.python.org/2/library/argparse.html#the-namespace-object
I'm using args.__dict__, which lets you access the underlying dict structure.
Then, its a simple key-value iteration:
for k in args.__dict__:
print k, args.__dict__[k]
Parsing the _actions from your parser seems like a decent idea. Instead of running parse_args() and then trying to pick stuff out of your Namespace.
import argparse
parser = argparse.ArgumentParser(
description='Text file conversion.')
parser.add_argument("inputfile", help="file to process", type=str)
parser.add_argument("-o", "--out", default="output.txt",
help="output name")
parser.add_argument("-t", "--type", default="detailed",
help="Type of processing")
options = parser._actions
for k in options:
print(getattr(k, 'dest'), getattr(k, 'default'))
You can modify the 'dest' part to be 'choices' for example if you need the preset defaults for a parameter in another script (by returning the options in a function for example).
Using the builtin dict() class to iterate over the args is straightforward and works great:
args = parser.parse_args()
args_dict = dict(vars(args))
for i, ii in args_dict.items():
print(i, ii)
From builtins.py:
def __init__(self, seq=None, **kwargs): # known special case of dict.__init__
"""
dict() -> new empty dictionary
dict(mapping) -> new dictionary initialized from a mapping object's
(key, value) pairs
dict(iterable) -> new dictionary initialized as if via:
d = {}
for k, v in iterable:
d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
in the keyword argument list. For example: dict(one=1, two=2)
# (copied from class doc)
"""
pass
ArgumentParser.parse_args returns a Namespace object instead of an iterable arrays.
For your reference, https://docs.python.org/3/library/argparse.html#parsing-arguments
ArgumentParser parses arguments through the parse_args() method. This will inspect the command line, convert each argument to the appropriate type and then invoke the appropriate action.
And it is not supposed to be used like that. Consider your use case, in the doc, it says argparse will figure out how to parse those out of sys.argv., which means you don't have to iterate over those arguments.
Related
Right now, I have a script that can accept command line arguments using argparse. For example, like this:
#foo.py
def function_with_args(optional_args=None):
parser = argparse.ArgumentParser()
# add some arguments
args = parser.parse_args(optional_args)
# do something with args
However, I'd like to be able to use this function with a dictionary instead, for example with something like this:
def function_using_dict(**kwargs):
# define parser and add some arguments
args = parser.parse_dict_args(kwargs)
# everything else is the same
Note that I have a lot of arguments with default values in argparse which I'd like to use, so the following wouldn't work:
def function_no_default_args(**kwargs):
args = kwargs # not using default values that we add to the parser!
argparse.Namespace is a relatively simple object subclass, with most of its code devoted to displaying the attributes (as print(args) shows). Internally parse_args uses get_attr and set_attr to access the namespace, minimizing the assumptions about attributes names.
When using subparsers, the subparser starts with a 'blank' namespace, and uses the following code to copy its values to the main namespace.
# In case this subparser defines new defaults, we parse them
# in a new namespace object and then update the original
# namespace for the relevant parts.
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
for key, value in vars(subnamespace).items():
setattr(namespace, key, value)
Originally the main namespace was passed to the subparser, eg. parser.parse_known_args(arg_strings, namespace), but the current version lets the subparser defaults take priority.
Handling defaults is a bit complicated. If you don't have any required arguments then
args = parser.parse_args([])
will set all the defaults. Or you could look at the start of parse.parse_known_args to see how defaults are inserted into the namespace at the start of parsing. Just beware that there's an added step at the end of parsing that runs remaining defaults through their respective type functions.
If you are trying to convert the result of parse_args into a dict, you can probably just do this:
kwargs = vars(args)
After your comment, I thought about it. Going to go with your existing function.
#foo.py
def function_with_args_and_default_kwargs(optional_args=None, **kwargs):
parser = argparse.ArgumentParser()
# add some arguments
# add the other arguments
for k, v in kwargs.items():
parser.add_argument('--' + k, default=v)
args = parser.parse_args(optional_args)
# do something with args
Attempting to pass an undetermined amount of integers using argparse. When I input: py main.py 3 2
%%writefile main.py
import sorthelper
import argparse
integers = 0
#top-level parser creation
parser = argparse.ArgumentParser("For sorting integers")
nargs = '+' #-> gathers cmd line arguments into a list
args = parser.add_argument('-f', metavar='N', type=int, nargs='+', help='yada yada yada')
args = parser.parse_args()
print(sorthelper.sortNumbers(args))
%%writefile sorthelper.py
def sortNumbers(args):
sorted(args)
Error Namespace Argument is not iterable
I think is is because I am passing an argument that is not of the correct type. After reading through all the documentation I could find I cannot figure out how to make this work. I want the program to sort the numbers I am passing.
parser.parse_args() returns a Namespace object, which is an object whose attributes represent the flags that were parsed. It is not iterable.
It seems like you want to get the command-line arguments given after -f, in which case you would take that particular flag out of the Namespace object:
print(sorthelper.sortNumbers(args.f))
Also, your code as you currently have it will print None, because sortNumbers() doesn't return anything. The built-in sorted() function does not sort in place (though list.sort() does, if you want to use that), so you have to actually do
def sortNumbers(args):
return sorted(args)
I have an argparse that is given a string:
def f():
return 'dummy2'
p = argparse.ArgumentParser()
p.add_argument('--a', nargs=1, type=str)
p.add_argument('--b', nargs='?', const=f(), default=f())
p.parse_args('--a dummy'.split())
The parser namespace is Namespace(a=['dummy'], b='dummy2').
How can I make the argument for a be stored as a string and not as a list of strings?
It's simple, just skip the argument for nargs. Try this:
p = argparse.ArgumentParser()
p.add_argument('--a', type=str)
p.add_argument('--b', nargs='?', const=f(), default=f())
I believe this is what you expected:
p.parse_args('--a dummy'.split())
=> Namespace(a='dummy', b='dummy2')
Quoting the docs:
ArgumentParser objects usually associate a single command-line argument with a single action to be taken. The nargs keyword argument associates a different number of command-line arguments with a single action. The supported values are:
N (an integer). N arguments from the command line will be gathered together into a list ... Note that nargs=1 produces a list of one item. This is different from the default, in which the item is produced by itself.
I am trying to create a script that takes multiple command line arguments of the same type, and then feeds them into a for loop for processing, the below is what I have in my code:
import argparse
parser = argparse.ArgumentParser(description='XYZ')
parser.add_argument('[values]', nargs='+', help='A list of values')
u = parser.parse_args(['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'])
for values in u:
... # do a bunch of stuff for each "value"
I understand that the values stored in parse_args are a NAMESPACE, hence the error below when I run my code:
TypeError: 'Namespace' object is not iterable
Is there any way to turn them into a list object? so that I can pass them to my for loop, or is there any other approach I should be using? Just getting started with python, so apologies in advance if this is a noob question
You need to iterate u.values (the argument name you specified in add_argument call) not u itself; To do that you need to rename [values] to values:
import argparse
parser = argparse.ArgumentParser(description='XYZ')
parser.add_argument('values', nargs='+', help='A list of values')
# ^^^^^^^^
u = parser.parse_args(['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'])
for value in u.values:
# ^^^^^^^^
# do a bunch of stuff for each "value"
The values from parsing are in a simple of object of class argparse.namespace (which can be customized - see the docs).
Usually you access those values by attribute name, e.g. args.foo. For unusual names you many have to use getattr(args, '[values]').
In the simple case argument values are a single string or number. With nargs like '*' or '+', the values will be in a list. With append action type you may even get lists within lists.
I encourage users to print the args namespace.
In [2]: parser = argparse.ArgumentParser(description='XYZ')
...: parser.add_argument('[values]', nargs='+', help='A list of values')....
In [3]: args =parser.parse_args(['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'])
In [4]: print(args)
Namespace([values]=['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'])
With your 'dest' I have to use the getattr to fetch the list of strings from args.
In [5]: getattr(args,'[values]')
Out[5]: ['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q']
args is a simple object, so you can add attributes or modify existing ones. e.g.
In [6]: args.values = ['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q']
In [7]: print(args)
Namespace([values]=['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'],
values=['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q'])
If you are more familiar with dictionaries, you can use vars():
In [9]: vars(args)['values']
Out[9]: ['3crlV5QaiuUjRz5kWze', 'F9Xw0rggHZ_PLs62q']
Play around with this sort of thing an interactive Python session.
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--pattern", help="Pattern file")
args = parser.parse_args()
Now is it possible to get back the string "--pattern" from args?
I need the string so that I can construct a cmd list to pass to Popen like Popen(['some_other_program', args.pattern.option_string, args.pattern], ...) without repeating it (and having to maintain it in two places) (Popen(['some_other_prog', '--pattern', args.pattern], ...)).
I need to create a wrapper for another program. Some of the args need to be passed to the wrapped program (via Popen) and some are required by the wrapper.
Is there a better method than the following example?
pass_1 = '--to-be-passed'
parser = argparse.ArgumentParser()
parser.add_argument("-p", pass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
...
process = Popen(['wrapped_program', pass_1, args.pass_1], ...)
...
This method of keeping the args in variables is not very good as:
Maintaining short options along with long options becomes difficult.
Popen if called in another function requires passing these variables(or a dict of them) to the function. This seems redundant as args passed to it should be sufficient.
Add a dest to your add_argument call.
parser.add_argmument("p", "--pattern", dest="pattern", help="your help text")
args = parser.parse_args()
args = vars(args)
The you can reference the pattern with args["pattern"] .
There doesn't seem to be an easy way to get the original option strings from the result of a parser.parse_args(), but you can get them from the parser object. You just need to peek into its __dict__, in order to retrieve the parser settings after it's created. In your case you want the _option_string_actions field. Unfortunately this doesn't seem officially supported, as I couldn't find a ArgumentParser method dedicated to this, so YMMV. On Python 3:
Demo:
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=int, default=1000, help='intensity of bar')
parser.add_argument('--bar', '-b', type=str, default='bla', help='whatever')
store_action_dict=vars(parser)['_option_string_actions']
print(store_action_dict.keys()) # dict_keys(['--help', '-b', '-f', '-h', '--foo', '--bar'])
The deleted answers and comments indicate there is some confusion as to what you want. So I'll add to that confusion.
Normally the parser does not record the option string. However it is provided to the Action __call__ method. So a custom Action class could save it. The FooAction custom class example in the argparse docs illustrates this.
If I define this action subclass:
In [324]: class PassThru(argparse._StoreAction):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, [values, option_string])
In [324]: p.add_argument('-o','--other',action=PassThru)
The option string is recorded along with the value ('-o' or '--other'):
In [322]: p.parse_args('-p test -o teseting'.split())
Out[322]: Namespace(other=['teseting', '-o'], pass_me_on='test')
In [323]: p.parse_args('-p test --other teseting'.split())
Out[323]: Namespace(other=['teseting', '--other'], pass_me_on='test')
Obviously the option_string and value could be recorded in a different order, in a dictionary, as seperate attributes in the Namespace, etc.
There are other ways of passing options to another program, particularly if the wrapping parser does not need to handle them itself.
argparse gets the arguments from sys.argv[1:], and does not change it. So even if your parser uses some of the arguments, you could pass that list on to popen (all or in part).
The argparse docs has an example, under nargs=REMAINDER, of parsing some arguments for itself, and collecting the rest to pass to another program. This is their example:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
So you could call popen with something like
plist = ['wrapped_program']
plist.extend(args.args)
popen(plist, ...)
Using parse.parse_known_args can also be used to collect unparsed words into an 'extras' list. That section of the docs talks about passing those strings on to another program (just as you are doing). In contrast with the REMAINDER case, the extra stuff does not have to be last.
These work, of course, only if this parser doesn't need --pattern for itself. If it parses it, then it won't appear appear in the REMAINDER or extras. In that case you will have to add it back to the list that you give popen
I would tweak your parser thus:
pass_1 = 'passed' # without the -- or internal -
dpass_` = '--'+pass_
parser = argparse.ArgumentParser()
parser.add_argument("-p", dpass_1, help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
process = Popen(['wrapped_program', dpass_1, getattr(args, pass_1)], ...)
another option:
parser = argparse.ArgumentParser()
pass_action = parser.add_argument("-p", '--pass-me-on', help="Pass me on")
parser.add_argument("-k", "--arg-for-wrapper")
args = parser.parse_args()
If you print pass_action (in a shell) you'll get something like:
_StoreAction(option_strings=['-p', '--pass-me-on'], dest='pass_me_on', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
So you could pull the --name and dest from that object, thus:
process = Popen(['wrapped_program', pass_action.option_strings[-1], getattr(args, pass_action.dest), ...], ...)
You have to look in sys.argv to see which option_string was used (the long, short or other). The parser does not record that anywhere.
Note '--pass-me-on' produced dest='pass_me_on'. The conversion of - to _ can complicate deriving one string from the other.
If you have a dest string, you have to use getattr to pull it from the args namespace, or use vars(args)[dest] (dictionary access).
Another issue. If --patten has nargs='+', its value will be a list, as opposed to a string. You'd have to careful when merging that into thepopen` argument list.