Develop Reference

argparse.ArgumentParser ArgumentError when adding arguments in multiple modules

I am working on automated test framework (using pytest) to test multiple flavors of an application. The test framework should be able to parse common (to all flavors) command line args and args specific to a flavor.
Here is how the code looks like:
parent.py:
import argparse
ARGS = None
PARSER = argparse.ArgumentParser()
PARSER.add_argument('--arg1', default='arg1', type=str, help='test arg1')
PARSER.add_argument('--arg2', default='arg2', type=str, help='test arg2')
def get_args():
global ARGS
if not ARGS:
ARGS = PARSER.parse_args()
return ARGS
MainScript.py:
import pytest
from parent import PARSER
ARGS = None
PARSER.conflict_handler = "resolve"
PARSER.add_argument('--arg3', default='arg3', type=str)
def get_args():
global ARGS
if not ARGS:
ARGS = PARSER.parse_args()
return ARGS
get_args()
def main():
pytest.main(['./Test_Cases.py', '-v'])
if __name__ == "__main__":
main()
Test_Cases.py
from MainScript import get_args
ARGS = get_args()
def test_case_one():
pass
Executing MainScript.py fails with following error:
E ArgumentError: argument --arg3: conflicting option string(s): --arg3

So the problem is that you have declared
PARSER.add_argument('--arg3', default='arg3', type=str)
in a global scope inside MainScript.py. That means that that line of code will be executed every time you import it like you do in Test_Cases.py hence why you get the conflict error, you're adding arg 3 to your argparse twice.
Easiest solution is to move PARSER.add_argument('--arg3', default='arg3', type=str) into your main() function as that will only get called once.
def main():
PARSER.add_argument('--arg3', default='arg3', type=str)
pytest.main(['./Test_Cases.py', '-v'])
But doing that causes another problem stemming from your multiple definition of get_args(). When you call get_args() before your main() it only has the two defined arguments from parent.py so it's missing arg3. If you move the call down into your main() or at least after your main() gets called it will work.
Personally I just removed both the definition and the call of get_args() from MainScript.py and it worked just fine.

How do I wrap argparse options into the python class?

I am pretty new to python OOP, so I got some confusion.
Currently I have:
parser = argparse.ArgumentParser(description='script 1.0')
parser.add_argument('-a', '--foo', help='specify foo')
parser.add_argument('-b', '--bar', type=int, help='specify bar')
parser.add_argument('-c', '--baz', help='specify baz')
parser.add_argument('-d', '--bar2', help='bar2')
args = parser.parse_args()
foo = args.foo
bar = args.bar
baz = args.baz
bar2 = args.bar2
which works pretty well, but I wan to create a class for the whole of my script and make argparse as a class's method (is it possible at all?).
So I tried:
import argparse
....
Class Program:
def __init__(self, foo, bar, baz, bar2):
self.foo = foo
self.bar = bar
...(so on for each arg)
def main():
parser = argparse.ArgumentParser(description='script 1.0')
parser.add_argument('-a', '--foo', help='specify foo')
parser.add_argument('-b', '--bar', type=int, help='specify bar')
parser.add_argument('-c', '--baz', help='specify baz')
parser.add_argument('-d', '--bar2', help='bar2')
args = parser.parse_args()
foo = self.foo
bar = self.bar
baz = self.baz
bar2 = self.bar2
I do not think I am doing right, though. I have not found too much info about it but one post on SO which did not clarify situation to me, so I want to have opinions for my specific case

argparse is already makes use of classes. argparse.ArgumentParser(...) creates a parser object. parser.add_argument(...) creates an Action object, and puts it in a parser list. It also returns it to your code, which may in some advanced uses be handy. parse_args() returns a argparse.Namespace object.
You can subclass the argparse classes to customize their performance.
But usually you don't need to create your own parser class - unless you need a lot of special behavior.
Here's what I'd consider to be a clean use of argparse:
import argparse
# your code
def main(args):
foo = args.foo
# other uses of args
def other(foo, bar):
# code using values
def make_parser():
parser = argparse.ArgumentParser(description='script 1.0')
parser.add_argument('-a', '--foo', help='specify foo')
parser.add_argument('-b', '--bar', type=int, help='specify bar')
parser.add_argument('-c', '--baz', help='specify baz')
parser.add_argument('-d', '--bar2', help='bar2')
return parser
if __name__ == '__main__':
parser = make_parser()
args = parser.parse_args()
Wrapping the parser definition in a function is a good idea. In more complex cases it could be a class. But normally you only need one parser per script, so a function is enough. That function can be in the main body of the script, but actual use should be in a if __name__ block, so the script can be imported without using the parser.
This puts the args namespace in the global environment, where is accessible to all your code. It can be passed as to your functions, or selected values can be passed:
main(args)
other(args.foo, args.bar)
foo = args.foo
do_things3(foo)
d = vars(args) # args in dictionary form
It's a good idea to write your code so it works as imported classes and/or functions, and when run as a script. The argparse part sets values when run as a script. When imported, the importing script sets the necessary control values, possibly with its own parser.

I would do it this way:
import argparse
Class Program:
def __init__(self, foo, bar, baz, bar2):
self.foo = foo
self.bar = bar
...(so on for each arg)
def do_things():
pass
def get_args():
parser = argparse.ArgumentParser(description='script 1.0')
parser.add_argument('-a', '--foo', help='specify foo')
parser.add_argument('-b', '--bar', type=int, help='specify bar')
parser.add_argument('-c', '--baz', help='specify baz')
parser.add_argument('-d', '--bar2', help='bar2')
return parser.parse_args()
def main(args):
instance = Program(args.foo,
args.bar,
args.baz,
args.bar2)
instance.do_things()
if __name__ == '__main__':
args = get_args()
main(args)
Explanation:
Try to separate things...
Parsing the arguments is done in the get_args function, and then pass the "args" to the main.
The main function is in charge to initialize the object by the parameters that been passed as args and perform some work on the object.

how to pass argparse arguments to a class

I'm very new to coding in general and Python in particular. I'm trying to learn how to pass argparse arguments I have created into a class for use the right/recommend way. In addition to learning python, I'm trying to learn how to do things in an OOP manner so that learning other, OOP-type languages comes a bit easier.
So here's a sample of what I am trying to do:
import argparse
class passyourcliargstome():
def __init__(self, whatdoiputheretogetmycliargs):
#how do I get my cli args here?
pass
def otherfunctionsthatdothings():
pass
if __name__ == '__main__':
#grab the arguments when the script is ran
parser = argparse.ArgumentParser(
description='Make things happen.')
parser.add_argument('-f', '--foo', action='store_true', default=False, help='here be dragons')
parser.add_argument('-b', '--bar', action='store_true', default=False, help='here be more dragons')
passyourcliargstome.otherfunctionsthatdothings()
So, I'm defining argparse arguments outside of the main class, and want to know how to get them inside the class. Is this even the right way to do it? should I just make argparse a function under my class?
Thank you in advance for any assistance, references, etc.
Edit: 11/16 2:18 EST
Note: Since I don't have enough rep to answer my own question, this is my only recourse for posting a proper answer.
Okay, it took me some doing, but I managed to piece this together. RyPeck's answers helped me in getting my arguments (something my code was missing), but then afterwards I was getting unbound method errors When I was trying to test the code. I had no idea what that meant. Did I mention that I live up to my screen name?
It didn't really click until I found and read this. Here is my working code. If anyone has anything to add to this, up to and including "You're still doing it wrong, do it this way, the right way." I'm all ears. In the meantime, thanks for your help.
import argparse
class Passyourcliargstome(object):
def __init__(self):
#here's how I got my args here
self.foo = args.foo
self.bar = args.bar
def otherfunctionsthatdothings(self):
print "args inside of the main class:"
print self.foo
print self.bar
if __name__ == '__main__':
#grab the arguments when the script is ran
parser = argparse.ArgumentParser(description='Make things happen.')
parser.add_argument('-f', '--foo', action='store_true', default=False, help='here be dragons')
parser.add_argument('-b', '--bar', action='store_true', default=False, help='here be more dragons')
args = parser.parse_args()
print "args outside of main:"
print args.foo
print args.bar
#this was the part that I wasn't doing, creating an instance of my class.
shell = Passyourcliargstome()
shell.otherfunctionsthatdothings()
Running this code with no arguments prints False four times. two times outside of the class instance, two times within the class instance.

Use parser.parse_args and wrap it with vars to convert the special argparse Namespace type to a regular Python dict. In general, you want this pattern:
def main():
parser = argparse.ArgumentParser()
parser.add_argument('foo')
parser.add_argument('bar')
args = parser.parse_args()
args_dict = vars(args)
After that, you can pass arguments explicitly or all at once to whatever class or function will take it. For a class, it is best to write the required arguments explicitly. Like so:
class MyClass(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def Print(self):
print self.foo
print self.bar
Now you can put those two together like this:
import argparse
class MyClass(object):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def Print(self):
print self.foo
print self.bar
def main():
parser = argparse.ArgumentParser()
parser.add_argument('foo')
parser.add_argument('bar')
args = parser.parse_args()
c1 = MyClass(args.foo, args.bar)
args_dict = vars(args)
c2 = MyClass(**args_dict)
Both c1 and c2 will be created. Best approach, though, is to create classes explicitly, as is done with c1.

A simple for loop can pass argument (or- set your attributes):
args_dict = vars(self.parser.parse_args())
# using argparse arguments as attributes of this (self) class
for item in args_dict:
setattr(self, item, args_dict[item])
but... maybe the elegant way would be to initialize your class with argparse and set them directly to the class by namespace:
class Foo:
def __init__(self)
self.parser = ArgumentParser()
self.parser.add_argument('-f', '--foo, default=False, action='store_true', help='foo or not?')
self.parser.add_argument('-b', '--bar', default=0, action='store', help='set the bar')
self.parser.parse_args(namespace=self)
an empty input is equivalent to:
class Foo:
def __init__(self)
self.foo = False
self.bar = 0

You have to do the following after you add your arguments.
args = parser.parse_args()
If you do a print on args, you'll see that you have all the arguments in a namespace argument.
You can then access them like so -
print args.foo
print args.bar
From there, you can treat them like normal variables. See the argparse documentation for greater detail and more info.

Python: How do I usefully unittest a method invoking OptionParser?

In my Python Unit Tests (via standard nosetests / unittest.TestCase) I have a class where I've created a method that does option parsing. I would like to write a nice unit testing routine that exercises it nicely.
More than that, with 4 questions below, I'm looking for the OPTIMAL way to do this. ... ?
from optparse import OptionParser
class Something(object):
def getOptionParser(self):
return OptionParser()
def parseMyOptions(self):
parser = self.getOptionParser()
parser.add_option('--debug', action='store_true', help='Run debug mode.')
parser.add_option('--logfile', default='log/logfile.log', help='what filename')
parser.add_option('--url', type='string', help='desired url')
(options, args) = parser.parse_args()
self.debug = options.debug
self.logfile = options.logfile
self.url = options.url
def main(self):
self.parseMyOptions()
class MockOptionParser(object):
def __init__(self):
self.url = None
self.debug = None
self.logfile = None
def add_option(self, *args, action=None, help=None, default=None):
pass
def parse_args(self):
pass
def error(self):
pass
class TestSomething(unittest.TestCase):
def test_something(self):
s = Something()
s.getOptionParser = MockOptionParser()
s.parseMyOptions()
assert True, "assert something useful?"
So the problems here are:
What useful thing can I assert?
How do I auto-generate MockOptionParser without reproducing the skeleton of OptionParser?
Notice that in MockOptionParser's add_option, I DON'T handle 'type' keyword, but I should. I'm sure I won't predict all the methods and keyword args that OptionParser uses and implement them in MockOptionParser. Is there a simpler way?
Initializing MockOptionParser with self vars for each option is UGLY. is there a way to do this nicely?
Is there an easy way to verify all the add_option() vars (options.debug, for instance) are used to set something like self.debug?

Python: Pass arguments from argparse to imported scripts

I'm building a command line tool which executes some python-scripts (k2_fig1 - k2_fig3) in one main *.py-file (let's call it "main_file.py"). In this "main_file.py" the user has to fill in some parameters for the database connection (username, dbname, etc.)
Now I don't know how to pass these parameters to every single python-script I am importing. What do I have to code to these imported files?
This is my code of the "main_file.py":
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-D', '--database', action="store", type=str, dest="my_dbname", required=True, help="DB name")
parser.add_argument('-U', '--username', action="store", type=str, dest="my_username", required=True, help="DB username")
args = parser.parse_args()
# Import different scripts
import k2_fig1
import k2_fig2
import k2_fig3
if __name__ == '__main__':
main()

Without knowing anything else about k2fig_1 et al., you'll need to call them using subprocess rather than importing them.
import argparse
import subprocess
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-D', '--database', action="store", type=str, dest="my_dbname", required=True, help="DB name")
parser.add_argument('-U', '--username', action="store", type=str, dest="my_username", required=True, help="DB username")
args = parser.parse_args()
for script in ['k2_fig1', 'k2_fig2', 'k2_fig3']:
subprocess.call([script, '-U', args.my_username, '-D', args.my_dbname])
if __name__ == '__main__':
main()

I think the best way is to copy the namespace attributes to a "config" module::
import argparse
from . import config
from . import other
def update_obj(dst, src):
for key, value in src.items():
setattr(dst, key, value)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-D', '--database')
parser.add_argument('-U', '--username')
args = parser.parse_args('-D foo'.split())
update_obj(config, args)
And the "other module"::
from . import config
def some_func():
assert config.database == 'foo'