I'm writing a test script in Python subclassing unittest.TestCase. There're multiple test methods starting with test_.
What I want to do know is to invoke different test methods according to input arguments, like trial test.py -option. I wonder if there's a way to do this in the unittest framework. Thanks a lot.
Yes you can! Straight from the unittest documentation on organizing your test code
Assume you have a TestCase that looks like this guy:
import unittest
class WidgetTestCase(unittest.TestCase):
def setUp(self):
self.widget = Widget('The widget')
def tearDown(self):
self.widget.dispose()
self.widget = None
def test_default_size(self):
self.assertEqual(self.widget.size(), (50,50),
'incorrect default size')
def test_resize(self):
self.widget.resize(100,150)
self.assertEqual(self.widget.size(), (100,150),
'wrong size after resize')
Now you can do something like this in your TestSuite setup/run code:
def suite():
suite = unittest.TestSuite()
suite.addTest(WidgetTestCase('test_default_size'))
suite.addTest(WidgetTestCase('test_resize'))
return suite
Obviously, you can use argparse to set your own options and customize the structure to your needs. Check out the documentation on the Organizing Your Test Code section. That example is just the most explicit one, there are plenty of other ways to test this cat.
Related
I have used python unittest quite a bit and there is a decorator method you can use to conditionally skip tests like this:
import unittest
class TestStringMethods(unittest.TestCase):
#unittest.skipIf(1 == 1, 'Skipped because 1 does indeed equal 1')
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
I wanted to add some features to the unittest.TestCase class though, so I subclassed it to start adding my own custom code:
import unittest
class CustomTestCase(unittest.TestCase):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs) # Just use whatever is in TestCase's init + our stuff
self.foo = 'foo' # our stuff
self.bar = 'bar'
def mymethod(self, param1, param2): # Some custom method I wanted to make for these custom test cases
pass
To continue using #unittest.skipIf I have been sticking import unittest at the top of any of my CustomTestCase test files, but I wonder if thats the correct way to be doing things. Am I importing more than I need? Or am I worried about nothing?
The fact that you've subclassed has little to do with what you're asking. In general, it's fine for subclasses to import or use their superclass: in fact, they must import their superclass when they're defined. It's the other way around that's the problem (superclass shouldn't know about its subclasses).
To continue using #unittest.skipIf I have been sticking import
unittest at the top of any of my CustomTestCase test files, but I
wonder if thats the correct way to be doing things. Am I importing
more than I need?
If you want to use any attribute off of the unittest module (including the skipIf decorator), then you have to import it into the module in question. It's no more complex than that.
If you're worried about something like header guards, like you need for C/C++ development, don't be. It doesn't work like the #include preprocessor directive (i.e. it's not actually including the source of the unittest module in your file).
If you're worried about importing unittest too many times, don't be. It's extremely common to import a module like unittest into many different modules of a given project.
Or am I worried about nothing?
Yes. Just import unittest whenever you need it and rid yourself of worry!
HTH.
I'm working on a module using sockets with hundreds of test cases. Which is nice. Except now I need to test all of the cases with and without socket.setdefaulttimeout( 60 )... Please don't tell me cut and paste all the tests and set/remove a default timeout in setup/teardown.
Honestly, I get that having each test case laid out on it's own is good practice, but i also don't like to repeat myself. This is really just testing in a different context not different tests.
i see that unittest supports module level setup/teardown fixtures, but it isn't obvious to me how to convert my one test module into testing itself twice with two different setups.
any help would be much appreciated.
you could do something like this:
class TestCommon(unittest.TestCase):
def method_one(self):
# code for your first test
pass
def method_two(self):
# code for your second test
pass
class TestWithSetupA(TestCommon):
def SetUp(self):
# setup for context A
do_setup_a_stuff()
def test_method_one(self):
self.method_one()
def test_method_two(self):
self.method_two()
class TestWithSetupB(TestCommon):
def SetUp(self):
# setup for context B
do_setup_b_stuff()
def test_method_one(self):
self.method_one()
def test_method_two(self):
self.method_two()
The other answers on this question are valid in as much as they make it possible to actually perform the tests under multiple environments, but in playing around with the options I think I like a more self contained approach. I'm using suites and results to organize and display results of tests. In order to run one tests with two environments rather than two tests I took this approach - create a TestSuite subclass.
class FixtureSuite(unittest.TestSuite):
def run(self, result, debug=False):
socket.setdefaulttimeout(30)
super().run(result, debug)
socket.setdefaulttimeout(None)
...
suite1 = unittest.TestSuite(testCases)
suite2 = FixtureSuite(testCases)
fullSuite = unittest.TestSuite([suite1,suite2])
unittest.TextTestRunner(verbosity=2).run(fullSuite)
I would do it like this:
Make all of your tests derive from your own TestCase class, let's call it SynapticTestCase.
In SynapticTestCase.setUp(), examine an environment variable to determine whether to set the socket timeout or not.
Run your entire test suite twice, once with the environment variable set one way, then again with it set the other way.
Write a small shell script to invoke the test suite both ways.
If your code does not call socket.setdefaulttimeout then you can run tests the following way:
import socket
socket.setdeaulttimeout(60)
old_setdefaulttimeout, socket.setdefaulttimeout = socket.setdefaulttimeout, None
unittest.main()
socket.setdefaulttimeout = old_setdefaulttimeout
It is a hack, but it can work
You could also inherit and rerun the original suite, but overwrite the whole setUp or a part of it:
class TestOriginal(TestCommon):
def SetUp(self):
# common setUp here
self.current_setUp()
def current_setUp(self):
# your first setUp
pass
def test_one(self):
# your test
pass
def test_two(self):
# another test
pass
class TestWithNewSetup(TestOriginal):
def current_setUp(self):
# overwrite your first current_setUp
I'm having trouble figuring out the proper way of using pythons unittest framework
I currently have 3 different implementations for a data structure class and unittests to test various things in the class as follows:
import fooHorse
import fooSnake
import fooMoose
import unittest
foo = fooHorse.foo()
#foo = fooSnake.foo()
#foo = fooMoose.foo()
class testFoo(unittest.TestCase):
def testSomething(self):
foo.do_something()
...
def testSomethingelse(self):
...
# etc
if __name__ == "__main__":
unittest.main()
How do I refactor the code so that all the tests are run for fooSnake.foo, fooMoose.foo and fooHorse.foo?
Just factor all the testing into a function and call it from the three tests :
class testFoo(unittest.TestCase):
def _doTest(self, foo):
foo.do_something()
# ...
def testFoorHorse(self):
foo = fooHorse.foo()
self._doTest(foo)
# and so on.
I wouldn't try to do anything more clever so that the test logic stays simple enought to be obviously bug-free.
I think it is better to keep the testing code inside the classes under test (so, inside the fooHorse classes and such). I can't really tell from the code segment, so maybe you are already doing that. Then to combine multiple tests to be run at the same time, or selecting a few of them, you could create a separate testing class that creates a test suite. In there, the individual unittests can be assigned. See the python docs for details on this.
I am unit testing mercurial integration and have a test class which currently creates a repository with a file and a clone of that repository in its setUp method and removes them in its tearDown method.
As you can probably imagine, this gets quite performance heavy very fast, especially if I have to do this for every test individually.
So what I would like to do is create the folders and initialize them for mercurial on loading the class, so each and every unittest in the TestCase class can use these repositories. Then when all the tests are run, I'd like to remove them. The only thing my setUp and tearDown methods then have to take care of is that the two repositories are in the same state between each test.
Basically what I'm looking for is a python equivalent of JUnit's #BeforeClass and #AfterClass annotations.
I've now done it by subclassing the TestSuite class, since the standard loader wraps all the test methods in an instance of the TestCase in which they're defined and puts them together in a TestSuite. I have the TestSuite call the before() and after() methods of the first TestCase. This of course means that you can't initialize any values to your TestCase object, but you probably want to do this in your setUp anyway.
The TestSuite looks like this:
class BeforeAfterSuite(unittest.TestSuite):
def run(self, result):
if len(self._tests) < 1:
return unittest.TestSuite.run(self, result)
first_test = self._tests[0]
if "before" in dir(first_test):
first_test.before()
result = unittest.TestSuite.run(self, result)
if "after" in dir(first_test):
first_test.after()
return result
For some slightly more finegrained control I also created the custom TestLoader which makes sure the BeforeAfterSuite is only used to wrap test-method-TestCase objects in, which looks like this:
class BeforeAfterLoader(unittest.TestLoader):
def loadTestsFromTestCase(self, testCaseClass):
self.suiteClass = BeforeAfterSuite
suite = unittest.TestLoader.loadTestsFromTestCase(self, testCaseClass)
self.suiteClass = unittest.TestLoader.suiteClass
return suite
Probably missing here is a try/except block around the before and after which could fail all the testcases in the suite or something.
from the Python unittest documentation :
setUpClass() :
A class method called before tests in an individual class run. setUpClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def setUpClass(cls):
...
New in version 2.7.
tearDownClass() :
A class method called after tests in an individual class have run. tearDownClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def tearDownClass(cls):
...
I have a test case:
class LoginTestCase(unittest.TestCase):
...
I'd like to use it in a different test case:
class EditProfileTestCase(unittest.TestCase):
def __init__(self):
self.t = LoginTestCase()
self.t.login()
This raises:
ValueError: no such test method in <class 'LoginTest: runTest`
I looked at the unittest code where the exception is being called, and it looks like the tests aren't supposed to be written this way. Is there a standard way to write something you'd like tested so that it can be reused by later tests? Or is there a workaround?
I've added an empty runTest method to LoginTest as a dubious workaround for now.
The confusion with "runTest" is mostly based on the fact that this works:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
unittest.main()
So there is no "runTest" in that class and all of the test-functions are being called. However if you look at the base class "TestCase" (lib/python/unittest/case.py) then you will find that it has an argument "methodName" that defaults to "runTest" but it does NOT have a default implementation of "def runTest"
class TestCase:
def __init__(self, methodName='runTest'):
The reason that unittest.main works fine is based on the fact that it does not need "runTest" - you can mimic the behaviour by creating a TestCase-subclass instance for all methods that you have in your subclass - just provide the name as the first argument:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
suite = unittest.TestSuite()
for method in dir(MyTest):
if method.startswith("test"):
suite.addTest(MyTest(method))
unittest.TextTestRunner().run(suite)
Here's some 'deep black magic':
suite = unittest.TestLoader().loadTestsFromTestCase(Test_MyTests)
unittest.TextTestRunner(verbosity=3).run(suite)
Very handy if you just want to test run your unit tests from a shell (i.e., IPython).
If you don't mind editing unit test module code directly, the simple fix is to add under case.py class TestCase a new method called runTest that does nothing.
The file to edit sits under pythoninstall\Lib\unittest\case.py
def runTest(self):
pass
This will stop you ever getting this error.
Guido's answer is almost there, however it doesn't explain the thing. I needed to look to unittest code to grasp the flow.
Say you have the following.
import unittest
class MyTestCase(unittest.TestCase):
def testA(self):
pass
def testB(self):
pass
When you use unittest.main(), it will try to discover test cases in current module. The important code is unittest.loader.TestLoader.loadTestsFromTestCase.
def loadTestsFromTestCase(self, testCaseClass):
# ...
# This will look in class' callable attributes that start
# with 'test', and return their names sorted.
testCaseNames = self.getTestCaseNames(testCaseClass)
# If there's no test to run, look if the case has the default method.
if not testCaseNames and hasattr(testCaseClass, 'runTest'):
testCaseNames = ['runTest']
# Create TestSuite instance having test case instance per test method.
loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
return loaded_suite
What the latter does, is converting test case class into test suite, that holds the instances of the class per its test method. I.e. my example will be turned into unittest.suite.TestSuite([MyTestCase('testA'), MyTestCase('testB')]). So if you would like to create a test case manually, you need to do the same thing.
#dmvianna's answer got me very close to being able to run unittest in a jupyter (ipython) notebook, but I had to do a bit more. If I wrote just the following:
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods)
unittest.TextTestRunner().run(suite)
I got
Ran 0 tests in 0.000s
OK
It's not broken, but it doesn't run any tests! If I instantiated the test class
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
(note the parens at the end of the line; that's the only change) I got
ValueError Traceback (most recent call last)
in ()
----> 1 suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
/usr/lib/python2.7/unittest/case.pyc in init(self, methodName)
189 except AttributeError:
190 raise ValueError("no such test method in %s: %s" %
--> 191 (self.class, methodName))
192 self._testMethodDoc = testMethod.doc
193 self._cleanups = []
ValueError: no such test method in : runTest
The fix is now reasonably clear: add runTest to the test class:
class TestStringMethods(unittest.TestCase):
def runTest(self):
test_upper (self)
test_isupper (self)
test_split (self)
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
unittest.TextTestRunner().run(suite)
Ran 3 tests in 0.002s
OK
It also works correctly (and runs 3 tests) if my runTest just passes, as suggested by #Darren.
This is a little yucchy, requiring some manual labor on my part, but it's also more explicit, and that's a Python virtue, isn't it?
I could not get any of the techniques via calling unittest.main with explicit arguments from here or from this related question Unable to run unittest's main function in ipython/jupyter notebook to work inside a jupyter notebook, but I am back on the road with a full tank of gas.
unittest does deep black magic -- if you choose to use it to run your unit-tests (I do, since this way I can use a very powerful battery of test runners &c integrated into the build system at my workplace, but there are definitely worthwhile alternatives), you'd better play by its rules.
In this case, I'd simply have EditProfileTestCase derive from LoginTestCase (rather than directly from unittest.TestCase). If there are some parts of LoginTestCase that you do want to also test in the different environment of EditProfileTestCase, and others that you don't, it's a simple matter to refactor LoginTestCase into those two parts (possibly using multiple inheritance) and if some things need to happen slightly differently in the two cases, factor them out into auxiliary "hook methods" (in a "Template Method" design pattern) -- I use all of these approaches often to diminish boilerplate and increase reuse in the copious unit tests I always write (if I have unit-test coverage < 95%, I always feel truly uneasy -- below 90%, I start to feel physically sick;-).