My code:
import unittest
class MyTestCase(unittest.TestCase):
def setUp(self):
print("setUp")
def tearDown(self):
print("tearDown")
def test_something(self):
for i in range(4):
with self.subTest():
self.assertEqual(True, True) # add assertion here
if __name__ == '__main__':
unittest.main()
Run the test test_something in it, and we get the result:
test_something (tests.ui.test_example.MyTestCase) ... setUp
tearDown
ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
You can see I use self.subTest() to parameterize my test, which is convenient. The problem is, I want python unittest to call setUp and tearDown for each subTest, i.e. they should be called 4 times each. However, they are actually only called once each (You can verify that from the test output above). Is there any way to achieve it?
This is not the usecase of subTest() method. The Setup() and TearDown() methods are executed before each test case, since you only have one test case, theses two methods will only be called once.
See : https://docs.python.org/3/library/unittest.html#test-cases
If your subTests need a set-up that means there are test cases,.
If you don't want to have many test cases, try to call the SetUp() methods manually inside your loop.
I've written a BaseTestCase class (which is meant to be inherited by all test cases) and a number of TestCase<Number>(BaseTestCase) classes (each in its own .py file).
The project structure is:
BaseTestCase.py
TestCase1.py
TestCase2.py
Example:
class TestCase2(BaseTestCase):
# methods
def execute(self):
self.method1()
self.method2()
self.method3()
The execute method should be called on an instance of TestCase<Number> to run a specific test. What is the proper way to run all test cases, i.e. to orchestrate them all?
I'm assuming you are using Unittest framework and want to run tests programmatically and have the ability to select which tests you run. If that's the correct assumption, here is the answer.
Lets assume you have those tests:
class TestExample1(unittest.TestCase):
def test_something(self):
pass
def test_something_else(self):
pass
class TestExample2(unittest.TestCase):
def test_something(self):
pass
def test_something_else(self):
pass
If those are in the same file, you can simply run them like this
# assuming your tests are above in this same file
unittest.main()
However, if they are in different files, you will have to extract the tests from them like so
for example in [TestExample1, TestExample2]:
tests = unittest.TestLoader().loadTestsFromTestCase(example)
unittest.TextTestRunner().run(tests)
This will result in two different executions like this
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
If you are considering other frameworks for running your tests, I recommend to take a look at Test Junkie: https://www.test-junkie.com/get-started/ as its much easier to get started with and is very powerful. (disclaimer: I'm the author!).
Background
In Python's unittest framework, it is a fairly common idiom to use inheritance on a base set of tests to apply an entire set of tests to a new problem, and occasionally to add additional tests. A trivial example would be:
from unittest import TestCase
class BaseTestCase(TestCase):
VAR = 3
def test_var_positive(self):
self.assertGreaterEqual(self.VAR, 0)
class SubTestCase(BaseTestCase):
VAR = 8
def test_var_even(self):
self.assertTrue(self.VAR % 2 == 0)
Which, when run, runs 3 tests:
$ python -m unittest -v
test_var_positive (test_unittest.BaseTestCase) ... ok
test_var_even (test_unittest.SubTestCase) ... ok
test_var_positive (test_unittest.SubTestCase) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.000s
This is particularly useful if you are testing a class hierarchy, where each subclass is a subtype of the parent classes, and should thus be able to pass the parent class's test suite in addition to its own.
Problem
I would like to switch over to using pytest, but I have a lot of tests that are structured this way. From what I can tell, pytest intends to replace most of the functionality of TestCase classes with fixtures, but is there a pytest idiom that allows test inheritance, and if so what is it?
I am aware that pytest can be used to run unittest-style tests, but the support is limited, and I would like to use some of the "will never be supported" features of pytest in my tests.
Pytest allows you to group test cases in classes, so it naturally has support for test case inheritance.
When rewriting your unittest tests to pytest tests, remember to follow pytest's naming guidelines:
class names must begin with Test
function/method names must begin with test_
Failing to comply with this naming scheme will prevent your tests from being collected and executed.
Your tests rewritten for pytest would look like this:
class TestBase:
VAR = 3
def test_var_positive(self):
assert self.VAR >= 0
class TestSub(TestBase):
VAR = 8
def test_var_even(self):
assert self.VAR % 2 == 0
I am a novice to pytest. I have a scenario wherein i wanted to test some test methods written in a subclass.
Assume that the following is my code structure
class Superclass:
def __init__(self, a):
self.a = a
def dummy_print():
print("This is a dummy function")
class TestSubClass(Superclass):
def test_1_eq_1():
assert 1 == 1
Upon executing the following command
py.test -s -v test_filename.py
I get the following error messgae:
cannot collect test class 'Test_Super' because it has a init constructor
The same is mentioned in the pytest documentation as well.
Is there a workaround for this?
I need the superclass' __init__() method because all my test files would need to inherit from the super class.
Pytest is different beast than, say unittests. It prohibits class hierarchies, which you can see in the warning message.
If you need some pre-initialization steps, do them using fixtures:
#pytest.fixture(autouse=True)
def pre_run_initialization():
# do before test
yield
# do after test
I have a test case:
class LoginTestCase(unittest.TestCase):
...
I'd like to use it in a different test case:
class EditProfileTestCase(unittest.TestCase):
def __init__(self):
self.t = LoginTestCase()
self.t.login()
This raises:
ValueError: no such test method in <class 'LoginTest: runTest`
I looked at the unittest code where the exception is being called, and it looks like the tests aren't supposed to be written this way. Is there a standard way to write something you'd like tested so that it can be reused by later tests? Or is there a workaround?
I've added an empty runTest method to LoginTest as a dubious workaround for now.
The confusion with "runTest" is mostly based on the fact that this works:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
unittest.main()
So there is no "runTest" in that class and all of the test-functions are being called. However if you look at the base class "TestCase" (lib/python/unittest/case.py) then you will find that it has an argument "methodName" that defaults to "runTest" but it does NOT have a default implementation of "def runTest"
class TestCase:
def __init__(self, methodName='runTest'):
The reason that unittest.main works fine is based on the fact that it does not need "runTest" - you can mimic the behaviour by creating a TestCase-subclass instance for all methods that you have in your subclass - just provide the name as the first argument:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
suite = unittest.TestSuite()
for method in dir(MyTest):
if method.startswith("test"):
suite.addTest(MyTest(method))
unittest.TextTestRunner().run(suite)
Here's some 'deep black magic':
suite = unittest.TestLoader().loadTestsFromTestCase(Test_MyTests)
unittest.TextTestRunner(verbosity=3).run(suite)
Very handy if you just want to test run your unit tests from a shell (i.e., IPython).
If you don't mind editing unit test module code directly, the simple fix is to add under case.py class TestCase a new method called runTest that does nothing.
The file to edit sits under pythoninstall\Lib\unittest\case.py
def runTest(self):
pass
This will stop you ever getting this error.
Guido's answer is almost there, however it doesn't explain the thing. I needed to look to unittest code to grasp the flow.
Say you have the following.
import unittest
class MyTestCase(unittest.TestCase):
def testA(self):
pass
def testB(self):
pass
When you use unittest.main(), it will try to discover test cases in current module. The important code is unittest.loader.TestLoader.loadTestsFromTestCase.
def loadTestsFromTestCase(self, testCaseClass):
# ...
# This will look in class' callable attributes that start
# with 'test', and return their names sorted.
testCaseNames = self.getTestCaseNames(testCaseClass)
# If there's no test to run, look if the case has the default method.
if not testCaseNames and hasattr(testCaseClass, 'runTest'):
testCaseNames = ['runTest']
# Create TestSuite instance having test case instance per test method.
loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
return loaded_suite
What the latter does, is converting test case class into test suite, that holds the instances of the class per its test method. I.e. my example will be turned into unittest.suite.TestSuite([MyTestCase('testA'), MyTestCase('testB')]). So if you would like to create a test case manually, you need to do the same thing.
#dmvianna's answer got me very close to being able to run unittest in a jupyter (ipython) notebook, but I had to do a bit more. If I wrote just the following:
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods)
unittest.TextTestRunner().run(suite)
I got
Ran 0 tests in 0.000s
OK
It's not broken, but it doesn't run any tests! If I instantiated the test class
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
(note the parens at the end of the line; that's the only change) I got
ValueError Traceback (most recent call last)
in ()
----> 1 suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
/usr/lib/python2.7/unittest/case.pyc in init(self, methodName)
189 except AttributeError:
190 raise ValueError("no such test method in %s: %s" %
--> 191 (self.class, methodName))
192 self._testMethodDoc = testMethod.doc
193 self._cleanups = []
ValueError: no such test method in : runTest
The fix is now reasonably clear: add runTest to the test class:
class TestStringMethods(unittest.TestCase):
def runTest(self):
test_upper (self)
test_isupper (self)
test_split (self)
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
unittest.TextTestRunner().run(suite)
Ran 3 tests in 0.002s
OK
It also works correctly (and runs 3 tests) if my runTest just passes, as suggested by #Darren.
This is a little yucchy, requiring some manual labor on my part, but it's also more explicit, and that's a Python virtue, isn't it?
I could not get any of the techniques via calling unittest.main with explicit arguments from here or from this related question Unable to run unittest's main function in ipython/jupyter notebook to work inside a jupyter notebook, but I am back on the road with a full tank of gas.
unittest does deep black magic -- if you choose to use it to run your unit-tests (I do, since this way I can use a very powerful battery of test runners &c integrated into the build system at my workplace, but there are definitely worthwhile alternatives), you'd better play by its rules.
In this case, I'd simply have EditProfileTestCase derive from LoginTestCase (rather than directly from unittest.TestCase). If there are some parts of LoginTestCase that you do want to also test in the different environment of EditProfileTestCase, and others that you don't, it's a simple matter to refactor LoginTestCase into those two parts (possibly using multiple inheritance) and if some things need to happen slightly differently in the two cases, factor them out into auxiliary "hook methods" (in a "Template Method" design pattern) -- I use all of these approaches often to diminish boilerplate and increase reuse in the copious unit tests I always write (if I have unit-test coverage < 95%, I always feel truly uneasy -- below 90%, I start to feel physically sick;-).