I need to use unittest in python to write some tests. I am testing the behavior of 2 classes, A and B, that have a lot of overlap in behavior because they are both subclasses of C, which is abstract. I would really like to be able to write 3 testing classes: ATestCase, BTestCase, and AbstractTestCase, where AbstractTestCase defines the common setup logic for ATestCase and BTestCase, but does not itself run any tests. ATestCase and BTestCase would be subclasses of AbstractTestCase and would define behavior/input data specific to A and B.
Is there a way to create an abstract class via python unittest that can take care of setup functionality by inheriting from TestCase, but not actually run any tests?
Sure, construct like that will surely work:
class BaseTestCase(unittest.TestCase):
def setUp(self):
pass # common teardown
def tearDown(self):
pass # common teardown
class ATestCase(BaseTestCase):
def test1(self):
pass
class BTestCase(BaseTestCase):
def test1(self):
pass
If knowledge from ATestCase or BTestCase is required in BaseTestCase simply override some method in subclasses but use it in superclass.
class BaseTestCase(unittest.TestCase):
def setUp(self):
self.instance = self._create_instance()
def _create_instance(self):
raise NotImplementedError()
class ATestCase(BaseTestCase):
def _create_instance(self):
return A()
class BestCase(BaseTestCase):
def _create_instance(self):
return B()
Note that if any test_(self) methods will be implemented in BaseTestCase, they'll run (and fail due to failing setUp) when discovered by automated runners.
As a workaround you may use skipTest in your setUp clause in abstract test and override it in subclasses.
class BaseTestCase(unittest.TestCase):
def setUp(self):
self.instance = self._create_instance()
def _create_instance(self):
self.skipTest("Abstract")
def test_fromBase(self):
self.assertTrue(True)
Note that skipping test_fromBase (e.g. via decorator) won't be good, since 'test should be skipped' logic will be inherited by all subclasses.
I tried Łukasz’s answer and it works, but I don’t like OK (SKIP=<number>) messages. For my own desires and aims for having a test suite I don’t want me or someone to start trusting any particular number of skipped tests, or not trusting and digging into the test suite and asking why something was skipped, and always?, and on purpose? For me that’s a non-starter.
I happen to use nosetests exclusively, and by convention test classes starting with _ are not run, so naming my base class _TestBaseClass is sufficient.
I tried this in Pycharm with Unittests and py.test and both of those tried to run my base class and its tests resulting in errors because there’s no instance data in the abstract base class. Maybe someone with specific knowledge of either of those runners could make a suite, or something, that bypasses the base class.
Related
I have a class and I need to test it
class ToBeTested():
def my_method():
pass
I have designed some unittests in python.
import unittest
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
# prepare some data
# run a method
# collect results
def test_one_assumption_about_results(self):
#self.assertEqual(...)
def test_another_assumption_about_results(self):
#self.assertEqual(...)
if __name__ == '__main__':
unittest.main()
I have designed this code looking at the examples.
Now I do not understand how to interface it the tesing class MyFirstUnitTest with the classe to be tested ToBeTested?
By inheritance?
class MyFirstUnitTest(unittest.TestCase,ToBeTested):
######
By creating a object of class ToBeTested as static member of
MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
mytestobj=ToBeTested()
def setUp(self):
By creating a object of class ToBeTested within each of the test
cases of MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
###
def test_one():
mytestobj=ToBeTested()
def test_two():
mytestobj=ToBeTested()
please suggest alternatives and why you would prefer any of them
There is a drawback to using inheritance and static member instead of creating a new object for every test: If tests change the state of the object, then the tests influence each other with their side effects. This gets even worse if the order of the test method calls are not guaranteed and changes (say, you rename a test and suddenly a different test case fails - these things have happened before!)
Additionally, having your test case inherit from the SUT (subject under test) means you only get to test one constructor.
I don't see any advantage to having the testcase inherit from the SUT, especially in python where there are no private members. If for some reason inheritance is necessary for the test (which sometimes IS the case), then having a seperate TestAdapter class that inherits and is instantiated by the test is the best practice.
There are other questions with answers which go more in depth:
https://softwareengineering.stackexchange.com/questions/366425/making-a-test-fixture-inherit-from-the-sut-class
https://softwareengineering.stackexchange.com/questions/154144/how-to-define-implementation-details
You can create an instance of your class(es) in your setup if it will not have to be reinstantiated.
And if you have to create a new instance in every test then just create a new instance in every test.
Consider this example:
class A:
def do_stuff(self):
# ...
some_var = self.helper_method()
# ...
def helper_method(self):
# This method must be implemented by subclass
raise NotImplementedError()
class B(A):
def helper_method(self):
# implementation for class B
class C(A):
def helper_method(self):
# implementation for class C
My task is to write unit tests for A, B and C classes (especially for do_stuff).
But how can I test A class if I cannot use some of it methods directly?
Should I just test only B and C classes (which have implementation for helper_method)
or maybe there is common way for testing abstract classes in Python?
You don't really have an abstract base class, at least as far as the language is concerned. Nothing is stopping you from instantiating it.
a = A()
If you were using the abc module to define classes that you can't instantiate:
class A(metaclass=abc.ABCMeta):
...
then you can make A instantiable by overriding its set of abstract methods:
A.__abstractmethods__ = frozenset()
a = A()
# test away
In either case, you can still test that an abstract method either raises NotImplementedError
try:
a.helper_method()
except NotImplementedError:
print("Test passed")
else:
print("Test failed")
or test its default implementation as needed.
do_stuff exists on A, so test it on A. The helper methods exist on the concrete classes so test them there. You can use the unittest.mock module to temporarily patch the abstract class so it will work with your test, and also patching the abstract method to return a specific value -- so that its logic is not under test. Given all that, this is how I would test an abstract class.
Given some abstract class:
from abc import abstractmethod, ABC
class MyAbstract(ABC):
def concrete_method(self):
i = self.abstract_method()
return i ** 2
#abstractmethod
def abstract_method(self):
"""return some int"""
pass
This is how I would test it.
from unittest import main, TestCase
from unittest.mock import patch, Mock
from module_under_test import MyAbstract
class TestMyAbstract(TestCase):
def test_cannot_instantiate(self):
"""showing we normally can't instantiate an abstract class"""
with self.assertRaises(TypeError):
MyAbstract()
#patch.multiple(MyAbstract,
__abstractmethods__=set(),
abstract_method=Mock(return_value=3))
def test_concrete_method(self):
"""patch abstract class and its abstract methods for duration of the test"""
# given
my_abstract = MyAbstract()
expected = 9
# when
actual = my_abstract.concrete_method()
# then
self.assertEqual(actual, expected)
if __name__ == "__main__":
main()
You should test logic, not the implementation. A's do_stuff() method has no logic itself, right? What it does depends on whether you are dealing with B or C. Instead it seems to me that it would make more sense to test B's and C's do_stuff() methods - you know what exactly they should do.
As #chepner has already answered your question, not to digress, but you should try to avoid using abstract classes in Python. Abstract classes don't or rather shouldn't serve much purpose much purpose in Dynamic DuckTyped languages like Python, Ruby etc. In Duck-typing as long as particular instance responds to a particular behavior, one shouldn't enforce it to be a child of a particular abstract class.
I'm using Pytest to test hardware via Python models.
My current conftest setup allows me to instantiate the Python model via funcargs, which I like.
def test_part1(target):
...
def test_part2(target):
...
etc.
My models have a deep, but simple single-inheritance structure:
class A():
def __init__:
self.attributes_added_by_class_A
...
class B(A):
def __init__:
super().__init__()
self.attributes_added_by_class_B
...
etc.
My pytests currently look like this:
def test_part1(target):
call_test_for_attributes_set_by_class_A(target)
call_test_for_attributes_set_by_class_B(target)
call_tests_specific_to_part1(target)
def test_part2(target):
call_test_for_attributes_set_by_class_A(target)
call_test_for_attributes_set_by_class_B(target)
call_tests_specific_to_part2(target)
...
I would like to avoid the need for repeating call_test_for_attributes_set_by_class_A(target), etc.
How can I organize/create my pytests so that I'm not constantly re-writing code to test attributes common to each target via their inheritance? In other words, if Part1 and Part2 both inherit from Class A, they both have attributes assigned by Class A.
Is there some way I could use Python's class inheritance structure to allow my pytests to reflect the same inheritance as my objects? In other words, is something like the following possible?
class PytestA():
def test_attributes_set_by_class_A()
class PytestB(PytestA):
def test_attributes_set_by_class_B()
test_all = PytestB(target)
I'm stuck trying to figure out how the above would receive the target argument as __init__() is now allowed (?) in pytest classes.
I'm using Python3
I think you want a fixture that depends on another fixture. See Modularity: using fixtures from a fixture function. Perhaps you want a generic target fixture, then a targetA fixture, and a targetB fixture. I don't entirely understand how your tests are working so I'm hesitant to give an example based on what you've written.
I am unit testing mercurial integration and have a test class which currently creates a repository with a file and a clone of that repository in its setUp method and removes them in its tearDown method.
As you can probably imagine, this gets quite performance heavy very fast, especially if I have to do this for every test individually.
So what I would like to do is create the folders and initialize them for mercurial on loading the class, so each and every unittest in the TestCase class can use these repositories. Then when all the tests are run, I'd like to remove them. The only thing my setUp and tearDown methods then have to take care of is that the two repositories are in the same state between each test.
Basically what I'm looking for is a python equivalent of JUnit's #BeforeClass and #AfterClass annotations.
I've now done it by subclassing the TestSuite class, since the standard loader wraps all the test methods in an instance of the TestCase in which they're defined and puts them together in a TestSuite. I have the TestSuite call the before() and after() methods of the first TestCase. This of course means that you can't initialize any values to your TestCase object, but you probably want to do this in your setUp anyway.
The TestSuite looks like this:
class BeforeAfterSuite(unittest.TestSuite):
def run(self, result):
if len(self._tests) < 1:
return unittest.TestSuite.run(self, result)
first_test = self._tests[0]
if "before" in dir(first_test):
first_test.before()
result = unittest.TestSuite.run(self, result)
if "after" in dir(first_test):
first_test.after()
return result
For some slightly more finegrained control I also created the custom TestLoader which makes sure the BeforeAfterSuite is only used to wrap test-method-TestCase objects in, which looks like this:
class BeforeAfterLoader(unittest.TestLoader):
def loadTestsFromTestCase(self, testCaseClass):
self.suiteClass = BeforeAfterSuite
suite = unittest.TestLoader.loadTestsFromTestCase(self, testCaseClass)
self.suiteClass = unittest.TestLoader.suiteClass
return suite
Probably missing here is a try/except block around the before and after which could fail all the testcases in the suite or something.
from the Python unittest documentation :
setUpClass() :
A class method called before tests in an individual class run. setUpClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def setUpClass(cls):
...
New in version 2.7.
tearDownClass() :
A class method called after tests in an individual class have run. tearDownClass is called with the class as the only argument and must be decorated as a classmethod():
#classmethod
def tearDownClass(cls):
...
I have a group of test cases that all should have exactly the same test done, along the lines of "Does method x return the name of an existing file?"
I thought that the best way to do it would be a base class deriving from TestCase that they all share, and simply add the test to that class. Unfortunately, the testing framework still tries to run the test for the base class, where it doesn't make sense.
class SharedTest(TestCase):
def x(self):
...do test...
class OneTestCase(SharedTest):
...my tests are performed, and 'SharedTest.x()'...
I tried to hack in a check to simply skip the test if it's called on an object of the base class rather than a derived class like this:
class SharedTest(TestCase):
def x(self):
if type(self) != type(SharedTest()):
...do test...
else:
pass
but got this error:
ValueError: no such test method in <class 'tests.SharedTest'>: runTest
First, I'd like any elegant suggestions for doing this. Second, though I don't really want to use the type() hack, I would like to understand why it's not working.
You could use a mixin by taking advantage that the test runner only runs tests inheriting from unittest.TestCase (which Django's TestCase inherits from.) For example:
class SharedTestMixin(object):
# This class will not be executed by the test runner (it inherits from object, not unittest.TestCase.
# If it did, assertEquals would fail , as it is not a method that exists in `object`
def test_common(self):
self.assertEquals(1, 1)
class TestOne(TestCase, SharedTestMixin):
def test_something(self):
pass
# test_common is also run
class TestTwo(TestCase, SharedTestMixin):
def test_another_thing(self):
pass
# test_common is also run
For more information on why this works do a search for python method resolution order and multiple inheritance.
I faced a similar problem. I couldn't prevent the test method in the base class being executed but I ensured that it did not exercise any actual code. I did this by checking for an attribute and returning immediately if it was set. This attribute was only set for the Base class and hence the tests ran everywhere else but the base class.
class SharedTest(TestCase):
def setUp(self):
self.do_not_run = True
def test_foo(self):
if getattr(self, 'do_not_run', False):
return
# Rest of the test body.
class OneTestCase(SharedTest):
def setUp(self):
super(OneTestCase, self).setUp()
self.do_not_run = False
This is a bit of a hack. There is probably a better way to do this but I am not sure how.
Update
As sdolan says a mixin is the right way. Why didn't I see that before?
Update 2
(After reading comments) It would be nice if (1) the superclass method could avoid the hackish if getattr(self, 'do_not_run', False): check; (2) if the number of tests were counted accurately.
There is a possible way to do this. Django picks up and executes all test classes in tests, be it tests.py or a package with that name. If the test superclass is declared outside the tests module then this won't happen. It can still be inherited by test classes. For instance SharedTest can be located in app.utils and then used by the test cases. This would be a cleaner version of the above solution.
# module app.utils.test
class SharedTest(TestCase):
def test_foo(self):
# Rest of the test body.
# module app.tests
from app.utils import test
class OneTestCase(test.SharedTest):
...