I have a code structure where we have a lot of sub classes from a common base class.
I want in an automated fashion test each subclass without having a separate class definition inheriting from TestCase for each subclass.
with a classs like
class MyBaseClass:
...
I can get a list of all subclasses that inherits from MyBaseClass with
import all_module_that_includes_subclasses
list_of_all_subclasses = MyBaseClass.__subclasses__()
But how do I connect to this list of scubclasses to avoid manually createing TestCases for each subclass like:
class TestSubClass1(TestCase):
def test_method_1(self):
...
...
class TestSubClass2(TestCase):
def test_method_1(self):
...
...
etc.
I thought of setting up an instance of all classes in setUp() method and looping through them in each def test_method_#(). This will somewhat work, I think, but it will break as soon as one single class fails. I want to be able to run through all classes and get a full report what class failed and in what tests.
Thankful for help :)
You can create those test cases dynamically using the load_tests protocol. For this you simply define a top-level function called load_tests which should return a TestSuite.
import unittest
class MyBaseClass:
pass
class Foo(MyBaseClass):
pass
class Bar(MyBaseClass):
pass
class TestBases:
# Wrapped into other class, so it won't be discovered by unittest.
class TestCase(unittest.TestCase):
subclass: MyBaseClass
def test(self):
self.assertIsInstance(self.subclass(), MyBaseClass)
def load_tests(loader, tests, pattern):
for cls in MyBaseClass.__subclasses__():
test_cls = type(f'Test{cls.__name__}', (TestBases.TestCase,), dict(subclass=cls))
tests.addTests(loader.loadTestsFromTestCase(test_cls))
return tests
if __name__ == '__main__':
unittest.main()
This runs the two dynamically generated test cases TestFoo and TestBar:
$ python /tmp/test.py -v
test (__main__.TestFoo) ... ok
test (__main__.TestBar) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
Related
I have an abstract class that I am using with the template pattern and some children that implement specific methods.
class TemplateClass(ABC):
#abstractmethod
def special_process_1():
pass
def common_process():
do_something()
def common_filter():
filter_something()
def __call__():
self.common_filter()
self.special_process_1()
self.common_process()
class classA(TemplateClass):
def special_process_1():
something_A_needs()
class classB(TemplateClass):
def special_process_1():
something_B_needs()
Now, I would like to test the __call__ method, but I am not sure what would be the best way. I think the best would be if I could test on the template so that I don't need to replicate test for classA and classB. However, I am not sure how to do it.
I have tried to test the template as follows:
#fixture
def template_mock():
with patch("TemplateClass.__abstractmethods__", set()):
t = TemplateClass()
t.special_process_1 = MagicMock(return_value=False)
yield t
The problem with the above is that on the tests, mypy would complain about template_mock.special_process_1 being a callable instead of a mock, so it does not have any return_value attribute.
Would be open to what other alternatives are there or if it makes sense at all to be testing this on the base class
Context
I have a test class where all my tests inherit from. It cant run by itself as it really doesnt contain any setup info
I wanted to add a test which is executed by ALL tests (adding it to the baseclass seems logical)
But now I notice the basetestclass( => Foo) which I import is being detected as a test itself and runs and is visible in the reports
Code
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the real test in test_something.py
from base import Foo # <= This is being detected as a testclass object and thus will be executed
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This triggers a testrun of the imported Foo
The Question
How can I import Foo without that its being detected as a 'test'
If I remove the test to run in all tests all is fine.
Adding #nottest decorator to Foo wont work since then also all inherited classes are defined nottest.
It needs to run on nose, pytest and unittest testrunners
I noticed if I changed the import statement like below that it also works. But that would mean adjusting a few hundreds of testfiles in different repos. (I'd like to avoid that)
import base
class TestFoo(base.Foo):
The key to the answer seems to be that each test has an attribute __test__ which is set to True when it is a test.
Setting it to False when the class should not be a test will then let the test collector ignore this class.
The answer assumes I can only do changes in the base.py
In python 3.9 classmethod and property decorators can be combined so I wrote a separate answer for that
answer for < py3.9
the base class in base.py
from unittest import TestCase
class MetaFoo(type):
#property
def __test__(cls):
return cls != Foo
class Foo(TestCase, metaclass=MetaFoo):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
answer for >= py3.9
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
#property
def __test__(cls):
return cls != Foo
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the actual test
test_something.py
from base import Foo # <= This will not be detected as a test anymore as __test__ returns False
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This doesnt trigger a testrun of the imported Foo anymore
With python and nosetests I have the following setup:
- package
- __init__.py
- test1.py
- test2.py
The __init__.py module contains a set up function
def setup():
print("Setup called")
var = 42
which will be used later to create a unique identified (different between running the tests, but the same for all the tests inside the package).
How can the tests itself access this variable (in this example case var)? The test scripts are just some stubs:
from nose.tools import assert_true
class TestSuite(object):
def test1(self):
# How to get content of 'var' here?
assert_true(True)
Is there some pythonic way to do this, or just use an environment variable to do this?
nose calls .setup() methods inside classes:
class Test:
def setup(self):
self.var = 1
def test_print_var(self):
print(self.var)
This also applies to methods inherited from elsewhere:
class TestBase:
def setup(self):
self.var = 1
class Test(TestBase):
def test_print_var(self):
print(self.var)
Suppose I need to implement an abstract Python interface which then will have many derived classes (each named equally but written in different modules), and in base class I heed to have a common method which will use a particular imported derived class' static method.
So my toy modules look like this:
abstract_class.py
from abc import ABCMeta, abstractmethod
from derived_class import Derived
class Abstract:
__metaclass__ = ABCMeta
#abstractmethod
def foo(self):
pass
def bar(self):
Derived.foo()
derived_class.py
from abstract_class import Abstract
class Derived(Abstract):
#staticmethod
def foo():
print 'Good news everyone!'
if __name__ == '__main__':
derived_object = Derived()
derived_object.bar()
Then of course when I'm trying to run derived_class.py, I get the Abstract name import error.
How do I properly organize this?
On the other hand, if you absolutely needed to do this without an object instance, it's possible to do with classmethods rather than staticmethods.
from abc import ABC, abstractmethod
class MyAbstractClass(ABC):
#staticmethod
#abstractmethod
def foo(label: str):
raise NotImplementedError()
#classmethod
def foo_agnostic(cls, label: str):
"""
NOTE: Here, this method doesn't have a reference to an instance of the class.
Instead, it only has a reference to the class itself; but that is enough
to call the abstract static foo() method.
"""
cls.foo(label)
class MyDerivedClass(MyAbstractClass):
#staticmethod
def foo(label: str):
print(label)
if __name__ == "__main__":
instance = MyDerivedClass()
instance.foo("Test 1") # Outputs "Test 1"
instance.foo_agnostic("Test 2") # Outputs "Test 2"
MyDerivedClass.foo_agnostic("Test 3") # Outputs "Test 3"
... in base class I heed to have a common method which will use a
particular imported derived class' static method
If I understand your question correctly, I'd say that this functionality is available out of the box with one small exception: Don't use a static method; just use a regular instance method.
Defining an abstract method in the base class will ensure that derived classes contain an implementation for that method. And, out of the box, the method defined in the derived class will get called when you call derived_object.bar().
I'm trying to implement an integration test framework using nose. At the core, I'd like a base class that all test classes inherit. I'd like to have a class setup function that is called as well as the per test setup function. When I use nosetests a_file.py -vs where a_file.py looks like this:
from nose import tools
class BaseClass(object):
def __init__(self):
print 'Initialize Base Class'
def setup(self):
print "\nBase Setup"
def teardown(self):
print "Base Teardown"
#tools.nottest
def a_test(self):
return 'This is a test.'
#tools.nottest
def another_test(self):
return 'This is another test'
class TestSomeStuff(BaseClass):
def __init__(self):
BaseClass.__init__(self)
print 'Initialize Inherited Class'
def setup(self):
BaseClass.setup(self)
print "Inherited Setup"
def teardown(self):
BaseClass.teardown(self)
print 'Inherited Teardown'
def test1(self):
print self.a_test()
def test2(self):
print self.another_test()
Outputs this:
Initialize Base Class
Initialize Inherited Class
Initialize Base Class
Initialize Inherited Class
cases.nose.class_super.TestSomeStuff.test1 ...
Base Setup
Inherited Setup
This is a test.
Base Teardown
Inherited Teardown
ok
cases.nose.class_super.TestSomeStuff.test2 ...
Base Setup
Inherited Setup
This is another test
Base Teardown
Inherited Teardown
ok
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK
How do I make the __init__, setup, and teardown functions class methods? When I attempt this:
from nose import tools
class BaseClass(object):
def __init__(self):
print 'Initialize Base Class'
#classmethod
def setup_class(self):
print "\nBase Setup"
#classmethod
def teardown_class(self):
print "Base Teardown"
#tools.nottest
def a_test(self):
return 'This is a test.'
#tools.nottest
def another_test(self):
return 'This is another test'
class TestSomeStuff(BaseClass):
def __init__(self):
BaseClass.__init__(self)
print 'Initialize Inherited Class'
#classmethod
def setup_class(self):
BaseClass.setup_class(self)
print "Inherited Setup"
#classmethod
def teardown_class(self):
BaseClass.teardown_class(self)
print 'Inherited Teardown'
def test1(self):
print self.a_test()
def test2(self):
print self.another_test()
I get this:
Initialize Base Class
Initialize Inherited Class
Initialize Base Class
Initialize Inherited Class
ERROR
======================================================================
ERROR: test suite for <class 'cases.nose.class_super.TestSomeStuff'>
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
self.setUp()
File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
self.setupContext(ancestor)
File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 314, in setupContext
try_run(context, names)
File "/usr/lib/python2.7/dist-packages/nose/util.py", line 478, in try_run
return func()
File "/home/ryan/project/python_testbed/cases/nose/class_super.py", line 30, in setup_class
BaseClass.setup_class(self)
TypeError: setup_class() takes exactly 1 argument (2 given)
----------------------------------------------------------------------
Ran 0 tests in 0.001s
FAILED (errors=1)
Removing the self from the super class calls (BaseClass.setup_class(self) -> BaseClass.setup_class()) seems to fix it...which I don't understand:
Initialize Base Class
Initialize Inherited Class
Initialize Base Class
Initialize Inherited Class
Base Setup
Inherited Setup
cases.nose.class_super.TestSomeStuff.test1 ... This is a test.
ok
cases.nose.class_super.TestSomeStuff.test2 ... This is another test
ok
Base Teardown
Inherited Teardown
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK
However, this doesn't help with the __init__ function. How can I make this a class method? Why does passing in self to the super class fail?
Does anyone have some info on this?
Class methods take a single implicit argument, (called cls by convention, although you have called it self too), like instance methods take self.
When you call
BaseClass.setup_class(self)
It's really more like
BaseClass.setup_class(BaseClass, self)
hence the warning over two arguments. Therefore it's fixed when you ditch self; as a reminder, change the definitions:
#classmethod
def setup_class(cls):
Oh, and __init__ makes no sense as a #classmethod; it's for setting up instances.
Looks like #Jonrsharpe already beat me, but yeah. I'll post it anyway.
This one might explain the setup_class() error. It is passing in an instance of BaseClass as well as 'self' (or TestSomeStuff) when you include the self. Nose must be hard coded to not allow more then 1 parameter in that function (or Unittest, not sure which one has that requirement).
As for the init part, after reading through the Unittest documentation, it appears that the code would be better printed as:
def __init__(self):
print 'Initialize Base Class Object'
And:
def __init__(self):
BaseClass.__init__(self)
print 'Initialize Inherited Class Object'
Since it's basically creating an object for every test case and running that init function to make sure the test case is ready.