I'm using Pytest to test hardware via Python models.
My current conftest setup allows me to instantiate the Python model via funcargs, which I like.
def test_part1(target):
...
def test_part2(target):
...
etc.
My models have a deep, but simple single-inheritance structure:
class A():
def __init__:
self.attributes_added_by_class_A
...
class B(A):
def __init__:
super().__init__()
self.attributes_added_by_class_B
...
etc.
My pytests currently look like this:
def test_part1(target):
call_test_for_attributes_set_by_class_A(target)
call_test_for_attributes_set_by_class_B(target)
call_tests_specific_to_part1(target)
def test_part2(target):
call_test_for_attributes_set_by_class_A(target)
call_test_for_attributes_set_by_class_B(target)
call_tests_specific_to_part2(target)
...
I would like to avoid the need for repeating call_test_for_attributes_set_by_class_A(target), etc.
How can I organize/create my pytests so that I'm not constantly re-writing code to test attributes common to each target via their inheritance? In other words, if Part1 and Part2 both inherit from Class A, they both have attributes assigned by Class A.
Is there some way I could use Python's class inheritance structure to allow my pytests to reflect the same inheritance as my objects? In other words, is something like the following possible?
class PytestA():
def test_attributes_set_by_class_A()
class PytestB(PytestA):
def test_attributes_set_by_class_B()
test_all = PytestB(target)
I'm stuck trying to figure out how the above would receive the target argument as __init__() is now allowed (?) in pytest classes.
I'm using Python3
I think you want a fixture that depends on another fixture. See Modularity: using fixtures from a fixture function. Perhaps you want a generic target fixture, then a targetA fixture, and a targetB fixture. I don't entirely understand how your tests are working so I'm hesitant to give an example based on what you've written.
Related
I need to define a variable in a fixture method setup_class for pytest like
class testClassForPytest:
#classmethod
def setup_class(cls):
m = aClassInstance(classInput)
def test_case_1(self):
#use of the instance m with a method of its class
I tried above approach but wasn't able to use the instance m.
It seems like the setting up of test suite (the class) for pytest is inherited from the standard python library unittest. With that library, the method is called setUpClass and should probably call the same method on its parent class. However since you are using pytest, which doesn't seem to follow the idea of test suites, I'm not entirely sure this is a problem. Still, it might be one source of problems if the method is supposed to be called something else.
In your code the method setup_class, you set a variable m to a value, but this variable is restricted to the local scope of the method. Only if you previously defined the variable on the scope of the class, or if you explicitly reference the variable in the class scope, can you later access the value again.
For example you could simply move the declaration into the class scope:
class testClassForPytest:
m = aClassInstance(classInput)
def test_case_1(self):
#use of the instance m with a method of its class
In this case you don't the setup method at all. If you need specific inputs that are not available on class declaration, you can also set the variable directly:
#classmethod
def setup_class(cls):
cls.m = aClassInstance(classInput)
I suppose that you meant, that in setup_class you meant to do cls.m = aClassInstance(classInput), not just m = ...
for that, you could modify your code to use pytest's class scope fixture to achieve the same result:
#pytest.fixture(scope='class')
def setup_class(request):
m = aClassInstance(classInput)
request.cls.m = m # makes pytest to add created instance to class to which fixture is applied to
#pytest.mark.usefixtures("setup_class")
class testClassForPytest:
def test_case_1(self):
#use of the instance m as self.m
class Test_visualization:
data: int = 0
#pytest.fixture(autouse=True)
def setup(self):
self.data = 3
#classmethod
def setup_class(cls):
Test_visualization.data = 3
setup_class Class will run only onces inside class
setup will run for every test creating a new instance of the class
From pytest documentation its written
Something to be aware of when grouping tests inside classes is that each test has a unique instance of the class. Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices
So it is better to define class variables that are common across all tests and instance variables that are created every time for a test
Example for Pytest auto use Fixture
Pytest Fixture Documentation
I have a class and I need to test it
class ToBeTested():
def my_method():
pass
I have designed some unittests in python.
import unittest
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
# prepare some data
# run a method
# collect results
def test_one_assumption_about_results(self):
#self.assertEqual(...)
def test_another_assumption_about_results(self):
#self.assertEqual(...)
if __name__ == '__main__':
unittest.main()
I have designed this code looking at the examples.
Now I do not understand how to interface it the tesing class MyFirstUnitTest with the classe to be tested ToBeTested?
By inheritance?
class MyFirstUnitTest(unittest.TestCase,ToBeTested):
######
By creating a object of class ToBeTested as static member of
MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
mytestobj=ToBeTested()
def setUp(self):
By creating a object of class ToBeTested within each of the test
cases of MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
###
def test_one():
mytestobj=ToBeTested()
def test_two():
mytestobj=ToBeTested()
please suggest alternatives and why you would prefer any of them
There is a drawback to using inheritance and static member instead of creating a new object for every test: If tests change the state of the object, then the tests influence each other with their side effects. This gets even worse if the order of the test method calls are not guaranteed and changes (say, you rename a test and suddenly a different test case fails - these things have happened before!)
Additionally, having your test case inherit from the SUT (subject under test) means you only get to test one constructor.
I don't see any advantage to having the testcase inherit from the SUT, especially in python where there are no private members. If for some reason inheritance is necessary for the test (which sometimes IS the case), then having a seperate TestAdapter class that inherits and is instantiated by the test is the best practice.
There are other questions with answers which go more in depth:
https://softwareengineering.stackexchange.com/questions/366425/making-a-test-fixture-inherit-from-the-sut-class
https://softwareengineering.stackexchange.com/questions/154144/how-to-define-implementation-details
You can create an instance of your class(es) in your setup if it will not have to be reinstantiated.
And if you have to create a new instance in every test then just create a new instance in every test.
I have a base contract class which can be inherited to provide plugin functionality. I'm adding the new plugins using setuptools entrypoints something like
entry_points="""
[plugins]
plugin1=plugins.plugin1:Plugin1
"""
And classes look like...
class Plugin:
__metaclass__ = abc.ABCMeta
#abstractmethod
def must_override_method():
pass
#abstractmethod
def must_override_method2():
pass
#./plugins/plugin1.py
#Actually the plugins could be anywhere
class Plugin1(Plugin):
def must_override_method():
print("Hello")
Although the #abstracmethod doesn't let me instantiate the class at runtime if must_override_methods are not defined but how should I go about adding unit tests for the plugins that are not yet written.
Is there a simple way to write generic test that catches "plugins" that don't implement abstract methods while testing?
I think, the best way is to use mocking for that abstract class. Mocking is a mechanism where it won't really creates an object or try to create. Rather, It will create a mock object which will have same properties. please use mock module for the same
I need to use unittest in python to write some tests. I am testing the behavior of 2 classes, A and B, that have a lot of overlap in behavior because they are both subclasses of C, which is abstract. I would really like to be able to write 3 testing classes: ATestCase, BTestCase, and AbstractTestCase, where AbstractTestCase defines the common setup logic for ATestCase and BTestCase, but does not itself run any tests. ATestCase and BTestCase would be subclasses of AbstractTestCase and would define behavior/input data specific to A and B.
Is there a way to create an abstract class via python unittest that can take care of setup functionality by inheriting from TestCase, but not actually run any tests?
Sure, construct like that will surely work:
class BaseTestCase(unittest.TestCase):
def setUp(self):
pass # common teardown
def tearDown(self):
pass # common teardown
class ATestCase(BaseTestCase):
def test1(self):
pass
class BTestCase(BaseTestCase):
def test1(self):
pass
If knowledge from ATestCase or BTestCase is required in BaseTestCase simply override some method in subclasses but use it in superclass.
class BaseTestCase(unittest.TestCase):
def setUp(self):
self.instance = self._create_instance()
def _create_instance(self):
raise NotImplementedError()
class ATestCase(BaseTestCase):
def _create_instance(self):
return A()
class BestCase(BaseTestCase):
def _create_instance(self):
return B()
Note that if any test_(self) methods will be implemented in BaseTestCase, they'll run (and fail due to failing setUp) when discovered by automated runners.
As a workaround you may use skipTest in your setUp clause in abstract test and override it in subclasses.
class BaseTestCase(unittest.TestCase):
def setUp(self):
self.instance = self._create_instance()
def _create_instance(self):
self.skipTest("Abstract")
def test_fromBase(self):
self.assertTrue(True)
Note that skipping test_fromBase (e.g. via decorator) won't be good, since 'test should be skipped' logic will be inherited by all subclasses.
I tried Łukasz’s answer and it works, but I don’t like OK (SKIP=<number>) messages. For my own desires and aims for having a test suite I don’t want me or someone to start trusting any particular number of skipped tests, or not trusting and digging into the test suite and asking why something was skipped, and always?, and on purpose? For me that’s a non-starter.
I happen to use nosetests exclusively, and by convention test classes starting with _ are not run, so naming my base class _TestBaseClass is sufficient.
I tried this in Pycharm with Unittests and py.test and both of those tried to run my base class and its tests resulting in errors because there’s no instance data in the abstract base class. Maybe someone with specific knowledge of either of those runners could make a suite, or something, that bypasses the base class.
I have a group of test cases that all should have exactly the same test done, along the lines of "Does method x return the name of an existing file?"
I thought that the best way to do it would be a base class deriving from TestCase that they all share, and simply add the test to that class. Unfortunately, the testing framework still tries to run the test for the base class, where it doesn't make sense.
class SharedTest(TestCase):
def x(self):
...do test...
class OneTestCase(SharedTest):
...my tests are performed, and 'SharedTest.x()'...
I tried to hack in a check to simply skip the test if it's called on an object of the base class rather than a derived class like this:
class SharedTest(TestCase):
def x(self):
if type(self) != type(SharedTest()):
...do test...
else:
pass
but got this error:
ValueError: no such test method in <class 'tests.SharedTest'>: runTest
First, I'd like any elegant suggestions for doing this. Second, though I don't really want to use the type() hack, I would like to understand why it's not working.
You could use a mixin by taking advantage that the test runner only runs tests inheriting from unittest.TestCase (which Django's TestCase inherits from.) For example:
class SharedTestMixin(object):
# This class will not be executed by the test runner (it inherits from object, not unittest.TestCase.
# If it did, assertEquals would fail , as it is not a method that exists in `object`
def test_common(self):
self.assertEquals(1, 1)
class TestOne(TestCase, SharedTestMixin):
def test_something(self):
pass
# test_common is also run
class TestTwo(TestCase, SharedTestMixin):
def test_another_thing(self):
pass
# test_common is also run
For more information on why this works do a search for python method resolution order and multiple inheritance.
I faced a similar problem. I couldn't prevent the test method in the base class being executed but I ensured that it did not exercise any actual code. I did this by checking for an attribute and returning immediately if it was set. This attribute was only set for the Base class and hence the tests ran everywhere else but the base class.
class SharedTest(TestCase):
def setUp(self):
self.do_not_run = True
def test_foo(self):
if getattr(self, 'do_not_run', False):
return
# Rest of the test body.
class OneTestCase(SharedTest):
def setUp(self):
super(OneTestCase, self).setUp()
self.do_not_run = False
This is a bit of a hack. There is probably a better way to do this but I am not sure how.
Update
As sdolan says a mixin is the right way. Why didn't I see that before?
Update 2
(After reading comments) It would be nice if (1) the superclass method could avoid the hackish if getattr(self, 'do_not_run', False): check; (2) if the number of tests were counted accurately.
There is a possible way to do this. Django picks up and executes all test classes in tests, be it tests.py or a package with that name. If the test superclass is declared outside the tests module then this won't happen. It can still be inherited by test classes. For instance SharedTest can be located in app.utils and then used by the test cases. This would be a cleaner version of the above solution.
# module app.utils.test
class SharedTest(TestCase):
def test_foo(self):
# Rest of the test body.
# module app.tests
from app.utils import test
class OneTestCase(test.SharedTest):
...