In my setUpClass I would like to create a resource in the database one time, which is then used for all of the tests in the class.
After I create the resource in setUpClass, I would like to perform assertions on it right then and there. However, I'm not sure how to call assertions in setUpClass, given that all of the assertion functions are instance methods, not class methods.
import unittest
class TestFoo(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
# How would I assert that cls.foo['name'] == 'bar'?
# The following does not work, as the assertEquals function is not a class method
# cls.assertEquals(cls.foo['name'], 'bar')
#classmethod
def make_foo(cls, name):
# Calls a POST API to create a resource in the database
return {'name': name}
def test_foo_0(self):
# Do something with cls.foo
pass
def test_foo_1(self):
# do something else with cls.foo
pass
The only alternative I can think of is to raise an exception in setUpClass:
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
if cls.foo['name'] != 'bar':
raise Exception("Failed to create resource, cannot do the tests")
Of course, I do not want to call the assertions from each test, as this will just duplicate the code.
Edit: I don't think this workaround is good, because the failure message will point to the self.assertFalse(self.flag) line, instead of the if cls.foo['name'] ~= 'bar' line. In addition, if you created multiple resources, this would need multiple flags to disambiguate.
flag=False
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
if cls.foo['name'] != 'bar':
cls.flag=True
def setUp(self):
self.assertFalse(self.flag)
Related
Context
I have a test class where all my tests inherit from. It cant run by itself as it really doesnt contain any setup info
I wanted to add a test which is executed by ALL tests (adding it to the baseclass seems logical)
But now I notice the basetestclass( => Foo) which I import is being detected as a test itself and runs and is visible in the reports
Code
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the real test in test_something.py
from base import Foo # <= This is being detected as a testclass object and thus will be executed
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This triggers a testrun of the imported Foo
The Question
How can I import Foo without that its being detected as a 'test'
If I remove the test to run in all tests all is fine.
Adding #nottest decorator to Foo wont work since then also all inherited classes are defined nottest.
It needs to run on nose, pytest and unittest testrunners
I noticed if I changed the import statement like below that it also works. But that would mean adjusting a few hundreds of testfiles in different repos. (I'd like to avoid that)
import base
class TestFoo(base.Foo):
The key to the answer seems to be that each test has an attribute __test__ which is set to True when it is a test.
Setting it to False when the class should not be a test will then let the test collector ignore this class.
The answer assumes I can only do changes in the base.py
In python 3.9 classmethod and property decorators can be combined so I wrote a separate answer for that
answer for < py3.9
the base class in base.py
from unittest import TestCase
class MetaFoo(type):
#property
def __test__(cls):
return cls != Foo
class Foo(TestCase, metaclass=MetaFoo):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
answer for >= py3.9
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
#property
def __test__(cls):
return cls != Foo
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the actual test
test_something.py
from base import Foo # <= This will not be detected as a test anymore as __test__ returns False
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This doesnt trigger a testrun of the imported Foo anymore
I have a class that I want to patch in my unittests.
class OriginalClass():
def method_a():
# do something
def method_b():
# do another thing
Now I created another class to patch it with, so the code for patching it is like
class MockClass(OriginalClass):
def method_a():
# This will override the original method and return custom response for testing.
patcher = patch('OriginalClass', new=MockClass)
mock_instance = patcher.start()
This works exactly as I want it to and I can return whatever responses required for my unittests.
Now this issue is when I want to verify that a method is called with the right parameters in the unittests.
I tried
mock_instance.method_a.assert_called_once()
But it fail with error AttributeError: 'function' object has no attribute 'assert_called_once'.
How can I test the method calls here?
AttributeError: 'function' object has no attribute 'assert_called_once'.
Once mock object is created, there is no method_a exists, you have to call once m.method_a() before assert.
m = mock.create_autospec(OriginalClass)
m.method_a()
m.method_a.assert_called_once()
patch mock entire class
I took it as mock the whole class and all its methods, I would take an example from here
https://docs.python.org/3.3/library/unittest.mock-examples.html
Applying the same patch to every test method, Here is my example, patch the entire Primary class as MockPrimay for every methods and every tests, setup or SetupClass could be added for the methods needed, even the whole class is mocked, but not every methods to be used in the tests.
from tests.lib.primary_secondary import Secondary
#mock.patch('tests.lib.primary_secondary.Primary')
class TestSecondaryMockPrimary(unittest.TestCase):
def test_method_d(self, MockPrimary):
MockPrimary().process()
MockPrimary().process.return_value = 1
oc = Secondary()
self.assertEqual(oc.method_d(), 1)
import tests
self.assertIs(tests.lib.primary_secondary.Primary, MockPrimary)
The Primary is needed for the Secondary for this test
class Primary(object):
def __init__(self, param):
self._param = param
def process(self):
if self._param == 1:
self._do_intermediate_process()
self._do_process()
class Secondary(object):
def __init__(self):
self.scl = Primary(1)
def method_d(self):
return self.scl.process
I think wraps can be useful here:
from unittest.mock import patch
class Person:
name = "Bob"
def age(self):
return 35
class Double(Person):
def age(self):
return 5
with patch('__main__.Person', wraps=Double()) as mock:
print(mock.name) # mocks data
print(mock.age()) # runs real methods, but still spies their calls
mock.age.assert_not_called()
Output:
<MagicMock name='Person.name' id='139815250247536'>
5
...
raise AssertionError(msg)
AssertionError: Expected 'age' to not have been called. Called 1 times.
Calls: [call()].
I have two test case modules TestClassA, TestClassB. On test_1_that_needs_resource_a in TeseClass A, I am using fixture so I can call resource_a and pass parameters before executing testcase. How can I use the same fixture on setup_class in TestClassB so all testcase
in TestClassB so the fixture can be called one time before all testcase in TestClassB
import pytest
#pytest.fixture(scope='function')
def resource(request):
print('resources_a_setup()')
return request.param
class TestClassA:
#classmethod
def setup_class(cls):
print('\nsetup_class()')
#classmethod
def teardown_class(cls):
print('\nteardown_class()')
#pytest.mark.parametrize('resource', [dict(), dict(name=1)], indirect=['resource'])
def test_1_that_needs_resource_a(self, resource):
assert resource == dict()
print(f'\ntest_1_that_needs_resource_a(), {resource}')
class TestClassB:
## I would like to call resource fixture with passing some parameter on setup_class here
#classmethod
def setup_class(cls):
print('\nsetup_class()')
#classmethod
def teardown_class(cls):
print('\nteardown_class()')
def test_1_that_needs_resource_a(self):
print('\ntest_1_that_needs_resource_a()')
def register_processor2(processor_name='SomeProcessor'):
def decorator(func):
class SomeProcessor(GenericPaymentProcessor, TriggeredProcessorMixin):
name = processor_name
transaction_class = Transaction
#staticmethod
def setup(data=None):
pass
#wraps(func)
def func_wrapper(*args, **kwargs):
PaymentProcessorManager.register(SomeProcessor)
result = func(*args, **kwargs)
PaymentProcessorManager.unregister(SomeProcessor)
return result
return func_wrapper
return decorator
def register_processor(func):
class SomeProcessor(GenericPaymentProcessor, TriggeredProcessorMixin):
name = 'SomeProcessor'
transaction_class = Transaction
#staticmethod
def setup(data=None):
pass
#wraps(func)
def func_wrapper(*args, **kwargs):
PaymentProcessorManager.register(SomeProcessor)
result = func(*args, **kwargs)
PaymentProcessorManager.unregister(SomeProcessor)
return result
return func_wrapper
class TestPaymentMethodEndpoints(APITestCase):
#register_processor
def test_put_detail_cannot_change_processor(self):
self.assertEqual(True, False)
Ok so the decorator register_processor works as expected. And the test fails, but I want to make the name of the inner class customizable so I went for a decorator factory implementation instead.
The thing is when running the test decorated with register_processor2 I get the following:
AttributeError: 'TestPaymentMethodEndpoints' object has no attribute '__name__'
This is from #wraps(func), my question is why is func here an instance of TestPaymentMethodEndpoints, and not the bound method?
Also if I remove the #wraps decorator then the test runs and passes.
I'd expect that the test would not be discovered as func_wrapper does not start with test_* and even if it is discovered then it should fail.
Any insight on what is happening and how I'd go about doing this?
EDIT
So I figured it out even if the decorator factory has arguments that have default values you still need to place () when calling it.
But would still love to hear an explanation of what happened in case of the tests passing / getting discovered in the first place.
class TestPaymentMethodEndpoints(APITestCase):
#register_processor()
def test_put_detail_cannot_change_processor(self):
self.assertEqual(True, False)
Makes sense now that I think about it :D, gosh you learn something new each day!
I think you're now asking "how come the unittest module can find test cases that have been wrapped in functions with names that don't start test?"
The answer to that is because unittest doesn't use the names of the functions to find the methods to run, it uses the attribute names of the test case classes to find them.
So try running the following code:
from unittest import TestCase
def apply_fixture(func):
def wrap_with_fixture(self):
print('setting up fixture...')
try:
func(self)
finally:
print('tearing down fixture')
return wrap_with_fixture
class MyTestCase(TestCase):
#apply_fixture
def test_something(self):
print('run test')
print('Attributes of MyTestCase: %s' % dir(MyTestCase))
print('test_something method: %s' % MyTestCase.test_something)
mtc = MyTestCase()
mtc.test_something()
You will see that the output from dir contains the name test_something:
Attributes of MyTestCase: ['__call__', ...lots of things..., 'test_something']
but that the value of that attribute is the wrapping function wrap_with_fixture:
test_something method: <function apply_fixture.<locals>.wrap_with_fixture at 0x10d90aea0>
This makes sense when you consider that when you create a function you are both creating a function with the name provided and a local variable with the same name, and that the decorator # syntax is just syntactic sugar, so the following would have been an equally valid albeit longer-winded way of creating your test case class:
class MyTestCase(TestCase):
def test_something(self):
print('run test')
# Overwrite existing 'local' (or 'class' variable in this context)
# with a new value. We haven't deleted the test_something function
# which still exists but now is owned by the function we've created.
test_something = apply_fixture(test_something)
Suppose I have this integration test
class TestClass(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.key = '123'
def test_01_create_acc(self):
user = create_account(...)
self.key = user.key
def test_02_check_account(self):
user = check_account(..)
self.assertEqual(self.key, user.key)
It looks like the attribute self.key is not mutable. It stays with the old value from setUpClass. But isn't setUpClass only called once?
The account function creates a key randomly for security reason, so I am not allowed to pass in my secret key. It returns the key, so I need to modify that attribute. Can I?
It looks like each test_ case is isolated.
my_gloabl = None
def setUpClass(cls):
cls.key = my_global
If I change my_global in test1, test2 will get None.
The class is set up only once. But each test method is actually called from a different instance of that test.
You can demonstrate this using the id function, which will return a different number for each object:
import unittest
class TestClass(unittest.TestCase):
#classmethod
def setUpClass(cls):
print "setup"
def test_01_create_acc(self):
print id(self)
def test_02_check_account(self):
print id(self)
unittest.main()
On my computer, this printed:
setup
4300479824
.4300479888
Note how the setup method was called only once, but the id of the instance for test1 and test2 are different.