Develop Reference

Test assert function order with mocks (pytest -> assert_has_calls)

I'm trying to test the order of the sub-functions inside of the main function:
def get_data():
pass
def process_data(data):
pass
def notify_admin(action):
pass
def save_data(data):
pass
def main_func():
notify_admin('start')
data = get_data()
processed_data = process_data(data)
save_data(processed_data)
notify_admin('finish')
I'm using pytest, so far I've come up with this:
import pytest
from unittest.mock import patch, Mock, call
from main_func import main_func
#patch('main_func.notify_admin')
#patch('main_func.get_data')
#patch('main_func.process_data')
#patch('main_func.save_data')
def test_main_func(mock_4, mock_3, mock_2, mock_1):
execution_order = [mock_1, mock_2, mock_3, mock_4]
order_mock = Mock()
for order, mock in enumerate(execution_order):
order_mock.attach_mock(mock, f'f_{order}')
main_func()
order_mock.assert_has_calls([
call.f_1(),
call.f_2(),
call.f_3(),
call.f_4(),
call.f_1(),
])
This is an error, which I'm not sure how to resolve:
E AssertionError: Calls not found.
E Expected: [call.f_1(), call.f_2(), call.f_3(), call.f_4(), call.f_1()]
E Actual: [call.f_1('start'),
E call.f_2(),
E call.f_3(<MagicMock name='mock.f_3()' id='2049968460848'>),
E call.f_4(<MagicMock name='mock.f_2()' id='2049968489424'>),
E call.f_1('finish')]
Could you please suggest ways to resolve it or maybe implement it in a different way?
I've read documentation of assert_has_calls but I'm still not sure how to use it for this particular case.

If you want to check the call order without the argument list, you can use the method_calls attribute of the mock, which contains a list of calls in the order they are made, and only check their name:
...
main_func()
assert len(order_mock.method_calls) == 4
assert order_mock.method_calls[0][0] == "f_1"
assert order_mock.method_calls[1][0] == "f_2"
assert order_mock.method_calls[2][0] == "f_3"
assert order_mock.method_calls[3][0] == "f_4"
Each method call is a tuple of name, positional arguments and keyword arguments, so if you want to check only the name you can just use the first index.
Note that the output of your test does not seem to match this, but this is a matter of your actual application logic.
If you are using has_calls, you have to provide each argument, which is also possible. This time taking the actual result of your test, something like this should work:
...
main_func()
order_mock.assert_has_calls([
call.f_1('start'),
call.f_2(),
call.f_3(mock1),
call.f_4(mock2),
call.f_1('finish')
])

Mock testing with nested function changes

I am adding testing to a pipeline project, code is already written and in production so it cannot be changed to accommodate the tests.
In simplest terms, if I have a function like so:
def other_foo():
return 1
def foo():
res = other_foo()
return res
In practicality, the other_foo call will return a variety of responses, but for testing, I want to create a fixed response to test foo.
So in my test I want to create a fixed response to other_foo of 2. and my test evaluation to be something like:
def test_foo():
# some mocking or nesting handle here for other_foo
res = foo()
assert res == 2

Use the patch decorator from unitest.mock and patch your module local variable.
from your.module import foo
from unitest.mock import patch
#patch('your.module.other_foo')
def test_foo(mock_other_foo):
mock_other_foo.return_value = 3
assert foo() == 3
mock_other_foo.return_value = 42
assert foo() == 42
You can find more information here and there.

Reset class and class variables for each test in Python via pytest

I created a class to make my life easier while doing some integration tests involving workers and their contracts. The code looks like this:
class ContractID(str):
contract_counter = 0
contract_list = list()
def __new__(cls):
cls.contract_counter += 1
new_entry = super().__new__(cls, f'Some_internal_name-{cls.contract_counter:10d}')
cls.contract_list.append(new_entry)
return new_entry
#classmethod
def get_contract_no(cls, worker_number):
return cls.contract_list[worker_number-1] # -1 so WORKER1 has contract #1 and not #0 etc.
When I'm unit-testing the class, I'm using the following code:
from test_helpers import ContractID
#pytest.fixture
def get_contract_numbers():
test_string_1 = ContractID()
test_string_2 = ContractID()
test_string_3 = ContractID()
return test_string_1, test_string_2, test_string_3
def test_contract_id(get_contract_numbers):
assert get_contract_ids[0] == 'Some_internal_name-0000000001'
assert get_contract_ids[1] == 'Some_internal_name-0000000002'
assert get_contract_ids[2] == 'Some_internal_name-0000000003'
def test_contract_id_get_contract_no(get_contract_numbers):
assert ContractID.get_contract_no(1) == 'Some_internal_name-0000000001'
assert ContractID.get_contract_no(2) == 'Some_internal_name-0000000002'
assert ContractID.get_contract_no(3) == 'Some_internal_name-0000000003'
with pytest.raises(IndexError) as py_e:
ContractID.get_contract_no(4)
assert py_e.type == IndexError
However, when I try to run these tests, the second one (test_contract_id_get_contract_no) fails, because it does not raise the error as there are more than three values. Furthermore, when I try to run all my tests in my folder test/, it fails even the first test (test_contract_id), which is probably because I'm trying to use this function in other tests that run before this test.
After reading this book, my understanding of fixtures was that it provides objects as if they were never called before, which is obviously not the case here. Is there a way how to tell the tests to use the class as if it hasn't been used before anywhere else?

If I understand that correctly, you want to run the fixture as setup code, so that your class has exactly 3 instances. If the fixture is function-scoped (the default) it is indeed run before each test, which will each time create 3 new instances for your class. If you want to reset your class after the test, you have to do this yourself - there is no way pytest can guess what you want to do here.
So, a working solution would be something like this:
#pytest.fixture(autouse=True)
def get_contract_numbers():
test_string_1 = ContractID()
test_string_2 = ContractID()
test_string_3 = ContractID()
yield
ContractID.contract_counter = 0
ContractID.contract_list.clear()
def test_contract_id():
...
Note that I did not yield the test strings, as you don't need them in the shown tests - if you need them, you can yield them, of course. I also added autouse=True, which makes sense if you need this for all tests, so you don't have to reference the fixture in each test.
Another possibility would be to use a session-scoped fixture. In this case the setup would be done only once. If that is what you need, you can use this instead:
#pytest.fixture(autouse=True, scope="session")
def get_contract_numbers():
test_string_1 = ContractID()
test_string_2 = ContractID()
test_string_3 = ContractID()
yield

How to run unittest test cases in the order they are declared

I fully realize that the order of unit tests should not matter. But these unit tests are as much for instructional use as for actual unit testing, so I would like the test output to match up with the test case source code.
I see that there is a way to set the sort order by setting the sortTestMethodsUsing attribute on the test loader. The default is a simple cmp() call to lexically compare names. So I tried writing a cmp-like function that would take two names, find their declaration line numbers and them return the cmp()-equivalent of them:
import unittest
class TestCaseB(unittest.TestCase):
def test(self):
print("running test case B")
class TestCaseA(unittest.TestCase):
def test(self):
print("running test case A")
import inspect
def get_decl_line_no(cls_name):
cls = globals()[cls_name]
return inspect.getsourcelines(cls)[1]
def sgn(x):
return -1 if x < 0 else 1 if x > 0 else 0
def cmp_class_names_by_decl_order(cls_a, cls_b):
a = get_decl_line_no(cls_a)
b = get_decl_line_no(cls_b)
return sgn(a - b)
unittest.defaultTestLoader.sortTestMethodsUsing = cmp_class_names_by_decl_order
unittest.main()
When I run this, I get this output:
running test case A
.running test case B
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
indicating that the test cases are not running in the declaration order.
My sort function is just not being called, so I suspect that main() is building a new test loader, which is wiping out my sort function.

The solution is to create a TestSuite explicitly, instead of letting unittest.main() follow all its default test discovery and ordering behavior. Here's how I got it to work:
import unittest
class TestCaseB(unittest.TestCase):
def runTest(self):
print("running test case B")
class TestCaseA(unittest.TestCase):
def runTest(self):
print("running test case A")
import inspect
def get_decl_line_no(cls):
return inspect.getsourcelines(cls)[1]
# get all test cases defined in this module
test_case_classes = list(filter(lambda c: c.__name__ in globals(),
unittest.TestCase.__subclasses__()))
# sort them by decl line no
test_case_classes.sort(key=get_decl_line_no)
# make into a suite and run it
suite = unittest.TestSuite(cls() for cls in test_case_classes)
unittest.TextTestRunner().run(suite)
This gives the desired output:
running test case B
.running test case A
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
It is important to note that the test method in each class must be named runTest.

You can manually build a TestSuite where your TestCases and all tests inside them run by line number:
# Python 3.8.3
import unittest
import sys
import inspect
def isTestClass(x):
return inspect.isclass(x) and issubclass(x, unittest.TestCase)
def isTestFunction(x):
return inspect.isfunction(x) and x.__name__.startswith("test")
class TestB(unittest.TestCase):
def test_B(self):
print("Running test_B")
self.assertEqual((2+2), 4)
def test_A(self):
print("Running test_A")
self.assertEqual((2+2), 4)
def setUpClass():
print("TestB Class Setup")
class TestA(unittest.TestCase):
def test_A(self):
print("Running test_A")
self.assertEqual((2+2), 4)
def test_B(self):
print("Running test_B")
self.assertEqual((2+2), 4)
def setUpClass():
print("TestA Class Setup")
def suite():
# get current module object
module = sys.modules[__name__]
# get all test className,class tuples in current module
testClasses = [
tup for tup in
inspect.getmembers(module, isTestClass)
]
# sort classes by line number
testClasses.sort(key=lambda t: inspect.getsourcelines(t[1])[1])
testSuite = unittest.TestSuite()
for testClass in testClasses:
# get list of testFunctionName,testFunction tuples in current class
classTests = [
tup for tup in
inspect.getmembers(testClass[1], isTestFunction)
]
# sort TestFunctions by line number
classTests.sort(key=lambda t: inspect.getsourcelines(t[1])[1])
# create TestCase instances and add to testSuite;
for test in classTests:
testSuite.addTest(testClass[1](test[0]))
return testSuite
if __name__ == '__main__':
runner = unittest.TextTestRunner()
runner.run(suite())
Output:
TestB Class Setup
Running test_B
.Running test_A
.TestA Class Setup
Running test_A
.Running test_B
.
----------------------------------------------------------------------
Ran 4 tests in 0.000s
OK

As stated in the name, sortTestMethodsUsing is used to sort test methods. It is not used to sort classes. (It is not used to sort methods in different classes either; separate classes are handled separately.)
If you had two test methods in the same class, sortTestMethodsUsing would be used to determine their order. (At that point, you would get an exception because your function expects class names.)

python mock global function that is used in class

I can't seem to get my head around mocking in Python. I have a global function:
a.py:
def has_permission(args):
ret_val = ...get-true-or-false...
return ret_val
b.py:
class MySerializer(HyperlinkedModelSerializer):
def get_fields():
fields = super().get_fields()
for f in :
if has_permission(...):
ret_val[f.name] = fields[f]
return ret_val
c.py:
class CountrySerializer(MySerializer):
class Meta:
model = Country
Question: Now i want to test c.py, but i want to mock the has_permission function that is defined in a.py, but is called in the get_fields-method of the class MySerializer that is defined in b.py ... How do i do that?
I've tried things like:
#patch('b.MySerializer.has_permission')
and
#patch('b.MySerializer.get_fields.has_permission')
and
#patch('a.has_permission')
But everything i try either just doesn't work and has_permission is still executed, or python complains about that it can't find the attribute 'has_permission'
with the patching done in:
test.py
class TestSerializerFields(TestCase):
#patch(... the above examples....)
def test_my_country_serializer():
s = CountrySerializer()
self..assertTrue(issubclass(my_serializer_fields.MyCharField, type(s.get_fields()['field1'])))

You need to patch the global in the b module:
#patch('b.has_permission')
because that's where your code looks for it.
Also see the Where to patch section of the mock documentation.

You need to patch the method where it exists at the time your test runs. If you try and patch the method where it is defined after the test code has already imported it, then the patch will have no effect. At the point where the #patch(...) executes, the test code under test has already grabbed the global method into its own module.
Here is an example:
app/util/config.py:
# This is the global method we want to mock
def is_search_enabled():
return True
app/service/searcher.py:
# Here is where that global method will be imported
# when this file is first imported
from app.util.config import is_search_enabled
class Searcher:
def __init__(self, api_service):
self._api_service = api_service
def search(self):
if not is_search_enabled():
return None
return self._api_service.perform_request('/search')
test/service/test_searcher.py:
from unittest.mock import patch, Mock
# The next line will cause the imports of `searcher.py` to execute...
from app.service.searcher import Searcher
# At this point, searcher.py has imported is_search_enabled into its module.
# If you later try and patch the method at its definition
# (app.util.config.is_search_enabled), it will have no effect because
# searcher.py won't look there again.
class MockApiService:
pass
class TestSearcher:
# By the time this executes, `is_search_enabled` has already been
# imported into `app.service.searcher`. So that is where we must
# patch it.
#patch('app.service.searcher.is_search_enabled')
def test_no_search_when_disabled(self, mock_is_search_enabled):
mock_is_search_enabled.return_value = False
mock_api_service = MockApiService()
mock_api_service.perform_request = Mock()
searcher = Searcher(mock_api_service)
results = searcher.search()
assert results is None
mock_api_service.perform_request.assert_not_called()
# (For completeness' sake, make sure the code actually works when search is enabled...)
def test_search(self):
mock_api_service = MockApiService()
mock_api_service.perform_request = mock_perform_request = Mock()
searcher = Searcher(mock_api_service)
expected_results = [1, 2, 3]
mock_perform_request.return_value = expected_results
actual_results = searcher.search()
assert actual_results == expected_results
mock_api_service.perform_request.assert_called_once_with('/search')