Similar to a setUp() method, is there a way to define in one place how a certain exception is handled for all the tests in a given TestCase test class?
My use-case: the stack trace of mock/patch's assert_any_call only gives the expected call it could not find, but I want to add the actual calls against the mock. Each unit test can add this info to the stack trace in an except clause, but I want it defined in one place to avoid bloated code.
As #Jérôme pointed out, this can be achieved by creating a decorator to wrap your tests (e.g. https://stackoverflow.com/revisions/43938162/1)
Here is the code I ended up using:
import mock
from unittest import TestCase
from foo.bar import method_that_calls_my_class_method
def _add_mock_context(test_func):
"""This decorator is only meant for use in the MyClass class for help debugging test failures.
It adds to the stack trace the context of what actual calls were made against the method_mock,
without bloating the tests with boilerplate code."""
def test_wrapper(self, *args, **kwargs):
try:
test_func(self, *args, **kwargs)
except AssertionError as e:
# Append the actual calls (mock's exception only includes expected calls) for debugging
raise type(e)('{}\n\nActual calls to my_method mock were:\n{}'.format(e.message, self.method_mock.call_args_list))
return test_wrapper
class TestMyStuff(TestCase):
def setUp(self):
class_patch = mock.patch('mine.MyClass', autospec=True)
self.addCleanup(class_patch.stop)
class_mock = class_patch.start()
self.method_mock = class_mock.return_value.my_method
#_add_mock_context
def test_my_method(self):
method_that_calls_my_class_method()
self.method_mock.assert_any_call(arg1='a', arg2='b')
self.method_mock.assert_any_call(arg1='c', arg2='d')
self.method_mock.assert_any_call(arg1='e', arg2='f')
self.assertEqual(self.method_mock.call_count, 3)
Related
TL;DR
How can I patch or mock "any functions that are not being called/used directly"?
Sceneario
I have a simple unit-test snippet as
# utils/functions.py
def get_user_agents():
# sends requests to a private network and pulls data
return pulled_data
# my_module/tasks.py
def create_foo():
from utils.functions import get_user_agents
value = get_user_agents()
# do something with value
return some_value
# test.py
class TestFooKlass(unittest.TestCase):
def setUp(self):
create_foo()
def test_foo(self):
...
Here in setUp() method I am calling get_user_agents() function indirectly by calling create_foo(). During this execution I have got socket.timeout exception since get_user_agents() tried to access a private network. So, how can I manipulate the return data or the entire get_user_agents function during the test?
Also, is there any way to persists this mock/patch during the whole test suite execution?
It does not matter that you call the function indirectly - important is to patch it as it is imported. In your example you import the function to be patched locally inside the tested function, so it will only be imported at function run time. In this case you have to patch the function as imported from its module (e.g. 'utils.functions.get_user_agents'):
class TestFooKlass(unittest.TestCase):
def setUp(self):
self.patcher = mock.patch('utils.functions.get_user_agents',
return_value='foo') # whatever it shall return in the test
self.patcher.start() # this returns the patched object, i case you need it
create_foo()
def tearDown(self):
self.patcher.stop()
def test_foo(self):
...
If you had imported the function instead at module level like:
from utils.functions import get_user_agents
def create_foo():
value = get_user_agents()
...
you should have patched the imported instance instead:
self.patcher = mock.patch('my_module.tasks.get_user_agents',
return_value='foo')
As for patching the module for all tests: you can start patching in setUp as shown above, and stop it in tearDown.
I'm writing some testing utility library and may want to do the following
def assertSomething(test_case, ...):
test_case.assertEqual()
Is there a way to skip passing test_case around? e.g.
def assertSomething(...):
get_current_unittest_case().assertEqual()
AssertionError
If you just want to do some checks and fail with custom message,
raising AssertionError (via raise or assert) is the way to
go. By default, TestCase.failureException
is AssertionError, and
fail
(which internally is used by convenience methods of unittest.TestCase)
just raises it.
test_things.py:
import unittest
def check_is_zero(number):
assert number == 0, "{!r} is not 0".format(number)
def check_is_one(number):
if number != 1:
raise AssertionError("{!r} is not 1".format(number))
class NumbersTest(unittest.TestCase):
def test_one(self):
check_is_one(1)
def test_zero(self):
check_is_zero(0)
TestCase mixin
An easy and relatively readable way to add new assertions is to
make a “mixin class” that test cases will subclass. Sometimes it
is good enough.
testutils.py that contains mixin:
class ArithmeticsMixin:
def check_is_one(self, number):
self.assertEqual(number, 1)
test_thing.py, actual tests:
import unittest
import testutils
class NumbersTest(unittest.TestCase, testutils.ArithmeticsMixin):
def test_one(self):
self.check_is_one(1)
If there will be many mixin classes, use of base one may help:
import unittest
import testutils
class BaseTestCase(unittest.TestCase, testutils.ArithmeticsMixin):
"""Test case with additional methods for testing arithmetics."""
class NumbersTest(BaseTestCase):
def test_one(self):
self.check_is_one(1)
Thread local and TestCase subclass
Less readable is use of special base class,
thread local
(like global, but is aware of threads) and getter function.
testutils.py:
import unittest
import threading
_case_data = threading.local() # thread local
class ImproperlyUsed(Exception):
"""Raised if get_case is called without cooperation of BaseTestCase."""
def get_case(): # getter function
case = getattr(_case_data, "case", None)
if case is None:
raise ImproperlyUsed
return case
class BaseTestCase(unittest.TestCase): # base class
def run(self, *args, **kwargs):
_case_data.case = self
super().run(*args, **kwargs)
_case_data.case = None
def check_is_one(number):
case = get_case()
get_case().assertEqual(number, 1)
When test case is running, self (test case instance) is saved as
_case_data.case, so later inside check_is_one (or any other function
that is called from inside of test method and wants to access self)
get_case will be able to get reference to test case instance. Note
that after running _case_data.case is set to None, and if
get_case is called after that, ImproperlyUsed will be raised.
test_thing.py:
import testutils
def check_is_zero(number): # example of out-of-testutils function
testutils.get_case().assertEqual(number, 0)
class NumbersTest(testutils.BaseTestCase):
def test_one(self):
testutils.check_is_one(1)
def test_zero(self):
check_is_zero(0)
s̻̙ỵ̞̜͉͎̩s.̠͚̱̹̦̩͓_̢̬ge̡̯̳̖t̞͚̖̳f̜̪̩̪r̙̖͚̖̼͉̹a͏ṃ̡̹e̞̝̱̠̙
Finally, sys._getframe.
Let’s just h́o̞̓̐p̆̇̊̇e you don’t need it, because it is part of CPython,
not Python.
testutils.py:
import sys
import unittest
class ImproperlyUsed(Exception):
"""Raised if get_case can't get "self" TestCase instance."""
def get_case():
case = sys._getframe().f_back.f_back.f_locals.get("self")
if not isinstance(case, unittest.TestCase):
raise ImproperlyUsed
return case
def check_is_one(number):
case = get_case()
get_case().assertEqual(number, 1)
sys._getframe returns frame from the top of call stack, then few
frames below f_locals is taken, and there is self: instance of
unittest.TestCase. Like in previous implementation option, there is
sanity check, but here it is done with isinstance.
test_things.py:
import unittest
import testutils
def check_is_zero(number): # example of out-of-testutils function
testutils.get_case().assertEqual(number, 0)
class NumbersTest(unittest.TestCase):
def test_one(self):
testutils.check_is_one(1)
def test_zero(self):
check_is_zero(0)
If you just want to provide assertEqual for some new type,
take a look at addTypeEqualityFunc.
I have run into a problem which I think might be a bug with the libraries I am using. However, I am fairly new to python, unittest, and unittest.mock libraries so this may just be a hole in my understanding.
While adding tests to some production code I have run into an error, I have produced a minimal sample that reproduces the issue:
import unittest
import mock
class noCtorArg:
def __init__(self):
pass
def okFunc(self):
raise NotImplemented
class withCtorArg:
def __init__(self,obj):
pass
def notOkFunc(self):
raise NotImplemented
def okWithArgFunc(self, anArgForMe):
raise NotImplemented
class BasicTestSuite(unittest.TestCase):
"""Basic test Cases."""
# passes
def test_noCtorArg_okFunc(self):
mockSUT = mock.MagicMock(spec=noCtorArg)
mockSUT.okFunc()
mockSUT.assert_has_calls([mock.call.okFunc()])
# passes
def test_withCtorArg_okWithArgFuncTest(self):
mockSUT = mock.MagicMock(spec=withCtorArg)
mockSUT.okWithArgFunc("testing")
mockSUT.assert_has_calls([mock.call.okWithArgFunc("testing")])
# fails
def test_withCtorArg_doNotOkFuncTest(self):
mockSUT = mock.MagicMock(spec=withCtorArg)
mockSUT.notOkFunc()
mockSUT.assert_has_calls([mock.call.notOkFunc()])
if __name__ == '__main__':
unittest.main()
How I run the tests and the output is as follows:
E:\work>python -m unittest testCopyFuncWithMock
.F.
======================================================================
FAIL: test_withCtorArg_doNotOkFuncTest (testCopyFuncWithMock.BasicTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
File "testCopyFuncWithMock.py", line 38, in test_withCtorArg_doNotOkFuncTest
mockSUT.assert_has_calls([mock.call.notOkFunc()])
File "C:\Python27\lib\site-packages\mock\mock.py", line 969, in assert_has_calls
), cause)
File "C:\Python27\lib\site-packages\six.py", line 718, in raise_from
raise value
AssertionError: Calls not found.
Expected: [call.notOkFunc()]
Actual: [call.notOkFunc()]
----------------------------------------------------------------------
Ran 3 tests in 0.004s
FAILED (failures=1)
I am using python 2.7.11, with mock version 2.0.0 installed via pip.
Any suggestions for what I am doing wrong? Or does this look like a bug in the library?
Interestingly, the way you chose to perform the assert has masked your issue.
Try, instead of this:
mockSUT.assert_has_calls(calls=[mock.call.notOkFunc()])
to do this:
mockSUT.assert_has_calls(calls=[mock.call.notOkFunc()], any_order=True)
You'll see the actual exception:
TypeError("'obj' parameter lacking default value")
This is because you tried to instantiate an instance of the class withCtorArg that has the parameter obj with no default value. If you had tried to actually instantiate it directly, you would've seen:
TypeError: __init__() takes exactly 2 arguments (1 given)
However, since you let the mock library handle the instantiation of a mock object, the error happens there - and you get the TypeError exception.
Modifying the relevant class:
class withCtorArg:
def __init__(self, obj = None):
pass
def notOkFunc(self):
pass
def okWithArgFunc(self, anArgForMe):
pass
and adding a default None value for obj solves the issue.
I don't think I can explain definitively why this is the case, I still suspect a bug in the Mock library, since the issue only occurs for a test case with no arguments on the called function. Thanks to advance512 for pointing out that the real error was hidden!
However to work around this issue, without having to modify the production code I am going to use the following approach:
# passes
#mock.patch ('mymodule.noCtorArg')
def test_noCtorArg_okFunc(self, noCtorArgMock):
mockSUT = noCtorArg.return_value
mockSUT.okFunc()
mockSUT.assert_has_calls([mock.call.okFunc()])
# passes
#mock.patch ('mymodule.withCtorArg')
def test_withCtorArg_okWithArgFuncTest(self, withCtorArgMock):
mockSUT = withCtorArg.return_value
mockSUT.okWithArgFunc("testing")
mockSUT.assert_has_calls([mock.call.okWithArgFunc("testing")])
# now passes
#mock.patch ('mymodule.withCtorArg')
def test_withCtorArg_doNotOkFuncTest(self, withCtorArgMock):
mockSUT = withCtorArg.return_value
mockSUT.notOkFunc()
mockSUT.assert_has_calls([mock.call.notOkFunc()], any_order=True)
Edit:
One problem with this is that the mock does not have spec set. This means that the SUT is able to call methods that do not exist on the original class definition.
An alternative approach is to wrap the class to be mocked:
class withCtorArg:
def __init__(self,obj):
pass
def notOkFunc(self):
raise NotImplemented
def okWithArgFunc(self, anArgForMe):
raise NotImplemented
class wrapped_withCtorArg(withCtorArg):
def __init__(self):
super(None)
class BasicTestSuite(unittest.TestCase):
"""Basic test Cases."""
# now passes
def test_withCtorArg_doNotOkFuncTest(self):
mockSUT = mock.MagicMock(spec=wrapped_withCtorArg)
mockSUT.notOkFunc()
#mockSUT.doesntExist() #causes the test to fail. "Mock object has no attribute 'doesntExist'"
assert isinstance (mockSUT, withCtorArg)
mockSUT.assert_has_calls([mock.call.notOkFunc()], any_order=True)
I want to test this class using python unittest framework and also mockito.
class ISightRequestEngine(object):
def __init__(self, pInputString=None):
self.__params = (pInputString)
def openHTTPConnection(self):
pass
def __closeHTTPConnection(self):
pass
def testFunc(self):
print 'test function called'
def startEngine(self):
self.__params.parseinputString()
self.openHTTPConnection()
self.testFunc()
def processRequest(self, header = None):
pass
I wanted to test that function startEngine() calls testFunc().
Similar to what we do in our mocked class,
obj = mock(ISightRequestEngine)
obj.startEngine()
try:
verify(obj).startEngine()
except VerificationError:
Unfortunately this only verifies whether the startEngine function is called or not, but it does not give the actual function call and I cannot verify that whether the call to testFunc() has been made or not.
Is there any way to test this scenario?
I am new to testing world and framework.
In your example you are testing your mock.
You create a mock of ISightRequestingEngine
You call startEngine() method of that mock
You verify that the mocked object was called
What you want to do is:
Mock out testFunc()
Call startEngine()
Verify that testFunc() was called
I'm not familiar with mockito, but what from what I can make up from the documentation, I think you have to do something like the following:
from mockito import mock, verify
# Setup ---------------------------------------------
my_mock = mock(ISightRequestingEngine)
system_under_test = ISightRequestingEngine()
system_under_test.testFunc = my_mock.testfunc # Mock out only testFunc()
# Exercise ------------------------------------------
system_under_test.startEngine()
# Verify --------------------------------------------
verify(my_mock).testFunc()
Having similar such issue, where I am bit lost in writing the test case
class UserCompanyRateLimitValidation:
def __init__(self, user_public_key):
self.adapter = UserAdapter(user_public_key)
container = self.adapter.get_user_company_rate_limit()
super(UserCompanyRateLimitValidation, self).__init__(container,\
UserCompanyRateLimitValidation.TYPE)
I have to test this function. I have written test case something like this. I have tried to mock the UserAdapter class but I am not able to do so completely.
def test_case_1():
self.user_public_key = 'TEST_USER_PUBLIC_KEY_XXXXXX1234567890XXXXX'
UserAdapter_mock = mock(UserAdapter)
when(UserAdapter_mock).get_user_company_rate_limit().\
thenReturn(self.get_container_object())
self.test_obj = UserCompanyRateLimitValidation(self.user_public_key)
Here if you see I have mocked get_user_company_rate_limit() call from the testable function, container = self.adapter.get_user_company_rate_limit()
but I am still not able to figure out the way in which I can mock this call,
self.adapter = UserAdapter(user_public_key)
I'm writing a test suite, and the code I'm testing makes excessive use of delayed module imports. So it's possible that with 5 different inputs to the same method, this may end up importing 5 additional modules. What I'd like to be able to do is set up tests so that I can assert that running the method with one input causes one import, and doesn't cause the other 4.
I had a few ideas of how to start on this, but none so far have been successful. I already have a custom importer, and I can put the logging code in the importer. But this doesn't work, because the import statements only run once. I need the log statement to be executed regardless of if the module has been previously imported. Just running del sys.modules['modname'] also doesn't work, because that runs in the test code, and I can't reload the module in the code being tested.
The next thing I tried was subclassing dict to do the monitoring, and replace sys.modules with this subclass. This subclass has a reimplemented __getitem__ method, but calling import module doesn't seem to trigger the __getitem__ call in the subclass. I also can't assign directly to sys.modules.__getitem__, because it is read-only.
Is what I'm trying to do even possible?
UPDATE
nneonneo's answer seems to only work if the implementation of logImports() is in the same module as where it is used. If I make a base test class containing this functionality, it has problems. The first is that it can't find just __import__, erroring with:
# old_import = __import__
# UnboundLocalError: local variable '__import__' referenced before assignment
When I change that to __builtin__.__import__, I another error:
myunittest.py:
import unittest
class TestCase(unittest.TestCase):
def logImports(self):
old_import = __builtins__.__import__
def __import__(*args, **kwargs):
print args, kwargs
return old_import(*args, **kwargs)
__builtins__.__import__ = __import__
test.py:
import myunittest
import unittest
class RealTest(myunittest.TestCase):
def setUp(self):
self.logImports()
def testSomething(self):
import unittest
self.assertTrue(True)
unittest.main()
# old_import = __builtins__.__import__
# AttributeError: 'dict' object has no attribute '__import__'
Try
old_import = __import__
def __import__(*args, **kwargs):
print args, kwargs
return old_import(*args, **kwargs)
__builtins__.__import__ = __import__
This overrides __import__ completely, allowing you to monitor every invocation of import.
Building on the previous answer, in Python 3, I've had success with the following.
import builtins
old_import = __import__
def importWithLog(*args, **kwargs):
print(args[0]) # This is the module name
print(args, kwargs)
return old_import(*args, **kwargs)
builtins.__import__ = importWithLog
# Rest of the code goes here.
import time
import myModule
...