I'm new to using the mock library and unit testing in general. I think I understand how the mock library works, but I think there is something wrong with my approach.
Lets say I have a class Foo that has a method addFoo(bar1, bar2) and addFoo calls other 'private' methods within foo, but can also raise exceptions at various parts in the code... i.e.
class Foo:
def __init__(self):
# This creates lots of dependencies
def _inner1(self, bar):
if this_doesnt_work:
return None
return modified_bar1
def _inner2(self, bar2):
if this_doesnt_work:
return None
return modified_bar2
def addFoo(bar1, bar2):
if self._inner1(bar1) and self._inner2(bar2):
#do something
else:
raise SomeException
When I unit test, what I currently do is mock out Foo.__init__ in my unit test setup and set return_value to None. This works and allows me to run tests against my methods in the class without all the external dependencies and just mock any class attributes with my mocked Foo.__init__ instance.
After that is done if I wanted to unit test addFoo I would do the following (skipping a bunch of steps):
#patch.object(Foo, '_inner2')
#patch.object(Foo, '_inner1')
def test_inner1_exception(self, mock_inner1, mock_inner2):
mock_inner1.side_effect = Exception
# then do some asserts to make sure it worked
QUESTION
How do I test an exception in addFoo if the code looks like this:
def addFoo(bar1, bar2):
bars = self._getBars() #something that returns a list
my_modified_bar_list = []
for bar in bars:
my_modified_bar_list.append(self._inner1(bar))
if len(my_modified_bar_list) == 0:
raise SomeException
In this instance how do I test SomeException when it is dependent upon the values produced by some array within my method?
If you are making use of pytest you can assert about expected exceptions as follows:
#patch('Foo._getBars', return_value=[])
def test_addFoo_exception(self):
foo = Foo()
with pytest.raises(SomeException):
foo.addFoo(Mock(), Mock())
As snippet above shows, by mocking _getBars you are in control of what it is returned and can therefore force the condition that triggers the exception in your code, in this case an empty list.
I have written a short post with common unit testing pitfalls in Python that you might find useful.
Related
I wrote a unit test for main_function and asserted that it calls the function get_things inside it with an instance of a class, mocked with patch as a parameter:
#patch("get_something")
#patch("MyClass.__new__")
def test(self, mock_my_class_instance, mock_get_something):
# Given
dummy_my_class_instance = MagicMock()
mock_my_class_instance.return_value = dummy_my_class_instance
dummy_my_class_instance.get_things.return_value = {}
# When
main_function(parameter)
# Then
dummy_my_class_instance.get_things.assert_called_once_with(parameter["key1"], parameter["key2"])
mock_get_something.assert_called_once_with(dummy_my_class_instance)
This is the main function:
def main_function(parameter):
properties = get_properties(parameter)
my_class_instance = MyClass()
list_of_things = my_class_instance.get_things(properties["key-1"], properties["key-2"])
an_object = get_something(my_class_instance)
return other_function(list_of_things, an_object)
It passes individually but when run along with other tests that patch MyClass.get_things() it fails. This is the message:
Unhandled exception occurred::'NoneType' object has no attribute 'client'
It seems like the patch decorators are affecting each other.
I have tried to create the mocks inside the test function as variables rather than decorators but the problem persists. I have also tried to create a tearDown() to stop the patches but it doesn't seem to work.
Is there any way to isolate the patches or successfully discard them when mocking class instances?
I've searched for hours. Can't find anyone even trying to do this. Hmmm.
I believe I have to override a single method within a class instance. I do not mean patch(return_value=). I need to make the method in question do something involving self.
I'll try to break it down. Liberally paraphrasing and including one of the many things I tried, which doesn't work ...
class SetupClass(object):
def set_some_stuff(self):
data_list = functon_cannot_be_run_on_test_platform()
self.something = data_list[0]
self.something_else = data_list[1]
class UUT(object):
self.config = SetupClass()
assert self.config.something == 'foo'
class UnitTests(TestCase):
#patch('SetupClass')
def test_UUT(self, mock1):
def WedgeClass(SetupClass):
def set_some_stuff(self):
self.something = 'foo'
pass # I'm a Python newbie, in too deep
wedge_class = WedgeClass()
mock1.return_value = wedge_class # doesn't work. context errors
uut = UUT() # <-- would crash here, because assert above
Assume that I cannot make changes to UUT or SetupClass.
Testing cannot even get off the ground because the assertion will fail, due to, SetupClass.functon_cannot_be_run_on_test_platform(). Note that simply mocking SetupClass.functon_cannot_be_run_on_test_platform will not solve the problem, because reasons.
ATM, I figure the only way to get around this mess is to somehow override SetupClass.set_some_stuff. I cannot simply mock the entire class, because UUT relies heavily on its other functionality as well. I need everything to work as is, except this one method and I need that method to be able to access, self in the same context as originally intend.
I tried various things involving subclassing and mock.return_value etc. I'd rather not recall the pain that caused. :p
My kingdom for test-driven code in the first place! This code contains a convolution of co-dependencies. :-/
Apart from the multiple errors in the example code I wrote up off the top of my head at the cafe ...
The problem was (mostly) that I was using return_value instead of side_effect to 'replace' the patched class with my own subclass.
My need, re-stated perhaps more clearly now, is to override a single method, set_some_stuff, in a class within the unit under test (UUT) but without mocking the class.
The method issues several 'self.foo = bar' statements, which I want to change for testing purposes. Thus, mocking the method's (unused) return value is not enough ... and patch.object seems to lose class context, "'self' unknown" or the like.
Finally, here is the working code, doing what I need ...
import unittest
import mock
class SetupClass(object):
def set_some_stuff(self):
data_list = ['data not available', 'on test_platform'] # CRASH!
self.something = data_list[0]
self.something_else = data_list[1]
class UUT:
def __init__(self):
self.config = SetupClass()
self.config.set_some_stuff()
assert self.config.something == 'foo' # <-- used to crash before here ...
self.got_here = True # ... but now the override method from
# WedgeClass is being used! :=)
"""
Subclass the original class, overriding just the method in question. Then set
the subclass as the side_effect of the patched original, effectively replacing it.
"""
class WedgeClass(SetupClass):
def set_some_stuff(self):
self.something = 'foo'
pass # I'm a Python newbie, in too deep
class UnitTests(unittest.TestCase):
#mock.patch(__module__+'.SetupClass')
def test_UUT(self, mock1):
wedge_class = WedgeClass()
mock1.side_effect = WedgeClass # <--- Ureka! 'side_effect' not 'return_value' was the ley!
uut = UUT()
self.assertTrue(uut.got_here)
I've got this production class:
class MyClass:
def __init__(self):
self.value = None
def set_value(self, value):
self.value = value
def foo(self):
# work with self.value here
# raise RuntimeError("error!")
return "a"
Which is being used from another place, like this:
class Caller:
def bar(self, smth):
obj = MyClass()
obj.set_value(smth)
# ...
# try:
obj.foo()
# except MyError:
# pass
obj.set_value("str2")
# obj.foo()
and I got this:
class MyError(Exception):
pass
In my test I want to make sure that Caller.bar calls obj.set_value, first with smth="a", then with smth="b", but I want it to really set the value (i.e. call the real set_value method). Is there any way for me to tell the mock to use the actual method, so I can later on read what it was called with?
P.S. I know that I can just change "foo" to require the parameter "smth" so I could get rid of "set_value", but I want to know if there is another option than this.
Okay, so I have tried this in my test:
def test_caller(self):
with patch('fullpath.to.MyClass', autospec=MyClass) as mock:
mock.foo.side_effect = [MyError("msg"), "text"]
caller = Caller()
caller.bar("str1")
calls = [call("str1"), call("str2")]
mock.set_value.assert_has_calls(calls)
But I see that the mock was not successful since the real "foo" is called when I wanted it to first raise MyError, then return "text".
Also, the assertion fails:
AssertionError: Calls not found.
Expected: [call('str1'), call('str2')]
Actual: []
The problem here is that you have mocked out your Class, and are not properly using the instance of your class. This is why things are not behaving as expected.
So, lets take a look at what is going on.
Right here:
with patch('fullpath.to.MyClass', autospec=MyClass) as mock:
So, what you are doing right here is mocking out your class MyClass only. So, when you are doing this:
mock.set_value.assert_has_calls(calls)
And inspect what is going on when you execute your unittest, your mock calls will actually contain this:
[call().set_value('str1'), call().foo(), call().set_value('str2')]
Pay attention to call as it is written as call(). call is with reference to your mock here. So, with that in mind, you need to use the called (aka return_value within context of the mocking world) mock to properly reference your mock object that you are trying to test with. The quick way to fix this is simply use mock(). So you would just need to change to this:
mock().set_value.assert_has_calls(calls)
However, to be more explicit on what you are doing, you can state that you are actually using the result of calling mock. Furthermore, it would actually be good to note to use a more explicit name, other than mock. Try MyClassMock, which in turn you name your instance my_class_mock_obj:
my_class_mock_obj = MyClassMock.return_value
So in your unit test it is more explicit that you are using a mocked object of your class. Also, it is always best to set up all your mocking before you make your method call, and for your foo.side_effect ensure that you are also using the instance mock object. Based on your recent update with your exception handling, keep your try/except without comments. Putting this all together, you have:
def test_caller(self):
with patch('tests.test_dummy.MyClass', autospec=MyClass) as MyClassMock:
my_class_mock_obj = MyClassMock.return_value
my_class_mock_obj.foo.side_effect = [MyError("msg"), "text"]
caller = Caller()
caller.bar("str1")
calls = [call("str1"), call("str2")]
my_class_mock_obj.set_value.assert_has_calls(calls)
I have a group of test cases that all should have exactly the same test done, along the lines of "Does method x return the name of an existing file?"
I thought that the best way to do it would be a base class deriving from TestCase that they all share, and simply add the test to that class. Unfortunately, the testing framework still tries to run the test for the base class, where it doesn't make sense.
class SharedTest(TestCase):
def x(self):
...do test...
class OneTestCase(SharedTest):
...my tests are performed, and 'SharedTest.x()'...
I tried to hack in a check to simply skip the test if it's called on an object of the base class rather than a derived class like this:
class SharedTest(TestCase):
def x(self):
if type(self) != type(SharedTest()):
...do test...
else:
pass
but got this error:
ValueError: no such test method in <class 'tests.SharedTest'>: runTest
First, I'd like any elegant suggestions for doing this. Second, though I don't really want to use the type() hack, I would like to understand why it's not working.
You could use a mixin by taking advantage that the test runner only runs tests inheriting from unittest.TestCase (which Django's TestCase inherits from.) For example:
class SharedTestMixin(object):
# This class will not be executed by the test runner (it inherits from object, not unittest.TestCase.
# If it did, assertEquals would fail , as it is not a method that exists in `object`
def test_common(self):
self.assertEquals(1, 1)
class TestOne(TestCase, SharedTestMixin):
def test_something(self):
pass
# test_common is also run
class TestTwo(TestCase, SharedTestMixin):
def test_another_thing(self):
pass
# test_common is also run
For more information on why this works do a search for python method resolution order and multiple inheritance.
I faced a similar problem. I couldn't prevent the test method in the base class being executed but I ensured that it did not exercise any actual code. I did this by checking for an attribute and returning immediately if it was set. This attribute was only set for the Base class and hence the tests ran everywhere else but the base class.
class SharedTest(TestCase):
def setUp(self):
self.do_not_run = True
def test_foo(self):
if getattr(self, 'do_not_run', False):
return
# Rest of the test body.
class OneTestCase(SharedTest):
def setUp(self):
super(OneTestCase, self).setUp()
self.do_not_run = False
This is a bit of a hack. There is probably a better way to do this but I am not sure how.
Update
As sdolan says a mixin is the right way. Why didn't I see that before?
Update 2
(After reading comments) It would be nice if (1) the superclass method could avoid the hackish if getattr(self, 'do_not_run', False): check; (2) if the number of tests were counted accurately.
There is a possible way to do this. Django picks up and executes all test classes in tests, be it tests.py or a package with that name. If the test superclass is declared outside the tests module then this won't happen. It can still be inherited by test classes. For instance SharedTest can be located in app.utils and then used by the test cases. This would be a cleaner version of the above solution.
# module app.utils.test
class SharedTest(TestCase):
def test_foo(self):
# Rest of the test body.
# module app.tests
from app.utils import test
class OneTestCase(test.SharedTest):
...
I have a test case:
class LoginTestCase(unittest.TestCase):
...
I'd like to use it in a different test case:
class EditProfileTestCase(unittest.TestCase):
def __init__(self):
self.t = LoginTestCase()
self.t.login()
This raises:
ValueError: no such test method in <class 'LoginTest: runTest`
I looked at the unittest code where the exception is being called, and it looks like the tests aren't supposed to be written this way. Is there a standard way to write something you'd like tested so that it can be reused by later tests? Or is there a workaround?
I've added an empty runTest method to LoginTest as a dubious workaround for now.
The confusion with "runTest" is mostly based on the fact that this works:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
unittest.main()
So there is no "runTest" in that class and all of the test-functions are being called. However if you look at the base class "TestCase" (lib/python/unittest/case.py) then you will find that it has an argument "methodName" that defaults to "runTest" but it does NOT have a default implementation of "def runTest"
class TestCase:
def __init__(self, methodName='runTest'):
The reason that unittest.main works fine is based on the fact that it does not need "runTest" - you can mimic the behaviour by creating a TestCase-subclass instance for all methods that you have in your subclass - just provide the name as the first argument:
class MyTest(unittest.TestCase):
def test_001(self):
print "ok"
if __name__ == "__main__":
suite = unittest.TestSuite()
for method in dir(MyTest):
if method.startswith("test"):
suite.addTest(MyTest(method))
unittest.TextTestRunner().run(suite)
Here's some 'deep black magic':
suite = unittest.TestLoader().loadTestsFromTestCase(Test_MyTests)
unittest.TextTestRunner(verbosity=3).run(suite)
Very handy if you just want to test run your unit tests from a shell (i.e., IPython).
If you don't mind editing unit test module code directly, the simple fix is to add under case.py class TestCase a new method called runTest that does nothing.
The file to edit sits under pythoninstall\Lib\unittest\case.py
def runTest(self):
pass
This will stop you ever getting this error.
Guido's answer is almost there, however it doesn't explain the thing. I needed to look to unittest code to grasp the flow.
Say you have the following.
import unittest
class MyTestCase(unittest.TestCase):
def testA(self):
pass
def testB(self):
pass
When you use unittest.main(), it will try to discover test cases in current module. The important code is unittest.loader.TestLoader.loadTestsFromTestCase.
def loadTestsFromTestCase(self, testCaseClass):
# ...
# This will look in class' callable attributes that start
# with 'test', and return their names sorted.
testCaseNames = self.getTestCaseNames(testCaseClass)
# If there's no test to run, look if the case has the default method.
if not testCaseNames and hasattr(testCaseClass, 'runTest'):
testCaseNames = ['runTest']
# Create TestSuite instance having test case instance per test method.
loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
return loaded_suite
What the latter does, is converting test case class into test suite, that holds the instances of the class per its test method. I.e. my example will be turned into unittest.suite.TestSuite([MyTestCase('testA'), MyTestCase('testB')]). So if you would like to create a test case manually, you need to do the same thing.
#dmvianna's answer got me very close to being able to run unittest in a jupyter (ipython) notebook, but I had to do a bit more. If I wrote just the following:
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods)
unittest.TextTestRunner().run(suite)
I got
Ran 0 tests in 0.000s
OK
It's not broken, but it doesn't run any tests! If I instantiated the test class
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
(note the parens at the end of the line; that's the only change) I got
ValueError Traceback (most recent call last)
in ()
----> 1 suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
/usr/lib/python2.7/unittest/case.pyc in init(self, methodName)
189 except AttributeError:
190 raise ValueError("no such test method in %s: %s" %
--> 191 (self.class, methodName))
192 self._testMethodDoc = testMethod.doc
193 self._cleanups = []
ValueError: no such test method in : runTest
The fix is now reasonably clear: add runTest to the test class:
class TestStringMethods(unittest.TestCase):
def runTest(self):
test_upper (self)
test_isupper (self)
test_split (self)
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
suite = unittest.TestLoader().loadTestsFromModule (TestStringMethods())
unittest.TextTestRunner().run(suite)
Ran 3 tests in 0.002s
OK
It also works correctly (and runs 3 tests) if my runTest just passes, as suggested by #Darren.
This is a little yucchy, requiring some manual labor on my part, but it's also more explicit, and that's a Python virtue, isn't it?
I could not get any of the techniques via calling unittest.main with explicit arguments from here or from this related question Unable to run unittest's main function in ipython/jupyter notebook to work inside a jupyter notebook, but I am back on the road with a full tank of gas.
unittest does deep black magic -- if you choose to use it to run your unit-tests (I do, since this way I can use a very powerful battery of test runners &c integrated into the build system at my workplace, but there are definitely worthwhile alternatives), you'd better play by its rules.
In this case, I'd simply have EditProfileTestCase derive from LoginTestCase (rather than directly from unittest.TestCase). If there are some parts of LoginTestCase that you do want to also test in the different environment of EditProfileTestCase, and others that you don't, it's a simple matter to refactor LoginTestCase into those two parts (possibly using multiple inheritance) and if some things need to happen slightly differently in the two cases, factor them out into auxiliary "hook methods" (in a "Template Method" design pattern) -- I use all of these approaches often to diminish boilerplate and increase reuse in the copious unit tests I always write (if I have unit-test coverage < 95%, I always feel truly uneasy -- below 90%, I start to feel physically sick;-).