Let's say I have the following modules:
# src/myapp/utils.py
import thirdparty
from myapp.secrets import get_super_secret_stuff
def get_thirdparty_client() -> return thirdparty.Client:
thirdparty.Client(**get_super_secret_stuff())
# src/myapp/things.py
from myapp.utils import get_thirdparty_client
from myapp.transformations import apply_fairydust, make_it_sparkle
def do_something(x):
thirdparty_client = get_thirdparty_client()
y = thidparty_client.query(apply_fairydust(x))
return make_it_sparkle(y)
Assume that myapp is lightly-tested legacy code, and refactoring is out of the question. Also assume (annoyingly) that thirdparty.Client does non-deterministic network I/O in its __init__ method. Therefore I intend to mock the thirdparty.Client class itself so as to make this do_something function testable.
Assume also that I must use unittest and cannot use another test framework like Pytest.
It seems like the patch function from unittest.mock is the right tool for the job. However, I'm unsure of how to apply the usual admonition to "patch where it is used."
Ideally I want to write a test that looks something like this:
# tests/test_myapp/test_things.py
from unittest import TestCase
from unittest.mock import patch
from myapp.things import do_something
def gen_test_pairs():
# Generate pairs of expected inputs and outputs
...
class ThingTest(unittest.TestCase):
#patch('????')
def test_do_something(self, mock_thirdparty_client):
for x, y in gen_test_pairs:
with self.subTest(params={'x': x, 'y': y}):
mock_thirdparty_client.query.return_value = y
self.assertEqual(do_something(x), y)
My problem is that I don't know what to write in place of the ????, because I never actually import thirdparty.Client in src/myapp/things.py.
Options I considered:
Apply the patch at myapp.utils.thirdparty.Client, which makes my test fragile and dependent on implementation details.
"Break the rules" and apply the patch at thirdparty.Client.
Import get_thirdparty_client in the test, use patch.object on it, and set its return_value to another MagicMock that I create separately, and this second mock would stand in for thirdparty.Client. This makes for more verbose testing code that can't easily be applied as a single decorator.
None of these options sounds particularly appealing, but I don't know which is considered the least bad.
Or is there another option available to me that I am not seeing?
The correct answer is 2: apply the patch to thirdparty.Client.
This is correct because it does NOT in fact break the "patch where it is used" rule.
The rule is intended to be descriptive, not literal. In this case, thirdparty.Client is considered to be "used" in the thirdparty module.
This concept is described in more detail in the 2018 PyCon talk "Demystifying the Patch Function" by Lisa Roach. The full recording is available on YouTube here: https://youtu.be/ww1UsGZV8fQ?t=537. The explanation of this particular case starts at approximately 9 minutes into the video.
Related
I have a complex function that calls many other 3rd party methods. I monkeypatched them out one by one:
import ThirdParty as tp
def my_method():
tp.func_3rd_party_1()
...
tp.func_3rd_party_5()
return "some_value"
In my test:
import pytest
def test_my_method(monkeypatch):
monkeypatch.setattr(ThirdParty, 'func_3rd_party_1', some_mock_1())
...
monkeypatch.setattr(ThirdParty, 'func_3rd_party_5', some_mock_5())
return_value = my_method()
assert return value
This runs just fine but the test feels too implicit for me in this form. I'd like to explicitly state that the monkeypatched methods were indeed called.
For the record, my mocked methods are not using any inbuilt Mock library resource. They are just redefined methods (smart stubs).
Is there any way to assert for that?
So the pytest monkeypatching fixture is specifically provided so you can change some global attributes like environment variables, stuff in third party libraries, etc, to provide some controlled and easy behavior for your test.
The Mock objects, on the other hand, are meant to provide all sorts of tracking and inspection on the object.
The two go hand in hand: You use patching to replace some third party function with a Mock object, then execute your code, and then ask the Mock object if it has indeed been invoked with the right arguments, for the right number of times.
Note that even though the mock module is part of unittest, it works perfectly fine with pytest.
Now as for the patching itself, it's up to your personal preference, and depends a bit on what exactly you want to patch, whether using unittest.mock.patch is more compact or pytest's monkeypatch fixture.
import pytest
from unittest.mock import Mock
def test_my_method(monkeypatch):
# refer to the mock module documentation for more complex
# set ups, where the mock object _also_ exhibits some behavior.
# as is, calling the function doesn't actually _do_ anything.
some_mock_1 = Mock()
...
some_mock_5 = Mock(return_value=66)
monkeypatch.setattr(ThirdParty, 'func_3rd_party_1', some_mock_1)
...
monkeypatch.setattr(ThirdParty, 'func_3rd_party_5', some_mock_5)
some_mock_1.assert_called_once()
some_mock_5.assert_called_with(42)
...
Now a note on this type of testing: Don't go overboard! It can quite easily lead to what's called brittle tests: Tests that break with the slightest change to your code. It can make refactoring an impossible neightmare.
These types of assertions are best when you use them in a message-focused object-oriented approach. If the whole point of the class or method under test is to invoke, in a particular way, the method or class of another object, then Mock away. If the calls to third party functions on the other hand are merely a means to an end, then go a level higher with your test and test for the desired behavior instead.
I haven't been able to found a good explanation of this in the net, I'm guessing that i'm missing something trivial but I haven't been able to find it, so I came here to ask the experts :)
I have a test were I need to patch a constructor call, reading the docs as I understand, something like this should work, but:
import unittest.mock as mocker
import some_module
mock_class1 = mocker.patch('some_module.some_class')
print(mock_class1 is some_module.some_class) # Returns False
print(mock_class1) # <unittest.mock._patch>
mock_instance1 = mock_class1.return_value # _patch object has no attr return_value
Instead I get a different output if I do this
with mocker.patch('some_module.some_class') as mock_class2:
print(mock_class2 is some_module.some_class) # Returns True
print(mock_class2) # <MagicMock name=...>
mock_instance2 = mock_class2.return_value # No problem
print(mock_instance2) # <NonCallableMagicMock name=...>
Now, for the test itself, i'm using pytest-mock module which gives a mocker fixture that behaves like the first code block.
I would like to know:
why the behavior differs depending on the way that one call the mock framework
is there a clean way to trigger the behavior of the second code block without the with clause?
1) pytest mocker plugin is being developer to avoid use of context managers; and probably not everybody fond of the way the standard mock plays with functions parameter 2) not really. It is intended to be used either as content manager or function decorator.
I think it is possible use mocker package without pytest
References
https://github.com/pytest-dev/pytest-mock
https://www.packtpub.com/mapt/book/application_development/9781847198846/5/ch05lvl1sec45/integrating-with-python-mocker
What about installing pytest-mock and creating a test like this
import itertools
def test1(mocker):
mock_class1 = mocker.patch('itertools.count')
print(mock_class1 is itertools.count)
print(mock_class1)
mock_instance1 = mock_class1.return_value # Magic staff...
or may be using monkeypatching? Just do not use the standard unittest.mock with pytest
I want to test an evolving SQLite database application, which is in parallel used "productively". In fact I am investigating a pile of large text files by importing them to the database and fiddling around with it. I am used to develop test-driven, and I do not want to drop that for this investigation. But running tests against the "production" database feels somewhat strange. So my objective is to run the tests against a test database (a real SQLite database, not a mock) containing a controlled, but considerable amount of real data showing all kinds of variability I have met during the investigation.
To support this approach, I have a central module myconst.py containing a function returning the name of the database that is used like so:
import myconst
conn = sqlite3.connect(myconst.get_db_path())
Now in the unittest TestCases, I thought about mocking like so:
#patch("myconst.get_db_name", return_value="../test.sqlite")
def test_this_and_that(self, mo):
...
where the test calls functions that will, in nested functions, access the database using myconst.get_db_path().
I have tried to do a little mocking for myself first, but it tends to be clumsy and error prone so I decided to dive into the python mock module as shown before.
Unfortunately, I found warnings all over, that I am supposed to "mock where it's used and not where it's defined" like so:
#patch("mymodule.myconst.get_db_name", return_value="../test.sqlite")
def test_this_and_that(self, mo):
self.assertEqual(mymodule.func_to_be_tested(), 1)
But mymodule will likely not call database functions itself but delegate that to another module. This in turn would imply that my unit tests have to know the call tree where the database is actually access – something I really want to avoid because it would lead to unnecessary test refactoring when the code is refactored.
So I tried to create a minimal example to understand the behavior of mock and where it fails to allow me to mock "at the source". Because a multi module setup is clumsy here, I have provided the original code also on github for everybody's convenience. See this:
myconst.py
----------
# global definition of the database name
def get_db_name():
return "../production.sqlite"
# this will replace get_db_name()
TEST_VALUE = "../test.sqlite"
def fun():
return TEST_VALUE
inner.py
--------
import myconst
def functio():
return myconst.get_db_name()
print "inner:", functio()
test_inner.py
-------------
from mock import patch
import unittest
import myconst, inner
class Tests(unittest.TestCase):
#patch("inner.myconst.get_db_name", side_effect=myconst.fun)
def test_inner(self, mo):
"""mocking where used"""
self.assertEqual(inner.functio(), myconst.TEST_VALUE)
self.assertTrue(mo.called)
outer.py
--------
import inner
def functio():
return inner.functio()
print "outer:", functio()
test_outer.py
-------------
from mock import patch
import unittest
import myconst, outer
class Tests(unittest.TestCase):
#patch("myconst.get_db_name", side_effect=myconst.fun)
def test_outer(self, mo):
"""mocking where it comes from"""
self.assertEqual(outer.functio(), myconst.TEST_VALUE)
self.assertTrue(mo.called)
unittests.py
------------
"""Deeply mocking a database name..."""
import unittest
print(__doc__)
suite = unittest.TestLoader().discover('.', pattern='test_*.py')
unittest.TextTestRunner(verbosity=2).run(suite)
test_inner.py works like the sources linked above say, and so I was expecting it to pass. test_outer.py should fail when I understand the caveats right. But all the tests pass without complaint! So my mock is drawn all the time, even when the mocked function is called from down the callstack like in test_outer.py. From that example I would conclude that my approach is safe, but on the other hand the warnings are consistent across quite some sources and I do not want to recklessly risk my "production" database by using concepts that I do not grok.
So my question is: Do I misunderstand the warnings or are these warnings just over-cautious?
Finally I sorted it out. Maybe this will help future visitors, so I will share my findings:
When changing the code like so:
inner.py
--------
from myconst import get_db_name
def functio():
return get_db_name()
test_inner.py
-------------
#patch("inner.get_db_name", side_effect=myconst.fun)
def test_inner(self, mo):
self.assertEqual(inner.functio(), myconst.TEST_VALUE)
test_inner will succeed, but test_outer will break with
AssertionError: '../production.sqlite' != '../test.sqlite'
This is because mock.patch will not replace the referenced object, which is function get_db_name in module myconst in both cases. mock will instead replace the usages of the name "myconst.get_db_name" by the Mock object passed as the second parameter to the test.
test_outer.py
-------------
#patch("myconst.get_db_name", side_effect=myconst.fun)
def test_outer(self, mo):
self.assertEqual(outer.functio(), myconst.TEST_VALUE)
Since I mock only "myconst.getdb_name" here and inner.py accesses get_db_name via "inner.get_db_name", the test will fail.
By using the proper name, however, this can be fixed:
#patch("outer.inner.get_db_name", return_value=myconst.TEST_VALUE)
def test_outer(self, mo):
self.assertEqual(outer.functio(), myconst.TEST_VALUE)
So the conclusion is that my approach will be safe when I make sure that all modules accessing the database include myconst and use myconst.get_db_name. Alternatively all modules could from myconst import get_db_name and use get_db_name. But I have to draw this decision globally.
Because I control all code accessing get_db_name I am safe. One can argue whether this is good style or not (assumingly the latter), but technically it's safe. Would I mock a library function instead, I could hardly control access to that function and so mocking "where it's defined" becomes risky. This is why the sources cited are warning.
I want to test a function in python, but it relies on a module-level "private" function, that I don't want called, but I'm having trouble overriding/mocking it. Scenario:
module.py
_cmd(command, args):
# do something nasty
function_to_be_tested():
# do cool things
_cmd('rm', '-rf /')
return 1
test_module.py
import module
test_function():
assert module.function_to_be_tested() == 1
Ideally, in this test I dont want to call _cmd. I've looked at some other threads, and I've tried the following with no luck:
test_function():
def _cmd(command, args):
# do nothing
pass
module._cmd = _cmd
although checking module._cmd against _cmd doesn't give the correct reference. Using mock:
from mock import patch
def _cmd_mock(command, args):
# do nothing
pass
#patch('module._cmd', _cmd_mock)
test_function():
...
gives the correct reference when checking module._cmd, although `function_to_be_tested' still uses the original _cmd (as evidenced by it doing nasty things).
This is tricky because _cmd is a module-level function, and I dont want to move it into a module
[Disclaimer]
The synthetic example posted in this question works and the described issue become from specific implementation in production code. Maybe this question should be closed as off topic because the issue is not reproducible.
[Note] For impatient people Solution is at the end of the answer.
Anyway that question given to me a good point to thought: how we can patch a method reference when we cannot access to the variable where the reference is?
Lot of times I found some issue like this. There are lot of ways to meet that case and the commons are
Decorators: the instance we would like replace is passed as decorator argument or used in decorator static implementation
What we would like to patch is a default argument of a method
In both cases maybe refactor the code is the best way to play with that but what about if we are playing with some legacy code or the decorator is a third part decorator?
Ok, we have the back on the wall but we are using python and in python nothing is impossible. What we need is just the reference of the function/method to patch and instead of patching its reference we can patch the __code__: yes I'm speaking about patching the bytecode instead the function.
Get a real example. I'm using default parameter case that is simple, but it works either in decorator case.
def cmd(a):
print("ORIG {}".format(a))
def cmd_fake(a):
print("NEW {}".format(a))
def do_work(a, c=cmd):
c(a)
do_work("a")
cmd=cmd_fake
do_work("b")
Output:
ORIG a
ORIG b
Ok In this case we can test do_work by passing cmd_fake but there some cases where is impossible do it: for instance what about if we need to call something like that:
def what_the_hell():
list(map(lambda a:do_work(a), ["c","d"]))
what we can do is patch cmd.__code__ instead of _cmd by
cmd.__code__ = cmd_fake.__code__
So follow code
do_work("a")
what_the_hell()
cmd.__code__ = cmd_fake.__code__
do_work("b")
what_the_hell()
Give follow output:
ORIG a
ORIG c
ORIG d
NEW b
NEW c
NEW d
Moreover if we want to use a mock we can do it by add follow lines:
from unittest.mock import Mock, call
cmd_mock = Mock()
def cmd_mocker(a):
cmd_mock(a)
cmd.__code__=cmd_mocker.__code__
what_the_hell()
cmd_mock.assert_has_calls([call("c"),call("d")])
print("WORKS")
That print out
WORKS
Maybe I'm done... but OP still wait for a solution of his issue
from mock import patch, Mock
cmd_mock = Mock()
#A closure for grabbing the right function code
def cmd_mocker(a):
cmd_mock(a)
#patch.object(module._cmd,'__code__', new=cmd_mocker.__code__)
test_function():
...
Now I should say never use this trick unless you are with the back on the wall. Test should be simple to understand and to debug ... try to debug something like this and you will become mad!
Say I want to test this module:
import osutils
def check_ip6(xml):
ib_output = osutils.call('iconfig ib0')
# process and validate ib_output (to be unit tested)
...
This method is dependent on the environment, because it makes a System call (which expects a specific network interface), so its not callable on a testmachine.
I want to write a Unit test for that method which checks if the processing of ib_output works as expected. Therefore I want to mock osutils.call and let it just return testdata. What is the preferred way to do that? Do I have to do mocking or (monkey) patching?
Example test:
def test_ib6_check():
from migration import check_ib6
# how to mock os_utils.call used by the check_ib6-method?
assert check_ib6(test_xml) == True
One solution would be to do from osutils import call and then when patching things replace yourmodule.call with something else before calling test_ib6_check.
Ok, I found out that this has nothing to do with mocks, afaik I just need a monkey patch: I need to import and change the osutils.call-method and then import the method under test (and NOT the whole module, since it would then import the original call-method) afterwards. So this method will then use my changed call-method:
def test_ib6_check():
def call_mock(cmd):
return "testdata"
osutils.call = call_mock
from migration import check_ib6
# the check_ib6 now uses the mocked method
assert check_ib6(test_xml) == True