I'm a beginner programmer, I've been stuck for the past week trying to write unit tests. I read through the unit test docs and watched two long tutorials on implementing unit testing with Mock. The docs refer to mocking classes extensively, but for functions, I'm not sure if I should be using #patch/with patch, patch.dict{}, side_effect, or some other option to mock a function, specifically the argument to a function.
mymodule.py
def regex():
'''Runs a regex, creates a dict 'data' and then calls scraper(data)'''
def scraper(data):
'''scrapes a website and then calls a function which submits data to a db'''
I would like to create a test that passes in test data to the function scraper . Thank you in advance.
Yes, you can also do unit test using mock for non-object_oriented code.
See example below:
from unittest.mock import MagicMock
def a():
return 10
def b():
print(a())
b()
a = MagicMock(return_value=3)
b()
And the output is:
10
3
In the previous example mock is used to fake/mock the function a(), so you can test function b() in isolation, b() is your SUT, without calling a() real implementation. This can be useful for more complex code, specially when function a() relies on data that might not be available in the unit test level.
You haven't given enough examples to really help, but a couple of observations:
Generally, it's better to structure your code to avoid the need for mocking in tests; to the extent possible, each function should be a self-contained piece of code, which can be called separately.
There's no need to mock arguments; simply pass the test value in.
I'm not sure what's intended with this code:
bar = {'key': 'value'}
def foo(bar):
pass
The bar defined at the outer level is a completely separate variable to the bar used as an argument in the function definition. It is confusing to give them the same name...
A function can be mocked using patch or patch.object with the return_value=... option; often, though, it suggests that the code needs to be refactored to reduce the dependency between the two functions.
I'm trying to mock a function used within the function I'm testing, but for some reason I'm not seeing the original function always runs, not the one I created as a mock.
The code I'm testing has this type of setup:
def function_being_tested(foo):
...
function_i_want_to_mock(bar)
...
def function_i_want_to_mock(bar)
print("Inside original function")
...
I've installed Mock and I've tried using unittest.mock patch
Currently the test file uses this setup:
import mock
from django.test import TestCase
def mock_function_i_want_to_mock(bar):
print(bar)
return True
class SupportFunctionsTestCases(TestCase):
#mock.patch("path.to.function.function_i_want_to_mock", mock_function_i_want_to_mock)
def test_function_being_tested(self):
# setup
result = function_being_tested(test_foo)
self.assertEqual(result, expected_result)
What then happens is when I run the test, I always get: "Inside original function", not the parameter printed so it's always running the original function.
I've used this exact setup before and it has worked so I'm not sure what's causing this. Probably some wrong setup...
If anyone has a different way of doing this or spots some error, it would be appreciated.
"path.to.function.function_i_want_to_mock" should be a path to where the function is used, not defined.
So if function_i_want_to_mock is defined in moduleA.py but imported and used in moduleB.py which you are testing then you should use #mock.patch("path.to.moduleB.function_i_want_to_mock", ...).
I am writing python tests for the first time. I am trying to test a basic mock. I want to return some value that I want when I call the function, rather than a mock object.
Here is the code:
In views:
def myfunction():
return "Actual data"
In test:
class TestBasic(unittest.TestCase):
#patch('trailblazer.views.myfunction')
def testMyFunction(self, val):
print(val)
val.return_value = "Test value"
print(val)
op = myfunction()
print(op)
output:
<MagicMock name='myfunction' id='4520521120'>
<MagicMock name='myfunction' id='4520521120'>
Actual data
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
PS: I don't have my methods in a class and I don't want to change that.
You have a direct reference to myfunction() in your test module, and that reference is never patched. You only patched the reference in the trailblazer.views module.
Your test would work if you used that reference instead of myfunction:
from trailblazer import views
class TestBasic(unittest.TestCase):
#patch('trailblazer.views.myfunction')
def testMyFunction(self, val):
print(val)
val.return_value = "Test value"
print(val)
op = views.myfunction()
print(op)
However, a more meaningful test is to test the code that uses myfunction(). You use mocking to be able to focus on the behaviour of a specific unit of code, where mocking lets you precisely control interactions with other units.
In other words, if you have code like:
def some_function_to_test():
# other things
result = myfunction()
# more things working on result
return final_result
then when testing some_function_to_test() it makes sense to patch myfunction().
I recommend you read up on how Python names work; I highly recommend Facts and myths about Python names and values, together with Where to patch in the unittest.mock documentation.
I want to test a function in python, but it relies on a module-level "private" function, that I don't want called, but I'm having trouble overriding/mocking it. Scenario:
module.py
_cmd(command, args):
# do something nasty
function_to_be_tested():
# do cool things
_cmd('rm', '-rf /')
return 1
test_module.py
import module
test_function():
assert module.function_to_be_tested() == 1
Ideally, in this test I dont want to call _cmd. I've looked at some other threads, and I've tried the following with no luck:
test_function():
def _cmd(command, args):
# do nothing
pass
module._cmd = _cmd
although checking module._cmd against _cmd doesn't give the correct reference. Using mock:
from mock import patch
def _cmd_mock(command, args):
# do nothing
pass
#patch('module._cmd', _cmd_mock)
test_function():
...
gives the correct reference when checking module._cmd, although `function_to_be_tested' still uses the original _cmd (as evidenced by it doing nasty things).
This is tricky because _cmd is a module-level function, and I dont want to move it into a module
[Disclaimer]
The synthetic example posted in this question works and the described issue become from specific implementation in production code. Maybe this question should be closed as off topic because the issue is not reproducible.
[Note] For impatient people Solution is at the end of the answer.
Anyway that question given to me a good point to thought: how we can patch a method reference when we cannot access to the variable where the reference is?
Lot of times I found some issue like this. There are lot of ways to meet that case and the commons are
Decorators: the instance we would like replace is passed as decorator argument or used in decorator static implementation
What we would like to patch is a default argument of a method
In both cases maybe refactor the code is the best way to play with that but what about if we are playing with some legacy code or the decorator is a third part decorator?
Ok, we have the back on the wall but we are using python and in python nothing is impossible. What we need is just the reference of the function/method to patch and instead of patching its reference we can patch the __code__: yes I'm speaking about patching the bytecode instead the function.
Get a real example. I'm using default parameter case that is simple, but it works either in decorator case.
def cmd(a):
print("ORIG {}".format(a))
def cmd_fake(a):
print("NEW {}".format(a))
def do_work(a, c=cmd):
c(a)
do_work("a")
cmd=cmd_fake
do_work("b")
Output:
ORIG a
ORIG b
Ok In this case we can test do_work by passing cmd_fake but there some cases where is impossible do it: for instance what about if we need to call something like that:
def what_the_hell():
list(map(lambda a:do_work(a), ["c","d"]))
what we can do is patch cmd.__code__ instead of _cmd by
cmd.__code__ = cmd_fake.__code__
So follow code
do_work("a")
what_the_hell()
cmd.__code__ = cmd_fake.__code__
do_work("b")
what_the_hell()
Give follow output:
ORIG a
ORIG c
ORIG d
NEW b
NEW c
NEW d
Moreover if we want to use a mock we can do it by add follow lines:
from unittest.mock import Mock, call
cmd_mock = Mock()
def cmd_mocker(a):
cmd_mock(a)
cmd.__code__=cmd_mocker.__code__
what_the_hell()
cmd_mock.assert_has_calls([call("c"),call("d")])
print("WORKS")
That print out
WORKS
Maybe I'm done... but OP still wait for a solution of his issue
from mock import patch, Mock
cmd_mock = Mock()
#A closure for grabbing the right function code
def cmd_mocker(a):
cmd_mock(a)
#patch.object(module._cmd,'__code__', new=cmd_mocker.__code__)
test_function():
...
Now I should say never use this trick unless you are with the back on the wall. Test should be simple to understand and to debug ... try to debug something like this and you will become mad!
Module A includes import B at its top. However under test conditions I'd like to mock B in A (mock A.B) and completely refrain from importing B.
In fact, B isn't installed in the test environment on purpose.
A is the unit under test. I have to import A with all its functionality. B is the module I need to mock. But how can I mock B within A and stop A from importing the real B, if the first thing A does is import B?
(The reason B isn't installed is that I use pypy for quick testing and unfortunately B isn't compatible with pypy yet.)
How could this be done?
You can assign to sys.modules['B'] before importing A to get what you want:
test.py:
import sys
sys.modules['B'] = __import__('mock_B')
import A
print(A.B.__name__)
A.py:
import B
Note B.py does not exist, but when running test.py no error is returned and print(A.B.__name__) prints mock_B. You still have to create a mock_B.py where you mock B's actual functions/variables/etc. Or you can just assign a Mock() directly:
test.py:
import sys
sys.modules['B'] = Mock()
import A
The builtin __import__ can be mocked with the 'mock' library for more control:
# Store original __import__
orig_import = __import__
# This will be the B module
b_mock = mock.Mock()
def import_mock(name, *args):
if name == 'B':
return b_mock
return orig_import(name, *args)
with mock.patch('__builtin__.__import__', side_effect=import_mock):
import A
Say A looks like:
import B
def a():
return B.func()
A.a() returns b_mock.func() which can be mocked also.
b_mock.func.return_value = 'spam'
A.a() # returns 'spam'
Note for Python 3:
As stated in the changelog for 3.0, __builtin__ is now named builtins:
Renamed module __builtin__ to builtins (removing the underscores, adding an ‘s’).
The code in this answer works fine if you replace __builtin__ by builtins for Python 3.
How to mock an import, (mock A.B)?
Module A includes import B at its top.
Easy, just mock the library in sys.modules before it gets imported:
if wrong_platform():
sys.modules['B'] = mock.MagicMock()
and then, so long as A doesn't rely on specific types of data being returned from B's objects:
import A
should just work.
You can also mock import A.B:
This works even if you have submodules, but you'll want to mock each module. Say you have this:
from foo import This, That, andTheOtherThing
from foo.bar import Yada, YadaYada
from foo.baz import Blah, getBlah, boink
To mock, simply do the below before the module that contains the above is imported:
sys.modules['foo'] = MagicMock()
sys.modules['foo.bar'] = MagicMock()
sys.modules['foo.baz'] = MagicMock()
(My experience: I had a dependency that works on one platform, Windows, but didn't work on Linux, where we run our daily tests.
So I needed to mock the dependency for our tests. Luckily it was a black box, so I didn't need to set up a lot of interaction.)
Mocking Side Effects
Addendum: Actually, I needed to simulate a side-effect that took some time. So I needed an object's method to sleep for a second. That would work like this:
sys.modules['foo'] = MagicMock()
sys.modules['foo.bar'] = MagicMock()
sys.modules['foo.baz'] = MagicMock()
# setup the side-effect:
from time import sleep
def sleep_one(*args):
sleep(1)
# this gives us the mock objects that will be used
from foo.bar import MyObject
my_instance = MyObject()
# mock the method!
my_instance.method_that_takes_time = mock.MagicMock(side_effect=sleep_one)
And then the code takes some time to run, just like the real method.
Aaron Hall's answer works for me.
Just want to mention one important thing,
if in A.py you do
from B.C.D import E
then in test.py you must mock every module along the path, otherwise you get ImportError
sys.modules['B'] = mock.MagicMock()
sys.modules['B.C'] = mock.MagicMock()
sys.modules['B.C.D'] = mock.MagicMock()
I realize I'm a bit late to the party here, but here's a somewhat insane way to automate this with the mock library:
(here's an example usage)
import contextlib
import collections
import mock
import sys
def fake_module(**args):
return (collections.namedtuple('module', args.keys())(**args))
def get_patch_dict(dotted_module_path, module):
patch_dict = {}
module_splits = dotted_module_path.split('.')
# Add our module to the patch dict
patch_dict[dotted_module_path] = module
# We add the rest of the fake modules in backwards
while module_splits:
# This adds the next level up into the patch dict which is a fake
# module that points at the next level down
patch_dict['.'.join(module_splits[:-1])] = fake_module(
**{module_splits[-1]: patch_dict['.'.join(module_splits)]}
)
module_splits = module_splits[:-1]
return patch_dict
with mock.patch.dict(
sys.modules,
get_patch_dict('herp.derp', fake_module(foo='bar'))
):
import herp.derp
# prints bar
print herp.derp.foo
The reason this is so ridiculously complicated is when an import occurs python basically does this (take for example from herp.derp import foo)
Does sys.modules['herp'] exist? Else import it. If still not ImportError
Does sys.modules['herp.derp'] exist? Else import it. If still not ImportError
Get attribute foo of sys.modules['herp.derp']. Else ImportError
foo = sys.modules['herp.derp'].foo
There are some downsides to this hacked together solution: If something else relies on other stuff in the module path this kind of screws it over. Also this only works for stuff that is being imported inline such as
def foo():
import herp.derp
or
def foo():
__import__('herp.derp')
I found fine way to mock the imports in Python. It's Eric's Zaadi solution found here which I just use inside my Django application.
I've got class SeatInterface which is interface to Seat model class.
So inside my seat_interface module I have such an import:
from ..models import Seat
class SeatInterface(object):
(...)
I wanted to create isolated tests for SeatInterface class with mocked Seat class as FakeSeat. The problem was - how tu run tests offline, where Django application is down. I had below error:
ImproperlyConfigured: Requested setting BASE_DIR, but settings are not
configured. You must either define the environment variable
DJANGO_SETTINGS_MODULE or call settings.configure() before accessing
settings.
Ran 1 test in 0.078s
FAILED (errors=1)
The solution was:
import unittest
from mock import MagicMock, patch
class FakeSeat(object):
pass
class TestSeatInterface(unittest.TestCase):
def setUp(self):
models_mock = MagicMock()
models_mock.Seat.return_value = FakeSeat
modules = {'app.app.models': models_mock}
patch.dict('sys.modules', modules).start()
def test1(self):
from app.app.models_interface.seat_interface import SeatInterface
And then test magically runs OK :)
.
Ran 1 test in 0.002s
OK
If you do an import ModuleB you are really calling the builtin method __import__ as:
ModuleB = __import__('ModuleB', globals(), locals(), [], -1)
You could overwrite this method by importing the __builtin__ module and make a wrapper around the __builtin__.__import__method. Or you could play with the NullImporter hook from the imp module. Catching the exception and Mock your module/class in the except-block.
Pointer to the relevant docs:
docs.python.org: __import__
Accessing Import internals with the imp Module
I hope this helps. Be HIGHLY adviced that you step into the more arcane perimeters of python programming and that a) solid understanding what you really want to achieve and b)thorough understanding of the implications is important.
I know this is a fairly old question, but I have found myself returning to it a few times recently, and wanted to share a concise solution to this.
import sys
from unittest import mock
def mock_module_import(module):
"""Source: https://stackoverflow.com/a/63584866/3972558"""
def _outer_wrapper(func):
def _inner_wrapper(*args, **kwargs):
orig = sys.modules.get(module) # get the original module, if present
sys.modules[module] = mock.MagicMock() # patch it
try:
return func(*args, **kwargs)
finally:
if orig is not None: # if the module was installed, restore patch
sys.modules[module] = orig
else: # if the module never existed, remove the key
del sys.modules[module]
return _inner_wrapper
return _outer_wrapper
It works by temporarily patching the key for the module in sys.modules, and then restoring the original module after the decorated function is called. This can be used in scenarios where a package may not be installed in the testing environment, or a more complex scenario where the patched module might actually perform some of its own internal monkey-patching (which was the case I was facing).
Here's an example of use:
#mock_module_import("some_module")
def test_foo():
# use something that relies upon "some_module" here
assert True
I found myself facing a similar problem today, and I've decided to solve it a bit differently. Rather than hacking on top of Python's import machinery, you can simply add the mocked module into sys.path, and have Python prefer it over the original module.
Create the replacement module in a subdirectory, e.g.:
mkdir -p test/mocked-lib
${EDITOR} test/mocked-lib/B.py
Before A is imported, insert this directory to sys.path. I'm using pytest, so in my test/conftest.py, I've simply done:
import os.path
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "mocked-lib"))
Now, when the test suite is run, the mocked-lib subdirectory is prepended into sys.path and import A uses B from mocked-lib.