Develop Reference

Mock not seen by pytest

I got a class that describes an entity and am trying to write unit tests that should check if certain fields are defaulted to the correct value. One of those fields uses datetime.now() to set the default state.
I am having trouble trying to mock this call to now() in my test. I am guessing it has to do with my folder structure.
src
classes
MyClass.py
pytests
test_MyClass.py
The MyClass definition:
from datetime import datetime
class MyClass:
def __init__(self):
self.mom = datetime.now()
I am using #mock.patch as follows (inside test_MyClass.py):
import pytest
import sys
from unittest import mock
sys.path.append(r'C:\python-projects\test\src')
from classes.MyClass import MyClass
#mock.patch('classes.MyClass.datetime.now', return_value = 'timestamp', autospec = True)
#pytest.fixture
def myclass():
myclass = MyClass()
yield myclass
def test_has_datetime(myclass):
assert myclass.mom == 'timestamp'
The test ignores the patch.

Try this:
make sure you have __init__.py files in src, classesand pytests directories;
remove sys.path.append(r'C:\python-projects\test\src')
in #mock.patch, replace classes.MyClass.datetime.now with src.classes.MyClass.datetime.now
make C:\python-projects\test\ the current working directory and run pytest ./pytests
Otherwise, mocking the datetime module can be easier with packages like freezegun: https://github.com/spulec/freezegun

Mock a class before importing it

I'm trying to mock a method in a parent class that is declared out of any class method. The problem es I can't figure out how to make the LoggerUtils class to be instantiated in the parent class. Puting it inside an __init__ is not an option due to the large size of the implementation and the cost of refactor.
Is there a way to mock the parent class prior to the import of the tested class?
At least mock the method before is loaded when importing the class.
Is there any approach to solve the problem of non-lazy methods loading on import?
I tried lazy loading TensorFlow LazyLoad Library methods but I just didnt get it to work; patching all methods with mock library, but the methods always load before I can mock anything. Below I have an example of the mocking tries, but LoggerUtils is always called.
Parent Class:
abstract_api.py
class AbstractApi:
logger = LoggerUtils.get_logger('AbstractApi')
def update(self):
<code>
Class to test:
api_map_asset_type.py
from abstract_api import AbstractApi
class ApiMapAssetType(AbstractApi):
def update(self):
<code>
Test Class:
test_api_map_asset_type.py
from unittest import TestCase
from mock imort patch
from api_map_asset_type import ApiMapAssetType
class TestApiMapAssetType(TestCase):
#patch('api_map_asset_type.AbstractApi.LoggerUtils')
#patch('api_map_asset_type.AbstractApi')
def setUp(self, mock2_abstract_loger, mock_3):
self.asset_api = ApiMapAssetType()
#patch('AbstractApi.update')
def test_update(self, mock_parent_update):
mock_orm = MagicMock()
self.asset_api.update()
mock_parent_update.assert_called_with()
EDITED
This is the only solution I found, since I am not able to mock parent classes or mock methods in class attributes before being imported, I decided to mock the whole test before importing, yet I think this is not an optimal or clean solution:
Test Class:
test_api_map_asset_type.py
from undetermined_project_library.LoggerUtils import LoggerUtils
with patch.object(LoggerUtils, 'get_logger') as mock_logger:
from unittest import TestCase
from mock imort patch
from api_map_asset_type import ApiMapAssetType
class TestApiMapAssetType(TestCase):
def setUp(self):
self.asset_api = ApiMapAssetType()
#patch('AbstractApi.update')
def test_update(self, mock_parent_update):
mock_orm = MagicMock()
self.asset_api.update()
mock_parent_update.assert_called_with()

Provided I understood your module layout, this should work:
class TestApiMapAssetType(TestCase):
#patch('undetermined_project_library.LoggerUtils')
def setUp(self, mock_abstract_logger):
self.asset_api = ApiMapAssetType()
#patch('api_map_asset_type.ApiMapAssetType.update')
def test_update(self, mock_parent_update):
self.asset_api.update()
mock_parent_update.assert_called_with()
Note a couple of things:
you have to mock LoggerUtils from the library they are defined in; you have used AbstractApi.LoggerUtils, which is not correct, as LoggerUtils does not belong to AbstractApi
I removed the mock for AbstractApi - at least for this you don't need it
update is patched for the method that is actually called - using the base class method may make sense if you call the base implementation (don't know if you do)
I guessed some of your module layout - you may have to adapt it

You can do this by using the sys module in python. You can register the imports with sys module and replace the implementation with Mock or MagicMock instances.
This is how it looks like in code -
import sys
from unittest.mock import MagicMock
sys.modules['numpy'] = MagicMock()
sys.modules['pandas'] = MagicMock()
sys.modules['torch'] = MagicMock()
sys.modules['torch.nn'] = MagicMock()
sys.modules['torch.nn.functional'] = MagicMock()
sys.modules['dgl'] = MagicMock()
sys.modules['dgl.ops'] = MagicMock()
sys.modules['dgl.function'] = MagicMock()
from path.to.module_under_test import ClassUnderTest
Assuming module_under_test.py starts like -
import numpy as np
import pandas as pd
import torch
import dgl
import torch.nn.functional as F
from dgl.ops import edge_softmax
class ClassUnderTest:
# code

Best practices using python mock for testing functions within sub modules

So,
consider I have a simple library that I am trying to write unit-tests for. This library talks to a database and then uses that data to call an SOAP API. I have three modules, and a testfile for each module.
dir structure:
./mypkg
../__init__.py
../main.py
../db.py
../api.py
./tests
../test_main
../test_db
../test_api
Code:
#db.py
import mysqlclient
class Db(object):
def __init__(self):
self._client = mysqlclient.Client()
#property
def data(self):
return self._client.some_query()
#api.py
import soapclient
class Api(object):
def __init__(self):
self._client = soapclient.Client()
#property
def call(self):
return self._client.some_external_call()
#main.py
from db import Db
from api import Api
class MyLib(object):
def __init__(self):
self.db = Db()
self.api = Api()
def caller(self):
return self.api.call(self.db.data)
Unit-Tests:
#test_db.py
import mock
from mypkg.db import Db
#mock.patch('mypkg.db.mysqlclient')
def test_db(mysqlclient_mock):
mysqlclient_mock.Client.return_value.some_query = {'data':'data'}
db = Db()
assert db.data == {'data':'data'}
#test_api.py
import mock
from mypkg.api import Api
#mock.patch('mypkg.db.soapclient')
def test_db(soap_mock):
soap_mock.Client.return_value.some_external_call = 'foo'
api = Api()
assert api.call == 'foo'
In the above example, mypkg.main.MyLib calls mypkg.db.Db() (uses third-party mysqlclient) and then mypkg.api.Api() (uses third-party soapclient)
I am using mock.patch to patch the third-party libraries to mock my db and api calls in test_db and test_api separately.
Now my question is, is it recommended to patch these external calls again in test_main OR simply patch db.Db and api.Api? (this example is pretty simple, but in larger libraries, the code becomes cumbersome when patching the external calls again or even using test helper functions that patch internal libraries).
Option1: patch external libraries in main again
#test_main.py
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.db.mysqlclient')
#mock.patch('mypkg.api.soapclient')
def test_main(soap_mock, mysqlcient_mock):
ml = MyLib()
soap_mock.Client.return_value.some_external_call = 'foo'
assert ml.caller() == 'foo'
Option2: patch internal libraries
#test_main.py
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.db.Db')
#mock.patch('mypkg.api.Api')
def test_main(api_mock, db_mock):
ml = MyLib()
api_mock.return_value = 'foo'
assert ml.caller() == 'foo'

mock.patch creates a mock version of something where it's imported, not where it lives. This means the string passed to mock.patch has to be a path to an imported module in the module under test. Here's what the patch decorators should look like in test_main.py:
#mock.patch('mypkg.main.Db')
#mock.patch('mypkg.main.Api')
Also, the handles you have on your patched modules (api_mock and db_mock) refer to the classes, not instances of those classes. When you write api_mock.return_value = 'foo', you're telling api_mock to return 'foo' when it gets called, not when an instance of it has a method called on it. Here are the objects in main.py and how they relate to api_mock and db_mock in your test:
Api is a class : api_mock
Api() is an instance : api_mock.return_value
Api().call is an instance method : api_mock.return_value.call
Api().call() is a return value : api_mock.return_value.call.return_value
Db is a class : db_mock
Db() is an instance : db_mock.return_value
Db().data is an attribute : db_mock.return_value.data
test_main.py should therefore look like this:
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.main.Db')
#mock.patch('mypkg.main.Api')
def test_main(api_mock, db_mock):
ml = MyLib()
api_mock.return_value.call.return_value = 'foo'
db_mock.return_value.data = 'some data' # we need this to test that the call to api_mock had the correct arguments.
assert ml.caller() == 'foo'
api_mock.return_value.call.assert_called_once_with('some data')
The first patch in Option 1 would work great for unit-testing db.py, because it gives the db module a mock version of mysqlclient. Similarly, #mock.patch('mypkg.api.soapclient') belongs in test_api.py.
I can't think of a way Option 2 could help you unit-test anything.
Edited: I was incorrectly referring to classes as modules. db.py and api.py are modules

Python Mock Patch multiple methods in a class

Im trying to patch multiple methods in a class. Here is my simplified set up
Hook.py is defined as
class Hook():
def get_key(self):
return "Key"
def get_value(self):
return "Value"
HookTransfer.py defined as
from Hook import Hook
class HookTransfer():
def execute(self):
self.hook = Hook()
key = self.hook.get_key()
value = self.hook.get_value()
print(key)
print(value)
I want to mock the methods get_key and get_value in the Hook class. The following works i.e. prints New_Key and New_Value
from HookTransfer import HookTransfer
import unittest
from unittest import mock
class TestMock(unittest.TestCase):
#mock.patch('HookTransfer.Hook.get_key', return_value="New_Key")
#mock.patch('HookTransfer.Hook.get_value', return_value="New_Value")
def test_execute1(self, mock_get_key, mock_get_value):
HookTransfer().execute()
if __name__ == '__main__':
unittest.main()
However this does not. It prints <MagicMock name='Hook().get_key()' id='4317706896'> and <MagicMock name='Hook().get_value()' id='4317826128'>
from HookTransfer import HookTransfer
import unittest
from unittest import mock
class TestMock(unittest.TestCase):
#mock.patch('HookTransfer.Hook', spec=True)
def test_execute2(self, mock_hook):
mock_hook.get_key = mock.Mock(return_value="New_Key")
mock_hook.get_value = mock.Mock(return_value="New_Value")
HookTransfer().execute()
if __name__ == '__main__':
unittest.main()
Intuitively it seems like the second one should work too but it doesnt. Could you help explain why it does not. I suspect it has something to do with "where to patch" but Im unable to get clarity.

You can patch multiple methods of a module or a class using patch.multiple(). Something like this should work for your case:
import unittest
from unittest.mock import MagicMock, patch
class TestMock(unittest.TestCase):
#patch.multiple('HookTransfer.Hook',
get_key=MagicMock(return_value='New_Key'),
get_value=MagicMock(return_value='New_Value'))
def test_execute1(self, **mocks):
HookTransfer().execute()
When patch.multiple() is used as a decorator, the mocks are passed into the decorated function by keyword, and a dictionary is returned when it's used as a context manager.

What you need to is:
mock the class Hook,
from HookTransfer import HookTransfer
from Hook import Hook
import unittest
try:
import mock
except ImportError:
from unittest import mock
class TestMock(unittest.TestCase):
#mock.patch.object(Hook, 'get_key', return_value="New_Key")
#mock.patch.object(Hook, 'get_value', return_value="New_Value")
def test_execute1(self, mock_get_value, mock_get_key):
HookTransfer().execute()
if __name__ == "__main__":
unittest.main()

After some testing I was able to find the issue.
In the second test case, the patch decorator creates a new instance of a Mock class and passes it via mock_hook argument to test_execute2 function. Lets refer to this as mock1. mock1 replaces the Hook class in HookTransfer.py. When self.hook = Hook() is run, it translates to calling __init__ of mock1. By design this returns yet another Mock instance - lets refer to this as mock2. So self.hook points to mock2. But mock_hook.get_key = mock.Mock(return_value="New_Key"), mocks the methods in mock1.
In order to mock correctly, mock2 needs to be patched. This can be done in 2 ways
By mocking the return_value of mock1 (which returns mock2) mock_hook.return_value.get_key = mock.Mock(return_value="New_Key")
Mocking the return value of constructor of mock1 (which returns mock2) mock_hook().get_key = mock.Mock(return_value="New_Key")
Under the wraps both options really do the same thing.

Python's mock.patch not patching class in unittest

I'm getting confused by using Mock in my python unittests. I've made this simplified version of my problem:
I have this dummy class and methods:
# app/project.py
class MyClass(object):
def method_a(self):
print(FetcherA)
results = FetcherA()
Which is using this class:
# app/fetch.py
class FetcherA(object):
pass
And then this test:
# app/tests/test.py
from mock import patch
from django.test import TestCase
from ..project import MyClass
class MyTestCase(TestCase):
#patch('app.fetch.FetcherA')
def test_method_a(self, test_class):
MyClass().method_a()
test_class.assert_called_once_with()
I would expect that running this test would pass and that print statement, for debugging, would output something like <MagicMock name=...>. Instead it prints out <class 'app.fetch.FetcherA'> and I get:
AssertionError: Expected to be called once. Called 0 times.
Why isn't FetcherA being patched?

OK, fourth time through I think I understood the 'Where to patch' section of the Mock docs.
So, instead of:
#patch('app.fetch.FetcherA')
I should use:
#patch('app.project.FetcherA')
Because we're testing the code in app.project.MyClass where FetcherA has been imported already. So at that point FetcherA is availably globally(?) within app.project.