I am using mockito to unit test a program in Python. I have a class like:
import boto3
import datetime
class Cache:
def __init__(self):
client = boto3.resource('s3')
self.bucket_name = 'name'
self.bucket = client.Bucket(self.bucket_name)
def setup_cache(self, cache_file='cache.csv', cache_filepath='cache'):
cache_object = self.bucket.Object(cache_file)
if cache_object.last_modified < datetime.datetime.now():
self.bucket.download_file(cache_filepath, cache_file)
else:
print('Cache already up to date')
def main():
cache = Cache()
cache.setup_cache()
And the test code I am getting stuck on is this:
from mockito import mock, when
import datetime
import boto3
import mock_cache
class TestMockCache:
def test_last_mod(self):
mock_client = mock()
when(boto3).resource('s3').thenReturn(mock_client)
mock_bucket = mock()
when(mock_client).Bucket('name').thenReturn(mock_bucket)
mock_bucket.last_modified = datetime.datetime.now()
mock_cache.main()
When running pytest on the unit test, I am getting thrown this attribute error:
AttributeError: 'NoneType' object has no attribute 'last_modified'
From the documentation it looked like I could assign 'cache_mock.last_modified' like this. However, I also tried:
when(cache_mock).last_modified.thenReturn(test_date)
and got:
AttributeError: 'StubbedInvocation' object has no attribute 'thenReturn'
Which I don't fully understand, but assume that means a mockito mock() object can't have multiple return values?
Any help with this would be appreciated. I feel like I am misunderstanding something fundamental about either how mockito's mock works or mocking in general.
The patch from the Mock can be used to mock the client from boto3. Do not have to return anything from the client, as it is being mocked on the whole.
e.g.:
from folder.file import your_func
from unittest.mock import patch
class TestSomething(unittest.TestCase):
#patch("botocore.client.BaseClient._make_api_call")
def test_something(self, _mock_client_boto3):
your_func()
.. do something
In case you want to return something from the client, it could be achieved by specifying the return_value in the patch as shown below:
from folder.file import your_func
from unittest.mock import patch
class TestSomething(unittest.TestCase):
#patch("botocore.client.BaseClient._make_api_call", return_value=None)
def test_something(self, _mock_client_boto3):
your_func()
.. do something
Related
I have a class ProductionClass with a method method_to_test which I want to test. Class ProductionClass has dependency to api, which I want to mock in the test.
from my_module.apis import api
class ProductionClass:
def method_to_test:
data = api.method_to_mock()
api.method_to_check_call(data)
The test code is as follows:
For api I have a mock class MockApi that I use by refering to it in the #patch decorator.
from unittest.mock import patch, MagicMock
class MockApi:
def method_to_mock():
return some_mock_data
def method_to_check_call(data):
pass
class TestClass:
#patch('my_module.apis.api', MagicMock(return_value=MockApi()))
def test_check_called_with(self):
from module_of_class_production_class.ProductionClass import method_to_test
mock_api = MockApi()
method_to_test()
some_data = { ... }
mock.method_to_check_call.assert_called_with(some_data)
The problem is that it does not work because mock_api is not the same instance of MockApi that is provided in the #patch decorator. Is there a better way to test that?
I have not tested this, but I think that your patch object will get passed as first argument to test_check_called_with like so:
#patch('my_module.apis.api', MagicMock(return_value=MockApi()))
def test_check_called_with(self, your_magic_mock):
# Rest of code
You can also use the with construct like so:
def test_check_called_with(self):
my_api = MockApi()
with patch('my_module.apis.api', MagicMock(return_value=my_api)) as my_mock_api:
# Your code here
You can checkout the official python documentation here for more details: https://docs.python.org/3/library/unittest.mock.html#quick-guide
I have this file structure
mocking_module
- sample_class.py
- class_util.py
- test_base_class_attribute_override.py
And I defined a class like this. It has a class variable called base_url
sample_class.py
class SampleClass():
base_url = "http://example.com"
def get_base_url(self):
return "base url {}".format(SampleClass.base_url)
And a utility class that uses the SampleClass
class_util.py
from mocking_module.sample_class import SampleClass
def do_something():
sample_class = SampleClass()
return sample_class.get_base_url()
I wanted to patch the SimpleClass base_url class instance variable so in my test case I am doing this
test_base_class_attribute_override.py
import unittest
from unittest.mock import patch
from unittest.mock import Mock
from mocking_module import class_util
class TestSomething(unittest.TestCase):
#patch.object("mocking_module.class_util.SampleClass", "base_url", "http://override.com")
def test_class_variable_override(self):
print("url = {}".format(class_util.do_something()))
But I am getting an error like this:
AttributeError: mocking_module.class_util.SampleClass does not have the attribute 'base_url'
Any thoughts what is wrong?
I'm trying to mock a method in a parent class that is declared out of any class method. The problem es I can't figure out how to make the LoggerUtils class to be instantiated in the parent class. Puting it inside an __init__ is not an option due to the large size of the implementation and the cost of refactor.
Is there a way to mock the parent class prior to the import of the tested class?
At least mock the method before is loaded when importing the class.
Is there any approach to solve the problem of non-lazy methods loading on import?
I tried lazy loading TensorFlow LazyLoad Library methods but I just didnt get it to work; patching all methods with mock library, but the methods always load before I can mock anything. Below I have an example of the mocking tries, but LoggerUtils is always called.
Parent Class:
abstract_api.py
class AbstractApi:
logger = LoggerUtils.get_logger('AbstractApi')
def update(self):
<code>
Class to test:
api_map_asset_type.py
from abstract_api import AbstractApi
class ApiMapAssetType(AbstractApi):
def update(self):
<code>
Test Class:
test_api_map_asset_type.py
from unittest import TestCase
from mock imort patch
from api_map_asset_type import ApiMapAssetType
class TestApiMapAssetType(TestCase):
#patch('api_map_asset_type.AbstractApi.LoggerUtils')
#patch('api_map_asset_type.AbstractApi')
def setUp(self, mock2_abstract_loger, mock_3):
self.asset_api = ApiMapAssetType()
#patch('AbstractApi.update')
def test_update(self, mock_parent_update):
mock_orm = MagicMock()
self.asset_api.update()
mock_parent_update.assert_called_with()
EDITED
This is the only solution I found, since I am not able to mock parent classes or mock methods in class attributes before being imported, I decided to mock the whole test before importing, yet I think this is not an optimal or clean solution:
Test Class:
test_api_map_asset_type.py
from undetermined_project_library.LoggerUtils import LoggerUtils
with patch.object(LoggerUtils, 'get_logger') as mock_logger:
from unittest import TestCase
from mock imort patch
from api_map_asset_type import ApiMapAssetType
class TestApiMapAssetType(TestCase):
def setUp(self):
self.asset_api = ApiMapAssetType()
#patch('AbstractApi.update')
def test_update(self, mock_parent_update):
mock_orm = MagicMock()
self.asset_api.update()
mock_parent_update.assert_called_with()
Provided I understood your module layout, this should work:
class TestApiMapAssetType(TestCase):
#patch('undetermined_project_library.LoggerUtils')
def setUp(self, mock_abstract_logger):
self.asset_api = ApiMapAssetType()
#patch('api_map_asset_type.ApiMapAssetType.update')
def test_update(self, mock_parent_update):
self.asset_api.update()
mock_parent_update.assert_called_with()
Note a couple of things:
you have to mock LoggerUtils from the library they are defined in; you have used AbstractApi.LoggerUtils, which is not correct, as LoggerUtils does not belong to AbstractApi
I removed the mock for AbstractApi - at least for this you don't need it
update is patched for the method that is actually called - using the base class method may make sense if you call the base implementation (don't know if you do)
I guessed some of your module layout - you may have to adapt it
You can do this by using the sys module in python. You can register the imports with sys module and replace the implementation with Mock or MagicMock instances.
This is how it looks like in code -
import sys
from unittest.mock import MagicMock
sys.modules['numpy'] = MagicMock()
sys.modules['pandas'] = MagicMock()
sys.modules['torch'] = MagicMock()
sys.modules['torch.nn'] = MagicMock()
sys.modules['torch.nn.functional'] = MagicMock()
sys.modules['dgl'] = MagicMock()
sys.modules['dgl.ops'] = MagicMock()
sys.modules['dgl.function'] = MagicMock()
from path.to.module_under_test import ClassUnderTest
Assuming module_under_test.py starts like -
import numpy as np
import pandas as pd
import torch
import dgl
import torch.nn.functional as F
from dgl.ops import edge_softmax
class ClassUnderTest:
# code
I am trying to mock the return of db.collection.find to avoid database calls. I tried to create a mock.patch for mongoclient() and attached on it a return_value. But when my_call() call db.collection.find it just return a Mock Object. Somebody have a idea how mock it?
#dao.py
class MyDao():
def my_call():
db = mongoclient().db_name
result = db.collection.find()
return result
#test_dao.py
import dao
def test_my_call():
result = dao.my_call()
assert result == list()
I think it would help to see your current mock to edit. In general, this is how I would mock the pymongo collection.find() as a standalone function.
test_dao.py
import unittest
import mock
from mock import Mock
import pymongo
class Test_Dao(unittest.TestCase):
"""
Set up the mock
"""
#classmethod
#mock.patch('pymongo.collection')
def setUpClass(self, mock_mongo):
a = Mock()
a.find.side_effect = self.findResponse # Assign side effect for mocked method
mock_mongo.return_value = a # Importing pymongo.collection returns the mock
"""
Response Data
"""
findResponse = 'some data'
def test_my_call():
result = dao.my_call()
assert result == list()
...
In your case that is worth a try but might not work because you are calling collection.find() from a variable.
You may need to mock the MongoClient() such that db_name has a side_effect to return a fake class. Then the fake class would need a collection.find() method you define. That would be a little longer and look a little bit like this:
class FakeCollection:
def find:
return 'data'
class FakeDatabase:
def collection:
return FakeCollection()
...
#classmethod
#mock.patch('pymongo.MongoClient')
def setUpClass(self, mock_mongo):
a = Mock()
a.db_name.side_effect = self.dbResponse
mock_mongo.return_value = a
dbResponse = FakeDatabase()
So,
consider I have a simple library that I am trying to write unit-tests for. This library talks to a database and then uses that data to call an SOAP API. I have three modules, and a testfile for each module.
dir structure:
./mypkg
../__init__.py
../main.py
../db.py
../api.py
./tests
../test_main
../test_db
../test_api
Code:
#db.py
import mysqlclient
class Db(object):
def __init__(self):
self._client = mysqlclient.Client()
#property
def data(self):
return self._client.some_query()
#api.py
import soapclient
class Api(object):
def __init__(self):
self._client = soapclient.Client()
#property
def call(self):
return self._client.some_external_call()
#main.py
from db import Db
from api import Api
class MyLib(object):
def __init__(self):
self.db = Db()
self.api = Api()
def caller(self):
return self.api.call(self.db.data)
Unit-Tests:
#test_db.py
import mock
from mypkg.db import Db
#mock.patch('mypkg.db.mysqlclient')
def test_db(mysqlclient_mock):
mysqlclient_mock.Client.return_value.some_query = {'data':'data'}
db = Db()
assert db.data == {'data':'data'}
#test_api.py
import mock
from mypkg.api import Api
#mock.patch('mypkg.db.soapclient')
def test_db(soap_mock):
soap_mock.Client.return_value.some_external_call = 'foo'
api = Api()
assert api.call == 'foo'
In the above example, mypkg.main.MyLib calls mypkg.db.Db() (uses third-party mysqlclient) and then mypkg.api.Api() (uses third-party soapclient)
I am using mock.patch to patch the third-party libraries to mock my db and api calls in test_db and test_api separately.
Now my question is, is it recommended to patch these external calls again in test_main OR simply patch db.Db and api.Api? (this example is pretty simple, but in larger libraries, the code becomes cumbersome when patching the external calls again or even using test helper functions that patch internal libraries).
Option1: patch external libraries in main again
#test_main.py
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.db.mysqlclient')
#mock.patch('mypkg.api.soapclient')
def test_main(soap_mock, mysqlcient_mock):
ml = MyLib()
soap_mock.Client.return_value.some_external_call = 'foo'
assert ml.caller() == 'foo'
Option2: patch internal libraries
#test_main.py
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.db.Db')
#mock.patch('mypkg.api.Api')
def test_main(api_mock, db_mock):
ml = MyLib()
api_mock.return_value = 'foo'
assert ml.caller() == 'foo'
mock.patch creates a mock version of something where it's imported, not where it lives. This means the string passed to mock.patch has to be a path to an imported module in the module under test. Here's what the patch decorators should look like in test_main.py:
#mock.patch('mypkg.main.Db')
#mock.patch('mypkg.main.Api')
Also, the handles you have on your patched modules (api_mock and db_mock) refer to the classes, not instances of those classes. When you write api_mock.return_value = 'foo', you're telling api_mock to return 'foo' when it gets called, not when an instance of it has a method called on it. Here are the objects in main.py and how they relate to api_mock and db_mock in your test:
Api is a class : api_mock
Api() is an instance : api_mock.return_value
Api().call is an instance method : api_mock.return_value.call
Api().call() is a return value : api_mock.return_value.call.return_value
Db is a class : db_mock
Db() is an instance : db_mock.return_value
Db().data is an attribute : db_mock.return_value.data
test_main.py should therefore look like this:
import mock
from mypkg.main import MyLib
#mock.patch('mypkg.main.Db')
#mock.patch('mypkg.main.Api')
def test_main(api_mock, db_mock):
ml = MyLib()
api_mock.return_value.call.return_value = 'foo'
db_mock.return_value.data = 'some data' # we need this to test that the call to api_mock had the correct arguments.
assert ml.caller() == 'foo'
api_mock.return_value.call.assert_called_once_with('some data')
The first patch in Option 1 would work great for unit-testing db.py, because it gives the db module a mock version of mysqlclient. Similarly, #mock.patch('mypkg.api.soapclient') belongs in test_api.py.
I can't think of a way Option 2 could help you unit-test anything.
Edited: I was incorrectly referring to classes as modules. db.py and api.py are modules