Develop Reference

Mocking Azure BlobServiceClient in Python

I am trying to write a unit test that will test azure.storage.blob.BlobServiceClient class and its methods. Below is my code
A fixture in the conftest.py
#pytest.fixture
def mock_BlobServiceClient(mocker):
azure_ContainerClient = mocker.patch("azure.storage.blob.ContainerClient", mocker.MagicMock())
azure_BlobServiceClient= mocker.patch("azure_module.BlobServiceClient", mocker.MagicMock())
azure_BlobServiceClient.from_connection_string.return_value
azure_BlobServiceClient.get_container_client.return_value = azure_ContainerClient
azure_ContainerClient.list_blob_names.return_value = "test"
azure_ContainerClient.get_container_client.list_blobs.return_value = ["test"]
yield azure_BlobServiceClient
Contents of the test file
from azure_module import AzureBlob
def test_AzureBlob(mock_BlobServiceClient):
azure_blob = AzureBlob()
# This assertion passes
mock_BlobServiceClient.from_connection_string.assert_called_once_with("testconnectionstring")
# This assertion fails
mock_BlobServiceClient.get_container_client.assert_called()
Contents of the azure_module.py
from azure.storage.blob import BlobServiceClient
import os
class AzureBlob:
def __init__(self) -> None:
"""Initialize the azure blob"""
self.azure_blob_obj = BlobServiceClient.from_connection_string(os.environ["AZURE_STORAGE_CONNECTION_STRING"])
self.azure_container = self.azure_blob_obj.get_container_client(os.environ["AZURE_CONTAINER_NAME"])
My test fails when I execute it with below error message
> mock_BlobServiceClient.get_container_client.assert_called()
E AssertionError: Expected 'get_container_client' to have been called.
I am not sure why it says that the get_container_client wasn't called when it was called during the AzureBlob's initialization.
Any help is very much appreciated.
Update 1
I believe this is a bug in the unittest's MagicMock itself. Per
Michael Delgado suggested that I dialed the code to a bare minimum to test and identify the issue, and I concluded that the MagicMock was causing the problem. Below are my findings:
conftest.py
#pytest.fixture
def mock_Blob(mocker):
yield mocker.patch("module.BlobServiceClient")
test_azureblob.py
def test_AzureBlob(mock_Blob):
azure_blob = AzureBlob()
print(mock_Blob)
print(mock_Blob.mock_calls)
print(mock_Blob.from_connection_string.mock_calls)
print(mock_Blob.from_connection_string.get_container_client.mock_calls)
assert False # <- Intentional fail
After running the test, I got the following results.
$ pytest -vv
.
.
.
------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------
<MagicMock name='BlobServiceClient' id='140704187870944'>
[call.from_connection_string('AZURE_STORAGE_CONNECTION_STRING'),
call.from_connection_string().get_container_client('AZURE_CONTAINER_NAME')]
[call('AZURE_STORAGE_CONNECTION_STRING'),
call().get_container_client('AZURE_CONTAINER_NAME')]
[]
.
.
.
The prints clearly show that the get_container_client was seen being called, but the mocked method did not register it at its level. That led me to conclude that the MagicMock has a bug which I will report to the developers for further investigation.

Mocking a function returning an object results in AssertionError: Expected '...' to have been called once. Called 0 times

I'm writing a simple example to help me understand how mocking works in unittest. I have a module with two functions:
# model.animals.py
def get_animals(animal_type):
db = connect_to_db()
result = db.query_all_data()
return list(filter(lambda x: x['animal_type'] == animal_type, result))
def connect_to_db():
pass # That would normally return a DB connection instance
I want to test the get_animals() function which uses a DB connection to retrieve information about all animals and then filters returned data based on animal type. Since I don't want to set up the whole database, I just want to mock the connect_to_db() function which returns a DB connection instance.
This is my test class:
# test_mock.py
from unittest import TestCase, main
from unittest.mock import Mock, patch
from model.animals import get_animals
class GetDataTest(TestCase):
#patch('model.animals.connect_to_db')
def test_get_animals(self, mock_db: Mock):
mock_db.return_value.query_all_data.return_value = [
{
'animal_type': 'meerkat',
'age': 5
},
{
'animal_type': 'meerkat',
'age': 11
},
{
'animal_type': 'cow',
'age': 3
}
]
result = get_animals('meerkat') # Run the function under test
mock_db.assert_called_once() # OK
mock_db.query_all_data.assert_called_once() # AssertionError
self.assertEqual(len(result), 2) # OK
self.assertEqual(result[0]['age'], 5) # OK
if __name__ == "__main__":
main()
As part of the test I wanted to not only check the filtering of animals based on their type but also whether all the methods inside get_animals() are called.
The test generally works as expected but I get an error when checking whether the query_all_data() function has been called:
AssertionError: Expected 'query_all_data' to have been called once. Called 0 times.
When I add spec=True to my patch I get another error:
AttributeError: Mock object has no attribute 'query_all_data'
Clearly, the function query_all_data is not visible inside the mock even though I set its return value in the test with mock_db.return_value.query_all_data.return_value = ....
What am I missing?

The reason that mock_db.query_all_data.assert_called_once() failed is that it should be mock_db.return_value.query_all_data.assert_called_once().
I have created a helper library to help me generate asserts for mocks so that I won't stumble on such issues as often.
To use it do: pip install mock-generator
Then, in your test place these lines after result = get_animals('meerkat'):
from mock_autogen import generate_asserts
generate_asserts(mock_db)
When you run the test, it would generate the asserts for you (printed to the console and copied to the clipboard):
assert 1 == mock_db.call_count
mock_db.assert_called_once_with()
mock_db.return_value.query_all_data.assert_called_once_with()
You can then edit the generated asserts and use whichever fits your test.

Why is my unit test returning False instead of True? [duplicate]

I'm having a fairly difficult time using mock in Python:
def method_under_test():
r = requests.post("http://localhost/post")
print r.ok # prints "<MagicMock name='post().ok' id='11111111'>"
if r.ok:
return StartResult()
else:
raise Exception()
class MethodUnderTestTest(TestCase):
def test_method_under_test(self):
with patch('requests.post') as patched_post:
patched_post.return_value.ok = True
result = method_under_test()
self.assertEqual(type(result), StartResult,
"Failed to return a StartResult.")
The test actually returns the right value, but r.ok is a Mock object, not True. How do you mock attributes in Python's mock library?

You need to use return_value and PropertyMock:
with patch('requests.post') as patched_post:
type(patched_post.return_value).ok = PropertyMock(return_value=True)
This means: when calling requests.post, on the return value of that call, set a PropertyMock for the property ok to return the value True.

A compact and simple way to do it is to use new_callable patch's attribute to force patch to use PropertyMock instead of MagicMock to create the mock object. The other arguments passed to patch will be used to create PropertyMock object.
with patch('requests.post.ok', new_callable=PropertyMock, return_value=True) as mock_post:
"""Your test"""

With mock version '1.0.1' the simpler syntax mentioned in the question is supported and works as is!
Example code updated (py.test is used instead of unittest):
import mock
import requests
def method_under_test():
r = requests.post("http://localhost/post")
print r.ok
if r.ok:
return r.ok
else:
raise Exception()
def test_method_under_test():
with mock.patch('requests.post') as patched_post:
patched_post.return_value.ok = True
result = method_under_test()
assert result is True, "mock ok failed"
Run this code with: (make sure you install pytest)
$ py.test -s -v mock_attributes.py
======= test session starts =======================
platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2 -- /home/developer/miniconda/bin/python
rootdir: /home/developer/projects/learn/scripts/misc, inifile:
plugins: httpbin, cov
collected 1 items
mock_attributes.py::test_method_under_test True
PASSED
======= 1 passed in 0.03 seconds =================

How to mock pytest.fixture decorator?

I wanted to write unit tests for pytest fixtures present in conftest.py
How do I mock decorator pytest.fixture?
Conftest.py
import pytest
#pytest.fixture(scope="session", autouse="True")
def get_ip(dict_obj):
"""Assume some functionality"""
return dict_obj.get('ip')
#pytest.fixture(scope="class")
def get_server(create_obj):
"""Assume some functionality"""
pass
test_conftest.py
mock_fixture = patch('pytest.fixture', lambda x : x).start()
from tests.conftest import get_ip
class TestConftestTests:
def test_mgmt_ip(self):
assert mgmt_ip({"ip": "10.192.174.15"}) == "10.192.174.15"
E TypeError: <lambda>() got an unexpected keyword argument 'scope'
When I tried to mock pytest.fixture at the starting of the test module before importing functions to be tested, I am getting error -
E TypeError: <lambda>() got an unexpected keyword argument 'scope'
If I remove the lambda function, I am getting E AssertionError: assert <MagicMock name='fixture()()()' id='4443565456'> == '10.192.174.15'
test_conftest.py
patch('pytest.fixture').start()
from tests.conftest import get_ip
class TestConftestTests:
def test_mgmt_ip(self):
assert mgmt_ip({"ip": "10.192.174.15"}) == "10.192.174.15"
E AssertionError: assert <MagicMock name='fixture()()()' id='4443565456'> == '10.192.174.15'
Could someone help me to resolve the error ? Thanks!

#pytest.fixture(scope='session') is going to call your lambda x: x with the kwarg scope. A mock covering both the no-args and some-args versions of pytest.fixture might be:
pytest_fixture = lambda x=None, **kw: x if callable(x) else fixture
But it should be noted pytest.fixture doesn't do any registration by itself. It only marks the function as a fixture, by setting the _pytestfixturefunction attribute on it; afterward, pytest collects the fixture.
I'm not sure you're barking up the right tree for whatever you might be trying to accomplish. If you want to change what value a fixture has in certain contexts, classes and subclasses can be used to override parent fixtures. If you're trying to see whether a fixture is used, creating a new class and overriding the fixture with your assertion can be useful.

AttributeError: while using monkeypatch of pytest

src/mainDir/mainFile.py
contents of mainFile.py
import src.tempDir.tempFile as temp
data = 'someData'
def foo(self):
ans = temp.boo(data)
return ans
src/tempDir/tempFile.py
def boo(data):
ans = data
return ans
Now I want to test foo() from src/tests/test_mainFile.py and I want to mock temp.boo(data) method in foo() method
import src.mainDir.mainFile as mainFunc
testData = 'testData'
def test_foo(monkeypatch):
monkeypatch.setattr('src.tempDir.tempFile', 'boo', testData)
ans = mainFunc.foo()
assert ans == testData
but I get error
AttributeError: 'src.tempDir.tempFile' has no attribute 'boo'
I expect ans = testData.
I would like to know if I am correctly mocking my tempDir.boo() method or I should use pytest's mocker instead of monkeypatch.

You're telling monkeypatch to patch the attribute boo of the string object you pass in.
You'll either need to pass in a module like monkeypatch.setattr(tempFile, 'boo', testData), or pass the attribute as a string too (using the two-argument form), like monkeypatch.setattr('src.tempDir.tempFile.boo', testData).

My use case was was slightly different but should still apply. I wanted to patch the value of sys.frozen which is set when running an application bundled by something like Pyinstaller. Otherwise, the attribute does not exist. Looking through the pytest docs, the raising kwarg controls wether or not AttributeError is raised when the attribute does not already exist. (docs)
Usage Example
import sys
def test_frozen_func(monkeypatch):
monkeypatch.setattr(sys, 'frozen', True, raising=False)
# can use ('fq_import_path.sys.frozen', ...)
# if what you are trying to patch is imported in another file
assert sys.frozen

Update: mocking function calls can be done with monkeypatch.setattr('package.main.slow_fun', lambda: False) (see answer and comments in https://stackoverflow.com/a/44666743/3219667) and updated snippet below
I don't think this can be done with pytest's monkeypatch, but you can use the pytest-mock package. Docs: https://github.com/pytest-dev/pytest-mock
Quick example with the two files below:
# package/main.py
def slow_fun():
return True
def main_fun():
if slow_fun():
raise RuntimeError('Slow func returned True')
# tests/test_main.py
from package.main import main_fun
# Make sure to install pytest-mock so that the mocker argument is available
def test_main_fun(mocker):
mocker.patch('package.main.slow_fun', lambda: False)
main_fun()
# UPDATE: Alternative with monkeypatch
def test_main_fun_monkeypatch(monkeypatch):
monkeypatch.setattr('package.main.slow_fun', lambda: False)
main_fun()
Note: this also works if the functions are in different files