I am trying to write a unit test that will test azure.storage.blob.BlobServiceClient class and its methods. Below is my code
A fixture in the conftest.py
#pytest.fixture
def mock_BlobServiceClient(mocker):
azure_ContainerClient = mocker.patch("azure.storage.blob.ContainerClient", mocker.MagicMock())
azure_BlobServiceClient= mocker.patch("azure_module.BlobServiceClient", mocker.MagicMock())
azure_BlobServiceClient.from_connection_string.return_value
azure_BlobServiceClient.get_container_client.return_value = azure_ContainerClient
azure_ContainerClient.list_blob_names.return_value = "test"
azure_ContainerClient.get_container_client.list_blobs.return_value = ["test"]
yield azure_BlobServiceClient
Contents of the test file
from azure_module import AzureBlob
def test_AzureBlob(mock_BlobServiceClient):
azure_blob = AzureBlob()
# This assertion passes
mock_BlobServiceClient.from_connection_string.assert_called_once_with("testconnectionstring")
# This assertion fails
mock_BlobServiceClient.get_container_client.assert_called()
Contents of the azure_module.py
from azure.storage.blob import BlobServiceClient
import os
class AzureBlob:
def __init__(self) -> None:
"""Initialize the azure blob"""
self.azure_blob_obj = BlobServiceClient.from_connection_string(os.environ["AZURE_STORAGE_CONNECTION_STRING"])
self.azure_container = self.azure_blob_obj.get_container_client(os.environ["AZURE_CONTAINER_NAME"])
My test fails when I execute it with below error message
> mock_BlobServiceClient.get_container_client.assert_called()
E AssertionError: Expected 'get_container_client' to have been called.
I am not sure why it says that the get_container_client wasn't called when it was called during the AzureBlob's initialization.
Any help is very much appreciated.
Update 1
I believe this is a bug in the unittest's MagicMock itself. Per
Michael Delgado suggested that I dialed the code to a bare minimum to test and identify the issue, and I concluded that the MagicMock was causing the problem. Below are my findings:
conftest.py
#pytest.fixture
def mock_Blob(mocker):
yield mocker.patch("module.BlobServiceClient")
test_azureblob.py
def test_AzureBlob(mock_Blob):
azure_blob = AzureBlob()
print(mock_Blob)
print(mock_Blob.mock_calls)
print(mock_Blob.from_connection_string.mock_calls)
print(mock_Blob.from_connection_string.get_container_client.mock_calls)
assert False # <- Intentional fail
After running the test, I got the following results.
$ pytest -vv
.
.
.
------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------
<MagicMock name='BlobServiceClient' id='140704187870944'>
[call.from_connection_string('AZURE_STORAGE_CONNECTION_STRING'),
call.from_connection_string().get_container_client('AZURE_CONTAINER_NAME')]
[call('AZURE_STORAGE_CONNECTION_STRING'),
call().get_container_client('AZURE_CONTAINER_NAME')]
[]
.
.
.
The prints clearly show that the get_container_client was seen being called, but the mocked method did not register it at its level. That led me to conclude that the MagicMock has a bug which I will report to the developers for further investigation.
Related
learning python mocks here. I need some helps to understand how the patch work when mocking a class.
In the code below, I mocked a class. the function under tests receives the mock and calls a function on it. In my assertions, the class is successfully called, but the function is reported as not being called.
I added a debug print to view the content in the function under tests and it is reported as called.
My expectation is the assertion assert facadeMock.install.called should be true. Why is it not reported as called and how do I achieve this?
Thank you.
install/__init__.py
from .facade import Facade
def main():
f = Facade()
f.install()
print('jf-debug-> "f.install.called": {value}'.format(
value=f.install.called))
test/install_tests.py
import os
import sys
# allow import of package
sys.path.insert(0,
os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from unittest.mock import patch
import install
#patch('install.Facade') # using autospec=True did not change the result
def test_main_with_links_should_call_facade_install_with_link_true(facadeMock):
install.main()
assert facadeMock.called
assert facadeMock.install is install.Facade.install
assert facadeMock.install.called # <-------------------- Fails here!
output:
============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/jfl/ubuntu-vim, configfile: pytest.ini
collected 1 item
test/install_tests.py jf-debug-> "f.install.called": True
F
=================================== FAILURES ===================================
________ test_main_with_links_should_call_facade_install_with_link_true ________
facadeMock = <MagicMock name='Facade' id='140679041900864'>
#patch('install.Facade')
def test_main_with_links_should_call_facade_install_with_link_true(facadeMock):
install.main()
assert facadeMock.called
assert facadeMock.install is install.Facade.install
> assert facadeMock.install.called
E AssertionError: assert False
E + where False = <MagicMock name='Facade.install' id='140679042325216'>.called
E + where <MagicMock name='Facade.install' id='140679042325216'> = <MagicMock name='Facade' id='140679041900864'>.install
test/install_tests.py:21: AssertionError
=========================== short test summary info ============================
FAILED test/install_tests.py::test_main_with_links_should_call_facade_install_with_link_true - AssertionError: assert False
============================== 1 failed in 0.09s ===============================
[edit]
Thank you to #chepner and #Daniil Fajnberg for their comments. I found the cause of the problem.
The problem can be reduced at:
install/__init__.py receives an instance of Facade when calling Facade() in main().
This instance is not the same as the one received in parameters of the test. They are different instances.
to retrieve the instance received in main(), do:
actualInstance = facadeMock.return_value
assert actualInstance.install.called
And it works!
Thank you. That really helps me understand the working of mocks in python.
[/edit]
I have found a method to solve your problem; it is empirical but it works.
To pass your test I have modified it as you can see below:
#patch('install.Facade') # using autospec=True did not change the result
def test_main_with_links_should_call_facade_install_with_link_true(facadeMock):
install.main()
assert facadeMock.called
assert facadeMock.install is install.Facade.install
#assert facadeMock.install.called # <-------------------- Fails here!
install_called = False
for call_elem in facadeMock.mock_calls:
if call_elem[0] == "().install":
install_called = True
break
assert install_called == True
Mock objects facadeMock and f are distinct
facadeMock is a mock object created in the test code and it is used by the production code during your test to create the mock object f by the instruction:
f = Facade()
In the production code f is a mock object (that is an instance of the class Mock) because it is created by the Mock object Facade that is exactly facadeMock.
But f and facadeMock are 2 different instances of the class Mock.
Below I show the id values of facadeMock, Facade and f:
facadeMock = <MagicMock name='Facade' id='140449467990536'>
Facade = <MagicMock name='Facade' id='140449467990536'>
f = <MagicMock name='Facade()' id='140449465274608'>
The id for facadeMock, Facade but are different from the id of f.
The attribute mock_calls
When your test code is executed the function install.main() execution causes the definition of the attribute mock_calls for the mock object facadeMock.
This attribute is a list of complex elements.
If you check the first field (I mean the field in position 0) of each one of this element you can find the name of the methods of the mock that are called.
In your case you have to found install and to do this you have to look for ().install.
So my test checks all the element of mock_calls and only if ().install is found set the variable install_called=True.
I hope that this answer can help you.
Context
I have started using pytest-vcr which is a pytest plugin wrapping VCR.py which I have documented in this blog post on Advanced Python Testing.
It records all HTTP traffic to cassettes/*.yml files on the first test run to save snapshots. Similar to Jest snapshot testing for web components.
On subsequent test runs, if a request is malformed, it won't find a match and throws an exception saying that recording new requests is forbidden and it did not find an existing recording.
Question
VCR.py raises a CannotOverwriteExistingCassetteException which is not particularly informative as to why it didn't match.
How do I leverage pytest pytest_exception_interact hooks to replace this exception with a more informative one leveraging fixture information?
I dove into my site-packages where VCR.py is pip installed and rewrote how I want it to handle the exception. I just need to know how to get this pytest_exception_interact hook to work correctly to access the fixtures from that test node (before it gets cleaned up) and raise a different exception.
Example
Lets get the dependencies.
$ pip install pytest pytest-vcr requests
test_example.py:
import pytest
import requests
#pytest.mark.vcr
def test_example():
r = requests.get("https://www.stackoverflow.com")
assert r.status_code == 200
$ pytest test_example.py --vcr-record=once
...
test_example.py::test_example PASSED
...
$ ls cassettes/
cassettes/test_example.yml
$ head cassettes/test_example.yml
interactions:
- request:
uri: https://wwwstackoverflow.com
body: null
headers:
Accept:
- '*/*'
$ pytest test_example.py --vcr-record=none
...
test_example.py::test_example PASSED
...
Now change the URI in the test to "https://www.google.com":
test_example.py:
import pytest
import requests
#pytest.mark.vcr
def test_example():
r = requests.get("https://www.google.com")
assert r.status_code == 200
And run the test again to detect the regression:
$ pytest test_example.py --vcr-record=none
E vcr.errors.CannotOverwriteExistingCassetteException: No match for the request (<Request (GET) https://www.google.com/>)
...
I can add a conftest.py file to the root of my test structure to create a local plugin, and I can verify that I can intercept the exception and inject my own using:
conftest.py
import pytest
from vcr.errors import CannotOverwriteExistingCassetteException
from vcr.config import VCR
from vcr.cassette import Cassette
class RequestNotFoundCassetteException(CannotOverwriteExistingCassetteException):
...
#pytest.fixture(autouse=True)
def _vcr_marker(request):
marker = request.node.get_closest_marker("vcr")
if marker:
cassette = request.getfixturevalue("vcr_cassette")
vcr = request.getfixturevalue("vcr")
request.node.__vcr_fixtures = dict(vcr_cassette=cassette, vcr=vcr)
yield
#pytest.hookimpl(hookwrapper=True)
def pytest_exception_interact(node, call, report):
excinfo = call.excinfo
if report.when == "call" and isinstance(excinfo.value, CannotOverwriteExistingCassetteException):
# Safely check for fixture pass through on this node
cassette = None
vcr = None
if hasattr(node, "__vcr_fixtures"):
for fixture_name, fx in node.__vcr_fixtures.items():
vcr = fx if isinstance(fx, VCR)
cassette = fx if isinstance(fx, Cassette)
# If we have the extra fixture context available...
if cassette and vcr:
match_properties = [f.__name__ for f in cassette._match_on]
cassette_reqs = cassette.requests
# filtered_req = cassette.filter_request(vcr._vcr_request)
# this_req, req_str = __format_near_match(filtered_req, cassette_reqs, match_properties)
# Raise and catch a new excpetion FROM existing one to keep the traceback
# https://stackoverflow.com/a/24752607/622276
# https://docs.python.org/3/library/exceptions.html#built-in-exceptions
try:
raise RequestNotFoundCassetteException(
f"\nMatching Properties: {match_properties}\n" f"Cassette Requests: {cassette_reqs}\n"
) from excinfo.value
except RequestNotFoundCassetteException as e:
excinfo._excinfo = (type(e), e)
report.longrepr = node.repr_failure(excinfo)
This is the part where the documentation on the internet gets pretty thin.
How do I access the vcr_cassette fixture and return a different exception?
What I want to do is get the filtered_request that was attempting to be requested and the list of cassette_requests and using the Python difflib standard library produce deltas against the information that diverged.
PyTest Code Spelunking
The internals of running a single test with pytest triggers pytest_runtest_protocol which effectively runs the following three call_and_report calls to get a collection of reports.
src/_pytest/runner.py:L77-L94
def runtestprotocol(item, log=True, nextitem=None):
# Abbreviated
reports = []
reports.append(call_and_report(item, "setup", log))
reports.append(call_and_report(item, "call", log))
reports.append(call_and_report(item, "teardown", log))
return reports
So I'm after modifying the report at the call stage... but still no clue how I get access to the fixture information.
src/_pytest/runner.py:L166-L174
def call_and_report(item, when, log=True, **kwds):
call = call_runtest_hook(item, when, **kwds)
hook = item.ihook
report = hook.pytest_runtest_makereport(item=item, call=call)
if log:
hook.pytest_runtest_logreport(report=report)
if check_interactive_exception(call, report):
hook.pytest_exception_interact(node=item, call=call, report=report)
return report
It looks like there are some helper methods for generating a new ExceptionRepresentation so I updated the conftest.py example.
src/_pytest/reports.py:L361
longrepr = item.repr_failure(excinfo)
UPDATE #1 2019-06-26: Thanks to some pointers from #hoefling in the comments I updated my conftest.py.
Correctly re-raising the exception using the raise ... from ... form.
Override the _vcr_marker to attach the vcr and vcr_cassette fixtures to the request.node which represent that individual test item.
Remaining: Get access to the intercepted request from the patched VCRConnection...
UPDATE #2 2019-06-26
It would seem impossible to get at the VCRHTTPConnections that were patched in creating the cassette context manager. I have opened up the following pull request to pass as arguments when the exception is thrown, to then catch and handle arbitrarily down stream.
https://github.com/kevin1024/vcrpy/pull/445
Related
Related questions that are informative but still don't answer this question.
Customizing error message for specific exceptions in pytest
Thanks to comments and guidance in the comments from #hoefling.
I could attach the cassette fixture to the request.node in a conftest.py local plugin overriding the pytest-vcr marker...
#pytest.fixture(autouse=True)
def _vcr_marker(request):
marker = request.node.get_closest_marker("vcr")
if marker:
cassette = request.getfixturevalue("vcr_cassette")
vcr = request.getfixturevalue("vcr")
request.node.__vcr_fixtures = dict(vcr_cassette=cassette, vcr=vcr)
yield
But I needed more than the cassette to get to my solution.
Ingredients
Use the pytest_exception_interact hook
Pull request to VCR.py https://github.com/kevin1024/vcrpy/pull/446
PR #439 and PR #441 by arthurHamon2 has been huge to fix the test suite and also integrate matcher differencing outputs.
BONUS: Use the raise ... from ... form of throwing an exception
Recipe
Get latest VCRpy
These patches were released in vcrpy v2.1.0
pip install vcrpy==2.1.0
Override the pytest_exception_interact hook
In the root of your test directory create a conftest.py to create a local plugin that overrides the pytest_exception_interact hook.
#pytest.hookimpl(hookwrapper=True)
def pytest_exception_interact(node, call, report):
"""Intercept specific exceptions from tests."""
if report.when == "call" and isinstance(call.excinfo.value, CannotOverwriteExistingCassetteException):
__handle_cassette_exception(node, call, report)
yield
Extract the Cassette and the Request from the exception.
# Define new exception to throw
class RequestNotFoundCassetteException(Exception):
...
def __handle_cassette_exception(node, call, report):
# Safely check for attributes attached to exception
vcr_request = None
cassette = None
if hasattr(call.excinfo.value, "cassette"):
cassette = call.excinfo.value.cassette
if hasattr(call.excinfo.value, "failed_request"):
vcr_request = call.excinfo.value.failed_request
# If we have the extra context available...
if cassette and vcr_request:
match_properties = [f.__name__ for f in cassette._match_on]
this_req, req_str = __format_near_match(cassette.requests, vcr_request, match_properties)
try:
raise RequestNotFoundCassetteException(f"{this_req}\n\n{req_str}\n") from call.excinfo.value
except RequestNotFoundCassetteException as e:
call.excinfo._excinfo = (type(e), e)
report.longrepr = node.repr_failure(call.excinfo)
I've gone through some of the previous questions on here about mocking sqlite3 in python when doing unit tests, and unfortunately none of them have helped me be able to successfully mock the result from fetchone().
The following is a quick test example I put together to try and get it working:
TETS.PY
import unittest
import sqlite3
from unittest import TestCase, mock
from unittest.mock import patch, MagicMock
class Foo:
def checkActive(self):
conn = sqlite3.connect('lll.db')
cur = conn.execute("SELECT * FROM SQLITE_MASTER")
value = cur.fetchone()
return value
class test_Foo(TestCase):
#patch('tets.sqlite3')
def test_shortTest(self, mock_sql):
mock_sql.connect().cursor().fetchall.return_value = ('Test',)
test_class = Foo()
return_mock = test_class.checkActive()
print(return_mock)
if __name__ == '__main__': # pragma: no cover -> local unittest main call
unittest.main()
I've tried variations of the above, as well as patching tets.sqlite3.connect and going from there but I always either of the below as result:
[Running] python -u "c:\Users\z003uwfm\Desktop\tets.py"
<MagicMock name='connect().execute().fetchone()' id='45622576'>
.
----------------------------------------------------------------------
Ran 1 test in 0.016s
OK
[Running] python -u "c:\Users\z003uwfm\Desktop\tets.py"
None
.
----------------------------------------------------------------------
Ran 1 test in 0.021s
OK
Does anyone have a real working example where they were able to mock the return from either fetchone() or fetchall()?
Thanks!
After further tinkering I found the following to be working:
#patch('sqlite3.connect')
def test_shortTest(self, mock_sql):
mock_sql.return_value.execute.return_value.fetchone.return_value = ('Test',)
test_class = Foo()
return_mock = test_class.checkActive()
print(return_mock)
All the other code remains the same as per the original post. Hope this helps others out if they ever have to come across this!
I don't know why I'm just not getting this, but I want to use mock in Python to test that my functions are calling functions in ftplib.FTP correctly. I've simplified everything down and still am not wrapping my head around how it works. Here is a simple example:
import unittest
import ftplib
from unittest.mock import patch
def download_file(hostname, file_path, file_name):
ftp = ftplib.FTP(hostname)
ftp.login()
ftp.cwd(file_path)
class TestDownloader(unittest.TestCase):
#patch('ftplib.FTP')
def test_download_file(self, mock_ftp):
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftp.cwd.assert_called_with('pub/files')
When I run this, I get:
AssertionError: Expected call: cwd('pub/files')
Not called
I know it must be using the mock object since that is a fake server name, and when run without patching, it throws a "socket.gaierror" exception.
How do I get the actual object the fuction is running? The long term goal is not having the "download_file" function in the same file, but calling it from a separate module file.
When you do patch(ftplib.FTP) you are patching FTP constructor. dowload_file() use it to build ftp object so your ftp object on which you call login() and cmd() will be mock_ftp.return_value instead of mock_ftp.
Your test code should be follow:
class TestDownloader(unittest.TestCase):
#patch('ftplib.FTP', autospec=True)
def test_download_file(self, mock_ftp_constructor):
mock_ftp = mock_ftp_constructor.return_value
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftp_constructor.assert_called_with('ftp.server.local')
self.assertTrue(mock_ftp.login.called)
mock_ftp.cwd.assert_called_with('pub/files')
I added all checks and autospec=True just because is a good practice
Like Ibrohim's answer, I prefer pytest with mocker.
I have went a bit further and have actually wrote a library which helps me to mock easily. Here is how to use it for your case.
You start by having your code and a basic pytest function, with the addition of my helper library to generate mocks to modules and the matching asserts generation:
import ftplib
from mock_autogen.pytest_mocker import PytestMocker
def download_file(hostname, file_path, file_name):
ftp = ftplib.FTP(hostname)
ftp.login()
ftp.cwd(file_path)
def test_download_file(mocker):
import sys
print(PytestMocker(mocked=sys.modules[__name__],
name=__name__).mock_modules().prepare_asserts_calls().generate())
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
When you run the test for the first time, it would fail due to unknown DNS, but the print statement which wraps my library would give us this valuable input:
...
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch('test_29817963.ftplib', new=mock_ftplib)
...
import mock_autogen
...
print(mock_autogen.generator.generate_asserts(mock_ftplib, name='mock_ftplib'))
I'm placing this in the test and would run it again:
def test_download_file(mocker):
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch('test_29817963.ftplib', new=mock_ftplib)
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
import mock_autogen
print(mock_autogen.generator.generate_asserts(mock_ftplib, name='mock_ftplib'))
This time the test succeeds and I only need to collect the result of the second print to get the proper asserts:
def test_download_file(mocker):
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch(__name__ + '.ftplib', new=mock_ftplib)
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftplib.FTP.assert_called_once_with('ftp.server.local')
mock_ftplib.FTP.return_value.login.assert_called_once_with()
mock_ftplib.FTP.return_value.cwd.assert_called_once_with('pub/files')
If you would like to keep using unittest while using my library, I'm accepting pull requests.
I suggest using pytest and pytest-mock.
from pytest_mock import mocker
def test_download_file(mocker):
ftp_constructor_mock = mocker.patch('ftplib.FTP')
ftp_mock = ftp_constructor_mock.return_value
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
ftp_constructor_mock.assert_called_with('ftp.server.local')
assert ftp_mock.login.called
ftp_mock.cwd.assert_called_with('pub/files')
I wanted to call setUpClass and tearDownClass so that setup and teardown would be performed only once for each test. However, it keeps failing for me when I call tearDownClass. I only want to record 1 test result, either PASS if both tests passed or FAIL if both tests failed. If I call only setup and tearDown then all works fine:
Calling setUpClass and tearDownClass:
#!/usr/bin/python
import datetime
import itertools
import logging
import os
import sys
import time
import unittest
LOGFILE = 'logfile.txt'
class MyTest(unittest.TestCase):
global testResult
testResult = None
#classmethod
def setUpClass(self):
## test result for DB Entry:
self.dbresult_dict = {
'SCRIPT' : 'MyTest.py',
'RESULT' : testResult,
}
def test1(self):
expected_number = 10
actual_number = 10
self.assertEqual(expected_number, actual_number)
def test2(self):
expected = True
actual = True
self.assertEqual(expected, actual)
def run(self, result=None):
self.testResult = result
unittest.TestCase.run(self, result)
#classmethod
def tearDownClass(self):
ok = self.testResult.wasSuccessful()
errors = self.testResult.errors
failures = self.testResult.failures
if ok:
self.dbresult_dict['RESULT'] = 'Pass'
else:
logging.info(' %d errors and %d failures',
len(errors), len(failures))
self.dbresult_dict['RESULT'] = 'Fail'
if __name__ == '__main__':
logger = logging.getLogger()
logger.addHandler(logging.FileHandler(LOGFILE, mode='a'))
stderr_file = open(LOGFILE, 'a')
runner = unittest.TextTestRunner(verbosity=2, stream=stderr_file, descriptions=True)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
sys.exit()
unittest.main(module=itersuite, exit=True)
stderr_file.close()
Error:
test1 (__main__.MyTest) ... ok
test2 (__main__.MyTest) ... ok
ERROR
===================================================================
ERROR: tearDownClass (__main__.MyTest)
-------------------------------------------------------------------
Traceback (most recent call last):
File "testTearDownClass.py", line 47, in tearDownClass
ok = self.testResult.wasSuccessful()
AttributeError: type object 'MyTest' has no attribute 'testResult'
----------------------------------------------------------------------
Ran 2 tests in 0.006s
FAILED (errors=1)
like #Marcin already pointed out, you're using the Unittest-Framework in a way it isn't intended.
To see if the tests are successful you check the given values with the expected, like you already did: assertEqual(given, expected). Unittest will then collect a summary of failed ones. you don't have to do this manually.
If you want to check that two tests need to be together successful or fail together, these should be combined in ONE Test, maybe as a additionally one, if the individual Tests need to be checked as well. This is nothing you want to save and load afterwards. The tests itself should be as stateless as possible.
When you say you want to run the SetUp and TearDown 'once per test', do you mean once per test-method or per test-run? This is different if you have more than one test-method inside your class:
setUp() Will be called before each test-method
tearDown() Will be called after each test-method
setUpClass() Will be called once per class (before the first test-method of this class)
tearDownClass() Will be called once per class (after the last test-method of this class)
Here's the official documentation
Here's a related answer
Change tearDownClass(self) to tearDownClass(cls) and setUpClass(self) to setUpClass(cls).