Develop Reference

Tests that verify the calling of remove function of the Python os module fail

I am trying to mock Python os module but my mocking steps are not working.
The code in the file os_mock.py:
import os
class MyTestMock:
def rm(self):
# some reason file is always hardcoded
file_path = "/tmp/file1"
if os.path.exists(file_path):
os.remove(file_path)
print(file_path, 'removed successfully')
else:
print(file_path, 'Does not exist')
The code in the test case file test_os_mock.py
import os
import unittest
from unittest.mock import patch
from os_mock import MyTestMock
class TestMyTestMock(unittest.TestCase):
#patch('os.path')
#patch('os.remove')
def test_rm(self, mock_remove, mock_path):
my_test_mock = MyTestMock()
mock_path.exists.return_vallue = False
my_test_mock.rm()
self.assertFalse(mock_remove.called)
mock_path.exists.return_vallue = True
my_test_mock.rm()
self.assertTrue(mock_remove.called)
I am getting below error when I execute test cases
F
======================================================================
FAIL: test_rm (__main__.TestMyTestMock)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/mock.py", line 1336, in patched
return func(*newargs, **newkeywargs)
File "/Users/vuser/code/MP-KT/mock/test_os_mock.py", line 15, in test_rm
self.assertFalse(mock_remove.called)
AssertionError: True is not false
----------------------------------------------------------------------
Ran 1 test in 0.008s
FAILED (failures=1)
I know I am doing something wrong while mocking, But I could not able to figure it out, I got couple of stack overflow links, I followed it, but no help.

I have made some change only to your test file, while the file os_mock.py remains unchanged.
Change return_vallue to return_value is enough
If you change return_vallue to return_value your test passes successfully so these changes (in the 2 points where the errors are present) are sufficient.
In particular the changes are the followings:
mock_path.exists.return_vallue=False --> mock_path.exists.return_value=False (return_vallue is not correct)
mock_path.exists.return_vallue=True --> mock_path.exists.return_value=True (return_vallue is not correct)
Improving tests by assert_not_called() and assert_called_once()
The most important change is return_vallue --> return_value, but in my opinion the package unittest.mock provides methods assert_not_called() and assert_called_once() which can improve your tests.
For example self.assertTrue(mock_remove.called) ensures you called the mocked method, instead mock_remove.assert_called_once() checks that you called the method exactly one time.
So I recommend the followings changes:
self.assertFalse(mock_remove.called) --> mock_remove.assert_not_called()
self.assertTrue(mock_remove.called) --> mock_remove.assert_called_once()
The new file test_os_mock.py
With the changes showed, the file test_os_mock.py becomes:
import os
import unittest
from unittest.mock import patch
from os_mock import MyTestMock
class TestMyTestMock(unittest.TestCase):
#patch('os.path')
#patch('os.remove')
def test_rm(self, mock_remove, mock_path):
my_test_mock = MyTestMock()
# return_vallue --> return_value
mock_path.exists.return_value = False
my_test_mock.rm()
mock_remove.assert_not_called()
#self.assertFalse(mock_remove.called)
# return_vallue --> return_value
mock_path.exists.return_value = True
my_test_mock.rm()
mock_remove.assert_called_once()
#self.assertTrue(mock_remove.called)
if __name__ == "__main__":
unittest.main()
assert_called_once_with()
In your test case better than assert_called_once() is the method assert_called_once_with() and with this other method your test becomes as follow:
#patch('os.path')
#patch('os.remove')
def test_rm(self, mock_remove, mock_path):
my_test_mock = MyTestMock()
mock_path.exists.return_value = False
my_test_mock.rm()
mock_remove.assert_not_called()
mock_path.exists.return_value = True
my_test_mock.rm()
# here is the NEW CHANGE
mock_remove.assert_called_once_with("/tmp/file1")
This link is very useful for understand Mocking object in Python.

How can I provide a non-fixture pytest parameter via a custom decorator?

We have unit tests running via Pytest, which use a custom decorator to start up a context-managed mock echo server before each test, and provide its address to the test as an extra parameter. This works on Python 2.
However, if we try to run them on Python 3, then Pytest complains that it can't find a fixture matching the name of the extra parameter, and the tests fail.
Our tests look similar to this:
#with_mock_url('?status=404&content=test&content-type=csv')
def test_file_not_found(self, url):
res_id = self._test_resource(url)['id']
result = update_resource(None, res_id)
assert not result, result
self.assert_archival_error('Server reported status error: 404 Not Found', res_id)
With a decorator function like this:
from functools import wraps
def with_mock_url(url=''):
"""
Start a MockEchoTestServer and call the decorated function with the server's address prepended to ``url``.
"""
def decorator(func):
#wraps(func)
def decorated(*args, **kwargs):
with MockEchoTestServer().serve() as serveraddr:
return func(*(args + ('%s/%s' % (serveraddr, url),)), **kwargs)
return decorated
return decorator
On Python 2 this works; the mock server starts, the test gets a URL similar to "http://localhost:1234/?status=404&content=test&content-type=csv", and then the mock is shut down afterward.
On Python 3, however, we get an error, "fixture 'url' not found".
Is there perhaps a way to tell Python, "This parameter is supplied from elsewhere and doesn't need a fixture"? Or is there, perhaps, an easy way to turn this into a fixture?

You can use url as args parameter
#with_mock_url('?status=404&content=test&content-type=csv')
def test_file_not_found(self, *url):
url[0] # the test url

Looks like Pytest is content to ignore it if I add a default value for the injected parameter, to make it non-mandatory:
#with_mock_url('?status=404&content=test&content-type=csv')
def test_file_not_found(self, url=None):
The decorator can then inject the value as intended.

consider separating the address from the service of the url. Using marks and changing fixture behavior based on the presence of said marks is clear enough. Mock should not really involve any communication, but if you must start some service, then make it separate from
with_mock_url = pytest.mark.mock_url('http://www.darknet.go')
#pytest.fixture
def url(request):
marker = request.get_closest_marker('mock_url')
if marker:
earl = marker.args[0] if args else marker.kwargs['fake']
if earl:
return earl
try:
#
earl = request.param
except AttributeError:
earl = None
return earl
#fixture
def server(request):
marker = request.get_closest_marker('mock_url')
if marker:
# start fake_server
#with_mock_url
def test_resolve(url, server):
server.request(url)

Twisted deferToThread, not working with Mock.patch()

I have 2 functions both wrapped with #defer.inlineCallbacks decorator.
For tests, I'm mocking various things, including the save_to_db() function.
logic.py
#defer.inlineCallbacks
def save_to_db(obj): # mocked for test.
raise Exception('Oh Noe!')
#defer.inlineCallbacks
def create_contact():
xero = yield get_xero_client()
data = {
# <contact info>
}
# Create a new contact
response = yield deferToThread(xero.contacts.put, data)
obj = {
# some data extracted from response
}
yield save_to_db(obj)
tests.py
import mock
from twisted.internet import defer
from twisted.trial import unittest
from .logic import create_contact
class TestContactCreation(unittest.TestCase):
#mock.patch('logic.save_to_db')
#mock.patch('logic.get_xero_client')
#defer.inlineCallbacks
def test_get_xero_client_is_called(self, mocked_xero_client, mocked_save_method):
yield create_contact()
mocked_get_xero_client.assert_called()
However when I run:
$ trial tests.TestContactCreation
save_to_db() is actually called and as expected its raises an Exception.
Traceback (most recent call last):
File "<file_path>/logic.py", line 93, in save_to_db
raise Exception('Oh Noe!')
exceptions.Exception: Oh Noe!
And I'm not sure why! I tried to debug using pdb.
import pdb; pdb.set_trace()
It looks like save_to_db() is mocked correctly before we use deferToThread()
(Pdb) save_to_db
<MagicMock name='save_to_db' id='4404276240'>
However after the line where I've used deferToThread()
(Pdb) save_to_db
<function save_to_db at 0x111c6f488>
save_to_db() is no longer mocked! Only way I can get around this is if I also mock deferToThread()
Is there a better option? Any tips will be appreciated. Many Thanks.

I encountered the same issue; the #mock.patch(...) decorator doesn't work (i.e., it doesn't actually mock the desired thing) when used in conjunction with the #inlineCallbacks decorator.
What did work for me was to mock via context manager:
#defer.inlineCallbacks
def test_get_xero_client_is_called(self):
with mock.patch('logic.save_to_db') as mocked_save_method, \
mock.patch('logic.get_xero_client') as mocked_xero_client:
yield create_contact()
mocked_get_xero_client.assert_called()
Does that work for you?

Python – How to mock a single function

From the Mock docs, I wasn't able to understand how to implement the following type of pattern successfully. fetch_url does not exist inside of a class.
My function in the auth.py file:
def fetch_url(url, method=urlfetch.GET, data=''):
"""Send a HTTP request"""
result = urlfetch.fetch(url=url, method=method, payload=data,
headers={'Access-Control-Allow-Origin': '*'})
return result.content
My test:
import unittest
from mock import Mock
class TestUrlFetch(unittest.TestCase):
def test_fetch_url(self):
from console.auth import fetch_url
# Create a mock object based on the fetch_url function
mock = Mock(spec=fetch_url)
# Mock the fetch_url function
content = mock.fetch_url('https://google.com')
# Test that content is not empty
self.assertIsNotNone(content)
If what I'm doing is completely in the wrong direction, please shed some light on the correct solution.
The test is not working, and is producing the following error:
======================================================================
ERROR: test_fetch_url (console.tests.test_auth.TestUrlFetch)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/bengrunfeld/Desktop/Work/code/wf-ghconsole/console/tests/test_auth.py", line 34, in test_fetch_url
content = mock.fetch_url('https://google.com')
File "/Users/bengrunfeld/.virtualenvs/env2/lib/python2.7/site-packages/mock.py", line 658, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'fetch_url'
-------------------- >> begin captured logging << --------------------
root: DEBUG: Using threading.local
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 1 test in 0.277s
FAILED (errors=1)

First of all, as univerio's comment suggests you should call you mock like this:
mock('https://google.com')
Your test should pass after that fix, but probably that mock doesn't do what you really want. I've encountered a few problems with spec and autospec.
Mocks created with Mock(spec=) don't check number of arguments they are called with. I've just looked through the docs and they don't state that, but for some reason I expected it to work. Autospecced mocks do check the arguments.
By default both spec and autospec function mocks return mock objects when you call them. This may be not what you want when you mock a function that does not return anything. In this case you can set the return_value manually:
def foo():
pass
mock_foo = Mock(spec=foo, return_value=None)
mock_foo()

Mocking file objects or iterables in python

Which way is proper for mocking and testing code that iters object returned by open(), using mock library?
whitelist_data.py:
WHITELIST_FILE = "testdata.txt"
format_str = lambda s: s.rstrip().lstrip('www.')
whitelist = None
with open(WHITELIST_FILE) as whitelist_data:
whitelist = set(format_str(line) for line in whitelist_data)
if not whitelist:
raise RuntimeError("Can't read data from %s file" % WHITELIST_FILE)
def is_whitelisted(substr):
return 1 if format_str(substr) in whitelist else 0
Here's how I try to test it.
import unittest
import mock
TEST_DATA = """
domain1.com
domain2.com
domain3.com
"""
class TestCheckerFunctions(unittest.TestCase):
def test_is_whitelisted_method(self):
open_mock = mock.MagicMock()
with mock.patch('__builtin__.open',open_mock):
manager = open_mock.return_value.__enter__.return_value
manager.__iter__ = lambda s: iter(TEST_DATA.splitlines())
from whitelist_data import is_whitelisted
self.assertTrue(is_whitelisted('domain1.com'))
if __name__ == '__main__':
unittest.main()
Result of python tests.py is:
$ python tests.py
E
======================================================================
ERROR: test_is_whitelisted_method (__main__.TestCheckerFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 39, in test_is_whitelisted_method
from whitelist_data import is_whitelisted
File "/Users/supa/Devel/python/whitelist/whitelist_data.py", line 20, in <module>
whitelist = set(format_str(line) for line in whitelist_data)
TypeError: 'Mock' object is not iterable
----------------------------------------------------------------------
Ran 1 test in 0.001s
UPD: Thanks to Adam, I've reinstalled mock library(pip install -e hg+https://code.google.com/p/mock#egg=mock) and updated tests.py. Works like a charm.

You're looking for a MagicMock. This supports iteration.
In mock 0.80beta4, patch returns a MagicMock. So this simple example works:
import mock
def foo():
for line in open('myfile'):
print line
#mock.patch('__builtin__.open')
def test_foo(open_mock):
foo()
assert open_mock.called
If you're running mock 0.7.x (It looks like you are), I don't think you can accomplish this with patch alone. You'll need to create the mock separately, then pass it into patch:
import mock
def foo():
for line in open('myfile'):
print line
def test_foo():
open_mock = mock.MagicMock()
with mock.patch('__builtin__.open', open_mock):
foo()
assert open_mock.called
Note - I've run these with py.test, however, these same approaches will work with unittest as well.