Develop Reference

Mocking ReviewBoard third party library using python and mock

I use ReviewBoard API library and today I moved the code to separate class and wanted to cover the logic with some tests. I understand mocks and testing but I am clearly not much experienced with the python and it's libraries. Here's the chunk of the real code:
<!-- language: python -->
from rbtools.api.client import RBClient
class ReviewBoardWrapper():
def __init__(self, url, username, password):
self.url = url
self.username = username
self.password = password
pass
def Connect(self):
self.client = RBClient(self.url, username=self.username, password=self.password)
self.root = self.client.get_root()
pass
And I want to assert the initialization as well as the get_root() methods are called. Here's how I try to accomplish that:
<!-- language: python -->
import unittest
import mock
from module_base import ReviewBoardWrapper as rb
class RbTestCase(unittest.TestCase):
#mock.patch('module_base.RBClient')
#mock.patch('module_base.RBClient.get_root')
def test_client_connect(self, mock_client, mock_method):
rb_client = rb('', '', '')
rb_client.Connect()
self.assertTrue(mock_method.called)
self.assertTrue(mock_client.called)
And here's the error I stuck on:
$ python -m unittest module_base_tests
F.
======================================================================
FAIL: test_client_connect (module_base_tests.RbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "module_base_tests.py", line 21, in test_client_connect
self.assertTrue(mock_client.called)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 2 tests in 0.002s
FAILED (failures=1)
What do I do wrong? Do I correctly mock the "local copy" of imported libraries? Does the issue lie completely in a different area?
I have also tried to do this:
#mock.patch('module_base.RBClient.__init__')
And / or this:
self.assertTrue(mock_client.__init__.called)

In the example from your post, the order of the mocking is reversed:
test_client_connect(self, mock_client, mock_method)
The client is actually being mocked as the second argument and the method call is being mocked as the first argument.
However, to properly mock the client, you want to mock the return value of the client call. An example of mocking the return value and making an assertion on the return value would like the following:
class RbTestCase(unittest.TestCase):
#mock.patch('module_base.RBClient')
def test_client_connect(self, mock_client):
client = mock.MagicMock()
mock_client.return_value = client
rb_client = rb('', '', '')
rb_client.Connect()
self.assertTrue(client.get_root.called)
self.assertTrue(mock_client.called)

Mock is not effective after twisted deferred is returned

I am having trouble with mocking in twisted trial unit test. The function under test returns a deferred, and I return that deferred from the unit test. However, it seems like the patches that I have applied are reverted once I return the deferred, it doesn't wait until the deferred is resolved.
Please see the following example:
schedule.py
import time
from twisted.internet import task, reactor
class Schedule():
def get_time(self):
t = time.time()
return task.deferLater(reactor, 2, lambda: t)
def get_time_future(self):
return task.deferLater(reactor, 2, lambda: time.time())
schedule_test.py
import mock
from twisted.trial import unittest
from schedule import Schedule
class ScheduleTest(unittest.TestCase):
#mock.patch('schedule.time')
def test_get_time(self, mock_time):
mock_time.time.return_value = 1450407660
d = Schedule().get_time()
d.addCallback(self.assertEqual, 1450407660)
return d
#mock.patch('schedule.time')
def test_get_time_future(self, mock_time):
mock_time.time.return_value = 1450407660
d = Schedule().get_time_future()
d.addCallback(self.assertEqual, 1450407660)
return d
And the output
test_schedule
ScheduleTest
test_get_time ... [OK]
test_get_time_future ... [FAIL]
===============================================================================
[FAIL]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/trial/_synctest.py", line 437, in assertEqual
super(_Assertions, self).assertEqual(first, second, msg)
File "/usr/lib/python2.7/unittest/case.py", line 515, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
raise self.failureException(msg)
twisted.trial.unittest.FailTest: 1456370638.190432 != 1450407660
test_schedule.ScheduleTest.test_get_time_future
-------------------------------------------------------------------------------
Ran 2 tests in 4.024s
FAILED (failures=1, successes=1)
Is there something wrong in the way that I am mocking time in the above example?

There is indeed something wrong with the way you are mocking time. The mock decorator sets up the patches on entry to the test function and then removes them on exit. But, you're returning a Deferred, which means that the time.time callback runs well after the test function has exited.
If you just want to continue using a mock-like patching interface, you can use Twisted's TestCase.patch, which I believe will take Deferreds into account.
However, mocking is a testing anti-pattern, and a much better way to deal with the passage of time is to use the documented API for testing the passage of time, twisted.internet.task.Clock.

As Glyph pointed out, using TestCase.patch can help. The patch needs to be performed as shown below:
def test_get_time_future(self):
mock_time = MagicMock()
mock_time.time.return_value = 1450407660
self.patch(sys.modules['schedule'], 'time', mock_time)
d = Schedule().get_time_future()
d.addCallback(self.assertEqual, 1450407660)
return d
This test passes.

Python – How to mock a single function

From the Mock docs, I wasn't able to understand how to implement the following type of pattern successfully. fetch_url does not exist inside of a class.
My function in the auth.py file:
def fetch_url(url, method=urlfetch.GET, data=''):
"""Send a HTTP request"""
result = urlfetch.fetch(url=url, method=method, payload=data,
headers={'Access-Control-Allow-Origin': '*'})
return result.content
My test:
import unittest
from mock import Mock
class TestUrlFetch(unittest.TestCase):
def test_fetch_url(self):
from console.auth import fetch_url
# Create a mock object based on the fetch_url function
mock = Mock(spec=fetch_url)
# Mock the fetch_url function
content = mock.fetch_url('https://google.com')
# Test that content is not empty
self.assertIsNotNone(content)
If what I'm doing is completely in the wrong direction, please shed some light on the correct solution.
The test is not working, and is producing the following error:
======================================================================
ERROR: test_fetch_url (console.tests.test_auth.TestUrlFetch)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/bengrunfeld/Desktop/Work/code/wf-ghconsole/console/tests/test_auth.py", line 34, in test_fetch_url
content = mock.fetch_url('https://google.com')
File "/Users/bengrunfeld/.virtualenvs/env2/lib/python2.7/site-packages/mock.py", line 658, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'fetch_url'
-------------------- >> begin captured logging << --------------------
root: DEBUG: Using threading.local
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 1 test in 0.277s
FAILED (errors=1)

First of all, as univerio's comment suggests you should call you mock like this:
mock('https://google.com')
Your test should pass after that fix, but probably that mock doesn't do what you really want. I've encountered a few problems with spec and autospec.
Mocks created with Mock(spec=) don't check number of arguments they are called with. I've just looked through the docs and they don't state that, but for some reason I expected it to work. Autospecced mocks do check the arguments.
By default both spec and autospec function mocks return mock objects when you call them. This may be not what you want when you mock a function that does not return anything. In this case you can set the return_value manually:
def foo():
pass
mock_foo = Mock(spec=foo, return_value=None)
mock_foo()

Python unit test how to use Mox to mock the gzip with statement

In Python, how do I mock an object created in a with statement using mox unit test library
Code
class MyCode:
def generate_gzip_file(self):
with gzip.GzipFile('file_name.txt.gz','wb') as f:
f.write('data')
Unit Test
class MyCodeTest(unittest.TestCase):
def test_generate_gzip_file(self):
mox = mox.Mox()
mock_gzip_file = self.mox.CreateMock(gzip.GzipFile)
mox.StubOutWithMock(gzip, 'GzipFile')
gzip.GzipFile('file_name.txt.gz','wb').AndReturn(mock_file)
mock_gzip_file.write('data')
mox.ReplayAll()
MyCode().generate_gzip_file()
mox.VerifyAll()
I get the error AttributeError: __exit__ on line
with gzip.GzipFile('file_name.txt.gz','wb') as f:

DSM is correct that the mocked instance of gzip.GzipFile isn't ending up with a __exit__ method for some reason. You'll get exactly the same error if you forget to define __exit__ on a class you use with a with statement. For example:
>>> class C(object):
... def __enter__(self):
... return self
...
>>> with C() as c:
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: __exit__
Fortunately, you can work around the problem by using Mox's CreateMockAnything() method to create a mock_gzip_file object that doesn't enforce a particular interface. You'll need to be careful to ensure that you set up the expectations for the mock_gzip_file object correctly (i.e. that you set up expectations for when and how the __enter__() and __exit__(...) methods will be called). Here's an example that worked for me:
import gzip
import mox
import unittest
class MyCode:
def generate_gzip_file(self):
with gzip.GzipFile('file_name.txt.gz', 'wb') as f:
f.write('data')
class MyCodeTest(unittest.TestCase):
def test_generate_gzip_file(self):
mymox = mox.Mox()
mock_gzip_file = mymox.CreateMockAnything()
mymox.StubOutWithMock(gzip, 'GzipFile')
gzip.GzipFile('file_name.txt.gz', 'wb').AndReturn(mock_gzip_file)
mock_gzip_file.__enter__().AndReturn(mock_gzip_file)
mock_gzip_file.write('data')
mock_gzip_file.__exit__(None, None, None).AndReturn(None)
mymox.ReplayAll()
MyCode().generate_gzip_file()
mymox.VerifyAll()
if __name__ == '__main__':
unittest.main()
When I run this I get:
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK

Mocking file objects or iterables in python

Which way is proper for mocking and testing code that iters object returned by open(), using mock library?
whitelist_data.py:
WHITELIST_FILE = "testdata.txt"
format_str = lambda s: s.rstrip().lstrip('www.')
whitelist = None
with open(WHITELIST_FILE) as whitelist_data:
whitelist = set(format_str(line) for line in whitelist_data)
if not whitelist:
raise RuntimeError("Can't read data from %s file" % WHITELIST_FILE)
def is_whitelisted(substr):
return 1 if format_str(substr) in whitelist else 0
Here's how I try to test it.
import unittest
import mock
TEST_DATA = """
domain1.com
domain2.com
domain3.com
"""
class TestCheckerFunctions(unittest.TestCase):
def test_is_whitelisted_method(self):
open_mock = mock.MagicMock()
with mock.patch('__builtin__.open',open_mock):
manager = open_mock.return_value.__enter__.return_value
manager.__iter__ = lambda s: iter(TEST_DATA.splitlines())
from whitelist_data import is_whitelisted
self.assertTrue(is_whitelisted('domain1.com'))
if __name__ == '__main__':
unittest.main()
Result of python tests.py is:
$ python tests.py
E
======================================================================
ERROR: test_is_whitelisted_method (__main__.TestCheckerFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 39, in test_is_whitelisted_method
from whitelist_data import is_whitelisted
File "/Users/supa/Devel/python/whitelist/whitelist_data.py", line 20, in <module>
whitelist = set(format_str(line) for line in whitelist_data)
TypeError: 'Mock' object is not iterable
----------------------------------------------------------------------
Ran 1 test in 0.001s
UPD: Thanks to Adam, I've reinstalled mock library(pip install -e hg+https://code.google.com/p/mock#egg=mock) and updated tests.py. Works like a charm.

You're looking for a MagicMock. This supports iteration.
In mock 0.80beta4, patch returns a MagicMock. So this simple example works:
import mock
def foo():
for line in open('myfile'):
print line
#mock.patch('__builtin__.open')
def test_foo(open_mock):
foo()
assert open_mock.called
If you're running mock 0.7.x (It looks like you are), I don't think you can accomplish this with patch alone. You'll need to create the mock separately, then pass it into patch:
import mock
def foo():
for line in open('myfile'):
print line
def test_foo():
open_mock = mock.MagicMock()
with mock.patch('__builtin__.open', open_mock):
foo()
assert open_mock.called
Note - I've run these with py.test, however, these same approaches will work with unittest as well.