I writing unit tests and for testing I want to mock a method of already existing object.
But looks like with asyncio corutines it is not so simple as it looks.
I tried to use MagickMock but it just doesn't work. There is no errors or exceptions, but with debugger I can see that f() is never being called.
My tests and object I want to patch look like this:
from unittest.mock import patch, MagicMock
class Service(object):
async def callback_handler(self, msg):
pass
async def handle(self, msg):
await self.callback_handler(msg)
class TestCase(object):
def setUp(self):
self.service = Service()
#patch('module.msg')
def test_my_case(self, msg_mock):
f_was_called = False
async def f():
global f_was_called
f_was_called = True
self.service.callback_handler = MagicMock(wraps=f) # here I try to mock
await self.service.handle(msg_mock)
assert f_was_called is True
How can I patch already instantiated object method with some custom one? Is there some issues with corutines?
Try using a context manager by replacing this line:
self.service.callback_handler = MagicMock(wraps=f) # here I try to mock
With this:
with mock.patch.object(self.service, 'callback_handler', side_effect=f) as mock_cb:
... # rest of code indented
I also faced the problem trying to mock asyncio and want into a lot of troubles,
I ended using pytest.asycio plugin .
#pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b'expected result' == res
Related
For instance, every time a test finds
database.db.session.using_bind("reader")
I want to remove the using_bind("reader")) and just work with
database.db.session
using mocker
Tried to use it like this in conftest.py
#pytest.fixture(scope='function')
def session(mocker):
mocker.patch('store.database.db.session.using_bind', return_value=_db.db.session)
But nothing has worked so far.
Code under test:
from store import database
results = database.db.session.using_bind("reader").query(database.Order.id).join(database.Shop).filter(database.Shop.deleted == False).all(),
and I get
AttributeError: 'scoped_session' object has no attribute 'using_bind' as an error.
Let's start with an MRE where the code under test uses a fake database:
from unittest.mock import Mock, patch
class Session:
def using_bind(self, bind):
raise NotImplementedError(f"Can't bind {bind}")
def query(self):
return "success!"
database = Mock()
database.db.session = Session()
def code_under_test():
return database.db.session.using_bind("reader").query()
def test():
assert code_under_test() == "success!"
Running this test fails with:
E NotImplementedError: Can't bind reader
So we want to mock session.using_bind in code_under_test so that it returns session -- that will make our test pass.
We do that using patch, like so:
#patch("test.database.db.session.using_bind")
def test(mock_bind):
mock_bind.return_value = database.db.session
assert code_under_test() == "success!"
Note that my code is in a file called test.py, so my patch call applies to the test module -- you will need to adjust this to point to the module under test in your own code.
Note also that I need to set up my mock before calling the code under test.
How do I mock async call from one native coroutine to other one using unittest.mock.patch?
I currently have quite an awkward solution:
class CoroutineMock(MagicMock):
def __await__(self, *args, **kwargs):
future = Future()
future.set_result(self)
result = yield from future
return result
Then
class TestCoroutines(TestCase):
#patch('some.path', new_callable=CoroutineMock)
def test(self, mock):
some_action()
mock.assert_called_with(1,2,3)
This works but looks ugly. Is there more pythonic way to do this?
The solution was actually quite simple:
I just needed to convert __call__ method of mock into coroutine:
class AsyncMock(MagicMock):
async def __call__(self, *args, **kwargs):
return super(AsyncMock, self).__call__(*args, **kwargs)
This works perfectly, when mock is called, code receives native coroutine
Example usage:
#mock.patch('my.path.asyncio.sleep', new_callable=AsyncMock)
def test_stuff(sleep):
# code
Everyone's missing what's probably the simplest and clearest solution:
#patch('some.path')
def test(self, mock):
f = asyncio.Future()
f.set_result('whatever result you want')
process_smtp_message.return_value = f
mock.assert_called_with(1, 2, 3)
remember a coroutine can be thought of as just a function which is guaranteed to return a future which can, in turn be awaited.
Based on #scolvin answer I created this (imo) cleaner way:
import asyncio
def async_return(result):
f = asyncio.Future()
f.set_result(result)
return f
That's it, just use it around whatever return you want to be async, as in
mock = MagicMock(return_value=async_return("Example return"))
await mock()
Subclassing MagicMock will propagate your custom class for all the mocks generated from your coroutine mock. For instance, AsyncMock().__str__ will also become an AsyncMock which is probably not what you're looking for.
Instead, you might want to define a factory that creates a Mock (or a MagicMock) with custom arguments, for instance side_effect=coroutine(coro). Also, it might be a good idea to separate the coroutine function from the coroutine (as explained in the documentation).
Here is what I came up with:
from asyncio import coroutine
def CoroMock():
coro = Mock(name="CoroutineResult")
corofunc = Mock(name="CoroutineFunction", side_effect=coroutine(coro))
corofunc.coro = coro
return corofunc
An explanation of the different objects:
corofunc: the coroutine function mock
corofunc.side_effect(): the coroutine, generated for each call
corofunc.coro: the mock used by the coroutine to get the result
corofunc.coro.return_value: the value returned by the coroutine
corofunc.coro.side_effect: might be used to raise an exception
Example:
async def coro(a, b):
return await sleep(1, result=a+b)
def some_action(a, b):
return get_event_loop().run_until_complete(coro(a, b))
#patch('__main__.coro', new_callable=CoroMock)
def test(corofunc):
a, b, c = 1, 2, 3
corofunc.coro.return_value = c
result = some_action(a, b)
corofunc.assert_called_with(a, b)
assert result == c
Another way of mocking coroutine is to make coroutine, that returns mock. This way you can mock coroutines that will be passed into asyncio.wait or asyncio.wait_for.
This makes more universal coroutines though makes setup of tests more cumbersome:
def make_coroutine(mock)
async def coroutine(*args, **kwargs):
return mock(*args, **kwargs)
return coroutine
class Test(TestCase):
def setUp(self):
self.coroutine_mock = Mock()
self.patcher = patch('some.coroutine',
new=make_coroutine(self.coroutine_mock))
self.patcher.start()
def tearDown(self):
self.patcher.stop()
One more variant of "simplest" solution to mock a async object, which is just a one liner.
In source:
class Yo:
async def foo(self):
await self.bar()
async def bar(self):
# Some code
In test:
from asyncio import coroutine
yo = Yo()
# Here bounded method bar is mocked and will return a customised result.
yo.bar = Mock(side_effect=coroutine(lambda:'the awaitable should return this'))
event_loop.run_until_complete(yo.foo())
I don't know why nobody mentioned the default option available.
python provides a Async version of MagicMock.
You can read more about this here.
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.AsyncMock
In case you are using patch then also you don't need to make any other change. It will automatically replace it with a async mock function if required.
Read more here
https://docs.python.org/3/library/unittest.mock.html#patch
You can set the return_value of an async method like so:
mock = unittest.mock.MagicMock()
mock.your_async_method.return_value = task_from_result(your_return_value)
async def task_from_result(result):
return result
The caller will have to do await your_async_method(..) just like as if the method wasn't mocked.
I like this approach, which also makes AsyncMock to behave exactly like Mock:
class AsyncMock:
def __init__(self, *args, **kwargs):
self.mock = Mock(*args, **kwargs)
async def __call__(self, *args, **kwargs):
return self.mock(*args, **kwargs)
def __getattr__(self, item):
return getattr(self.mock, item)
Then you can work with it in the same way like with Mock, i.e:
#pytest.mark.asyncio
async def test_async_mock_example(monkeypatch):
fn = AsyncMock(side_effect=ValueError)
with pytest.raises(ValueError):
await fn()
assert fn.call_count == 1
I'm quite stuck on structuring the code in this scenario. Can anyone help me with this?
| module.py
import asyncio
class Server:
def __init__(self):
self.d = {}
#classmethod
async def create(cls):
self = cls()
await self.func()
return self
async def func(self):
await asyncio.sleep(5) # Some other async code here
self.a = 12
def reg(self, ev):
def decorator(func):
self.d[ev] = func()
retun func
return decorator
def reg2(self, ev, func):
self.d[ev] = func
| main.py
import asyncio
from module import Server
async def main():
ser = await Server.create()
# This would be another way... but i find the other way one neater
serv.reg2('msg', some_handler)
# I want to decorate and register this using
# reg func; but since object is not created yet
# how do i acomplish this?
# #ser.reg('msg')
async def some_handler():
...
if __name__ == "__main__":
asyncio.run(main())
Some key points of my aim:
The function 'some_handler' is never used other than the time for register. That is, the function soley exists to be registered and is not used anywhere else.
Since Server class needs an asynchronous initialisation, it cannot be done globally.
(I dont know whether this point is helpful) Generally only one Server instance is created for a single program. There wont be any other instance even in other modules.
How do I model my code to satisfy this senario? I have mentioned an alternate way to register the function, but I feel I am missing something, as some_handler isn't used anywhere else. I have thought about making Server class into a metaclass to do registering and converting the main() and some_handler() as parts of the metclass's class but I'm seeking for different views and opinions.
How do I mock async call from one native coroutine to other one using unittest.mock.patch?
I currently have quite an awkward solution:
class CoroutineMock(MagicMock):
def __await__(self, *args, **kwargs):
future = Future()
future.set_result(self)
result = yield from future
return result
Then
class TestCoroutines(TestCase):
#patch('some.path', new_callable=CoroutineMock)
def test(self, mock):
some_action()
mock.assert_called_with(1,2,3)
This works but looks ugly. Is there more pythonic way to do this?
The solution was actually quite simple:
I just needed to convert __call__ method of mock into coroutine:
class AsyncMock(MagicMock):
async def __call__(self, *args, **kwargs):
return super(AsyncMock, self).__call__(*args, **kwargs)
This works perfectly, when mock is called, code receives native coroutine
Example usage:
#mock.patch('my.path.asyncio.sleep', new_callable=AsyncMock)
def test_stuff(sleep):
# code
Everyone's missing what's probably the simplest and clearest solution:
#patch('some.path')
def test(self, mock):
f = asyncio.Future()
f.set_result('whatever result you want')
process_smtp_message.return_value = f
mock.assert_called_with(1, 2, 3)
remember a coroutine can be thought of as just a function which is guaranteed to return a future which can, in turn be awaited.
Based on #scolvin answer I created this (imo) cleaner way:
import asyncio
def async_return(result):
f = asyncio.Future()
f.set_result(result)
return f
That's it, just use it around whatever return you want to be async, as in
mock = MagicMock(return_value=async_return("Example return"))
await mock()
Subclassing MagicMock will propagate your custom class for all the mocks generated from your coroutine mock. For instance, AsyncMock().__str__ will also become an AsyncMock which is probably not what you're looking for.
Instead, you might want to define a factory that creates a Mock (or a MagicMock) with custom arguments, for instance side_effect=coroutine(coro). Also, it might be a good idea to separate the coroutine function from the coroutine (as explained in the documentation).
Here is what I came up with:
from asyncio import coroutine
def CoroMock():
coro = Mock(name="CoroutineResult")
corofunc = Mock(name="CoroutineFunction", side_effect=coroutine(coro))
corofunc.coro = coro
return corofunc
An explanation of the different objects:
corofunc: the coroutine function mock
corofunc.side_effect(): the coroutine, generated for each call
corofunc.coro: the mock used by the coroutine to get the result
corofunc.coro.return_value: the value returned by the coroutine
corofunc.coro.side_effect: might be used to raise an exception
Example:
async def coro(a, b):
return await sleep(1, result=a+b)
def some_action(a, b):
return get_event_loop().run_until_complete(coro(a, b))
#patch('__main__.coro', new_callable=CoroMock)
def test(corofunc):
a, b, c = 1, 2, 3
corofunc.coro.return_value = c
result = some_action(a, b)
corofunc.assert_called_with(a, b)
assert result == c
Another way of mocking coroutine is to make coroutine, that returns mock. This way you can mock coroutines that will be passed into asyncio.wait or asyncio.wait_for.
This makes more universal coroutines though makes setup of tests more cumbersome:
def make_coroutine(mock)
async def coroutine(*args, **kwargs):
return mock(*args, **kwargs)
return coroutine
class Test(TestCase):
def setUp(self):
self.coroutine_mock = Mock()
self.patcher = patch('some.coroutine',
new=make_coroutine(self.coroutine_mock))
self.patcher.start()
def tearDown(self):
self.patcher.stop()
One more variant of "simplest" solution to mock a async object, which is just a one liner.
In source:
class Yo:
async def foo(self):
await self.bar()
async def bar(self):
# Some code
In test:
from asyncio import coroutine
yo = Yo()
# Here bounded method bar is mocked and will return a customised result.
yo.bar = Mock(side_effect=coroutine(lambda:'the awaitable should return this'))
event_loop.run_until_complete(yo.foo())
I don't know why nobody mentioned the default option available.
python provides a Async version of MagicMock.
You can read more about this here.
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.AsyncMock
In case you are using patch then also you don't need to make any other change. It will automatically replace it with a async mock function if required.
Read more here
https://docs.python.org/3/library/unittest.mock.html#patch
You can set the return_value of an async method like so:
mock = unittest.mock.MagicMock()
mock.your_async_method.return_value = task_from_result(your_return_value)
async def task_from_result(result):
return result
The caller will have to do await your_async_method(..) just like as if the method wasn't mocked.
I like this approach, which also makes AsyncMock to behave exactly like Mock:
class AsyncMock:
def __init__(self, *args, **kwargs):
self.mock = Mock(*args, **kwargs)
async def __call__(self, *args, **kwargs):
return self.mock(*args, **kwargs)
def __getattr__(self, item):
return getattr(self.mock, item)
Then you can work with it in the same way like with Mock, i.e:
#pytest.mark.asyncio
async def test_async_mock_example(monkeypatch):
fn = AsyncMock(side_effect=ValueError)
with pytest.raises(ValueError):
await fn()
assert fn.call_count == 1
How do I mock async call from one native coroutine to other one using unittest.mock.patch?
I currently have quite an awkward solution:
class CoroutineMock(MagicMock):
def __await__(self, *args, **kwargs):
future = Future()
future.set_result(self)
result = yield from future
return result
Then
class TestCoroutines(TestCase):
#patch('some.path', new_callable=CoroutineMock)
def test(self, mock):
some_action()
mock.assert_called_with(1,2,3)
This works but looks ugly. Is there more pythonic way to do this?
The solution was actually quite simple:
I just needed to convert __call__ method of mock into coroutine:
class AsyncMock(MagicMock):
async def __call__(self, *args, **kwargs):
return super(AsyncMock, self).__call__(*args, **kwargs)
This works perfectly, when mock is called, code receives native coroutine
Example usage:
#mock.patch('my.path.asyncio.sleep', new_callable=AsyncMock)
def test_stuff(sleep):
# code
Everyone's missing what's probably the simplest and clearest solution:
#patch('some.path')
def test(self, mock):
f = asyncio.Future()
f.set_result('whatever result you want')
process_smtp_message.return_value = f
mock.assert_called_with(1, 2, 3)
remember a coroutine can be thought of as just a function which is guaranteed to return a future which can, in turn be awaited.
Based on #scolvin answer I created this (imo) cleaner way:
import asyncio
def async_return(result):
f = asyncio.Future()
f.set_result(result)
return f
That's it, just use it around whatever return you want to be async, as in
mock = MagicMock(return_value=async_return("Example return"))
await mock()
Subclassing MagicMock will propagate your custom class for all the mocks generated from your coroutine mock. For instance, AsyncMock().__str__ will also become an AsyncMock which is probably not what you're looking for.
Instead, you might want to define a factory that creates a Mock (or a MagicMock) with custom arguments, for instance side_effect=coroutine(coro). Also, it might be a good idea to separate the coroutine function from the coroutine (as explained in the documentation).
Here is what I came up with:
from asyncio import coroutine
def CoroMock():
coro = Mock(name="CoroutineResult")
corofunc = Mock(name="CoroutineFunction", side_effect=coroutine(coro))
corofunc.coro = coro
return corofunc
An explanation of the different objects:
corofunc: the coroutine function mock
corofunc.side_effect(): the coroutine, generated for each call
corofunc.coro: the mock used by the coroutine to get the result
corofunc.coro.return_value: the value returned by the coroutine
corofunc.coro.side_effect: might be used to raise an exception
Example:
async def coro(a, b):
return await sleep(1, result=a+b)
def some_action(a, b):
return get_event_loop().run_until_complete(coro(a, b))
#patch('__main__.coro', new_callable=CoroMock)
def test(corofunc):
a, b, c = 1, 2, 3
corofunc.coro.return_value = c
result = some_action(a, b)
corofunc.assert_called_with(a, b)
assert result == c
Another way of mocking coroutine is to make coroutine, that returns mock. This way you can mock coroutines that will be passed into asyncio.wait or asyncio.wait_for.
This makes more universal coroutines though makes setup of tests more cumbersome:
def make_coroutine(mock)
async def coroutine(*args, **kwargs):
return mock(*args, **kwargs)
return coroutine
class Test(TestCase):
def setUp(self):
self.coroutine_mock = Mock()
self.patcher = patch('some.coroutine',
new=make_coroutine(self.coroutine_mock))
self.patcher.start()
def tearDown(self):
self.patcher.stop()
One more variant of "simplest" solution to mock a async object, which is just a one liner.
In source:
class Yo:
async def foo(self):
await self.bar()
async def bar(self):
# Some code
In test:
from asyncio import coroutine
yo = Yo()
# Here bounded method bar is mocked and will return a customised result.
yo.bar = Mock(side_effect=coroutine(lambda:'the awaitable should return this'))
event_loop.run_until_complete(yo.foo())
I don't know why nobody mentioned the default option available.
python provides a Async version of MagicMock.
You can read more about this here.
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.AsyncMock
In case you are using patch then also you don't need to make any other change. It will automatically replace it with a async mock function if required.
Read more here
https://docs.python.org/3/library/unittest.mock.html#patch
You can set the return_value of an async method like so:
mock = unittest.mock.MagicMock()
mock.your_async_method.return_value = task_from_result(your_return_value)
async def task_from_result(result):
return result
The caller will have to do await your_async_method(..) just like as if the method wasn't mocked.
I like this approach, which also makes AsyncMock to behave exactly like Mock:
class AsyncMock:
def __init__(self, *args, **kwargs):
self.mock = Mock(*args, **kwargs)
async def __call__(self, *args, **kwargs):
return self.mock(*args, **kwargs)
def __getattr__(self, item):
return getattr(self.mock, item)
Then you can work with it in the same way like with Mock, i.e:
#pytest.mark.asyncio
async def test_async_mock_example(monkeypatch):
fn = AsyncMock(side_effect=ValueError)
with pytest.raises(ValueError):
await fn()
assert fn.call_count == 1