I'm trying to test this kind of method that alters my incoming object, but it needs to be mocked because I can't run the code in a unit test case (requires massive files that aren't in the repo). I want to mock method_to_mock so that it sets the file_wrapper_object.status to whatever I want for that test case, notice the method doesn't return anything.
def method_to_mock(file_wrapper_object):
try:
process(file_wrapper_object) #<—- I can't run this in a test case
file_wrapper_object.status = "COMPLETE"
except:
file_wrapper_object.status = "FAILED"
def TestProcessFileWrapperObject(self):
file_wrapper_object = create_wrapper_object(arbitrary_data)
method_to_mock(file_wrapper_object)
self.assertEqual(file_wrapper_object.status, "COMPLETE")
How can I mock a method that doesn't return anything but alters the incoming object?
You can use the side_effect to specify a custom behavior for the mock object. Here is an example:
import unittest
from unittest.mock import MagicMock
def TestProcessFileWrapperObject(self):
file_wrapper_object = create_wrapper_object(arbitrary_data)
my_method_mock = MagicMock(return_value=None)
def side_effect(file_wrapper_object):
file_wrapper_object.status = "COMPLETE"
my_method_mock.side_effect = side_effect
my_method = my_method_mock
my_method(file_wrapper_object)
self.assertEqual(file_wrapper_object.status, "COMPLETE")
Please check this for more details.
I looked around a bit on SO but didn't find what I am looking for, I am pretty sure this has been answered elsewhere. So I have two file writes in my function like this:
def write_files():
with open("a.txt", 'w') as f_h:
f_h.write("data1")
with open("b.txt", 'w') as f_h:
f_h.write("data2")
How do I mock the f_h.write() selectively so that one returns an exception and the other does not? I tried to set side_effect but its not clear where that fits. The test code that I have experimented with has something like this:
from unittest.mock import patch, call, mock_open
import unittest
class Tester(unittest.TestCase):
def test_analyze(self):
with patch("builtins.open", mock_open(read_data="data")) as mf:
# mf.side_effect = [None, Exception()] ?
write_files()
if __name__ == '__main__':
unittest.main()
Two things: you have to mock the context manager, e.g. the result of __enter__, and you have to put the side effect on the write method of the mocked file handle (e.g. the result of the __enter__ call):
class Tester(unittest.TestCase):
def test_analyze(self):
with patch("builtins.open", mock_open(read_data="data")) as mf:
fh_mock = mf.return_value.__enter__.return_value
fh_mock.write.side_effect = [None, Exception]
with self.assertRaises(Exception):
write_files()
src/mainDir/mainFile.py
contents of mainFile.py
import src.tempDir.tempFile as temp
data = 'someData'
def foo(self):
ans = temp.boo(data)
return ans
src/tempDir/tempFile.py
def boo(data):
ans = data
return ans
Now I want to test foo() from src/tests/test_mainFile.py and I want to mock temp.boo(data) method in foo() method
import src.mainDir.mainFile as mainFunc
testData = 'testData'
def test_foo(monkeypatch):
monkeypatch.setattr('src.tempDir.tempFile', 'boo', testData)
ans = mainFunc.foo()
assert ans == testData
but I get error
AttributeError: 'src.tempDir.tempFile' has no attribute 'boo'
I expect ans = testData.
I would like to know if I am correctly mocking my tempDir.boo() method or I should use pytest's mocker instead of monkeypatch.
You're telling monkeypatch to patch the attribute boo of the string object you pass in.
You'll either need to pass in a module like monkeypatch.setattr(tempFile, 'boo', testData), or pass the attribute as a string too (using the two-argument form), like monkeypatch.setattr('src.tempDir.tempFile.boo', testData).
My use case was was slightly different but should still apply. I wanted to patch the value of sys.frozen which is set when running an application bundled by something like Pyinstaller. Otherwise, the attribute does not exist. Looking through the pytest docs, the raising kwarg controls wether or not AttributeError is raised when the attribute does not already exist. (docs)
Usage Example
import sys
def test_frozen_func(monkeypatch):
monkeypatch.setattr(sys, 'frozen', True, raising=False)
# can use ('fq_import_path.sys.frozen', ...)
# if what you are trying to patch is imported in another file
assert sys.frozen
Update: mocking function calls can be done with monkeypatch.setattr('package.main.slow_fun', lambda: False) (see answer and comments in https://stackoverflow.com/a/44666743/3219667) and updated snippet below
I don't think this can be done with pytest's monkeypatch, but you can use the pytest-mock package. Docs: https://github.com/pytest-dev/pytest-mock
Quick example with the two files below:
# package/main.py
def slow_fun():
return True
def main_fun():
if slow_fun():
raise RuntimeError('Slow func returned True')
# tests/test_main.py
from package.main import main_fun
# Make sure to install pytest-mock so that the mocker argument is available
def test_main_fun(mocker):
mocker.patch('package.main.slow_fun', lambda: False)
main_fun()
# UPDATE: Alternative with monkeypatch
def test_main_fun_monkeypatch(monkeypatch):
monkeypatch.setattr('package.main.slow_fun', lambda: False)
main_fun()
Note: this also works if the functions are in different files
My question is how to mock open in python, such that it reacts differently depending on the argument open() is called with. These are some different scenario's that should be possible:
open a mocked file; read preset contents, the basic scenario.
open two mocked files and have them give back different values for the read() method. The order in which the files are opened/read from should not influence the results.
Furthermore, if I call open('actual_file.txt') to open an actual file, I want the actual file to be opened, and not a magic mock with mocked behavior. Or if I just don't want the access to a certain file mocked, but I do want other files to be mocked, this should be possible.
I know about this question: Python mock builtin 'open' in a class using two different files.
But that answer only partially answers up to the second requirement. The part about order independent results is not included and it does not specify how to mock only some calls, and allow other calls to go through to the actual files (default behavior).
A bit late, but I just recently happened upon the same need, so I'd like to share my solution, based upon this answer from the referred-to question:
import pytest
from unittest.mock import mock_open
from functools import partial
from pathlib import Path
mock_file_data = {
"file1.txt": "some text 1",
"file2.txt": "some text 2",
# ... and so on ...
}
do_not_mock: {
# If you need exact match (see note in mocked_file(),
# you should replace these with the correct Path() invocations
"notmocked1.txt",
"notmocked2.txt",
# ... and so on ...
}
# Ref: https://stackoverflow.com/a/38618056/149900
def mocked_file(m, fn, *args, **kwargs):
m.opened_file = Path(fn)
fn = Path(fn).name # If you need exact path match, remove this line
if fn in do_not_mock:
return open(fn, *args, **kwargs)
if fn not in mock_file_data:
raise FileNotFoundError
data = mock_file_data[fn]
file_obj = mock_open(read_data=data).return_value
file_obj.__iter__.return_value = data.splitlines(True)
return file_obj
def assert_opened(m, fn):
fn = Path(fn)
assert m.opened_file == fn
#pytest.fixture()
def mocked_open(mocker):
m = mocker.patch("builtins.open")
m.side_effect = partial(mocked_file, m)
m.assert_opened = partial(assert_opened, m)
return m
def test_something(mocked_open):
...
# Something that should NOT invoke open()
mocked_open.assert_not_called()
...
# Something that SHOULD invoke open()
mocked_open.assert_called_once()
mocked_open.assert_opened("file1.txt")
# Depends on how the tested unit handle "naked" filenames,
# you might have to change the arg to:
# Path.cwd() / "file1.txt"
# ... and so on ...
Do note that (1) I am using Python 3, and (2) I am using pytest.
This can be done by following the approach in the other question's accepted answer (Python mock builtin 'open' in a class using two different files) with a few alterations.
First off. Instead of just specifying a side_effect that can be popped. We need to make sure the side_effect can return the correct mocked_file depending on the parameters used with the open call.
Then if the file we wish to open is not among the files we wish to mock, we instead return the original open() of the file instead of any mocked behavior.
The code below demonstrates how this can be achieved in a clean, repeatable way. I for instance have this code inside of a file that provides some utility functions to make testing easier.
from mock import MagicMock
import __builtin__
from mock import patch
import sys
# Reference to the original open function.
g__test_utils__original_open = open
g__test_utils__file_spec = None
def create_file_mock(read_data):
# Create file_spec such as in mock.mock_open
global g__test_utils__file_spec
if g__test_utils__file_spec is None:
# set on first use
if sys.version_info[0] == 3:
import _io
g__test_utils__file_spec = list(set(dir(_io.TextIOWrapper)).union(set(dir(_io.BytesIO))))
else:
g__test_utils__file_spec = file
file_handle = MagicMock(spec=g__test_utils__file_spec)
file_handle.write.return_value = None
file_handle.__enter__.return_value = file_handle
file_handle.read.return_value = read_data
return file_handle
def flexible_mock_open(file_map):
def flexible_side_effect(file_name):
if file_name in file_map:
return file_map[file_name]
else:
global g__test_utils__original_open
return g__test_utils__original_open(file_name)
global g__test_utils__original_open
return_value = MagicMock(name='open', spec=g__test_utils__original_open)
return_value.side_effect = flexible_side_effect
return return_value
if __name__ == "__main__":
a_mock = create_file_mock(read_data="a mock - content")
b_mock = create_file_mock(read_data="b mock - different content")
mocked_files = {
'a' : a_mock,
'b' : b_mock,
}
with patch.object(__builtin__, 'open', flexible_mock_open(mocked_files)):
with open('a') as file_handle:
print file_handle.read() # prints a mock - content
with open('b') as file_handle:
print file_handle.read() # prints b mock - different content
with open('actual_file.txt') as file_handle:
print file_handle.read() # prints actual file contents
This borrows some code straight from the mock.py (python 2.7) for the creating of the file_spec.
side note: if there's any body that can help me in how to hide these globals if possible, that'd be very helpful.
How do I test the following code with unittest.mock:
def testme(filepath):
with open(filepath) as f:
return f.read()
Python 3
Patch builtins.open and use mock_open, which is part of the mock framework. patch used as a context manager returns the object used to replace the patched one:
from unittest.mock import patch, mock_open
with patch("builtins.open", mock_open(read_data="data")) as mock_file:
assert open("path/to/open").read() == "data"
mock_file.assert_called_with("path/to/open")
If you want to use patch as a decorator, using mock_open()'s result as the new= argument to patch can be a little bit weird. Instead, use patch's new_callable= argument and remember that every extra argument that patch doesn't use will be passed to the new_callable function, as described in the patch documentation:
patch() takes arbitrary keyword arguments. These will be passed to the Mock (or new_callable) on construction.
#patch("builtins.open", new_callable=mock_open, read_data="data")
def test_patch(mock_file):
assert open("path/to/open").read() == "data"
mock_file.assert_called_with("path/to/open")
Remember that in this case patch will pass the mocked object as an argument to your test function.
Python 2
You need to patch __builtin__.open instead of builtins.open and mock is not part of unittest, you need to pip install and import it separately:
from mock import patch, mock_open
with patch("__builtin__.open", mock_open(read_data="data")) as mock_file:
assert open("path/to/open").read() == "data"
mock_file.assert_called_with("path/to/open")
The way to do this has changed in mock 0.7.0 which finally supports mocking the python protocol methods (magic methods), particularly using the MagicMock:
http://www.voidspace.org.uk/python/mock/magicmock.html
An example of mocking open as a context manager (from the examples page in the mock documentation):
>>> open_name = '%s.open' % __name__
>>> with patch(open_name, create=True) as mock_open:
... mock_open.return_value = MagicMock(spec=file)
...
... with open('/some/path', 'w') as f:
... f.write('something')
...
<mock.Mock object at 0x...>
>>> file_handle = mock_open.return_value.__enter__.return_value
>>> file_handle.write.assert_called_with('something')
With the latest versions of mock, you can use the really useful mock_open helper:
mock_open(mock=None, read_data=None)
A helper function to create a
mock to replace the use of open. It works for open called directly or
used as a context manager.
The mock argument is the mock object to configure. If None (the
default) then a MagicMock will be created for you, with the API
limited to methods or attributes available on standard file handles.
read_data is a string for the read method of the file handle to
return. This is an empty string by default.
>>> from mock import mock_open, patch
>>> m = mock_open()
>>> with patch('{}.open'.format(__name__), m, create=True):
... with open('foo', 'w') as h:
... h.write('some stuff')
>>> m.assert_called_once_with('foo', 'w')
>>> handle = m()
>>> handle.write.assert_called_once_with('some stuff')
To use mock_open for a simple file read() (the original mock_open snippet already given on this page is geared more for write):
my_text = "some text to return when read() is called on the file object"
mocked_open_function = mock.mock_open(read_data=my_text)
with mock.patch("__builtin__.open", mocked_open_function):
with open("any_string") as f:
print f.read()
Note as per docs for mock_open, this is specifically for read(), so won't work with common patterns like for line in f, for example.
Uses python 2.6.6 / mock 1.0.1
The top answer is useful but I expanded on it a bit.
If you want to set the value of your file object (the f in as f) based on the arguments passed to open() here's one way to do it:
def save_arg_return_data(*args, **kwargs):
mm = MagicMock(spec=file)
mm.__enter__.return_value = do_something_with_data(*args, **kwargs)
return mm
m = MagicMock()
m.side_effect = save_arg_return_array_of_data
# if your open() call is in the file mymodule.animals
# use mymodule.animals as name_of_called_file
open_name = '%s.open' % name_of_called_file
with patch(open_name, m, create=True):
#do testing here
Basically, open() will return an object and with will call __enter__() on that object.
To mock properly, we must mock open() to return a mock object. That mock object should then mock the __enter__() call on it (MagicMock will do this for us) to return the mock data/file object we want (hence mm.__enter__.return_value). Doing this with 2 mocks the way above allows us to capture the arguments passed to open() and pass them to our do_something_with_data method.
I passed an entire mock file as a string to open() and my do_something_with_data looked like this:
def do_something_with_data(*args, **kwargs):
return args[0].split("\n")
This transforms the string into a list so you can do the following as you would with a normal file:
for line in file:
#do action
I might be a bit late to the game, but this worked for me when calling open in another module without having to create a new file.
test.py
import unittest
from mock import Mock, patch, mock_open
from MyObj import MyObj
class TestObj(unittest.TestCase):
open_ = mock_open()
with patch.object(__builtin__, "open", open_):
ref = MyObj()
ref.save("myfile.txt")
assert open_.call_args_list == [call("myfile.txt", "wb")]
MyObj.py
class MyObj(object):
def save(self, filename):
with open(filename, "wb") as f:
f.write("sample text")
By patching the open function inside the __builtin__ module to my mock_open(), I can mock writing to a file without creating one.
Note: If you are using a module that uses cython, or your program depends on cython in any way, you will need to import cython's __builtin__ module by including import __builtin__ at the top of your file. You will not be able to mock the universal __builtin__ if you are using cython.
If you don't need any file further, you can decorate the test method:
#patch('builtins.open', mock_open(read_data="data"))
def test_testme():
result = testeme()
assert result == "data"
To patch the built-in open() function with unittest:
This worked for a patch to read a json config.
class ObjectUnderTest:
def __init__(self, filename: str):
with open(filename, 'r') as f:
dict_content = json.load(f)
The mocked object is the io.TextIOWrapper object returned by the open() function
#patch("<src.where.object.is.used>.open",
return_value=io.TextIOWrapper(io.BufferedReader(io.BytesIO(b'{"test_key": "test_value"}'))))
def test_object_function_under_test(self, mocker):
I'm using pytest in my case, and the good news is that in Python 3 the unittest library can also be imported and used without issue.
Here is my approach. First, I create a conftest.py file with reusable pytest fixture(s):
from functools import cache
from unittest.mock import MagicMock, mock_open
import pytest
from pytest_mock import MockerFixture
class FileMock(MagicMock):
def __init__(self, mocker: MagicMock = None, **kwargs):
super().__init__(**kwargs)
if mocker:
self.__dict__ = mocker.__dict__
# configure mock object to replace the use of open(...)
# note: this is useful in scenarios where data is written out
_ = mock_open(mock=self)
#property
def read_data(self):
return self.side_effect
#read_data.setter
def read_data(self, mock_data: str):
"""set mock data to be returned when `open(...).read()` is called."""
self.side_effect = mock_open(read_data=mock_data)
#property
#cache
def write_calls(self):
"""a list of calls made to `open().write(...)`"""
handle = self.return_value
write: MagicMock = handle.write
return write.call_args_list
#property
def write_lines(self) -> str:
"""a list of written lines (as a string)"""
return ''.join([c[0][0] for c in self.write_calls])
#pytest.fixture
def mock_file_open(mocker: MockerFixture) -> FileMock:
return FileMock(mocker.patch('builtins.open'))
Where I decided to make the read_data as a property, in order to be more pythonic. It can be assigned in a test function with whatever data that open() needs to return.
In my test file, named something like test_it_works.py, I have a following test case to confirm intended functionality:
from unittest.mock import call
def test_mock_file_open_and_read(mock_file_open):
mock_file_open.read_data = 'hello\nworld!'
with open('/my/file/here', 'r') as in_file:
assert in_file.readlines() == ['hello\n', 'world!']
mock_file_open.assert_called_with('/my/file/here', 'r')
def test_mock_file_open_and_write(mock_file_open):
with open('/out/file/here', 'w') as f:
f.write('hello\n')
f.write('world!\n')
f.write('--> testing 123 :-)')
mock_file_open.assert_called_with('/out/file/here', 'w')
assert call('world!\n') in mock_file_open.write_calls
assert mock_file_open.write_lines == """\
hello
world!
--> testing 123 :-)
""".rstrip()
Check out the gist here.
Sourced from a github snippet to patch read and write functionality in python.
The source link is over here
import configparser
import pytest
simpleconfig = """[section]\nkey = value\n\n"""
def test_monkeypatch_open_read(mockopen):
filename = 'somefile.txt'
mockopen.write(filename, simpleconfig)
parser = configparser.ConfigParser()
parser.read(filename)
assert parser.sections() == ['section']
def test_monkeypatch_open_write(mockopen):
parser = configparser.ConfigParser()
parser.add_section('section')
parser.set('section', 'key', 'value')
filename = 'somefile.txt'
parser.write(open(filename, 'wb'))
assert mockopen.read(filename) == simpleconfig
SIMPLE #patch with assert
If you're wanting to use #patch. The open() is called inside the handler and is read.
#patch("builtins.open", new_callable=mock_open, read_data="data")
def test_lambda_handler(self, mock_open_file):
lambda_handler(event, {})