I have this Python code normally running on a distant server that I want to test locally:
from af.data import Variable
def create_file():
main_client = file.Client()
directory = main_client.create(Variable.get("DIR_NAME"))
As I am developing locally and do not have access to the remote service providing the af.data.Variable class, I'd like to mock the Variable.get(str) function, but I'd like to be able—in my own mock—to return some value based on the passed the str parameter. So far, I've found only ways to mock a function to some pre-defined static values using side_effect of unittest.
How can I do that?
after the Peter's comment, here is the solution (pretty straightforward):
from unittest.mock import Mock
from af.data import Variable
import os
def side_effect(arg):
"""Put here whatever logic you want to put in your mock, here's an example of reading a system environment variable"""
return os.getenv(arg)
mock = Mock()
mock.get.side_effect = side_effect
Variable = mock
"""Then just use the Variable.get method as you would do in production environment, it should be mocked to the side_effect function declared above"""
Related
I'm using pytest to patch the os.makedirs method for a test. In a particular test I wanted to add a side effect of an exception.
So I import the os object that I've imported in my script under test, patch it, and then set the side effect in my test:
from infrastructure.scripts.src.server_administrator import os
def mock_makedirs(self, mocker):
mock = MagicMock()
mocker.patch.object(os, "makedirs", return_value=mock)
return mock
def test_if_directory_exist_exception_is_not_raised(self, administrator, mock_makedirs):
mock_makedirs.side_effect = Exception("Directory already exists.")
with pytest.raises(Exception) as exception:
administrator.initialize_server()
assert exception.value == "Directory already exists."
The problem I ran into was that when the mock gets called in my script under test, the side effect no longer existed. While troubleshooting I stopped the tests in the debugger to look at the ID values for the mock I created and the mock that the patch should have set as the return value and found that they are different instances:
I'm still relatively new to some of the testing tools in python, so this may be me missing something in the documentation, but shouldn't the returned mock patched in here be the mock I created?? Am I patching it wrong?
UPDATE
I even adjusted the import style to grab makedirs directly to patch it:
def mock_makedirs(self, mocker):
mock = MagicMock()
mocker.patch("infrastructure.scripts.src.server_administrator.makedirs", return_value=mock)
return mock
And I still run into the same "different mocks" issue.
:facepalm:
I was patching incorrectly. I'm considering just deleting the whole question/answer, but I figured I'd leave it here in case someone runs into the same situation.
I'm defining the patch like this:
mocker.patch.object(os, "makedirs", return_value=mock)
Which would be a valid structure if I was patching the result a function/method. That is, what this patch is saying is "when you call the makedirs, return this.
What I actually want to do is return a mock in place of the method. In it's current form it makes sense that I see two different mocks because the patch logic is currently "replace makedirs with a new mock and then when that mock is called, return this other mock (the mock I made)"
What I really want is just:
mocker.patch.object(os, "makedirs", mock)
Where my third argument (in the patch.object form) is the mock module parameter (vs the named return_value parameter).
In retrospect, it's pretty obvious when I think about it which is why I'm considering deleting the question, but it's an easy enough trip-up that I'm going to leave it live for now.
I have a lamba handler that uses an environment variable. How can I set that value using pytest. I'm getting the error
tests/test_kinesis.py:3: in <module>
from runner import kinesis
runner/kinesis.py:6: in <module>
DATA_ENGINEERING_BUCKET = os.environ["BUCKET"]
../../../../../.pyenv/versions/3.8.8/lib/python3.8/os.py:675: in __getitem__
raise KeyError(key) from None
E KeyError: 'BUCKET'
7:03
I tried setting in the test like this
class TestHandler(unittest.TestCase):
#mock_s3
#mock_lambda
def test_handler(monkeypatch):
monkeypatch.setenv("BUCKET", "test-bucket")
actual = kinesis.handler(kinesis_stream_event, "")
expected = {"statusCode": 200, "body": "OK"}
assert actual == expected
DATA_ENGINEERING_BUCKET = os.environ["BUCKET"]
def handler(event, context):
...
You're getting the failure before your monkeypatch is able to run. The loading of the environment variable will happen when the runner module is first imported.
If this is a module you own, I'd recommend modifying the code to use a default value if DATA_ENGINEERING_BUCKET isn't set. Then you can modify it's value to whatever you want at runtime by calling module.DATA_ENGINEERING_BUCKET = "my_bucket".
DATA_ENGINEERING_BUCKET = os.environ.get("BUCKET", default="default_bucket")
If you can't modify that file then things are more complicated.
I looked into creating a global fixture that monkeypatches the environment and loads the module once, before any tests load and received a pytest error about using function level fixtures within a session level fixture. Which makes sense monkeypatch really isn't intended to fake things long term. You can stick the module load into your test after the monkeypatch but that will generate a lot of boilerplate.
What eventually worked creating a fixture that will provide the class in lieu of importing it. The fixture; sets os.environ to the desired value, loads the module, resets os.environ to it's origional value then yields the module. Any tests that need this module can request the fixture to have access to it within their scope. A word of caution, because test files are imported before fixtures are run any test files that don't use the fixture and import the module normally will raise a KeyError and cause pytest to crash before running any tests.
conftest.py
import os, pytest
#pytest.fixture(scope='session')
def kinesis():
old_environ = os.environ
os.environ = {'BUCKET': 'test-bucket'}
import kinesis
os.environ = old_environ
yield kinesis
tests.py
# Do NOT import kinesis in any test file. Rely on the fixture.
class TestHandler(unittest.TestCase):
#mock_s3
#mock_lambda
def test_handler(kinesis):
actual = kinesis.handler(kinesis_stream_event, "")
expected = {"statusCode": 200, "body": "OK"}
assert actual == expected
A potentially simpler method
os.environ is a dictionary of environment variables that is created when os first loads. If you want a single value for every test then you just need to add the value you want to it before loading any test modules. If you put os.environ['BUCKET'] = 'test-bucket' at the top of conftest.py you will set the environment variable for the rest of the test session. Then as long as the first import of the module happens afterwards you won't have a key error. The big downside to this approach is that unless you know to look in conftest.py or grep the code it will be difficult to determine where the environment variable is getting set when troubleshooting.
I expect the following call to which_user to return self.user no matter what is passed into it but it's behaving as if it is not mocked at all.
def test_user_can_retrieve_favs_using_impersonation(self):
with mock.patch('impersonate.helpers.which_user', return_value=self.user):
user = which_user(self.user2)
What am I doing wrong here? I imported which_user like so: from impersonate.helpers import which_user if that helps.
You need to mock function by it's path to which it's imported.
For example, you use your function inside myapp's views. So you need to do it in following way:
with mock.patch('myapp.views.which_user', return_value=self.user):
user = which_user(self.user2)
This is because it patches only variables defined in testable module, not patching original libraries and packages in the system or venv.
I'm trying to mock a function used within the function I'm testing, but for some reason I'm not seeing the original function always runs, not the one I created as a mock.
The code I'm testing has this type of setup:
def function_being_tested(foo):
...
function_i_want_to_mock(bar)
...
def function_i_want_to_mock(bar)
print("Inside original function")
...
I've installed Mock and I've tried using unittest.mock patch
Currently the test file uses this setup:
import mock
from django.test import TestCase
def mock_function_i_want_to_mock(bar):
print(bar)
return True
class SupportFunctionsTestCases(TestCase):
#mock.patch("path.to.function.function_i_want_to_mock", mock_function_i_want_to_mock)
def test_function_being_tested(self):
# setup
result = function_being_tested(test_foo)
self.assertEqual(result, expected_result)
What then happens is when I run the test, I always get: "Inside original function", not the parameter printed so it's always running the original function.
I've used this exact setup before and it has worked so I'm not sure what's causing this. Probably some wrong setup...
If anyone has a different way of doing this or spots some error, it would be appreciated.
"path.to.function.function_i_want_to_mock" should be a path to where the function is used, not defined.
So if function_i_want_to_mock is defined in moduleA.py but imported and used in moduleB.py which you are testing then you should use #mock.patch("path.to.moduleB.function_i_want_to_mock", ...).
So let's say I have this bit of code:
import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
Now it's a simple enough method, and as you can see we are using an external library(coolObject).
In unit tests, I have to create a mock of this object that roughly replicates it. Let's call this mock object coolMock.
My question is how would I tell the code when to use coolMock or coolObject? I've looked it up online, and a few people have suggested dependency injection, but I'm not sure I understand it correctly.
Thanks in advance!
def doSomething(cool_object=None):
cool_object = cool_object or coolObject()
...
In you test:
def test_do_something(self):
cool_mock = mock.create_autospec(coolObject, ...)
cool_mock.coolOperation.side_effect = ...
doSomthing(cool_object=cool_mock)
...
self.assertEqual(cool_mock.coolOperation.call_count, ...)
As Dan's answer says, one option is to use dependency injection: have the function accept an optional argument, if it's not passed in use the default class, so that a test can pass in a moc.
Another option is to use the mock library (here or here) to replace your coolObject.
Let's say you have a foo.py that looks like
from somewhere.else import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
In your test_foo.py you can do:
import mock
def test_thing():
path = 'foo.coolObject' # The fully-qualified path to the module, class, function, whatever you want to mock.
with mock.patch('foo.coolObject') as m:
doSomething()
# Whatever you want to assert here.
assert m.called
The path you use can include properties on objects, e.g. module1.module2.MyClass.my_class_method. A big gotcha is that you need to mock the object in the module being tested, not where it is defined. In the example above, that means using a path of foo.coolObject and not somwhere.else.coolObject.