I'm having a hell of a time getting pytest, relative import, and patch to cooperate.
I have the following:
from .. import cleanup
class CleanupTest(unittest.TestCase):
#patch.object(cleanup, 'get_s3_url_components', MagicMock())
#patch.object(cleanup, 'get_db_session', MagicMock(return_value={'bucket', 'key'}))
#patch('time.time')
def test_cleanup(self, mock_time, mock_session, mock_components):
...
I've tried a few variations.
The currently shown one. Pytest returns 'TypeError: test_cleanup() takes exactly 4 arguments (2 given)'. It's not recognizing the patch.objects, even though I'm pretty sure I'm using them correctly according to https://docs.python.org/3/library/unittest.mock-examples.html
Changing the patch.objects over to simple patches causes them to throw 'no module named cleanup'. I can't figure out how to use patch with a relative import.
Edit: I'm using Python 2.7 in case that's relevant.
this actually doesn't have anything to do with pytest and is merely the behaviour of the mock decorators
From the docs
If patch() is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function
that is, because you're passing a replacement object the signature-mutation doesn't occur
For example:
from unittest import mock
class x:
def y(self):
return 'q'
#mock.patch.object(x, 'y', mock.Mock(return_value='z'))
def t(x): ...
t() # TypeError: t() missing 1 required positional argument: 'x'
Fortunately, if you're producing a mock object, you rarely need to actually pass in the new argument and can use the keyword options
For your particular example this should work fine:
#patch.object(cleanup, 'get_s3_url_components')
#patch.object(cleanup, 'get_db_session', return_value={'bucket', 'key'})
#patch('time.time')
def test_cleanup(self, mock_time, mock_session, mock_components): ...
If you absolutely need them to be MagicMocks, you can use the new_callable=MagicMock keyword argument
Related
Introduction:
We know that, in python, a function is a class. To some extent, We can look at it as a data type which can be called and return a value. So it is a callable. We also know that Python classes are callables. When they are called, we are actually making objects as their instances.
My implementation: In a current task, I have defined the following class with two methods:
class SomeClass():
def some_method_1():
some_code
def some_method_2():
some_code
self.some_method_1()
some_code
To describe the code above, some_method_2 is using some_method_1 inside it.
Now I am seeking to test some_method_2. In this case, I need to replace some_method_1 with a mock object and specify the mock object to return what I define:
from unittest.mock import Mock
import unittest
class TestSomeClass(unittest.TestCase):
some_object=Some_Class()
some_object.some_method_1 = Mock(return_value=foo)
self.assertEqual(an_expected_value, some_object.some_method_2())
This works totally fine and the script runs without error and the test result is also ok.
A fast Intro about Mypy: Mypy is a python type checking tool which can run and check if the script is written correctly based on variable types. It does this process by some criteria such as type annotations, variable assignments and libraries stub files. This process is done without interpreting and running the code.
What is the Problem?
When I try to check my code with mypy, it gives error for this line:
some_object.some_method_1 = Mock(return_value=foo)
The error indicates that I am not allowed to assign an object to a callable in python. Sometimes mypy does not report real errors and I doubt if this is the case. Especially because I can run my code with no problem.
Now, my question is, have I done my job wrong or just the mypy report is wrong? If I have done wrong, how can I implement the same scenario in a correct manner?
Background
I have two modules. Module 1 (example) defines TestClass. Module 2 (example2) defines functions change_var which takes an argument TestClass. example has a method change which calls change_var from example2 and passes self as argument.
example2 uses TYPE_CHECKING from typing to ensure cyclic import does not appear at run-time, but still allows MYPY to check types.
At the call to change_var from within change, MYPY gives the error Argument 1 to "change_var" has incompatible type "__main__.TestClass"; expected "example.TestClass".
Python Version: 3.7.3,
MYPY Version: 0.701
Example Code
example.py
from example2 import change_var
class TestClass:
def __init__(self) -> None:
self.test_var = 1
def change(self) -> None:
change_var(self)
example2.py
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from example import TestClass
def change_var(obj: "TestClass") -> None:
obj.test_var = 2
This code is a minimal example of the actual problem I am experiencing in a larger python project.
What I Expect this to do
These types should match as they are (as far as I can tell) the same.
My intuition as to why this doesn't work is that TestClass, at the point of calling to change_var isn't fully defined? For the same reason I can't refer to TestClass as a type within TestClass itself, I can't pass a TestClass object to a function that expects a TestClass object from withing the class itself. To MYPY, this is not a full class yet so it uses some kind of placeholder type. This is only an intuition though.
Questions
What exactly is the problem here?
What is the best work around to achieve this general code structure (two modules, class in one, function that takes class in other, method calls to function) while still making MYPY happy?
I am also open to refactoring this example entirely but I'd like to try to stick to this general structure.
This is one of many cases of breakage from treating a module as a script (whether via -m, -c (or --command for mypy), or simply python …/module.py). It works only for trivial applications that do not care about the identity of the types or functions they create. (They must also avoid side effects on import and mutable global state, but those are good ideas anyway.)
The solution, beyond “don’t do that”, is to use __main__.py in a package. Even that is not without issues, since certain naïve recursive importers will import it as if it were a true module.
I am attempting to mock a method and running into issues with mock actually overwriting it.
app/tests/test_file.py <- contains the the unit test, currently using:
#mock.patch('app.method', return_value='foo')
def test(self, thing):
...
do some stuff with app/main/server.py
and get its response, assert a few values
...
assert 'foo' is in return value of some stuff
The method being mocked is being called by another file that server.py is calling.
app/main/server.py <- what the unit test is actually interacting with
app/main/route.py <- where method being mocked is called
app/main/thing.py <- contains method to be mocked
This is with python 2.7 and each package has an init file. The parent folder (app) contains imports for every class and method. I've tried app.method which doesn't give problems, but doesnt work. I've tried thing.method, throws an error. I've tried app.main.thing.method which does nothing.
I've had success in this same test suite mocking an object and one of its methods, but that object is created and used directly in the server.py file. I'm wondering if it's because the method being called is so far down the chain. Mocking is pretty magical to me, especially in Python.
After more digging finally figured it out, will leave it up for any others that have problems (especially since it's not easily Google'able).
As #Gang specified the full path should work, however the module needs to be the module where the method is called, not the module where it is located, as this person points out. . Using #Gang example:
#mock.patch('app.main.route.method')
def test(self, mock_method):
mock_method.return_value = 'foo'
# call and assert
So let's say I have this bit of code:
import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
Now it's a simple enough method, and as you can see we are using an external library(coolObject).
In unit tests, I have to create a mock of this object that roughly replicates it. Let's call this mock object coolMock.
My question is how would I tell the code when to use coolMock or coolObject? I've looked it up online, and a few people have suggested dependency injection, but I'm not sure I understand it correctly.
Thanks in advance!
def doSomething(cool_object=None):
cool_object = cool_object or coolObject()
...
In you test:
def test_do_something(self):
cool_mock = mock.create_autospec(coolObject, ...)
cool_mock.coolOperation.side_effect = ...
doSomthing(cool_object=cool_mock)
...
self.assertEqual(cool_mock.coolOperation.call_count, ...)
As Dan's answer says, one option is to use dependency injection: have the function accept an optional argument, if it's not passed in use the default class, so that a test can pass in a moc.
Another option is to use the mock library (here or here) to replace your coolObject.
Let's say you have a foo.py that looks like
from somewhere.else import coolObject
def doSomething():
x = coolObject()
x.coolOperation()
In your test_foo.py you can do:
import mock
def test_thing():
path = 'foo.coolObject' # The fully-qualified path to the module, class, function, whatever you want to mock.
with mock.patch('foo.coolObject') as m:
doSomething()
# Whatever you want to assert here.
assert m.called
The path you use can include properties on objects, e.g. module1.module2.MyClass.my_class_method. A big gotcha is that you need to mock the object in the module being tested, not where it is defined. In the example above, that means using a path of foo.coolObject and not somwhere.else.coolObject.
I have a test as follows:
import mock
# other test code, test suite class declaration here
#mock.patch("other_file.another_method")
#mock.patch("other_file.open", new=mock.mock_open(read=["First line", "Second line"])
def test_file_open_and_read(self, mock_open_method, mock_another_method):
self.assertTrue(True) # Various assertions.
I'm getting the following error:
TypeError: test_file_open_and_read() takes exactly 3 arguments (2 given)
I'm trying to specify that I want the other file's __builtin__.open method to be mocked with mock.mock_open rather than mock.MagicMock which is the default behavior for the patch decorator. How can I do this?
Should use new_callable instead of new. That is,
#mock.patch("other_file.open", new_callable=mock.mock_open)
def test_file_open_and_read(self, mock_open_method):
# assert on the number of times open().write was called.
self.assertEqual(mock_open_method().write.call_count,
num_write_was_called)
Note that we are passing the function handle mock.mock_open to new_callable, not the resulting object. This allows us to do mock_open_method().write to access the write function, just like the example in the documentation of mock_open shows.
You missed the parameter create from open builtin.
#mock.patch("other_file.open", new=mock.mock_open(read=["First line", "Second line"], create=True)