I'm writing a unit test to make sure my class object is created correctly, and this object relies on getting contents from s3. I want to mock out the function that calls s3 inside it completely,:
class SomeClassTest(unittest.TestCase):
#patch('someDir.util._call_to_s3')
def test_someclass_load(self, magic_mock):
magic_mock.return_value = {"host": "bogus.com"}
some_class = SomeClass()
self.assertGreater(len(some_class), 0)
class SomeClass():
def __init__():
try:
content = _call_to_s3(bucket_name, file_name)
except:
rest of code ...
How do I mock out the function _call_to_s3 which is defined in another library file?
When you monkeypatch, you're changing a name so that it points at a different value. You're not changing the value itself. The key is to patch the name that's being used by the unit you are testing.
Every time you do "from foo import bar", you're creating a new, local copy of a name. In this case, it looks like SomeClass is not in the someDir.util module. Let's say it's in someDir.other_mod
someDir.other_mod would be doing something like "from someDir.util import _call_to_s3". This creates a new name someDir.other_mod._call_to_s3. That's the name that SomeClass uses, so that's the name that you need to patch.
e.g. #patch('someDir.other_mod._call_to_s3')
There is no way to patch every name that points at a particular value.
Related
I am trying to patch/mock a method (AsyncHTTPClient.fetch) that is being called in two different places in my program: Firstly in tornado_openapi3.testing and secondly in my_file. The problem is that the method is being patched in the first location, which breaks the functionality of my tests.
my_file.py:
import tornado
class Handler(tornado.web.RequestHandler, ABC):
def initialize(self):
<some_code>
async def get(self, id):
<some_code>
client = AsyncHTTPClient()
response = await client.fetch(<some_path>)
<some_code>
test_handler.py:
from tornado_openapi3.testing import AsyncOpenAPITestCase
class HandlerTestCase(AsyncOpenAPITestCase):
def get_app(self) -> Application:
return <some_app>
def test_my_handler(self):
with patch.object(my_file.AsyncHTTPClient, 'fetch') as mock_fetch:
f = asyncio.Future()
f.set_result('some_result_for_testing')
mock_fetch.return_value = f
self.fetch(<some_path>)
From what I understood from various mocking tutorials (e.g. https://docs.python.org/3/library/unittest.mock.html), fetch should only be patched/mocked in my_file. How can I make sure that is the case?
Cause of the issue
The imported class AsyncHTTPClient in my_file.py is actually just a reference to tornado's original AsyncHTTPClient class.
Basically, from x import y statements are variable assignments in that a new variable called y is created in the current file referencing the original object x.y.
And, since, classes are mutable objects, when you patch the fetch method in the imported class, you are actually patching the fetch method on the original class.
Here's an example using variable assignment to illustrate this issue:
class A:
x = 1
b = A # create a variable 'b' referencing the class 'A'
b.x = 2 # change the value of 'x' attribute' of 'b'
print(A.x)
# Outputs -> 2 (not 1 because classes are mutable)
Like I said earlier, from ... import ... statements are basically variable assignments. So the above illustration is what's really happening when you patch the fetch method.
Solution
Instead of patching a single method, patch the whole class:
with patch.object(my_file, 'AsyncHTTPClient') as mock_client:
f = asyncio.Future()
f.set_result('some_result_for_testing')
mock_client.fetch.return_value = f
self.fetch(<some_path>)
What's happening this time is Python is reassigning the value of the local variable AsyncHTTPClient to a mock object. There's no mutation going on this time and, so, the original class is not affected.
I have a method foo in Python that creates a Service class. I want to mock the Service class but when I run the test, it still attempts to instantiate the class. Here is the simplified version of my setup:
class Service:
def __init__(self, service):
self.service_stuff = service
def run_service(self):
do_service_stuff()
def foo:
new_service = Service("bar")
new_service.run_service()
Then my unit test:
#patch('mymodule.service_file.Service')
def test_foo(self, mock_service):
foo()
I would like to run foo, but have it use my mocked object instead of creating an actual Service instance, but instead, when I run it, it tries to instantiate an actual instance of Service() and runs foo() as usual even though it seems to recognize the string signature I've put into patch. Why is this happening?
Figured it out: The patch reference to the class had to be of its imported name in the method itself rather than the original class, similar to https://stackoverflow.com/a/32461515/4234853
So the patch should look like:
#patch('mymodule.foo_file.Service')
instead of trying to patch the class directly.
In this case, it might be easier to make your function more test-friendly:
def foo(cls=Service):
new_service = cls("bar")
new_service.run_service()
Then your test doesn't need to patch anything.
def test_foo(self):
mock_service = Mock()
foo(mock_service)
I am using Python's mock library along with unittest. I am writing unit tests for a class that uses a function of an external library in one of its methods. Depending on the case, this function returns different values.
So let's say I wanna test class A:
from external_library import function_foo
class A(object):
...
In my test class, in order to use the values returned by the function from the external library, I create a patch, and only import class A after defining the patch. However, I need to use this function in all my test methods, and in each method it returns different values.
My test class is as follows:
class TestA(TestCase):
#patch('external_library.function_foo', side_effect=[1, 2, 3])
def test_1(self, *patches):
from module import class A
obj = A()
...
#patch('external_library.function_foo', side_effect=[1, 1, 2, 2, 3, 3])
def test_2(self, *patches):
from module import class A
obj = A()
...
...
I have 10 tests and only 1 (the first one) passes when I run all of them together, for the rest, I get StopIteration error. However, if I run each one of them individually, they all pass.
I have tried using with patch('external_library.function_foo', side_effect=[...]) in each method, but the outcome was the same. I also tried creating only once the patch in the setUp method, starting it, reassigning the side_effect within each method, and stopping in tearDown, but it didn't work.
Any ideas on what might work in this case?
Thanks!
The caveat is, the second time you import a module, it would not be loaded again, you get the same module object as the first time you imported.
When you first run "test_1", external_library.function_foo replaced by a Mock object, let's name it mock_a. Then your "module" get imported for the first time, python will load it, means, execute code inside "module", which will bind name "function_foo" to object "mock_a" in the namespace of "module", save "module" object to sys.modules. This time your test will pass, and side_effect of mock_a get consumed.
Next is "test_2", external_library.function_foo replaced by a Mock object, name it to mock_b. Then import "module", this time it would not be loaded again, but populate from sys.modules, you get the same module object as in "test_1". In the namespace of this module object, name "function_foo" is still bound to object mock_a, not the newly created mock_b. Because side_effect of mock_a has already consumed, StopIteration error raised.
You should apply patch to where a name is looked up, but not where it is defined:
#patch('module.function_foo', side_effect=[1, 2, 3])
def test_1(self, patch):
...
Read more detail on the "Where to patch" section of the manual of patch.
So I have a python 2.7 project with three modules. Two contain classes and one is a script. An example files structure is shown below
project/
__main__.py
__init__.py
- serial_connect/
ser_conn.py
__init__.py
- ui/
parse_file.py
__init__.py
ser_conn.py contains a class which handles all interaction with some RS-232 hardware device. It contains methods such as open(), close(), send_go() just basically everything required for this connection.
parse_file.py contains a class which has methods relating to parsing a file and getting text commands which are associated with serial commands.
e.g. if the text file contains the command "send_go" parse_file.py will parse this command and call Ser_Conn.send_go()
__main.py__ contains the main() function. in main() an instance of the Ser_Conn class is created and then passed to the Parse_File class as there exists only one instance of a serial connection in the program and it is required by both main() and the Parse_File class.
My question is as follows
In methods of Parse_File I call methods of the Ser_Conn instance, such as Ser_Conn.send_go() but parse_file.py does not complain about there being no
from serial_connect.ser_conn import Ser_Conn
There only exists a ser_conn import in __main__.py. why does this work?
In python, an instance carries all the "stuff" that it needs to do it's work along with it. At a high level, (and somewhat simplified) when you write:
qux.whatever
python looks at the object qux and then finds its whatever attribute (if it has one). Note that python doesn't care what type qux is, only that it has a whatever attribute. I could create multiple objects that satisfy this interface:
class Foo(object):
whatever = 'Hey Dawg!'
class Bar(object):
whatever = 'I satisfy the interface too!'
Also note that these objects could be defined anywhere. As long as you manage to get a reference to the object (e.g. it was passed into a function) you can use the object with all of it's attributes and methods.
maybe a more concrete example would help -- Say you have 4 modules, foo, bar, call_method and driver. foo, bar and call_method don't know anything about each other, but driver imports the other 3. Then you can do something like this:
# foo.py
class Foo(object):
def method(self):
return 'foo!'
# bar.py
class Bar(object):
def method(self):
return 'bar!'
# call_method.py
def go(object_with_method):
print(object_with_method.method())
# driver.py
import call_method
import foo
import bar
call_method.go(Foo()) # call the method on a foo instance
call_method.go(Bar()) # call the method on a bar instance
You're passing the connection instance to parse_file which means Python already knows the class and other details of that object. So that's why you don't need to import the class again in the parse_file code.
You only need to import something if you wish to use that something in that file.
When you created the object Python used the class to construct the object and that's sufficient. You can of course add that import line in parse_file but only if you actually need to use the class in that file (otherwise the import line it's very useful).
I am new to unit testing and mocking in python, in below scenario I know how to mock the get_name() which doesn't have any parameter mock example, but I am not able to mock the below scenario which take age as an argument. Could you please help me to fix mock test_name function below?
# data_source.py
def get_name(a):
return "Alice"+str(a)
def get_age():
return 30
The Person class exposes a method that fetches data from the data source:
# person.py
from data_source import get_name
class Person(object):
def name(self):
age = get_age()
return get_name(age)
from mock import patch
from person import Person
#patch('person.get_name')
def test_name(mock_get_name):
mock_get_name.return_value = "Bob"
person = Person()
name = person.name()
assert name == "Bob"
Thanks in advance!
First of all, apart from importing get_name, you also need to import get_age, since you use it in the Person class. Second, there is nothing like person.get_name defined ever. So you can't patch that. Instead, you probably wanted to patch data_souce.get_name.
But unfortunately, that doesn't work, because data_source is a python module, and get_name is a method within it that you want to redefine. The patch decorator would be good if get_name were a method within a DataSource class, for example. I don't know the exact reason why it doesn't work, because I did not take a deep dive into the implemenation details of the mock library, but the intention of the library is different than how you want to use it.
To solve the problem, you can eiter turn data_source into a class, or you can temporarily change the meaning of the name get_name, by assigning a different function or a lambda to it:
import data_source
def test_person():
# replace the function
data_source._real_get_name = data_source.get_name
data_source.get_name=lamba a: "Bob"
# do some work
person = Person()
name = person.name()
# after use of the get_name function, put the original function back
data_source.get_name=data_source._real_get_name
# assert post condition
assert name=="Bob"
Of course, changing what functions do severely affects your program and should be done with care, probably only for testing. Make sure you always put the original function back, even if an error is thrown unexpectedly.