Develop Reference

Mock function call inside init method Python

I'm writing a unit test to make sure my class object is created correctly, and this object relies on getting contents from s3. I want to mock out the function that calls s3 inside it completely,:
class SomeClassTest(unittest.TestCase):
#patch('someDir.util._call_to_s3')
def test_someclass_load(self, magic_mock):
magic_mock.return_value = {"host": "bogus.com"}
some_class = SomeClass()
self.assertGreater(len(some_class), 0)
class SomeClass():
def __init__():
try:
content = _call_to_s3(bucket_name, file_name)
except:
rest of code ...
How do I mock out the function _call_to_s3 which is defined in another library file?

When you monkeypatch, you're changing a name so that it points at a different value. You're not changing the value itself. The key is to patch the name that's being used by the unit you are testing.
Every time you do "from foo import bar", you're creating a new, local copy of a name. In this case, it looks like SomeClass is not in the someDir.util module. Let's say it's in someDir.other_mod
someDir.other_mod would be doing something like "from someDir.util import _call_to_s3". This creates a new name someDir.other_mod._call_to_s3. That's the name that SomeClass uses, so that's the name that you need to patch.
e.g. #patch('someDir.other_mod._call_to_s3')
There is no way to patch every name that points at a particular value.

Python: mocks in unittests

I have situation similar to:
class BaseClient(object):
def __init__(self, api_key):
self.api_key = api_key
# Doing some staff.
class ConcreteClient(BaseClient):
def get_some_basic_data(self):
# Doing something.
def calculate(self):
# some staff here
self.get_some_basic_data(param)
# some calculations
Then I want to test calculate function using mocking of get_some_basic_data function.
I'm doing something like this:
import unittest
from my_module import ConcreteClient
def my_fake_data(param):
return [{"key1": "val1"}, {"key2": "val2"}]
class ConcreteClientTest(unittest.TestCase):
def setUp(self):
self.client = Mock(ConcreteClient)
def test_calculate(self):
patch.object(ConcreteClient, 'get_some_basic_data',
return_value=my_fake_data).start()
result = self.client.calculate(42)
But it doesn't work as I expect.. As I thought, self.get_some_basic_data(param) returns my list from my_fake_data function, but it looks like it's still an Mock object, which is not expected for me.
What is wrong here?

There are two main problems that you are facing here. The primary issue that is raising the current problem you are experiencing is because of how you are actually mocking. Now, since you are actually patching the object for ConcreteClient, you want to make sure that you are still using the real ConcreteClient but mocking the attributes of the instance that you want to mock when testing. You can actually see this illustration in the documentation. Unfortunately there is no explicit anchor for the exact line, but if you follow this link:
https://docs.python.org/3/library/unittest.mock-examples.html
The section that states:
Where you use patch() to create a mock for you, you can get a
reference to the mock using the “as” form of the with statement:
The code in reference is:
class ProductionClass:
def method(self):
pass
with patch.object(ProductionClass, 'method') as mock_method:
mock_method.return_value = None
real = ProductionClass()
real.method(1, 2, 3)
mock_method.assert_called_with(1, 2, 3)
The critical item to notice here is how the everything is being called. Notice that the real instance of the class is created. In your example, when you are doing this:
self.client = Mock(ConcreteClient)
You are creating a Mock object that is specced on ConcreteClient. So, ultimately this is just a Mock object that holds the attributes for your ConcreteClient. You will not actually be holding the real instance of ConcreteClient.
To solve this problem. simply create a real instance after you patch your object. Also, to make your life easier so you don't have to manually start/stop your patch.object, use the context manager, it will save you a lot of hassle.
Finally, your second problem, is your return_value. Your return_value is actually returning the uncalled my_fake_data function. You actually want the data itself, so it needs to be the return of that function. You could just put the data itself as your return_value.
With these two corrections in mind, your test should now just look like this:
class ConcreteClientTest(unittest.TestCase):
def test_calculate(self):
with patch.object(ConcreteClient, 'get_some_basic_data',
return_value=[{"key1": "val1"}, {"key2": "val2"}]):
concrete_client = ConcreteClient(Mock())
result = concrete_client.calculate()
self.assertEqual(
result,
[{"key1": "val1"}, {"key2": "val2"}]
)
I took the liberty of actually returning the result of get_some_basic_data in calculate just to have something to compare to. I'm not sure what your real code looks like. But, ultimately, the structure of your test in how you should be doing this, is illustrated above.

Test if some field has been initialized in python

I am trying to write a test in Python that checks if a method in a class that I am writing sets the attribute value for a dataset in some Hdf file. The logic is the following: An instance of the class is constructed by passing an instance of h5py.File, then one method creates a dataset inside this file. In the next step I have another method that sets certain attributes for this dataset.
What I am trying to test is if my class method create_attributes(self,attributes) sets the field hdf_file[dset_name].attrs[attr_name] to some value that is passed in the variable attributes. However, I would like to avoid to actually create a Hdf file. So far I have tried to mock an instance of a hdf file and work with that. The minimal working code example would be the following:
import h5py
class TestSomething:
#mock.patch('h5py.File')
def test_if_attr_is_initialized(self,mock_hdf):
# Here I would like to call a function that basically executes
# the following line:
mock_hdf['test_dset'].attrs['test_field']='value'
# Then I want to check if the attribute field has been assigned
assert mock_hdf['test_dset'].attrs['test_field']=='value'
Can anybody help me finding the correct thing to do to check whether or not the attribute in the hdf file is set correctly? Any help would be greatly appreciated, I am a complete newbie to all the mocking techniques.
Edit:
In the following I am providing a minimal code example for both the class, and the respective test as requeseted by wwii:
import h5py
class HdfWriter():
def __init__(self,hdf_file):
self.hdf_file=hdf_file
def create_attrs(self,attributes):
dset_name=attributes.keys()[0]
attrs=attributes[dset_name]
for key in attrs:
self.hdf_file[dset_name].attrs[key]=attrs[key]
Please note here that with a real hdf file I would first have to create a dataset but I would like to leave that for another test. The following test should just check, whether for a hypothetical hdf file, which has the dataset test_dset the attributes for this data set are written:
import h5py
import HdfWriter
class TestSomething:
#mock.patch('h5py.File')
def test_if_attr_is_initialized(self,mock_hdf):
writer=hw.HdfWriter(mock_hdf)
attr={'test_dset':{'test_field':'test_value'}}
writer.create_attrs(attr)
assert writer.hdf_file['test_dset'].attrs['test_field']=='value'

Mocking h5py.File
class HdfWriter():
def __init__(self,hdf_file):
self.hdf_file=hdf_file
def create_attrs(self,attributes):
dset_name=attributes.keys()[0]
attrs=attributes[dset_name]
for key in attrs:
self.hdf_file[dset_name].attrs[key]=attrs[key]
For the purpose of the create_attrs method, hdf_file behaves as a dictionary that returns an object that also behaves like a dictionary. The docs explain pretty clearly how to mock a dictionary.
You need a mock that has an attrs attribute that behaves like a dictionary:
import mock
attrs_d = {}
def setattrs(name, value):
## print 'setattrs', name, value
attrs_d[name] = value
def getattrs(name):
## print 'getattrs', name
return attrs_d[name]
mock = mock.MagicMock()
mock.attrs.__setitem__.side_effect = setattrs
mock.attrs.__getitem__.side_effect = getattrs
You need a mock for hdf_file that behaves like a dictionary and will return the mock object created above.
hdf_d = {'test_dset':mock}
def getitem(name):
## print 'getitem', name
return hdf_d[name]
def setitem(name, value):
hdf_d[name] = value
mock_hdf = mock.MagicMock()
mock_hdf.__getitem__.side_effect = getitem
mock_hdf.__setitem__.side_effect = setitem
hdf_d, as implemented, only works for the key 'test_dset'. Depending on your needs it may be better for getitems to just return mock regardless of the name argument.
def test_if_attr_is_initialized(mock_hdf):
writer=HdfWriter(mock_hdf)
attr={'test_dset':{'test_field':'test_value'}}
writer.create_attrs(attr)
print writer.hdf_file['test_dset'].attrs['test_field'], '==', attr['test_dset']['test_field']
assert writer.hdf_file['test_dset'].attrs['test_field']=='test_value'
test_if_attr_is_initialized(mock_hdf)
>>>
test_value == test_value
>>>
This should suffice to test create_attrs but it may not be optimal - maybe someone will chime in with some refinements.

How to mock multiple functions in python?

I am new to unit testing and mocking in python, in below scenario I know how to mock the get_name() which doesn't have any parameter mock example, but I am not able to mock the below scenario which take age as an argument. Could you please help me to fix mock test_name function below?
# data_source.py
def get_name(a):
return "Alice"+str(a)
def get_age():
return 30
The Person class exposes a method that fetches data from the data source:
# person.py
from data_source import get_name
class Person(object):
def name(self):
age = get_age()
return get_name(age)
from mock import patch
from person import Person
#patch('person.get_name')
def test_name(mock_get_name):
mock_get_name.return_value = "Bob"
person = Person()
name = person.name()
assert name == "Bob"
Thanks in advance!

First of all, apart from importing get_name, you also need to import get_age, since you use it in the Person class. Second, there is nothing like person.get_name defined ever. So you can't patch that. Instead, you probably wanted to patch data_souce.get_name.
But unfortunately, that doesn't work, because data_source is a python module, and get_name is a method within it that you want to redefine. The patch decorator would be good if get_name were a method within a DataSource class, for example. I don't know the exact reason why it doesn't work, because I did not take a deep dive into the implemenation details of the mock library, but the intention of the library is different than how you want to use it.
To solve the problem, you can eiter turn data_source into a class, or you can temporarily change the meaning of the name get_name, by assigning a different function or a lambda to it:
import data_source
def test_person():
# replace the function
data_source._real_get_name = data_source.get_name
data_source.get_name=lamba a: "Bob"
# do some work
person = Person()
name = person.name()
# after use of the get_name function, put the original function back
data_source.get_name=data_source._real_get_name
# assert post condition
assert name=="Bob"
Of course, changing what functions do severely affects your program and should be done with care, probably only for testing. Make sure you always put the original function back, even if an error is thrown unexpectedly.

Defining Python Functions in Google App Engine

I am new to python and am trying to define a function and then use it in Google App Engine - but I keep getting the error "Error: global name 'cache_email_received_list' is not defined" when I try to execute the function. Any help would be greatly appreciated, thanks.
Here is my function:
class EmailMessageHandler(BaseHandler2):
def cache_email_sent_list(): #set email_sent_list to memcache
email_sent_list = db.GqlQuery("SELECT * FROM EmailMessage WHERE sender =:1 ORDER BY created DESC", user_info.username)
if email_sent_list:
string1 = "email_sent_list"
email_sent_list_cache_id = "_".join((user_info.username, string1))
memcache.set('%s' % email_sent_list_cache_id, email_sent_list, time=2000000)
logging.info('**************email_sent_list added to memcache*********')
Here is where I am trying to call it:
if email_received_list is None and email_sent_list is not None:
params = {
'email_sent_list': email_sent_list,
}
cache_email_sent_list()

cache_email_sent_list() is a method of the class EmailMessageHandler therfore the method needs to pass in self a a parameter it will therefore look like this:
class EmailMessageHandler(BaseHandler2):
def cache_email_sent_list(self): #set email_sent_list to memcache
email_sent_list = db.GqlQuery("SELECT * FROM EmailMessage WHERE sender =:1 ORDER BY created DESC", user_info.username)
if email_sent_list:
string1 = "email_sent_list"
email_sent_list_cache_id = "_".join((user_info.username, string1))
memcache.set('%s' % email_sent_list_cache_id, email_sent_list, time=2000000)
logging.info('**************email_sent_list added to memcache*********')
Then when you call it from within the class EmailMessageHandler you have to do it like this:
self.cache_email_sent_list()
If however you are calling it from outside the class EmailMessageHandler you need to first create an instance and then call it using:
instanceName.cache_email_sent_list()

Just as an addition to the previous answers: In your post you define cache_email_sent_list() as a function defined in a class definition, which will not work. I think you are confusing instance methods, static methods and functions. There's a prominent difference between these three.
So, as a stylised example:
# instance method:
class MyClass(MySuperClass):
def my_instance_method(self):
#your code here
# call the instance method:
instance = MyClass() # creates a new instance
instance.my_instance_method() # calls the method on the instance
# static method:
class MyClass(MySuperClass):
#staticmethod # use decorator to nominate a static method
def my_static_method()
#your code here
# call the static method:
MyClass.my_static_method() # calls the static method
# function
def my_function():
# your code here
# call the function:
my_function() # calls your function
Indentation is part of Python syntax and determines how the interpreter handles your code. It takes a bit getting used to but once you've got the hang of it, it's actually really handy and makes your code very readable. I think you have an indentation error in your original post. Just add the correct indentation for the method cache_email_sent_list() and call it on an instance of EmailMessageHandler and you're good to go.

The problem has nothing to do with GAE.
The problem is that you've defined cache_email_sent_list as a method of the class EmailMessageHandler, but you're trying to call it as a top-level function. You can't do that. You need to have an instance of a EmailMessageHandler to call it on.
If you're trying to call it from another method of EmailMessageHandler, that instance should be available as self. For example:
self.cache_email_sent_list()
If you're trying to call it from elsewhere, it's up to you to figure out what instance you should be calling it on. For example:
handler_passed_as_param_to_this_function.cache_email_sent_list()
Note that your error message is about cache_email_received_list, but your code only has cache_email_sent_list. I'm guessing that you have parallel code, and the exact same error for both cases, but of course I could be guessing wrong—in which case you'll have to actually show us either the code that goes with your displayed error, or the error that goes with your displayed code…