I have a function func1() that is in production and cannot be modified. It calls a function ,function_to_be_mocked(), in another module. This takes input parameters.
I have another function func2() which calls func1().
I am writing unit tests to test func2(), and trying to mock function_to_be_mocked (as it depends on some keys I don't have (and should't have) on my local system). The only thing I can modify is test_func2().
I have a set up like the following (minimum example):
from othermodule import function_to_be_mocked
import pytest
import mock
def func1():
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
#mock.patch('othermodule.function_to_be_mocked', return_value = 3)
def test_func2(mocker):
func2()
And othermodule.py is:
def function_to_be_mocked(arg1):
if not arg1 == 'foo':
raise ValueError
My output:
Calling func2 directly:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/blah/temp.py", line 9, in func2
ret = func1()
File "/Users/blah/temp.py", line 6, in func1
function_to_be_mocked(None)
File "/Users/blah/othermodule.py", line 3, in function_to_be_mocked
raise ValueError
ValueError
Calling test_func2() which I would expect to be mocked:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/blah/venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/Users/blah/temp.py", line 14, in test_func2
func2()
File "/Users/blah/temp.py", line 9, in func2
ret = func1()
File "/Users/blah/temp.py", line 6, in func1
function_to_be_mocked(None)
File "/Users/blah/othermodule.py", line 3, in function_to_be_mocked
raise ValueError
ValueError
So the mock doesn't seem to be working. Does anyone have any thoughts how to achieve this?
============ Edited below this line ===========
It doesn't sound like I can do what I thought I could (as I cannot modify anything related to function 1, or 2 in fact. All I have control over is the test.
Let me pose the following problem then as perhaps more experienced eyes than mine can see a way forward.
I have a function:
def function_to_be_tested(args):
# Some processing steps
# Function call that works locally
function_that_returns_something_1()
# Some logic
# Function call that works locally
function_that_returns_something_2()
# Function that raises an exception when running locally,
# and since I need to test the logic after this function
# (and cannot edit this code here to bypass it) I would
# (naively) like to mock it.
function_I_would_like_to_mock()
# Much more logic that follows this function.
# And this logic needs to be unit tested.
return some_value_based_on_the_logic
Tests:
def test_function_to_be_tested():
assert function_to_be_tested(args) == some_expected_value
I can easily unit test anything before function_I_would_like_to_mock().
But since this function crashes locally (and I cannot edit the code to stop it crashing locally), I feel like the correct approach would be to mock it and force a sensible return value. So that I can unit tests the code paths beyond this.
What would you suggest as a good approach?
Please note, the only thing I can modify is the test function. I can't add even decorators to the main functions.
Option A)
The function you are willing to mock is loaded into func1. Therefore you have to apply the #patch decorator to func1
import pytest
from unittest import mock
#mock.patch('othermodule.function_to_be_mocked', return_value = 3)
def func1(mocker):
from othermodule import function_to_be_mocked
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
def test_func2():
func2()
test_func2()
=========Edit===========
Option B)
import pytest
from unittest import mock
def func1():
from othermodule import function_to_be_mocked
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
def test_func2():
with mock.patch('othermodule.function_to_be_mocked', return_value = 3) as irrelevant:
func2()
test_func2()
The Where to patch section of the official "unittest.mock — mock object library" documentation explains this quite clearly:
a.py
-> Defines SomeClass
b.py
-> from a import SomeClass
-> some_function instantiates SomeClass
Now we want to test some_function but we want to mock out SomeClass using patch(). The problem is that when we import module b, which we will have to do then it imports SomeClass from module a. If we use patch() to mock out a.SomeClass then it will have no effect on our test; module b already has a reference to the real SomeClass and it looks like our patching had no effect.
The key is to patch out SomeClass where it is used (or where it is looked up ). In this case some_function will actually look up SomeClass in module b, where we have imported it. The patching should look like:
#patch('b.SomeClass')
So I think in your case the patching should look like:
#patch("module_of_func2.function_to_be_mocked_as_it_is_imported_there", return_value=3)
def test_func2():
...
If suppose you want to mock a function from module inside function in another module in python you can try this.
# application2.py
def app2_func(a):
print(a)
# application1.py
import application2
def app1_func(a):
application2.app2_func(a) # func to be mocked
the test file to test the function
# application_test.py
import application1
def test_app1_func(mocker):
app2_mocker = mocker.patch('application1.application2.app2_func')
application1.app1_func('mock call')
app2_mocker.assert_called_once() # to check if mocked function is called once
Related
I have a file named file.py containing the following script:
def b():
print("b")
def proc():
print("this is main")
b()
proc()
And I have another file named caller.py, which contains this script:
text = open('file.py').read()
exec(text)
When I run it, I get the expected output:
this is main
b
However, if I change caller.py to this:
def main():
text = open('file.py').read()
exec(text)
main()
I get the following error:
this is main
Traceback (most recent call last):
File "./caller.py", line 7, in <module>
main()
File "./caller.py", line 5, in main
exec(text)
File "<string>", line 10, in <module>
File "<string>", line 8, in main
NameError: global name 'b' is not defined
How is function b() getting lost? It looks to me like I'm not violating any scope rules. I need to make something similar to the second version of caller.py work.
exec(text) executes text in the current scope, but modifying that scope (as def b does via the implied assignment) is undefined.
The fix is simple:
def main():
text = open('file.py').read()
exec(text, {})
This causes text to run in an empty global scope (augmented with the default __builtins object), the same way as in a regular Python file.
For details, see the exec documentation. It also warns that modifying the default local scope (which is implied when not specifying any arguments besides text) is unsound:
The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.
Would it work for you if you imported and called the function instead?
myfile.py
def b():
print("b")
def proc():
print("this is main")
b()
caller.py
import myfile
myfile.proc()
Not wanting to test manually my code, I'm trying to write a test which mocks/patches one of my dependencies (PyInquirer is a pretty neat package which handles the CLI for me - question dicts in, answer dicts out).
However, being very new to Python, I'm having difficulties with mocking that dependency. Here's the code I'm testing:
from PyInquirer import prompt
class Foo:
def bar(self):
# this line is asking the user for inpit, and that's what I want to mock.
a = prompt({'name': 'q',
'type': 'input',
'message': 'well, foo'})
print("f is", f)
return a
And this is the test:
import unittest
from unittest.mock import patch
from Foo import Foo
class TestFoo(unittest.TestCase):
#patch('PyInquirer.prompt', return_value=24)
def test_bar(self):
f = Foo()
a = f.bar()
assert a == 24
if __name__ == '__main__':
unittest.main()
(the real code is obviously more complicated, but this is the essence of the problem). Which manifests itself as:
Error
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 605, in run
testMethod()
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/mock.py", line 1179, in patched
return func(*args, **keywargs)
TypeError: test_bar() takes 1 positional argument but 2 were given
I'm quite confused.
If I omit the patch decorator, the invocation fails with an expected assertion error - the dictionary produced by prompt isn't equal to 24. But if I do provide the decorator, I get the argument mismatch above. And indeed the last line in the stacktrace does show the function "func", which I presume is what the decorator was applied to, is invoked with two arguments... but... why? Can't be the essence of a problem? That only functions with arity of two can be thus patched =)
Your class uses the name Foo.prompt (because of how you import it), so that's what you need to patch.
class TestFoo(unittest.TestCase):
#patch('Foo.prompt', return_value=24)
def test_bar(self, mock_prompt):
f = Foo()
a = f.bar()
assert a == 24
You also need to add a parameter to test_bar to receive the patched object, whether or not you plan to use it. If you don't want to do that,
you can move the call to patch inside the method, using it with a with statement.
class TestFoo(unittest.TestCase):
def test_bar(self):
with patch('Foo.prompt', return_value=24):
f = Foo()
a = f.bar()
assert a == 24
When I use HTMLTestRunner for Python 3.5，it shows an error.
I have changed the HTMLTestRunner for support python 3.5.
The code :
import pymysql
import pymysql
import unittest
import time
import unittest.suite
import HTMLTestRunner
import sys
def hell(a):
print(a)
return a
testunit = unittest.TestSuite()
testunit.addTest(hell('ad'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'print', description=u'简单')
runner.run(testunit)
When I run it, I got this error:
Traceback (most recent call last):
File "/Applications/Python 3.5/……/B.py", line 30, in <module>
testunit.addTest(hell('ad'))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py", line 47, in addTest
raise TypeError("{} is not callable".format(repr(test)))
TypeError: 'ad' is not callable
What should I do to make the script works?
You're adding the result of the call to hell('ad') to your tests, not the hell function itself. Since the hell function returns its argument, it returns the string ad, which is not a callable (function or the like).
Use
testunit.addTest(hell)
instead.
What about that argument then, how do you pass that?
Well, there are ways to do that, but generally, try not to let your unit test functions take an argument. Thus, hell() should siply not take an argument.
If you code your unit test correctly, you'll find that you rarely need to pass it an argument.
The line testunit.addTest(hell('ad')) doesn't do what you intend it to do. It doesn't tell the test suite to run hell('ad') later on. Rather, it calls that function immediately and passed the return value (which happens to be the string 'ad' you gave it as an argument) to addTest. That causes the exception later on, since a string is not a valid test case.
I'm not exactly sure how you should fix this. Your hell function doesn't appear to actually test anything, so there's not an obvious way to transform it into a proper test. There is a unittest.FunctionTestCase class that wraps a function up as a TestCase, but it doesn't appear to have any way of passing arguments to the function. Probably you should write a TestCase subclass class, and add various test_-prefixed methods that actually test things.
I had a right answer for my question, give the code:
# -*- coding: utf-8 -*-
import unittest
import HTMLTestRunner
def hell(a):
print(a)
return a
class HellTest(unittest.TestCase):
def setUp(self):
self.hell = hell
def tearDown(self):
pass
def testHell(self):
self.assertEqual(self.hell('aaa'), 'aaa')
if __name__ == '__main__':
testunit = unittest.TestSuite()
testunit.addTest(HellTest('testHell'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'不要生成error啦！', description=u'简单1112')
runner.run(testunit)
fp.close()
But, I do not know why add the class code 'class HellTest()'.The answer comes from a Chinese people whoes name is '幽谷奇峰'。Source code reference：https://segmentfault.com/q/1010000007427143?_ea=1345414
I have code for cassandra python driver.
from cassandra.cqlengine.management import sync_table
def sync_my_tables():
print sync_table
print "*" * 80
sync_table(my_models.student)
When I try to write UT for this, i mocked sync_table using #patch.
from unittest import TestCase
from mock import patch
class TestCassandraSetup(TestCase):
#patch('cassandra.cqlengine.management.sync_table', return_value=True)
def test_sync_my_tables(self, _):
from cassandra.cqlengine.management import sync_table
print "*"*80
print sync_table
print "*"*80
cass_setup.sync_my_tables()
After patch, it call actual function and give error.
Traceback (most recent call last):
File "/venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/venv/tests/test_cassandra_setup.py", line 26, in test_sync_database_tables
cassandra_client.sync_database_tables()
File "/venv/code/cass_setup.py", line 18, in sync_my_tables
sync_table(my_tables.student)
File "/venv/lib/python2.7/site-packages/cassandra/cqlengine/management.py", line 200, in sync_table
cluster = get_cluster()
File "/venv/lib/python2.7/site-packages/cassandra/cqlengine/connection.py", line 182, in get_cluster
raise CQLEngineException("%s.cluster is not configured. Call one of the setup or default functions first." % __name__)
CQLEngineException: cassandra.cqlengine.connection.cluster is not configured. Call one of the setup or default functions first.
-------------------- >> begin captured stdout << ---------------------
********************************************************************************
<MagicMock name='sync_table' id='4490003152'>
********************************************************************************
<function sync_table at 0x10b8075f0>
********************************************************************************
In print statement, it print MagicMock first time, but when print same in actual code, it print actual function not mocked object.
Whey it change in between ?
When using patch you need to patch the object where it is used. So if sync_my_tables is located in a file with a path of foo/bar/baz.py you will need to call patch like this:
#patch('foo.bar.baz.sync_table')
def test_sync_table(self, _):
# test code
Python cannot mock (replace) things that are already in the module scope. Once imported, you cannot change it from the outside. To make the code testable, you need to import the higher level module and use the method from there as then it becomes just a property that can be changed.
from cassandra.cqlengine import management
def sync_my_tables():
print management.sync_table
print "*" * 80
management.sync_table(my_models.student)
This way mock is capable of replacing function at runtime (just does management.sync_table = MagicMock()).
In your test function, you are doing the import after the function has been replaced so it works as expected.
I am attempting to use multiprocessing to call derived class member function defined in a different module. There seem to be several questions dealing with calling class methods from the same module, but none from different modules. For example, if I have the following structure:
main.py
multi/
__init__.py (empty)
base.py
derived.py
main.py
from multi.derived import derived
from multi.base import base
if __name__ == '__main__':
base().multiFunction()
derived().multiFunction()
base.py
import multiprocessing;
# The following two functions wrap calling a class method
def wrapPoolMapArgs(classInstance, functionName, argumentLists):
className = classInstance.__class__.__name__
return zip([className] * len(argumentLists), [functionName] * len(argumentLists), [classInstance] * len(argumentLists), argumentLists)
def executeWrappedPoolMap(args, **kwargs):
classType = eval(args[0])
funcType = getattr(classType, args[1])
funcType(args[2], args[3:], **kwargs)
class base:
def multiFunction(self):
mppool = multiprocessing.Pool()
mppool.map(executeWrappedPoolMap, wrapPoolMapArgs(self, 'method', range(3)))
def method(self,args):
print "base.method: " + args.__str__()
derived.py
from base import base
class derived(base):
def method(self,args):
print "derived.method: " + args.__str__()
Output
base.method: (0,)
base.method: (1,)
base.method: (2,)
Traceback (most recent call last):
File "e:\temp\main.py", line 6, in <module>
derived().multiFunction()
File "e:\temp\multi\base.py", line 15, in multiFunction
mppool.map(executeWrappedPoolMap, wrapPoolMapArgs(self, 'method', range(3)))
File "C:\Program Files\Python27\lib\multiprocessing\pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "C:\Program Files\Python27\lib\multiprocessing\pool.py", line 567, in get
raise self._value
NameError: name 'derived' is not defined
I have tried fully qualifying the class name in the wrapPoolMethodArgs method, but that just gives the same error, saying multi is not defined.
Is there someway to achieve this, or must I restructure to have all classes in the same package if I want to use multiprocessing with inheritance?
This is almost certainly caused by the ridiculous eval based approach to dynamically invoking specific code.
In executeWrappedPoolMap (in base.py), you convert a str name of a class to the class itself with classType = eval(args[0]). But eval is executed in the scope of executeWrappedPoolMap, which is in base.py, and can't find derived (because the name doesn't exist in base.py).
Stop passing the name, and pass the class object itself, passing classInstance.__class__ instead of classInstance.__class__.__name__; multiprocessing will pickle it for you, and you can use it directly on the other end, instead of using eval (which is nearly always wrong; it's code smell of the strongest sort).
BTW, the reason the traceback isn't super helpful is that the exception is raised in the worker, caught, pickle-ed, and sent back to the main process and re-raise-ed. The traceback you see is from that re-raise, not where the NameError actually occurred (which was in the eval line).