I don't know why I'm just not getting this, but I want to use mock in Python to test that my functions are calling functions in ftplib.FTP correctly. I've simplified everything down and still am not wrapping my head around how it works. Here is a simple example:
import unittest
import ftplib
from unittest.mock import patch
def download_file(hostname, file_path, file_name):
ftp = ftplib.FTP(hostname)
ftp.login()
ftp.cwd(file_path)
class TestDownloader(unittest.TestCase):
#patch('ftplib.FTP')
def test_download_file(self, mock_ftp):
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftp.cwd.assert_called_with('pub/files')
When I run this, I get:
AssertionError: Expected call: cwd('pub/files')
Not called
I know it must be using the mock object since that is a fake server name, and when run without patching, it throws a "socket.gaierror" exception.
How do I get the actual object the fuction is running? The long term goal is not having the "download_file" function in the same file, but calling it from a separate module file.
When you do patch(ftplib.FTP) you are patching FTP constructor. dowload_file() use it to build ftp object so your ftp object on which you call login() and cmd() will be mock_ftp.return_value instead of mock_ftp.
Your test code should be follow:
class TestDownloader(unittest.TestCase):
#patch('ftplib.FTP', autospec=True)
def test_download_file(self, mock_ftp_constructor):
mock_ftp = mock_ftp_constructor.return_value
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftp_constructor.assert_called_with('ftp.server.local')
self.assertTrue(mock_ftp.login.called)
mock_ftp.cwd.assert_called_with('pub/files')
I added all checks and autospec=True just because is a good practice
Like Ibrohim's answer, I prefer pytest with mocker.
I have went a bit further and have actually wrote a library which helps me to mock easily. Here is how to use it for your case.
You start by having your code and a basic pytest function, with the addition of my helper library to generate mocks to modules and the matching asserts generation:
import ftplib
from mock_autogen.pytest_mocker import PytestMocker
def download_file(hostname, file_path, file_name):
ftp = ftplib.FTP(hostname)
ftp.login()
ftp.cwd(file_path)
def test_download_file(mocker):
import sys
print(PytestMocker(mocked=sys.modules[__name__],
name=__name__).mock_modules().prepare_asserts_calls().generate())
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
When you run the test for the first time, it would fail due to unknown DNS, but the print statement which wraps my library would give us this valuable input:
...
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch('test_29817963.ftplib', new=mock_ftplib)
...
import mock_autogen
...
print(mock_autogen.generator.generate_asserts(mock_ftplib, name='mock_ftplib'))
I'm placing this in the test and would run it again:
def test_download_file(mocker):
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch('test_29817963.ftplib', new=mock_ftplib)
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
import mock_autogen
print(mock_autogen.generator.generate_asserts(mock_ftplib, name='mock_ftplib'))
This time the test succeeds and I only need to collect the result of the second print to get the proper asserts:
def test_download_file(mocker):
mock_ftplib = mocker.MagicMock(name='ftplib')
mocker.patch(__name__ + '.ftplib', new=mock_ftplib)
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
mock_ftplib.FTP.assert_called_once_with('ftp.server.local')
mock_ftplib.FTP.return_value.login.assert_called_once_with()
mock_ftplib.FTP.return_value.cwd.assert_called_once_with('pub/files')
If you would like to keep using unittest while using my library, I'm accepting pull requests.
I suggest using pytest and pytest-mock.
from pytest_mock import mocker
def test_download_file(mocker):
ftp_constructor_mock = mocker.patch('ftplib.FTP')
ftp_mock = ftp_constructor_mock.return_value
download_file('ftp.server.local', 'pub/files', 'wanted_file.txt')
ftp_constructor_mock.assert_called_with('ftp.server.local')
assert ftp_mock.login.called
ftp_mock.cwd.assert_called_with('pub/files')
Related
I need to mock the object whose method I call in a function. The object is initialized inside the function and that's the problem. How to replace its implementation with mock?
Function code:
def handler (event, context):
"""Function, when call Yandex Server less"""
function_heandler = Handler(event=event, context=context)
response = function_heandler.run()
return response
Test code:
def test_main_call_handler():
with mock.patch('function.handler.Handler', new=mock.MagicMock()) as mock_handler:
handler({}, object())
mock_handler.run.assert_called()
And this, as expected, does not work. The function will be called in another module and I cannot pass the mock object there. Any ideas on how to fix this?
You should mock the class Handler instead.
Assuming Handler is imported from package.module, you can simply patch package.module.Handler:
def test_main_call_handler():
with mock.patch('package.module.Handler') as mock_handler:
handler({}, object())
mock_handler.run.assert_called()
One thing to remember when mocking, is that you mock (replace) the attribute of the tested module and not the called object.
To be concrete, let's say that your tested module is named using_my_handler_module.py and it looks like this (the first import line is important):
from my_handler_module import Handler
def handler(event, context):
"""Function, when call Yandex Server less"""
function_heandler = Handler(event=event, context=context)
response = function_heandler.run()
return response
Now, you need to mock the Handler attribute of using_my_handler_module, so when it's used, it's mocked.
So the test function would look like so:
import mock
from using_my_handler_module import handler
def test_main_call_handler():
with mock.patch('using_my_handler_module.Handler', new=mock.MagicMock()) as mock_handler:
handler({}, object())
mock_handler.return_value.run.assert_called()
Note we patch 'using_my_handler_module.Handler' .
Next, we check the assert called like this: mock_handler.return_value.run.assert_called()
The reason is that mock_handler.return_value is matching the return object of Handler(event=event, context=context) and you execute the run method on that object, so need to append .run.assert_called().
I myself find that mocking is confusing at times. This is why I wrote a helper library on top of pytest mock, called pytest-mock-generator. Here is how you can use it for your case:
def test_main_call_handler_using_pytest_mock_generator(mocker, mg):
mg.generate_uut_mocks(handler)
handler({}, object())
You use the mg (mock generator) fixture to generate the mocks for you - simply send the tested function as a parameter to mg.generate_uut_mocks. When you execute the test function this output would be printed and copied to your clipboard:
# mocked dependencies
mock_Handler = mocker.MagicMock(name='Handler')
mocker.patch('using_my_handler_module.Handler', new=mock_Handler)
Copy it to the beginning of your test and you now have:
def test_main_call_handler_using_pytest_mock_generator(mocker, mg):
# mocked dependencies
mock_Handler = mocker.MagicMock(name='Handler')
mocker.patch('using_mock_fixture.using_my_handler_module.Handler', new=mock_Handler)
handler({}, object())
Next, you want to generate the asserts section, so use another mg function called generate_asserts - send it your new mock:
def test_main_call_handler_using_pytest_mock_generator(mocker, mg):
# mocked dependencies
mock_Handler = mocker.MagicMock(name='Handler')
mocker.patch('using_mock_fixture.using_my_handler_module.Handler', new=mock_Handler)
handler({}, object())
mg.generate_asserts(mock_Handler)
When you execute your test now, you would get a bunch of generated asserts:
assert 1 == mock_Handler.call_count
mock_Handler.assert_called_once_with(context=_object_object_at_0x7f7fa60c2790_, event={})
mock_Handler.return_value.run.assert_called_once_with()
You don't need most of them, copy the last line into your test and the final version of your working test would be like this one:
def test_main_call_handler_using_pytest_mock_generator(mocker):
# mocked dependencies
mock_Handler = mocker.MagicMock(name='Handler')
mocker.patch('using_mock_fixture.using_my_handler_module.Handler', new=mock_Handler)
handler({}, object())
mock_Handler.return_value.run.assert_called_once_with()
I have read many article over the last 6 hours and i still don't understand mocking and unit-testing. I want to unit test a open function, how can i do this correctly?
i am also concerned as the bulk of my code is using external files for data import and manipulation. I understand that i need to mock them for testing, but I am struggling to understand how to move forward.
Some advice please. Thank you in advance
prototype5.py
import os
import sys
import io
import pandas
pandas.set_option('display.width', None)
def openSetupConfig (a):
"""
SUMMARY
Read setup file
setup file will ONLY hold the file path of the working directory
:param a: str
:return: contents of the file stored as str
"""
try:
setupConfig = open(a, "r")
return setupConfig.read()
except Exception as ve:
ve = (str(ve) + "\n\nPlease ensure setup file " + str(a) + " is available")
sys.exit(ve)
dirPath = openSetupConfig("Setup.dat")
test_prototype5.py
import prototype5
import unittest
class TEST_openSetupConfig (unittest.TestCase):
"""
Test the openSetupConfig function from the prototype 5 library
"""
def test_open_correct_file(self):
result = prototype5.openSetupConfig("Setup.dat")
self.assertTrue(result)
if __name__ == '__main__':
unittest.main()
So the rule of thumb is to mock, stub or fake all external dependencies to the method/function under test. The point is to test the logic in isolation. So in your case you want to test that it can open a file or log an error message if it can't be opened.
import unittest
from mock import patch
from prototype5 import openSetupConfig # you don't want to run the whole file
import __builtin__ # needed to mock open
def test_openSetupConfig_with_valid_file(self):
"""
It should return file contents when passed a valid file.
"""
expect = 'fake_contents'
with patch('__builtin__.open', return_value=expect) as mock_open:
actual = openSetupConfig("Setup.dat")
self.assertEqual(expect, actual)
mock_open.assert_called()
#patch('prototype5.sys.exit')
def test_openSetupConfig_with_invalid_file(self, mock_exit):
"""
It should log an error and exit when passed an invalid file.
"""
with patch('__builtin__.open', side_effect=FileNotFoundError) as mock_open:
openSetupConfig('foo')
mock_exit.assert_called()
When I use HTMLTestRunner for Python 3.5，it shows an error.
I have changed the HTMLTestRunner for support python 3.5.
The code :
import pymysql
import pymysql
import unittest
import time
import unittest.suite
import HTMLTestRunner
import sys
def hell(a):
print(a)
return a
testunit = unittest.TestSuite()
testunit.addTest(hell('ad'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'print', description=u'简单')
runner.run(testunit)
When I run it, I got this error:
Traceback (most recent call last):
File "/Applications/Python 3.5/……/B.py", line 30, in <module>
testunit.addTest(hell('ad'))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py", line 47, in addTest
raise TypeError("{} is not callable".format(repr(test)))
TypeError: 'ad' is not callable
What should I do to make the script works?
You're adding the result of the call to hell('ad') to your tests, not the hell function itself. Since the hell function returns its argument, it returns the string ad, which is not a callable (function or the like).
Use
testunit.addTest(hell)
instead.
What about that argument then, how do you pass that?
Well, there are ways to do that, but generally, try not to let your unit test functions take an argument. Thus, hell() should siply not take an argument.
If you code your unit test correctly, you'll find that you rarely need to pass it an argument.
The line testunit.addTest(hell('ad')) doesn't do what you intend it to do. It doesn't tell the test suite to run hell('ad') later on. Rather, it calls that function immediately and passed the return value (which happens to be the string 'ad' you gave it as an argument) to addTest. That causes the exception later on, since a string is not a valid test case.
I'm not exactly sure how you should fix this. Your hell function doesn't appear to actually test anything, so there's not an obvious way to transform it into a proper test. There is a unittest.FunctionTestCase class that wraps a function up as a TestCase, but it doesn't appear to have any way of passing arguments to the function. Probably you should write a TestCase subclass class, and add various test_-prefixed methods that actually test things.
I had a right answer for my question, give the code:
# -*- coding: utf-8 -*-
import unittest
import HTMLTestRunner
def hell(a):
print(a)
return a
class HellTest(unittest.TestCase):
def setUp(self):
self.hell = hell
def tearDown(self):
pass
def testHell(self):
self.assertEqual(self.hell('aaa'), 'aaa')
if __name__ == '__main__':
testunit = unittest.TestSuite()
testunit.addTest(HellTest('testHell'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'不要生成error啦！', description=u'简单1112')
runner.run(testunit)
fp.close()
But, I do not know why add the class code 'class HellTest()'.The answer comes from a Chinese people whoes name is '幽谷奇峰'。Source code reference：https://segmentfault.com/q/1010000007427143?_ea=1345414
I'm trying to mock one particular boto3 function. My module, Cleanup, imports boto3. Cleanup also has a class, "cleaner". During init, cleaner creates an ec2 client:
self.ec2_client = boto3.client('ec2')
I want to mock the ec2 client method: desribe_tags(), which python says is:
<bound method EC2.describe_tags of <botocore.client.EC2 object at 0x7fd98660add0>>
the furthest I've gotten is importing botocore in my test file and trying:
mock.patch(Cleaner.botocore.client.EC2.describe_tags)
which fails with:
AttributeError: 'module' object has no attribute 'EC2'
How do I mock this method?
Cleanup looks like:
import boto3
class cleaner(object):
def __init__(self):
self.ec2_client = boto3.client('ec2')
The ec2_client object is the one that has the desribe_tags() method. It's a botocore.client.EC2 object, but I never directly import botocore.
I found a solution to this when trying to mock a different method for the S3 client
import botocore
from mock import patch
import boto3
orig = botocore.client.BaseClient._make_api_call
def mock_make_api_call(self, operation_name, kwarg):
if operation_name == 'DescribeTags':
# Your Operation here!
print(kwarg)
return orig(self, operation_name, kwarg)
with patch('botocore.client.BaseClient._make_api_call', new=mock_make_api_call):
client = boto3.client('ec2')
# Calling describe tags will perform your mocked operation e.g. print args
e = client.describe_tags()
Hope it helps :)
You should be mocking with respect to where you are testing. So, if you are testing your cleaner class (Which I suggest you use PEP8 standards here, and make it Cleaner), then you want to mock with respect to where you are testing. So, your patching should actually be something along the lines of:
class SomeTest(Unittest.TestCase):
#mock.patch('path.to.Cleaner.boto3.client', return_value=Mock())
def setUp(self, boto_client_mock):
self.cleaner_client = boto_client_mock.return_value
def your_test(self):
# call the method you are looking to test here
# simple test to check that the method you are looking to mock was called
self.cleaner_client.desribe_tags.assert_called_with()
I suggest reading through the mocking documentation which has many examples to do what you are trying to do
I'm converting a code base from Ruby to Python. In Ruby/RSpec I wrote custom "matchers" which allow me to black-box test web services like this:
describe 'webapp.com' do
it 'is configured for ssl' do
expect('www.webapp.com').to have_a_valid_cert
end
end
I'd like to write code to extend a Python testing framework with the same functionality. I realize that it will probably not look the same, of course. It doesn't need to be BDD. "Assert..." is just fine. Is pytest a good candidate for extending? Are there any examples of writing extensions like these?
Yes, pytest is good framework for doing what are you needs. We are using pytest with requests and PyHamcrest. Look at this example:
import pytest
import requests
from hamcrest import *
class SiteImpl:
def __init__(self, url):
self.url = url
def has_valid_cert(self):
return requests.get(self.url, verify=True)
#pytest.yield_fixture
def site(request):
# setUp
yield SiteImpl('https://' + request.param)
# tearDown
def has_status(item):
return has_property('status_code', item)
#pytest.mark.parametrize('site', ['google.com', 'github.com'], indirect=True)
def test_cert(site):
assert_that(site.has_valid_cert(), has_status(200))
if __name__ == '__main__':
pytest.main(args=[__file__, '-v'])
The code above uses parametrization for fixture site. Also yeld_fixture give you possibility to setUp and tearDown. Also you can write inline matcher has_status, which can used for easy read test assert.