Develop Reference

Writing a unit test for Python REST API function

I'm currently learning Python REST API (side project). I've been reading a lot of tutorials from RealPython, Python Requests documentation, etc. I found this post on how to write try/except properly in Python (Correct way to try/except using Python requests module?). One thing that still confuses me though is how to create a unit test for a function like this since it is not returning anything. Any help?
def google_do_something(blahblah):
url='http://www.google.com/' + blahblah
try:
r = requests.get(url,timeout=3)
r.raise_for_status()
except requests.exceptions.HTTPError as errh:
print (errh)
except requests.exceptions.ConnectionError as errc:
print (errc)
except requests.exceptions.Timeout as errt:
print (errt)
except requests.exceptions.RequestException as err:
print (err)
I could think of this but I don't know what to assert with.
def test_google_do_something():
g = google_do_something('blahblah')
# assert??

There are several unit test frameworks available in Python. Try/except blocks are good for error handling, but you still need a separate unit test around the call if you want to unit test it.
You do have something you can test, you can just return it and test that in your unit test.
Example Unit test using unittest:
import unittest
import requests
class RestCalls():
def google_do_something(blahblah):
url= blahblah
try:
r = requests.get(url,timeout=1)
r.raise_for_status()
return r.status_code
except requests.exceptions.Timeout as errt:
print (errt)
raise
except requests.exceptions.HTTPError as errh:
print (errh)
raise
except requests.exceptions.ConnectionError as errc:
print (errc)
raise
except requests.exceptions.RequestException as err:
print (err)
raise
class TestRESTMethods(unittest.TestCase):
def test_valid_url(self):
self.assertEqual(200,RestCalls.google_do_something('http://www.google.com/search'))
def test_exception(self):
self.assertRaises(requests.exceptions.Timeout,RestCalls.google_do_something,'http://localhost:28989')
if __name__ == '__main__':
unittest.main()
Executing should show (made some edits to this post, updated output included at bottom of post):
> python .\Tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.192s
OK
If you asserted a different response code from your request, it would fail (the request is just returning http response codes):
python .\Tests.py
F
======================================================================
FAIL: test_upper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\Tests.py", line 25, in test_upper
self.assertEqual(404,RestCalls.google_do_something('search'))
AssertionError: 404 != 200
----------------------------------------------------------------------
Ran 1 test in 0.245s
FAILED (failures=1)
Which is expected.
Edit: Included exception testing. You can test these by just including raise in the except block, which will show this after running:
> python .\Tests.py
HTTPConnectionPool(host='localhost', port=28989): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x03688598>, 'Connection to localhost timed out. (connect timeout=1)'))
..
----------------------------------------------------------------------
Ran 2 tests in 2.216s
OK
References:
Unit tests in Python
https://docs.python.org/3/library/unittest.html
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

I am not sure that your approach is such a good idea (just printing something in case of an error) but you could mock the print function to see if it was really called (and with what arguments):
https://docs.python.org/3/library/unittest.mock.html?highlight=mock#module-unittest.mock
Edit:
Working with mocks is a bit tricky as far as I remember. You would have to mock the print function in the current module. Perhaps something like this (not tested ...):
from unittest.mock import patch
from unittest import TestCase
class TestGoogleDoSomething(TestCase)
#patch("nameOfYourModule.print")
def test_google_do_something(self, print_mock): # the decorator will pass the mock object into the function
g = google_do_something('blahblah')
print_mock.assert_called_with("your error message here ...")

It seems that you are using print instead of all exception handlers. I don’t think that is a good practice. From my perspective, I prefer to raise those Exceptions out again if not sure how to deal with them right now.
With that said, when any error occurs, an exception will be thrown out; if there’re no exceptions, that means this function work well. Therefore you can design your unit test cases base on that.

To Kill A Mocking Object: A Python Story

I have been having trouble with Python mock and have been going crazy. I've held off on this question due to fear of down voting for not enough research. I have a cumulative 24 hours over the last week trying to figure out how to get this work and cannot.
I have read numerous examples and have created this one from those. I know mock objects are supposed to be easy to use, but this has taken too long. Now I am out of time.
I am trying to do two simple things here:
1. Override a request.ok status code inside another function
2. Cause an urllib2.HTTPError exception to be thrown
I have distilled these two tasks into the simplest possible example for your convenience:
#ExampleModule.py
import requests
import urllib2
def hello_world():
try:
print "BEGIN TRY"
r = requests.request('GET', "http://127.0.0.1:80")
print r.ok
if r.ok:
print "PATCH 1 FAILED"
else:
print "PATCH 1 SUCCESSFUL"
except urllib2.HTTPError:
print "PATCH 2 SUCCESSFUL"
print "EXCEPTION 2 HIT\n"
else:
print "PATCH 2 FAILED\n"
and
#in TestModule.py
import mock
import ExampleModule
def test_function_try():
with mock.patch('ExampleModule.hello_world') as patched_request:
patched_request.requests.request.ok = False
result = ExampleModule.hello_world()
print result
def test_function_exception():
with mock.patch('ExampleModule.hello_world') as patched_exception:
patched_exception.urllib2.side_effect = HTTPError
result = ExampleModule.hello_world()
print result
test_function_try()
test_function_exception()
A normal call to hello_world() outputs:
BEGIN TRY
True
<Response [200]>
A normal call to test_function_try() outputs:
<MagicMock name='hello_world()' id='70272816'>
#From the "print result" inside test_function_try()
A normal call to test_function_exception() outputs:
<MagicMock name='hello_world()' id='62320016'>
#From the "print result" inside test_function_exception()
Obviously, I am not actually returning anything from hello_world() so it looks like the patched object is the hello_world() function instead of the requests or urllib2 patched modules.
It should be noted that when I try to patch with 'ExampleModule.hello_world.requests' or 'ExampleModule.hello_world.urllib2' I get an error saying they cannot be found in hello_world()
QUESTION SUMMARY
What is wrong with the two functions test_function_try() and test_function_exception()? What needs to be modified so that I can manually assign the value of request.ok inside hello_world() and also manually raise the exception HTTPError so that I can test the code in that block... Bonus points for explaining 'when' exactly the exception gets thrown: as soon as the try: is entered, or when request is called, or some other time?
Something that has been a concern of mine: will my print statements inside the ExampleModule.py reveal whether my patching and mock tests are working or do I HAVE to use assert methods to get the truth? I am not sure whether assert is a necessity when people mention 'use assertions to find out if the actual patched object was called, etc.' or if this is for convenience/convention/practicality.
UPDATE
After changing the patch target to the requests.request() function, as per #chepner's suggestion, I receive the following output:
BEGIN TRY
False
PATCH 1 SUCCESSFUL
PATCH 2 FAILED
BEGIN TRY
Traceback (most recent call last):
File "C:\LOCAL\ECLIPSE PROJECTS\MockingTest\TestModule.py", line 44, in <module>
test_function_exception()
File "C:\LOCAL\ECLIPSE PROJECTS\MockingTest\TestModule.py", line 19, in test_function_exception
ExampleModule.hello_world()
File "C:\LOCAL\ECLIPSE PROJECTS\MockingTest\ExampleModule.py", line 12, in hello_world
r = requests.request('GET', "http://127.0.0.1:8080")
File "C:\Python27\lib\site-packages\mock\mock.py", line 1062, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "C:\Python27\lib\site-packages\mock\mock.py", line 1118, in _mock_call
raise effect
TypeError: __init__() takes exactly 6 arguments (1 given)

You can't mock local variables in a function. The thing you want to mock is requests.request itself, so that when hello_world calls it, a mock object is returned, rather than actually making an HTTP request.
import mock
import urllib2
import ExampleModule
def test_function_try():
with mock.patch('ExampleModule.requests.request') as patched_request:
patched_request.return_value.ok = False
ExampleModule.hello_world()
def test_function_exception():
with mock.patch('ExampleModule.requests.request') as patched_request:
patched_request.side_effect = urllib2.HTTPError
ExampleModule.hello_world()
test_function_try()
test_function_exception()

Can't catch mocked exception because it doesn't inherit BaseException

I'm working on a project that involves connecting to a remote server, waiting for a response, and then performing actions based on that response. We catch a couple of different exceptions, and behave differently depending on which exception is caught. For example:
def myMethod(address, timeout=20):
try:
response = requests.head(address, timeout=timeout)
except requests.exceptions.Timeout:
# do something special
except requests.exceptions.ConnectionError:
# do something special
except requests.exceptions.HTTPError:
# do something special
else:
if response.status_code != requests.codes.ok:
# do something special
return successfulConnection.SUCCESS
To test this, we've written a test like the following
class TestMyMethod(unittest.TestCase):
def test_good_connection(self):
config = {
'head.return_value': type('MockResponse', (), {'status_code': requests.codes.ok}),
'codes.ok': requests.codes.ok
}
with mock.patch('path.to.my.package.requests', **config):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.SUCCESS
)
def test_bad_connection(self):
config = {
'head.side_effect': requests.exceptions.ConnectionError,
'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError
}
with mock.patch('path.to.my.package.requests', **config):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.FAILURE
)
If I run the function directly, everything happens as expected. I even tested by adding raise requests.exceptions.ConnectionError to the try clause of the function. But when I run my unit tests, I get
ERROR: test_bad_connection (test.test_file.TestMyMethod)
----------------------------------------------------------------
Traceback (most recent call last):
File "path/to/sourcefile", line ###, in myMethod
respone = requests.head(address, timeout=timeout)
File "path/to/unittest/mock", line 846, in __call__
return _mock_self.mock_call(*args, **kwargs)
File "path/to/unittest/mock", line 901, in _mock_call
raise effect
my.package.requests.exceptions.ConnectionError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Path/to/my/test", line ##, in test_bad_connection
mypackage.myMethod('some_address',
File "Path/to/package", line ##, in myMethod
except requests.exceptions.ConnectionError:
TypeError: catching classes that do not inherit from BaseException is not allowed
I tried to change the exception I was patching in to BaseException and I got a more or less identical error.
I've read https://stackoverflow.com/a/18163759/3076272 already, so I think it must be a bad __del__ hook somewhere, but I'm not sure where to look for it or what I can even do in the mean time. I'm also relatively new to unittest.mock.patch() so it's very possible that I'm doing something wrong there as well.
This is a Fusion360 add-in so it is using Fusion 360's packaged version of Python 3.3 - as far as I know it's a vanilla version (i.e. they don't roll their own) but I'm not positive of that.

I could reproduce the error with a minimal example:
foo.py:
class MyError(Exception):
pass
class A:
def inner(self):
err = MyError("FOO")
print(type(err))
raise err
def outer(self):
try:
self.inner()
except MyError as err:
print ("catched ", err)
return "OK"
Test without mocking :
class FooTest(unittest.TestCase):
def test_inner(self):
a = foo.A()
self.assertRaises(foo.MyError, a.inner)
def test_outer(self):
a = foo.A()
self.assertEquals("OK", a.outer())
Ok, all is fine, both test pass
The problem comes with the mocks. As soon as the class MyError is mocked, the expect clause cannot catch anything and I get same error as the example from the question :
class FooTest(unittest.TestCase):
def test_inner(self):
a = foo.A()
self.assertRaises(foo.MyError, a.inner)
def test_outer(self):
with unittest.mock.patch('foo.MyError'):
a = exc2.A()
self.assertEquals("OK", a.outer())
Immediately gives :
ERROR: test_outer (__main__.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "...\foo.py", line 11, in outer
self.inner()
File "...\foo.py", line 8, in inner
raise err
TypeError: exceptions must derive from BaseException
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#78>", line 8, in test_outer
File "...\foo.py", line 12, in outer
except MyError as err:
TypeError: catching classes that do not inherit from BaseException is not allowed
Here I get a first TypeErrorthat you did not have, because I am raising a mock while you forced a true exception with 'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError in config. But the problem remains that the except clause tries to catch a mock.
TL/DR: as you mock the full requests package, the except requests.exceptions.ConnectionError clause tries to catch a mock. As the mock is not really a BaseException, it causes the error.
The only solution I can imagine is not to mock the full requests but only the parts that are not exceptions. I must admit I could not find how to say to mock mock everything except this but in your example, you only need to patch requests.head. So I think that this should work :
def test_bad_connection(self):
with mock.patch('path.to.my.package.requests.head',
side_effect=requests.exceptions.ConnectionError):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.FAILURE
)
That is : only patch the head method with the exception as side effect.

I just ran into the same issue while trying to mock sqlite3 (and found this post while looking for solutions).
What Serge said is correct:
TL/DR: as you mock the full requests package, the except requests.exceptions.ConnectionError clause tries to catch a mock. As the mock is not really a BaseException, it causes the error.
The only solution I can imagine is not to mock the full requests but only the parts that are not exceptions. I must admit I could not find how to say to mock mock everything except this
My solution was to mock the entire module, then set the mock attribute for the exception to be equal to the exception in the real class, effectively "un-mocking" the exception. For example, in my case:
#mock.patch(MyClass.sqlite3)
def test_connect_fail(self, mock_sqlite3):
mock_sqlite3.connect.side_effect = sqlite3.OperationalError()
mock_sqlite3.OperationalError = sqlite3.OperationalError
self.assertRaises(sqlite3.OperationalError, MyClass, self.db_filename)
For requests, you could assign exceptions individually like this:
mock_requests.exceptions.ConnectionError = requests.exceptions.ConnectionError
or do it for all of the requests exceptions like this:
mock_requests.exceptions = requests.exceptions
I don't know if this is the "right" way to do it, but so far it seems to work for me without any issue.

For those of us who need to mock an exception and can't do that by simply patching head, here is an easy solution that replaces the target exception with an empty one:
Say we have a generic unit to test with an exception we have to have mocked:
# app/foo_file.py
def test_me():
try:
foo()
return "No foo error happened"
except CustomError: # <-- Mock me!
return "The foo error was caught"
We want to mock CustomError but because it is an exception we run into trouble if we try to patch it like everything else. Normally, a call to patch replaces the target with a MagicMock but that won't work here. Mocks are nifty, but they do not behave like exceptions do. Rather than patching with a mock, let's give it a stub exception instead. We'll do that in our test file.
# app/test_foo_file.py
from mock import patch
# A do-nothing exception we are going to replace CustomError with
class StubException(Exception):
pass
# Now apply it to our test
#patch('app.foo_file.foo')
#patch('app.foo_file.CustomError', new_callable=lambda: StubException)
def test_foo(stub_exception, mock_foo):
mock_foo.side_effect = stub_exception("Stub") # Raise our stub to be caught by CustomError
assert test_me() == "The error was caught"
# Success!
So what's with the lambda? The new_callable param calls whatever we give it and replaces the target with the return of that call. If we pass our StubException class straight, it will call the class's constructor and patch our target object with an exception instance rather than a class which isn't what we want. By wrapping it with lambda, it returns our class as we intend.
Once our patching is done, the stub_exception object (which is literally our StubException class) can be raised and caught as if it were the CustomError. Neat!

I faced a similar issue while trying to mock the sh package. While sh is very useful, the fact that all methods and exceptions are defined dynamically make it more difficult to mock them. So following the recommendation of the documentation:
import unittest
from unittest.mock import Mock, patch
class MockSh(Mock):
# error codes are defined dynamically in sh
class ErrorReturnCode_32(BaseException):
pass
# could be any sh command
def mount(self, *args):
raise self.ErrorReturnCode_32
class MyTestCase(unittest.TestCase):
mock_sh = MockSh()
#patch('core.mount.sh', new=mock_sh)
def test_mount(self):
...

I just ran into the same problem when mocking struct.
I get the error:
TypeError: catching classes that do not inherit from BaseException is not allowed
When trying to catch a struct.error raised from struct.unpack.
I found that the simplest way to get around this in my tests was to simply set the value of the error attribute in my mock to be Exception. For example
The method I want to test has this basic pattern:
def some_meth(self):
try:
struct.unpack(fmt, data)
except struct.error:
return False
return True
The test has this basic pattern.
#mock.patch('my_module.struct')
def test_some_meth(self, struct_mock):
'''Explain how some_func should work.'''
struct_mock.error = Exception
self.my_object.some_meth()
struct_mock.unpack.assert_called()
struct_mock.unpack.side_effect = struct_mock.error
self.assertFalse(self.my_object.some_meth()
This is similar to the approach taken by #BillB, but it is certainly simpler as I don't need to add imports to my tests and still get the same behavior. To me it would seem this is the logical conclusion to the general thread of reasoning in the answers here.

Use patch.object to partially mock a class.
My use case:
import unittest
from unittest import mock
import requests
def test_my_function(self):
response = mock.MagicMock()
response.raise_for_status.side_effect = requests.HTTPError
with mock.patch.object(requests, 'get', return_value=response):
my_function()

Issue testing user-defined exception when imported from module into __init__.py

I am attempting to write a large module split into multiple files while keeping it unified in a single logical module, as described in Ch. 10 of the Python Cookbook
However, in my unit test, I'm receiving an unexpected error when testing for an exception. When I am attempting to force an exception, I expect the exception package.MyExceptionError to be raised. However, my test is failing, because I am getting the exception package.module.MyExceptionError. On the other hand, when I call a function from a module that has been imported via __init__.py, I'm able to call it as package.my_function() as expected.
I've looked through the code for several PyPi modules that are similarly laid-out, but I'm not able to determine what I'm doing incorrectly, or am I misunderstanding how this should work?
Below is a dummy package, which includes the minimum code necessary to replicate the issue.
My file structure is:
Spam/
spam/
__init__.py
eggs.py
test/
test_eggs.py
Here are the contents of spam/eggs.py:
class EggError(Exception):
pass
def egg(b=True):
if b == True:
return 0
else:
raise EggError('Spam!')
if __name__ == '__main__':
pass
Here are the contents of spam/__init__.py:
from .egg import egg
from .egg import EggError
__all__ = ['egg', 'EggError', ]
if __name__ == '__main__':
pass
Here are the contents of test/test_spam.py:
import unittest
import spam
class Test(unittest.TestCase):
def test_egg(self):
self.assertEquals(spam.egg(), 0)
def test_EggError(self):
self.assertRaises(spam.EggError, spam.egg(False))
if __name__ == "__main__":
unittest.main()
These are the results I get when I run the test:
Finding files... done.
Importing test modules ... done.
======================================================================
ERROR: test_EggError (test.test_spam.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "K:\Scripting\Python\err_test\test\test_spam.py", line 11, in test_EggError
self.assertRaises(spam.EggError, spam.egg(False))
File "K:\Scripting\Python\err_test\spam\eggs.py", line 12, in egg
raise EggError('Spam!')
spam.eggs.EggError: Spam!
----------------------------------------------------------------------
Ran 2 tests in 0.415s
FAILED (errors=1)

You are calling spam.egg() before assertRaises() has a chance to catch the exception. You should write that test as follows:
self.assertRaises(spam.EggError, spam.egg, False)
assertRaises() will call spam.egg() for you.
If you find this hard to read, you can use the context manager style instead:
with self.assertRaises(spam.EggError):
spam.egg(False)

Python: assertRaises( ) not catching ldap.SERVER_DOWN error when raised

Thanks for your help in advance.
I've got the following class method that I'm trying to test:
def _get_ldap_connection(self):
"""
Instantiate and return simpleldap.Connection object.
Raises:
ldap.SERVER_DOWN: When ldap_url is invalid or server is
not reachable.
"""
try:
ldap_connection = simpleldap.Connection(
self.ldap_url, encryption='ssl', require_cert=False,
debug=False, dn=self.ldap_login_dn,
password=self.ldap_login_password)
except ldap.SERVER_DOWN:
raise ldap.SERVER_DOWN(
"The LDAP server specified, {}, did not respond to the "
"connection attempt.".format(self.ldap_url))
And here's the unittest:
def test__get_ldap_connection(self):
"""
VERY IMPORTANT: This test refers to your actual config.json file.
If it is correctly populated, you can expect this test to fail.
"""
# Instantiate Class
test_extractor = SakaiLdapExtractor('config_files/config.json')
# Monkey with ldap server url to ensure error.
test_extractor.ldap_url = "invalid_ldap_url"
self.assertRaises(
ldap.SERVER_DOWN, test_extractor._get_ldap_connection())
So far, so good. But when I execute the unit tests (via nose) test_extractor._get_ldap_connection() is called from the assertRaises statement, but the exception is not caught and the test fails.
Here is the output:
vagrant#precise64:/vagrant/sakai-directory-integration$ nosetests
...E..
======================================================================
ERROR: VERY IMPORTANT: This test refers to your actual config.json file.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/vagrant/sakai-directory-integration/test_sakaiLdapExtractor.py", line 77, in test__get_ldap_connection
ldap.SERVER_DOWN, test_extractor._get_ldap_connection())
File "/vagrant/sakai-directory-integration/sakai_ldap_integration.py", line 197, in _get_ldap_connection
"connection attempt.".format(self.ldap_url))
SERVER_DOWN: The LDAP server specified, invalid_ldap_url, did not respond to the connection attempt.
----------------------------------------------------------------------
Ran 6 tests in 0.159s
Help me!

Don't call, just pass function (method) itself; drop ():
self.assertRaises(
ldap.SERVER_DOWN, test_extractor._get_ldap_connection)
Alternatively, you can use with self.assertRaises(..) form if you are using recent version of python (Python 2.7+ / Python 3.1+):
with self.assertRaises(ldap.SERVER_DOWN):
test_extractor._get_ldap_connection()

You are not using assertRaises correctly.
You can use it as a context manager:
with self.assertRaises(ldap.SERVER_DOWN):
test_extractor._get_ldap_connection()
or the usual way (self.assertRaises(exception, function, args):
self.assertRaises(ldap.SERVER_DOWN, test_extractor._get_ldap_connection)
Also see:
How to properly use unit-testing's assertRaises() with NoneType objects?
Testing in Python - how to use assertRaises in testing using unittest?
documentation