From what I understand in test_main_version_exception unit test, the compare_version.version_match() function would raise an exception and ideally this exception should get caught by the except block. But for some reason, the flow is not going into except block.
(1st one seems to be working fine)
Could someone please help me with this? Any suggestions to improve theses tests are also welcome.
main.py file
try:
is_version_same = compare_version.version_match()
if is_version_same:
LOGGER.info("Upgrade not required")
sys.exit()
except compare_version.VersionError as error: #custom error
#### these two statements are not getting covered when I run test_main_version_exception(mocker):
LOGGER.exception("Exception occurred in compare_version: %s", error)
sys.exit(f"Exception occurred in compare_version: {error}")
UNIT TESTS:
def test_main_same_version(mocker):
mocker.patch(
"compare_version.version_match",
return_value=True,
)
with pytest.raises(SystemExit) as ex:
main.main()
assert ex.type == SystemExit
def test_main_version_exception(mocker):
mocker.patch(
"compare_version.version_match",
side_effect=VersionError,
)
with pytest.raises(VersionError) as ex:
main.main()
assert ex.type == VersionError
You have to patch the compare_version used in main.py specifically via mocker.patch("main.compare_version.version_match", ...) regardless if it was:
Imported into main.py:
from some_file import compare_version
Or defined within main.py:
class compare_version: # Sample implementation only as you haven't included this in the question
version_match = lambda: None
VersionError = Exception
As documented:
a.py
-> Defines SomeClass
b.py
-> from a import SomeClass
-> some_function instantiates SomeClass
Now we want to test some_function but we want to mock out SomeClass using patch()... If we use patch() to mock out a.SomeClass then it will have no effect on our test; module b already has a reference to the real SomeClass and it looks like our patching had no effect.
The key is to patch out SomeClass where it is used (or where it is looked up). In this case some_function will actually look up SomeClass in module b, where we have imported it. The patching should look like:
#patch('b.SomeClass')
Also, your assertion with pytest.raises(VersionError) as ex: is incorrect because your implementation doesn't raise VersionError anymore, instead it catches that error and raises SystemExit too. So what you need to assert here is if it raises SystemExit. If what you really need is the VersionError, then replace this line:
sys.exit(f"Exception occurred in compare_version: {error}")
To:
raise # Will re-raise the previous exception, which is the VersionError
Sample run you might want to use as reference.
main.py
import sys
# # Whether it is defined in another file
# from some_file import compare_version
# # Or it is defined here
class compare_version:
version_match = lambda: None
VersionError = ValueError
def main():
try:
is_version_same = compare_version.version_match()
if is_version_same:
print("Upgrade not required")
sys.exit()
except compare_version.VersionError as error: #custom error
print("Exception occurred in compare_version: %s", error)
# If you want this block to raise SystemExit
sys.exit(f"Exception occurred in compare_version: {error}")
# If you want this block to re-raise VersionError
# raise
test_main.py
import pytest
import main
def test_main_same_version(mocker):
mocker.patch(
"main.compare_version.version_match",
return_value=True,
)
with pytest.raises(SystemExit) as ex:
main.main()
assert ex.type == SystemExit
def test_main_version_exception(mocker):
mocker.patch(
"main.compare_version.version_match",
side_effect=main.compare_version.VersionError,
)
with pytest.raises(SystemExit) as ex:
main.main()
assert ex.type == SystemExit
Output
$ pytest -q -rP
================================================================================================= PASSES ==================================================================================================
_________________________________________________________________________________________ test_main_same_version __________________________________________________________________________________________
------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------
Upgrade not required
_______________________________________________________________________________________ test_main_version_exception _______________________________________________________________________________________
------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------
Exception occurred in compare_version: %s
2 passed in 0.04s
Related
I'm using pytest in a project with a good many custom exceptions.
pytest provides a handy syntax for checking that an exception has been raised, however, I'm not aware of one that asserts that the correct exception message has been raised.
Say I had a CustomException that prints "boo!", how could I assert that "boo!" has indeed been printed and not, say, "<unprintable CustomException object>"?
#errors.py
class CustomException(Exception):
def __str__(self): return "ouch!"
#test.py
import pytest, myModule
def test_custom_error(): # SHOULD FAIL
with pytest.raises(myModule.CustomException):
raise myModule.CustomException == "boo!"
I think what you're looking for is:
def failer():
raise myModule.CustomException()
def test_failer():
with pytest.raises(myModule.CustomException) as excinfo:
failer()
assert str(excinfo.value) == "boo!"
You can use match keyword in raises. Try something like
with pytest.raises(
RuntimeError, match=<error string here>
):
pass
Background
I would like to test for the following behaviours of a function:
If condition is met function raises None
If condition is not met function:
Outputs log message of ERROR level
Raises SystemExit
Example
The following trivial function checks if directory is accessible. The logging configuration logging.getLogger('__main__.' + __name__) is defined in __main__.py as the function is part of a package.
###########
# Modules #
###########
import os
###########
# Logging #
###########
import logging
logger = logging.getLogger('__main__.' + __name__)
#############
# Functions #
#############
def check_directory_access(folder_path):
"""Test if directory is readable"""
if os.access(folder_path, os.R_OK) is not True:
logger.error("Provided folder %s is invalid", folder_path)
raise SystemExit
Testing
#########
# Tests #
#########
class TestDirectoryChecking(unittest.TestCase):
"""Unit test checking utility functions"""
def test_return_none_correct_path(self):
"""Test if function checking for valid directory works"""
# Should return None
self.assertIsNone(utilities.check_directory_access(folder_path="/"))
def test_raise_exception_wrong_path(self):
"""Raises sception for wrong path"""
# Should raise a system exception
with self.assertRaises(SystemExit):
utilities.check_directory_access(folder_path="/wrong_path")
def test_outputting_log_message(self):
"""Function returns log message in case of wrong directory"""
with self.assertLogs(level='ERROR'):
utilities.check_directory_access(folder_path="/wrong_path")
if __name__ == '__main__':
unittest.main()
Problem
The last test errors:
======================================================================
ERROR: test_outputting_log_message (test_utility_functions.TestDirectoryChecking)
Function returns log message in case of wrong directory
----------------------------------------------------------------------
Traceback (most recent call last):
File "/module_path/tests/test_utility_functions.py", line 38, in test_outputting_log_message
utilities.check_directory_access(folder_path="/wrong_path")
File "/module_path/sample_module/utilities.py", line 30, in check_directory_access
raise SystemExit
SystemExit
----------------------------------------------------------------------
Ran 4 tests in 0.001s
It seems to me that your error is actually on test_outputting_log_message, which you're not adding the with self.assertRaises context. Since /wrong_path doesn't exist, the exception is raised just like in the previous test, but this time it's not expected in the test, so it breaks.
Working solution
def test_outputting_log_message(self):
"""Function returns log message in case of wrong directory"""
with self.assertRaises(SystemExit), self.assertLogs(level='ERROR') as log:
utilities.check_directory_access(folder_path="/wrong_path")
self.assertRegex(str(log), '.*Provided folder /wrong_path is invalid')
I'm working on a project that involves connecting to a remote server, waiting for a response, and then performing actions based on that response. We catch a couple of different exceptions, and behave differently depending on which exception is caught. For example:
def myMethod(address, timeout=20):
try:
response = requests.head(address, timeout=timeout)
except requests.exceptions.Timeout:
# do something special
except requests.exceptions.ConnectionError:
# do something special
except requests.exceptions.HTTPError:
# do something special
else:
if response.status_code != requests.codes.ok:
# do something special
return successfulConnection.SUCCESS
To test this, we've written a test like the following
class TestMyMethod(unittest.TestCase):
def test_good_connection(self):
config = {
'head.return_value': type('MockResponse', (), {'status_code': requests.codes.ok}),
'codes.ok': requests.codes.ok
}
with mock.patch('path.to.my.package.requests', **config):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.SUCCESS
)
def test_bad_connection(self):
config = {
'head.side_effect': requests.exceptions.ConnectionError,
'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError
}
with mock.patch('path.to.my.package.requests', **config):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.FAILURE
)
If I run the function directly, everything happens as expected. I even tested by adding raise requests.exceptions.ConnectionError to the try clause of the function. But when I run my unit tests, I get
ERROR: test_bad_connection (test.test_file.TestMyMethod)
----------------------------------------------------------------
Traceback (most recent call last):
File "path/to/sourcefile", line ###, in myMethod
respone = requests.head(address, timeout=timeout)
File "path/to/unittest/mock", line 846, in __call__
return _mock_self.mock_call(*args, **kwargs)
File "path/to/unittest/mock", line 901, in _mock_call
raise effect
my.package.requests.exceptions.ConnectionError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Path/to/my/test", line ##, in test_bad_connection
mypackage.myMethod('some_address',
File "Path/to/package", line ##, in myMethod
except requests.exceptions.ConnectionError:
TypeError: catching classes that do not inherit from BaseException is not allowed
I tried to change the exception I was patching in to BaseException and I got a more or less identical error.
I've read https://stackoverflow.com/a/18163759/3076272 already, so I think it must be a bad __del__ hook somewhere, but I'm not sure where to look for it or what I can even do in the mean time. I'm also relatively new to unittest.mock.patch() so it's very possible that I'm doing something wrong there as well.
This is a Fusion360 add-in so it is using Fusion 360's packaged version of Python 3.3 - as far as I know it's a vanilla version (i.e. they don't roll their own) but I'm not positive of that.
I could reproduce the error with a minimal example:
foo.py:
class MyError(Exception):
pass
class A:
def inner(self):
err = MyError("FOO")
print(type(err))
raise err
def outer(self):
try:
self.inner()
except MyError as err:
print ("catched ", err)
return "OK"
Test without mocking :
class FooTest(unittest.TestCase):
def test_inner(self):
a = foo.A()
self.assertRaises(foo.MyError, a.inner)
def test_outer(self):
a = foo.A()
self.assertEquals("OK", a.outer())
Ok, all is fine, both test pass
The problem comes with the mocks. As soon as the class MyError is mocked, the expect clause cannot catch anything and I get same error as the example from the question :
class FooTest(unittest.TestCase):
def test_inner(self):
a = foo.A()
self.assertRaises(foo.MyError, a.inner)
def test_outer(self):
with unittest.mock.patch('foo.MyError'):
a = exc2.A()
self.assertEquals("OK", a.outer())
Immediately gives :
ERROR: test_outer (__main__.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "...\foo.py", line 11, in outer
self.inner()
File "...\foo.py", line 8, in inner
raise err
TypeError: exceptions must derive from BaseException
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#78>", line 8, in test_outer
File "...\foo.py", line 12, in outer
except MyError as err:
TypeError: catching classes that do not inherit from BaseException is not allowed
Here I get a first TypeErrorthat you did not have, because I am raising a mock while you forced a true exception with 'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError in config. But the problem remains that the except clause tries to catch a mock.
TL/DR: as you mock the full requests package, the except requests.exceptions.ConnectionError clause tries to catch a mock. As the mock is not really a BaseException, it causes the error.
The only solution I can imagine is not to mock the full requests but only the parts that are not exceptions. I must admit I could not find how to say to mock mock everything except this but in your example, you only need to patch requests.head. So I think that this should work :
def test_bad_connection(self):
with mock.patch('path.to.my.package.requests.head',
side_effect=requests.exceptions.ConnectionError):
self.assertEqual(
mypackage.myMethod('some_address',
mypackage.successfulConnection.FAILURE
)
That is : only patch the head method with the exception as side effect.
I just ran into the same issue while trying to mock sqlite3 (and found this post while looking for solutions).
What Serge said is correct:
TL/DR: as you mock the full requests package, the except requests.exceptions.ConnectionError clause tries to catch a mock. As the mock is not really a BaseException, it causes the error.
The only solution I can imagine is not to mock the full requests but only the parts that are not exceptions. I must admit I could not find how to say to mock mock everything except this
My solution was to mock the entire module, then set the mock attribute for the exception to be equal to the exception in the real class, effectively "un-mocking" the exception. For example, in my case:
#mock.patch(MyClass.sqlite3)
def test_connect_fail(self, mock_sqlite3):
mock_sqlite3.connect.side_effect = sqlite3.OperationalError()
mock_sqlite3.OperationalError = sqlite3.OperationalError
self.assertRaises(sqlite3.OperationalError, MyClass, self.db_filename)
For requests, you could assign exceptions individually like this:
mock_requests.exceptions.ConnectionError = requests.exceptions.ConnectionError
or do it for all of the requests exceptions like this:
mock_requests.exceptions = requests.exceptions
I don't know if this is the "right" way to do it, but so far it seems to work for me without any issue.
For those of us who need to mock an exception and can't do that by simply patching head, here is an easy solution that replaces the target exception with an empty one:
Say we have a generic unit to test with an exception we have to have mocked:
# app/foo_file.py
def test_me():
try:
foo()
return "No foo error happened"
except CustomError: # <-- Mock me!
return "The foo error was caught"
We want to mock CustomError but because it is an exception we run into trouble if we try to patch it like everything else. Normally, a call to patch replaces the target with a MagicMock but that won't work here. Mocks are nifty, but they do not behave like exceptions do. Rather than patching with a mock, let's give it a stub exception instead. We'll do that in our test file.
# app/test_foo_file.py
from mock import patch
# A do-nothing exception we are going to replace CustomError with
class StubException(Exception):
pass
# Now apply it to our test
#patch('app.foo_file.foo')
#patch('app.foo_file.CustomError', new_callable=lambda: StubException)
def test_foo(stub_exception, mock_foo):
mock_foo.side_effect = stub_exception("Stub") # Raise our stub to be caught by CustomError
assert test_me() == "The error was caught"
# Success!
So what's with the lambda? The new_callable param calls whatever we give it and replaces the target with the return of that call. If we pass our StubException class straight, it will call the class's constructor and patch our target object with an exception instance rather than a class which isn't what we want. By wrapping it with lambda, it returns our class as we intend.
Once our patching is done, the stub_exception object (which is literally our StubException class) can be raised and caught as if it were the CustomError. Neat!
I faced a similar issue while trying to mock the sh package. While sh is very useful, the fact that all methods and exceptions are defined dynamically make it more difficult to mock them. So following the recommendation of the documentation:
import unittest
from unittest.mock import Mock, patch
class MockSh(Mock):
# error codes are defined dynamically in sh
class ErrorReturnCode_32(BaseException):
pass
# could be any sh command
def mount(self, *args):
raise self.ErrorReturnCode_32
class MyTestCase(unittest.TestCase):
mock_sh = MockSh()
#patch('core.mount.sh', new=mock_sh)
def test_mount(self):
...
I just ran into the same problem when mocking struct.
I get the error:
TypeError: catching classes that do not inherit from BaseException is not allowed
When trying to catch a struct.error raised from struct.unpack.
I found that the simplest way to get around this in my tests was to simply set the value of the error attribute in my mock to be Exception. For example
The method I want to test has this basic pattern:
def some_meth(self):
try:
struct.unpack(fmt, data)
except struct.error:
return False
return True
The test has this basic pattern.
#mock.patch('my_module.struct')
def test_some_meth(self, struct_mock):
'''Explain how some_func should work.'''
struct_mock.error = Exception
self.my_object.some_meth()
struct_mock.unpack.assert_called()
struct_mock.unpack.side_effect = struct_mock.error
self.assertFalse(self.my_object.some_meth()
This is similar to the approach taken by #BillB, but it is certainly simpler as I don't need to add imports to my tests and still get the same behavior. To me it would seem this is the logical conclusion to the general thread of reasoning in the answers here.
Use patch.object to partially mock a class.
My use case:
import unittest
from unittest import mock
import requests
def test_my_function(self):
response = mock.MagicMock()
response.raise_for_status.side_effect = requests.HTTPError
with mock.patch.object(requests, 'get', return_value=response):
my_function()
I have two modules, main and notmain. I declared my custom exception in main module and want to catch it. This exception is raised in notmain module. The problem is I can't catch my exception raised in notmain module.
main.py:
class MyException(Exception):
pass
m = __import__('notmain')
try:
m.func()
except MyException as e:
print(type(e))
print('ops')
notmain.py:
def func():
import main # 1
# from main import MyException # 2
# from main import MyException as MyException # 3
raise main.MyException # 1
# raise MyException # 2, 3
I tried different import methods, with the same result. When I run main.py, I see this:
<class 'main.MyException'>
ops
Traceback (most recent call last):
File "D:\exception\main.py", line 6, in <module>
m.func()
File "D:\exception\notmain.py", line 6, in func
raise main.MyException # 1
main.MyException
This means exception is somehow caught, but why I see traceback? And why exception class has "main.MyException" name?
Now if I modify main.py a little bit and run it:
try:
raise MyException
except MyException as e:
print(type(e))
print('ops')
I'll see what's expected:
<class '__main__.MyException'>
ops
I don't know why this class MyException has different names in main.py and in notmain.py? And why Python can't catch it as expected?
Thank you :)
Your module main is imported twice (as main and __main__), each having its own class MyException. You should consider redesigning your application to avoid circular imports.
The __main__ name, with underscores, is an automatic namespace for the program being called. A workaround would be to declare the exception in a third file (or have a third file that is the user-called program, and just a single method in your "real" program).
Also, the way you import notmain may have something to do with it. Why not just "from notmain import func" ?
I'm trying to unit test some code that looks like this:
def main():
parser = optparse.OptionParser(description='This tool is cool', prog='cool-tool')
parser.add_option('--foo', action='store', help='The foo option is self-explanatory')
options, arguments = parser.parse_args()
if not options.foo:
parser.error('--foo option is required')
print "Your foo is %s." % options.foo
return 0
if __name__ == '__main__':
sys.exit(main())
With code that looks like this:
#patch('optparse.OptionParser')
def test_main_with_missing_p4clientsdir_option(self, mock_optionparser):
#
# setup
#
optionparser_mock = Mock()
mock_optionparser.return_value = optionparser_mock
options_stub = Mock()
options_stub.foo = None
optionparser_mock.parse_args.return_value = (options_stub, sentinel.arguments)
def parser_error_mock(message):
self.assertEquals(message, '--foo option is required')
sys.exit(2)
optionparser_mock.error = parser_error_mock
#
# exercise & verify
#
self.assertEquals(sut.main(), 2)
I'm using Michael Foord's Mock, and nose to run the tests.
When I run the test, I get:
File "/Users/dspitzer/Programming/Python/test-optparse-error/tests/sut_tests.py", line 27, in parser_error_mock
sys.exit(2)
SystemExit: 2
----------------------------------------------------------------------
Ran 1 test in 0.012s
FAILED (errors=1)
The problem is that OptionParser.error does a sys.exit(2), and so main() naturally relies on that. But nose or unittest detects the (expected) sys.exit(2) and fails the test.
I can make the test pass by adding "return 2" under the parser.error() call in main() and removing the sys.exit() call from parser_error_mock(), but I find it distasteful to modify the code under test to allow a test to pass. Is there a better solution?
Update: df's answer works, although the correct call is "self.assertRaises(SystemExit, sut.main)".
Which means the test passes whatever the number is in the sys.exit() in parser_error_mock(). Is there any way to test for the exit code?
BTW, the test is more robust if I add:
self.assertEquals(optionparser_mock.method_calls, [('add_option', ('--foo',), {'action': 'store', 'help': 'The foo option is self-explanatory'}), ('parse_args', (), {})])
at the end.
Update 2: I can test for the exit code by replacing "self.assertRaises(SystemExit, sut.main)" with:
try:
sut.main()
except SystemExit, e:
self.assertEquals(type(e), type(SystemExit()))
self.assertEquals(e.code, 2)
except Exception, e:
self.fail('unexpected exception: %s' % e)
else:
self.fail('SystemExit exception expected')
Will this work instead of assertEquals?
self.assertRaises(SystemExit, sut.main, 2)
This should catch the SystemExit exception and prevent the script from terminating.
As noted in my updates to my question, I had to modify dF's answer to:
self.assertRaises(SystemExit, sut.main)
...and I came up with a few longer snippet to test for the exit code.
[Note: I accepted my own answer, but I will delete this answer and accept dF's if he updates his.]
Probably this question contains some new information:
Java: How to test methods that call System.exit()?
Add #raises to your test function. I.e:
from nose.tools import raises
#raises(SystemExit)
#patch('optparse.OptionParser')
def test_main_with_missing_p4clientsdir_option(self, mock_optionparser):
# Setup
...
sut.main()