In the Python unittest framework, is there a way to pass a unit test if an exception wasn't raised, and fail with an AssertRaise otherwise?
If I understand your question correctly, you could do something like this:
def test_does_not_raise_on_valid_input(self):
raised = False
try:
do_something(42)
except:
raised = True
self.assertFalse(raised, 'Exception raised')
...assuming that you have a corresponding test that the correct Exception gets raised on invalid input, of course:
def test_does_raise_on_invalid_input(self):
self.assertRaises(OutOfCheese, do_something, 43)
However, as pointed out in the comments, you need to consider what it is that you are actually testing. It's likely that a test like...
def test_what_is_42(self):
self.assertEquals(do_something(42), 'Meaning of life')
...is better because it tests the desired behaviour of the system and will fail if an exception is raised.
Many of the comments on this page treat errors and failures as equivalent, which they are not. The right solution in my opinion is to explicitly fail the test if the exception is raised. E.g.:
def test_does_not_error(self):
try:
code_under_test()
except ThatException:
self.fail("code_under_test raised ThatException")
Simply call your functionality, e.g. do_something(). If an unhandled exception gets raised, the test automatically fails! There is really no reason to do anything else. This is also the reason why assertDoesNotRaise() does not exist.
Credit: comment by Sven
Related
Coming from this thread, it seems that is not possible with pytest to check if a specific exception is raised: if you mention the parent exception, the test passes as well.
I made this little example to illustrate, all three tests passed just fine. I expect only one to pass.
Could you confirm ? Should I check the excinfo ?
(Why is this a problem? The function I was testing can, in some cases, return a true TypeError exception, and I'd like my tests to detect precisely when one or the other is raised)
Thanks
import pytest
class CustomExc(TypeError):
pass
def a_random_function():
raise CustomExc
def test_random_function1():
with pytest.raises(TypeError):
a_random_function()
def test_random_function2():
with pytest.raises(CustomExc):
a_random_function()
def test_random_function3():
with pytest.raises(BaseException):
a_random_function()
CustomExc is both a TypeError and a BaseException, so this is expected behavior. The raised exception is checked for its type using isinstance, which means that all base classes will also pass.
If you want to test for a concrete exception, and not any derived exception, you have to do the check yourself. As you mentioned, you can check the exception info:
def test_random_function():
with pytest.raises(TypeError) as e:
a_random_function()
assert e.type == CustomExc
Both these snippets do the same thing:
Try/except in function declaration:
def something():
try:
# code goes here
except:
print("Error")
sys.exit(1)
something()
Try/except in function call:
def something():
# code goes here
try:
something()
except:
print("Error")
sys.exit(1)
Is there one that is better/more Pythonic/recommended by PEP8 or is it just up to personal preference? I understand that the second method would get tedious and repetitive if the function needs to be called more than once, but assuming the function is only called once, which one should I use?
the general rule is "only catch exceptions you can handle", see here for an explanation
note that an uncaught exception (in most languages) will cause the program to exit with an unsuccessful status code (i.e. your sys.exit(1)), it will probably also print out a message saying that an exception occurred. your demo therefore is emulating default behaviour, but doing it worse
further, you're catching every exception and this is generally bad style, e.g. you'll implicitly catch SystemExit and other internal exceptions that you probably shouldn't be dealing interacting with
I have a function that has try/except as follows:
def func_A():
try:
# do some stuff
except Exception as e:
log.error("there was an exception %s", str(e))
I want to write a unit test for this func_A()
More importantly, I want to ensure that
No exception was caught inside A
I have try/except just for safety.
Unless there is a bug, there should be no exception thrown inside A (although it will be caught with try/except) and that's what I want to validate with my unit test.
What is the best way for unit test to catch the case where there was an exception thrown and caught?
If you really need this, one possible way is to mock out the log.error object. After invoking the func_A function, you can make an assertion that your mock wasn't called.
Note that you should not catch exceptions at all if you don't intend to actually handle them. For proper test coverage, you should provide 2 tests here - one which checks each branching of the try/except.
Another possible solution is to split implementation into two functions:
Function foo() with logic without try statement. This way you can make sure that no exception is thrown in your implementation.
safe_foo() which wraps foo() into try statement. Then you can mock foo() to simulate throwing an exception by it and make sure every exception is caught.
Drawback is that either foo() will be part of a public interface or you will write tests for a private function.
You can have one variable which will track function executed properly or ended in exception.
def func_A():
function_state = True
try:
# do some stuff
except Exception as e:
log.error("there was an exception %s", str(e))
function_state = False
return function_state
Use assertTrue to validate function_state.
Option 1: Don't. This is testing an implementation detail. Try to write your test suite so that you very the function does everything you need it to do. If it does what you want with the inputs you want, you're good.
Option 2: You can modify the function to take a logger as a parameter. Then in the test case, pass in a mock object and check that the logging method gets called.
There is a simple scenario that I seem to encounter quite often: I invoke a function that can raise any number of exceptions. I won't do anything different if it is one exception versus another, I just want to log the exception information and either re-raise the exception or indicate in some other way that something didn't go as planned (such as returning None), otherwise proceed normally. So I use some form of the exception handling shown below.
Please note:
Imagine his code is running in a daemon that processes messages, so it needs to keep running, even if one of the messages causes some kind of exception.
I am aware that there is a rule of thumb that it is not generally advisable to catch a generic Exception because that may hide specfic errors that should be handled differently. (This is true in other languages as well.) This case is different because I don't care what exception is raised, the handling is the same.
Is there a better way?
def my_func(p1):
retval = None
try:
valx = other_func1(p1)
except Exception as ex:
log.error('other_func1 failed. {}: {}'.format(type(ex).__name__, ex))
else:
retval = ...
return retval
Is there a better way?
Doubt it, Python has these built-in Base Exception Classes so creating something on your own is really just being redundant. If you handle everything in the same way, generalizing in your except with Exception is most likely the best way to tackle this.
Small caveat here: Exception isn't the most general you can get, from the documentation:
All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class.
So, it won't catch all exceptions:
In [4]: try:
...: raise SystemExit
...: except Exception as b:
...: print("Catch All")
To exit: use 'exit', 'quit', or Ctrl-D.
An exception has occurred, use %tb to see the full traceback.
SystemExit
Which, do note, is of course something you should want. A SystemExit should exit. But, if some edge case requires it, to also catch system-exiting exceptions you can use BaseException which is as loose as you can get with exception matching:
In [2]: try:
...: raise SystemExit
...: except BaseException as b:
...: print("Catch All")
Catch All
Use it at your own discretion but, it probably makes zero sense to actually use it, and this case does not seem to require it. I just mentioned it because it is the most general you can get. I believe the way you have done it is more than sufficient.
That looks like a fine way to catch them if you're handling them all the same way. If you want to check what kind of exception was raised, you can use the built-in function type and compare the result to an exception class (for example, one from the list of built-in exception types):
try:
f()
except Exception as ex:
if type(ex)==ValueError:
handle_valueerror()
else:
handle_other_exception()
If you're handling them differently, use except <SpecificExceptionClass>. I'm not sure what I was thinking before.
In a unittest I want to do something like:
result = myFunction()
self.assertFalse(result) OR self.assertRaises(Exception, myFunction)
with the idea being that if the implementation of myFunction changes to return false instead of raise an exception (or vice-versa), the test will still behave correctly (i.e. indicate a non-positive result).
Is something like this possible?
You can check if myFunction() raises a specific exception:
try:
self.assertFalse(myFunction())
except SpecificException:
pass
In this case if myFunction() raises SpecificException it would be silently ignored.
In other words, you would see the test failing in two cases: either myFunction() raises the exception you are not waiting for, or the result of myFunction() call doesn't evaluate to False.
In case if your function can throw any exception, you can check whether you caught an AssertionError in the except block and reraise it, silently ignoring other exceptions:
try:
self.assertFalse(myFunction())
except AssertionError:
raise
except:
pass
Note that this doesn't seem very intuitive and clear to me. In an ideal world, you would need to handle these two situations separately: create a test that would make myFunction() result into False and another one that would make it raise an exception.
You could, but that will make your test weaker. Every other unit test that uses myFunction will have to assume that it could either raise an exception or return False -- or else that test may break when the behavior changes.
Making all your other tests accomodate this uncertainty will make all those pieces of code harder to write. And for what purpose? The benefit is dubious.
Instead, make a definite test which uses assertRaises. Let the test fail if the implementation changes, and change the test then to assertFalse. That change may also cause other tests to fail, and that is a good thing. It will force you to review use cases for myFunction, and the repercussions for changing the behavior.