Develop Reference

Python unittest: Accessing decorated test attributes in setUp

I'm trying to figure out how to decorate a test function in a way that makes the information from the decorator available to setUp. The code looks something like this:
import unittest
class MyTest(unittest.TestCase):
def setUp(self):
stopService()
eraseAllPreferences()
setTestPreferences()
startService()
#setPreference("abc", 5)
def testPreference1(self):
pass
#setPreference("xyz", 5)
def testPreference2(self):
pass
The goal is for setUp to understand it's running testPreference1 and to know that it needs to set preference "abc" to 5 before starting the service (& similarly regarding "xyz" and testPreference2).
I can of course just use a conditional on the the test name (if self._testMethodName == "testPreference1") but that doesn't feel quite as maintainable as the number of tests grows (+ refactoring is more error-prone). I'm hoping to solve this in setUp rather than overriding the run implementation. I'm also having
I'm running python3.6 although if there are creative solutions depending on newer python features happy to learn about that too.

Decorators work well but there's no real "official" way to get the underlying method so I just did what the unittest source does: method = getattr(self, self._testMethodName)
import functools
import unittest
def setFoo(value):
def inner(func):
print(f"Changing foo for function {func}")
func.foo = value
#functools.wraps(func)
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
return wrapper
return inner
class Foo(unittest.TestCase):
def setUp(self):
method = getattr(self, self._testMethodName)
print(f"Foo = {method.foo}")
#setFoo("abc")
def testFoo(self):
self.assertEqual(self.testFoo.foo, "abc")
#setFoo("xyz")
def testBar(self):
self.assertEqual(self.testBar.foo, "xyz")
if __name__ == "__main__":
unittest.main()

How can I stop the python process from terminating after a failed unit test so that I can call pdb.pm [duplicate]

Is there a way to automatically start the debugger at the point at which a unittest fails?
Right now I am just using pdb.set_trace() manually, but this is very tedious as I need to add it each time and take it out at the end.
For Example:
import unittest
class tests(unittest.TestCase):
def setUp(self):
pass
def test_trigger_pdb(self):
#this is the way I do it now
try:
assert 1==0
except AssertionError:
import pdb
pdb.set_trace()
def test_no_trigger(self):
#this is the way I would like to do it:
a=1
b=2
assert a==b
#magically, pdb would start here
#so that I could inspect the values of a and b
if __name__=='__main__':
#In the documentation the unittest.TestCase has a debug() method
#but I don't understand how to use it
#A=tests()
#A.debug(A)
unittest.main()

I think what you are looking for is nose. It works like a test runner for unittest.
You can drop into the debugger on errors, with the following command:
nosetests --pdb

import unittest
import sys
import pdb
import functools
import traceback
def debug_on(*exceptions):
if not exceptions:
exceptions = (AssertionError, )
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
try:
return f(*args, **kwargs)
except exceptions:
info = sys.exc_info()
traceback.print_exception(*info)
pdb.post_mortem(info[2])
return wrapper
return decorator
class tests(unittest.TestCase):
#debug_on()
def test_trigger_pdb(self):
assert 1 == 0
I corrected the code to call post_mortem on the exception instead of set_trace.

Third party test framework enhancements generally seem to include the feature (nose and nose2 were already mentioned in other answers). Some more:
pytest supports it.
pytest --pdb
Or if you use absl-py's absltest instead of unittest module:
name_of_test.py --pdb_post_mortem

A simple option is to just run the tests without result collection and letting the first exception crash down the stack (for arbitrary post mortem handling) by e.g.
try: unittest.findTestCases(__main__).debug()
except:
pdb.post_mortem(sys.exc_info()[2])
Another option: Override unittest.TextTestResult's addError and addFailure in a debug test runner for immediate post_mortem debugging (before tearDown()) - or for collecting and handling errors & tracebacks in an advanced way.
(Doesn't require extra frameworks or an extra decorator for test methods)
Basic example:
import unittest, pdb
class TC(unittest.TestCase):
def testZeroDiv(self):
1 / 0
def debugTestRunner(post_mortem=None):
"""unittest runner doing post mortem debugging on failing tests"""
if post_mortem is None:
post_mortem = pdb.post_mortem
class DebugTestResult(unittest.TextTestResult):
def addError(self, test, err):
# called before tearDown()
traceback.print_exception(*err)
post_mortem(err[2])
super(DebugTestResult, self).addError(test, err)
def addFailure(self, test, err):
traceback.print_exception(*err)
post_mortem(err[2])
super(DebugTestResult, self).addFailure(test, err)
return unittest.TextTestRunner(resultclass=DebugTestResult)
if __name__ == '__main__':
##unittest.main()
unittest.main(testRunner=debugTestRunner())
##unittest.main(testRunner=debugTestRunner(pywin.debugger.post_mortem))
##unittest.findTestCases(__main__).debug()

To apply #cmcginty's answer to the successor nose 2 (recommended by nose available on Debian-based systems via apt-get install nose2), you can drop into the debugger on failures and errors by calling
nose2
in your test directory.
For this, you need to have a suitable .unittest.cfg in your home directory or unittest.cfg in the project directory; it needs to contain the lines
[debugger]
always-on = True
errors-only = False

To address the comment in your code "In the documentation the unittest.TestCase has a debug() method but I don't understand how to use it", you can do something like this:
suite = unittest.defaultTestLoader.loadTestsFromModule(sys.modules[__name__])
suite.debug()
Individual test cases are created like:
testCase = tests('test_trigger_pdb') (where tests is a sub-class of TestCase as per your example). And then you can do testCase.debug() to debug one case.

Here's a built-in, no extra modules, solution:
import unittest
import sys
import pdb
####################################
def ppdb(e=None):
"""conditional debugging
use with: `if ppdb(): pdb.set_trace()`
"""
return ppdb.enabled
ppdb.enabled = False
###################################
class SomeTest(unittest.TestCase):
def test_success(self):
try:
pass
except Exception, e:
if ppdb(): pdb.set_trace()
raise
def test_fail(self):
try:
res = 1/0
#note: a `nosetests --pdb` run will stop after any exception
#even one without try/except and ppdb() does not not modify that.
except Exception, e:
if ppdb(): pdb.set_trace()
raise
if __name__ == '__main__':
#conditional debugging, but not in nosetests
if "--pdb" in sys.argv:
print "pdb requested"
ppdb.enabled = not sys.argv[0].endswith("nosetests")
sys.argv.remove("--pdb")
unittest.main()
call it with python myunittest.py --pdb and it will halt. Otherwise it won't.

Some solution above modifies business logic:
try: # <-- new code
original_code() # <-- changed (indented)
except Exception as e: # <-- new code
pdb.post_mortem(...) # <-- new code
To minimize changes to the original code, we can define a function decorator, and simply decorate the function that's throwing:
def pm(func):
import functools, pdb
#functools.wraps(func)
def func2(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
pdb.post_mortem(e.__traceback__)
raise
return func2
Use:
#pm
def test_xxx(...):
...

Buildt a module with a decorator which post mortems into every type of error except AssertionError. The decorator can be triggered by the logging root level
#!/usr/bin/env python3
'''
Decorator for getting post mortem on errors of a unittest TestCase
'''
import sys
import pdb
import functools
import traceback
import logging
import unittest
logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG)
def debug_on(log_level):
'''
Function decorator for post mortem debugging unittest functions.
Args:
log_level (int): logging levels coesponding to logging stl module
Usecase:
class tests(unittest.TestCase):
#debug_on(logging.root.level)
def test_trigger_pdb(self):
assert 1 == 0
'''
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
try:
return f(*args, **kwargs)
except BaseException as err:
info = sys.exc_info()
traceback.print_exception(*info)
if log_level < logging.INFO and type(err) != AssertionError:
pdb.post_mortem(info[2])
return wrapper
return decorator
class Debug_onTester(unittest.TestCase):
#debug_on(logging.root.level)
def test_trigger_pdb(self):
assert 1 == 0
if __name__ == '__main__':
unittest.main()

How to mock function only in tested class and not in unittest case

So i have unit test case class and a class that I want to test. There is a function in this class that has a function that I'm using to run this method that i am testing. To visualize the problem i will write some abstract code below:
import needed_module
from casual_class import CasualClass
class CasualClassTests(unittest.TestCase):
def setUp(self):
self.casual_class = CasualClass()
def _run(self, func):
result = needed_module.runner(func)
return result
#mock.patch('needed_module.runner', return_value='test_run_output')
def test_get_that_item(self, mocked_runner):
result = self._run(self.casual_class.get_that_item())
And the tested class:
import needed_module
class CasualClass:
def get_that_item(self):
#..some code..
run_output = needed_module.runner(something)
return something2
In this code get_that_item code wont even run because the runner is mocked. What i want to achieve is to run needed_module.runner original in test case and mocked one in the tested class. I've searched the internet for too long, to solve this...

To achieve such a task I had to create an object of needed_module before patching the function and pass it to _run function in test case to use it.
class CasualClassTests(unittest.TestCase):
def setUp(self):
self.casual_class = CasualClass()
def _run(self, func, runner=None):
if not runner:
result = needed_module.runner(func)
else:
result = runner(func)
return result
def test_get_that_item(self, mocked_runner):
runner = needed_module.runner
with mock.patch('needed_module.runner', return_value='test_run_output'):
result = self._run(self.casual_class.get_that_item(), runner)

How to detect when pytest test case failed?

I am using pytest with selenium to automate a website. I want to take some screen shot only when a test case fails. I have previosly used TestNG and with TestNG it's quite east using the ITestListner.
Do we have something like that in pytest.
I have tried to achieve this using the teardown_method()
But this method is not getting executed when a test case fails.
import sys
from unittestzero import Assert
class TestPY:
def setup_method(self, method):
print("in setup method")
print("executing " + method.__name__)
def teardown_method(self, method):
print(".....teardown")
if sys.exc_info()[0]:
test_method_name = method
print test_method_name
def test_failtest(self):
Assert.fail("failed test")
teardown_method() get executed only when there are no fails

According to you post on stackoverflow, I can share that I is something on my mind, I hope it will help:wink:
What you're trying to do is to handle standard AssertionError exception that can be raised by assert keyword or by any assertion method implemented in unittest.TestCase or maybe any custom assertion method that raises custom exception.
There are 3 ways to do that:
Use try-except-finally construction. Some basic example:
try:
Assert.fail("failed test")
except AssertionError:
get_screenshot()
raise
Or use with statement, as context manager:
class TestHandler:
def __enter__(self):
# maybe some set up is expected before assertion method call
pass
def __exit__(self, exc_type, exc_val, exc_tb):
# catch whether exception was raised
if isinstance(exc_val, AssertionError):
get_screenshot()
with TestHandler():
Assert.fail("failed test")
here you can dive deeper on how to play with it
The last one, in my opinion, is the most elegant approach. Using decorators. With this decorator you can decorate any testing method:
def decorator_screenshot(func):
def wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except AssertionError:
get_screenshot()
raise
return wrapper
#decorator_screenshot
def test_something():
Assert.fail("failed test")

After some struggle, eventually this worked for me.
In conftest.py:
#pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
outcome = yield
rep = outcome.get_result()
setattr(item, "rep_" + rep.when, rep)
return rep
And, in your code, in a fixture (e.g., in a teardown fixture for tests) use it like so:
def tear_down(request):
method_name = request.node.name
if request.node.rep_call.failed:
print('test {} failed :('.format(method_name))
# do more stuff like take a selenium screenshot
Note that "request" is a fixture "funcarg" that pytest provides in the context of your tests. You don't have to define it yourself.
Sources: pytest examples and thread on (not) making this easier.

This is how we do it , note __multicall__ has very less documentation and I remember reading __multicall__ is going to be deprecated, please use this with a pinch of salt and experiment with replacing __multicall__ with 'item, call' as per the examples.
def pytest_runtest_makereport(__multicall__):
report = __multicall__.execute()
if report.when == 'call':
xfail = hasattr(report, 'wasxfail')
if (report.skipped and xfail) or (report.failed and not xfail):
try:
screenshot = APP_DRIVER.take_screen_shot(format="base64")
except Exception as e:
LOG.debug("Error saving screenshot !!")
LOG.debug(e)
return report

def pytest_runtest_makereport(item, call):
if call.when == 'call':
if call.excinfo is not None:
# if excinfor is not None, indicate that this test item is failed test case
error("Test Case: {}.{} Failed.".format(item.location[0], item.location[2]))
error("Error: \n{}".format(call.excinfo))

Python magic method to alter the way raising an object is handled

I was wondering, is there a simple magic method in python that allows customization of the behaviour of an exception-derived object when it is raised? I'm looking for something like __raise__ if that exists. If no such magic methods exist, is there any way I could do something like the following (it's just an example to prove my point):
class SpecialException(Exception):
def __raise__(self):
print('Error!')
raise SpecialException() #this is the part of the code that must stay
Is it possible?

I don't know about such magic method but even if it existed it is just some piece of code that gets executed before actually raising the exception object. Assuming that its a good practice to raise exception objects that are instantiated in-place you can put such code into the __init__ of the exception. Another workaround: instead of raising your exception directly you call an error handling method/function that executes special code and then finally raises an exception.

import time
from functools import wraps
def capture_exception(callback=None, *c_args, **c_kwargs):
"""捕获到异常后执行回调函数"""
assert callable(callback), "callback 必须是可执行对象"
def _out(func):
#wraps(func)
def _inner(*args, **kwargs):
try:
res = func(*args, **kwargs)
return res
except Exception as e:
callback(*c_args, **c_kwargs)
raise e
return _inner
return _out
def send_warning():
print("warning message..............")
class A(object):
#capture_exception(callback=send_warning)
def run(self):
print('run')
raise SystemError("测试异常捕获回调功能")
time.sleep(0.2)
if __name__ == '__main__':
a = A()
a.run()