I'm trying to write a nice error handler for my code, so that when it fails, the logs, traceback and other relevant info get emailed to me.
I can't figure out how to take an exception object and extract the traceback.
I find the traceback module pretty confusing, mostly because it doesn't deal with exceptions at all. It just fetches some global variables from somewhere, assuming that I want the most recent exception. But what if I don't? What if I want to ignore some exception in my error handler? (e.g. if I fail to send me email and want to retry.)
What I want
import traceback as tb
# some function that will fail after a few recursions
def myfunc(x):
assert x > 0, "oh no"
return myfunc(x-1)
try:
myfunc(3)
except Exception as e:
traceback_str = tb.something(e)
Note that tb.something takes e as an argument.
There's lots of questions on Stack Overflow about using the traceback module to get a traceback string. The unique thing about this question is how to get it from the caught exception, instead of global variables.
Result:
traceback_str contains the string:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in myfunc
File "<stdin>", line 3, in myfunc
File "<stdin>", line 3, in myfunc
File "<stdin>", line 2, in myfunc
AssertionError: oh no
Note that it contains not just the most recent function call, but the whole stack, and includes the "AssertionError" at the end
The correct function for tb.something(e) is
''.join(tb.format_exception(None, e, e.__traceback__))
Related
I'm currently learning how to code in python and I was trying to raise exceptions in my code to make my tool more user-friendly. However, I find the template given by Maya to be too simple and I'd like to have more control over it.
Normally, the base template looks a little like this:
raise Exception(errorTitle)
>>> Result
# Error: errorTitle
# Traceback (most recent call last):
# File "<maya console>", line 1, in <module>
# Exception: errorTitle #
It works fine, but it doesn't let you give a more detailed description of the error. What I'm trying to achieve would look a little more like this:
raise Exception(errorTitle, errorDescription)
>>> Result
# Error: errorTitle
# errorDescription
# Traceback (most recent call last):
# File "<maya console>", line 1, in <module>
# Exception: errorTitle #
I also tried putting the errorDescription into errorTitle by adding a new line to the string, but since errorTitle is repeated twice, it becomes confusing quickly when trying to sort errors.
I know you can't really use multiple arguments in the raise function, so I was wondering if there was an error template in Maya I could reference instead of raising an error.
Alternatively, is there a way to create my own error template? If I can reuse the Traceback function and change the script bar color, it might be easier to just do that.
Thanks in advance!
You can get your desired result just like that:
>>> raise Exception("Error Title\nError description that explains the problem")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: Error Title
Error description that explains the problem
Or you may want to declare your custom exception class:
class YourException(Exception):
def __init__(self, title, description):
super(YourException, self).__init__(title + "\n" + description)
and then you can use your original syntax:
>>> raise YourException("Error Title", "Error description that explains the problem")
File "<stdin>", line 1, in <module>
__main__.YourException: Error Title
Error description that explains the problem
I wrote this program:
def fun():
try: 1/0
except: fun()
fun()
I thought that I will get an exception, but instead, I got the following fatal error:
Fatal Python error: Cannot recover from stack overflow.
Current thread 0x00003bec (most recent call first):
File "<stdin>", line 2 in fun
File "<stdin>", line 3 in fun
(the File "<stdin>", line 3 in fun line is shown 98 times) and then the program crushes (instead of raising an exception).
I don't really get why this happens.
When I run the above program without errors it just raises an exception:
def fun():
fun()
fun()
Raises the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in fun
File "<stdin>", line 2, in fun
File "<stdin>", line 2, in fun
[Previous line repeated 995 more times]
RecursionError: maximum recursion depth exceeded
But when the code is erroneous, the program just crashes.
Can anyone explain to me why this happens?
Yes you call your function within the function of the same name leading you down a rabbit hole of recursion (a function calling itself) with no escape
def fun():
try: 1/0
except: fun()
This means that when you call fun() if 1/0 raises an error it will move to the except branch and call the function fun which if 1/0 raises an error it will move to the except branch and call the function fun which if 1/0 raises an error it will move to the except branch and call the function fun which if 1/0 raises an error it will move to the except branch and call the function fun which...
If you get the picture.
So if it's error handling you are learning you might simple want to return some value like:
def fun():
try:
1/0
except:
return "Error handling worked"
fun()
Consider the following Python program:
code = """
def test():
1/0
"""
filename = "<test>"
c = compile(code, filename, 'exec')
exec(c)
import linecache
linecache.cache[filename] = (len(code), None, code.splitlines(keepends=True), filename)
import traceback
print("Traceback from the traceback module:")
print()
try:
test()
except:
traceback.print_exc()
print()
print("Regular traceback:")
print()
test()
I am dynamically defining a function that raises an exception and adding it to the linecache. The output of the code is
Traceback from the traceback module:
Traceback (most recent call last):
File "test.py", line 20, in <module>
test()
File "<test>", line 3, in test
1/0
ZeroDivisionError: division by zero
Regular traceback:
Traceback (most recent call last):
File "test.py", line 28, in <module>
test()
File "<test>", line 3, in test
ZeroDivisionError: division by zero
If I then get a traceback from that function using the traceback module, the line of code from the function is shown (the 1/0 part of the first traceback). But if I just let the code raise an exception and get the regular traceback from the interpreter, it doesn't show the code.
Why doesn't the regular interpreter traceback use the linecache? Is there a way to make the code appear in regular tracebacks?
The default sys.excepthook uses a separate, C-level implementation of traceback printing, not the traceback module. (Perhaps this is so it still works even if the system is too borked to use traceback.py.) The C implementation doesn't try to use linecache. You can see the code it uses to retrieve source lines in _Py_DisplaySourceLine.
If you want tracebacks to use the traceback module's implementation, you could replace sys.excepthook with traceback.print_exception:
import sys
import traceback
sys.excepthook = traceback.print_exception
In the module warnings (https://docs.python.org/3.5/library/warnings.html) there is the ability to raise a warning that appears to come from somewhere earlier in the stack:
warnings.warn('This is a test', stacklevel=2)
Is there an equivalent for raising errors? I know I can raise an error with an alternative traceback, but I can't create that traceback within the module since it needs to come from earlier. I imagine something like:
tb = magic_create_traceback_right_here()
raise ValueError('This is a test').with_traceback(tb.tb_next)
The reason is that I am developing a module that has a function module.check_raise that I want to raise an error that appears to originate from where the function is called. If I raise an error within the module.check_raise function, it appears to originate from within module.check_raise, which is undesired.
Also, I've tried tricks like raising a dummy exception, catching it, and passing the traceback along, but somehow the tb_next becomes None. I'm out of ideas.
Edit:
I would like the output of this minimal example (called tb2.py):
import check_raise
check_raise.raise_if_string_is_true('True')
to be only this:
Traceback (most recent call last):
File "tb2.py", line 10, in <module>
check_raise.raise_if_string_is_true(string)
RuntimeError: An exception was raised.
I can't believe I am posting this
By doing this you are going against the zen.
Special cases aren't special enough to break the rules.
But if you insist here is your magical code.
check_raise.py
import sys
import traceback
def raise_if_string_is_true(string):
if string == 'true':
#the frame that called this one
f = sys._getframe().f_back
#the most USELESS error message ever
e = RuntimeError("An exception was raised.")
#the first line of an error message
print('Traceback (most recent call last):',file=sys.stderr)
#the stack information, from f and above
traceback.print_stack(f)
#the last line of the error
print(*traceback.format_exception_only(type(e),e),
file=sys.stderr, sep="",end="")
#exit the program
#if something catches this you will cause so much confusion
raise SystemExit(1)
# SystemExit is the only exception that doesn't trigger an error message by default.
This is pure python, does not interfere with sys.excepthook and even in a try block it is not caught with except Exception: although it is caught with except:
test.py
import check_raise
check_raise.raise_if_string_is_true("true")
print("this should never be printed")
will give you the (horribly uninformative and extremely forged) traceback message you desire.
Tadhgs-MacBook-Pro:Documents Tadhg$ python3 test.py
Traceback (most recent call last):
File "test.py", line 3, in <module>
check_raise.raise_if_string_is_true("true")
RuntimeError: An exception was raised.
Tadhgs-MacBook-Pro:Documents Tadhg$
If I understand correctly, you would like the output of this minimal example:
def check_raise(function):
try:
return function()
except Exception:
raise RuntimeError('An exception was raised.')
def function():
1/0
check_raise(function)
to be only this:
Traceback (most recent call last):
File "tb2.py", line 10, in <module>
check_raise(function)
RuntimeError: An exception was raised.
In fact, it's a lot more output; there is exception chaining, which could be dealt with by handling the RuntimeError immediately, removing its __context__, and re-raising it, and there is another line of traceback for the RuntimeError itself:
File "tb2.py", line 5, in check_raise
raise RuntimeError('An exception was raised.')
As far as I can tell, it is not possible for pure Python code to substitute the traceback of an exception after it was raised; the interpreter has control of adding to it but it only exposes the current traceback whenever the exception is handled. There is no API (not even when using tracing functions) for passing your own traceback to the interpreter, and traceback objects are immutable (this is what's tackled by that Jinja hack involving C-level stuff).
So further assuming that you're interested in the shortened traceback not for further programmatic use but only for user-friendly output, your best bet will be an excepthook that controls how the traceback is printed to the console. For determining where to stop printing, a special local variable could be used (this is a bit more robust than limiting the traceback to its length minus 1 or such). This example requires Python 3.5 (for traceback.walk_tb):
import sys
import traceback
def check_raise(function):
__exclude_from_traceback_from_here__ = True
try:
return function()
except Exception:
raise RuntimeError('An exception was raised.')
def print_traceback(exc_type, exc_value, tb):
for i, (frame, lineno) in enumerate(traceback.walk_tb(tb)):
if '__exclude_from_traceback_from_here__' in frame.f_code.co_varnames:
limit = i
break
else:
limit = None
traceback.print_exception(
exc_type, exc_value, tb, limit=limit, chain=False)
sys.excepthook = print_traceback
def function():
1/0
check_raise(function)
This is the output now:
Traceback (most recent call last):
File "tb2.py", line 26, in <module>
check_raise(function)
RuntimeError: An exception was raised.
EDIT: The previous version did not provide quotes or explanations.
I suggest referring to PEP 3134 which states in the Motivation:
Sometimes it can be useful for an exception handler to intentionally
re-raise an exception, either to provide extra information or to
translate an exception to another type. The __cause__ attribute
provides an explicit way to record the direct cause of an exception.
When an Exception is raised with a __cause__ attribute the traceback message takes the form of:
Traceback (most recent call last):
<CAUSE TRACEBACK>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
<MAIN TRACEBACK>
To my understanding this is exactly what you are trying to accomplish; clearly indicate that the reason for the error is not your module but somewhere else. If you are instead trying to omit information to the traceback like your edit suggests then the rest of this answer won't do you any good.
Just a note on syntax:
The __cause__ attribute on exception objects is always initialized
to None. It is set by a new form of the 'raise' statement:
raise EXCEPTION from CAUSE
which is equivalent to:
exc = EXCEPTION
exc.__cause__ = CAUSE
raise exc
so the bare minimum example would be something like this:
def function():
int("fail")
def check_raise(function):
try:
function()
except Exception as original_error:
err = RuntimeError("An exception was raised.")
raise err from original_error
check_raise(function)
which gives an error message like this:
Traceback (most recent call last):
File "/PATH/test.py", line 7, in check_raise
function()
File "/PATH/test.py", line 3, in function
int("fail")
ValueError: invalid literal for int() with base 10: 'fail'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/PATH/test.py", line 12, in <module>
check_raise(function)
File "/PATH/test.py", line 10, in check_raise
raise err from original_error
RuntimeError: An exception was raised.
However the first line of the cause is the statement in the try block of check_raise:
File "/PATH/test.py", line 7, in check_raise
function()
so before raising err it may (or may not) be desirable to remove the outer most traceback frame from original_error:
except Exception as original_error:
err = RuntimeError("An exception was raised.")
original_error.__traceback__ = original_error.__traceback__.tb_next
raise err from original_error
This way the only line in the traceback that appears to come from check_raise is the very last raise statement which cannot be omitted with pure python code although depending on how informative the message is you can make it very clear that your module was not the cause of the problem:
err = RuntimeError("""{0.__qualname__} encountered an error during call to {1.__module__}.{1.__name__}
the traceback for the error is shown above.""".format(function,check_raise))
The advantage to raising exception like this is that the original Traceback message is not lost when the new error is raised, which means that a very complex series of exceptions can be raised and python will still display all the relevant information correctly:
def check_raise(function):
try:
function()
except Exception as original_error:
err = RuntimeError("""{0.__qualname__} encountered an error during call to {1.__module__}.{1.__name__}
the traceback for the error is shown above.""".format(function,check_raise))
original_error.__traceback__ = original_error.__traceback__.tb_next
raise err from original_error
def test_chain():
check_raise(test)
def test():
raise ValueError
check_raise(test_chain)
gives me the following error message:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/test.py", line 16, in test
raise ValueError
ValueError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/test.py", line 13, in test_chain
check_raise(test)
File "/Users/Tadhg/Documents/test.py", line 10, in check_raise
raise err from original_error
RuntimeError: test encountered an error during call to __main__.check_raise
the traceback for the error is shown above.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/test.py", line 18, in <module>
check_raise(test_chain)
File "/Users/Tadhg/Documents/test.py", line 10, in check_raise
raise err from original_error
RuntimeError: test_chain encountered an error during call to __main__.check_raise
the traceback for the error is shown above.
Yes it is long but it is significantly more informative then:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/test.py", line 18, in <module>
check_raise(test_chain)
RuntimeError: An exception was raised.
not to mention that the original error is still usable even if the program doesn't end:
import traceback
def check_raise(function):
...
def fail():
raise ValueError
try:
check_raise(fail)
except RuntimeError as e:
cause = e.__cause__
print("check_raise failed because of this error:")
traceback.print_exception(type(cause), cause, cause.__traceback__)
print("and the program continues...")
I understand 'Don't do this'. On the other hand, there may be some special use cases i believe. I'm generating own errors (just deleting some defined frames...) this way
def get_traceback_with_removed_frames_by_line_string(lines):
"""In traceback call stack, it is possible to remove particular level defined by some line content.
Args:
lines (list): Line in call stack that we want to hide.
Returns:
string: String traceback ready to be printed.
"""
exc = trcbck.TracebackException(*sys.exc_info())
for i in exc.stack[:]:
if i.line in lines:
exc.stack.remove(i)
return "".join(exc.format())
I return just string.
If there is concrete function that is raising, you can add it to ignored frames.
Though have in mind, that if you hide something, somebody may not understand why is something happening...
My use case was to hide only first level - decorator from my library that was decorating all user functions in framework, so error from user side was on level 1.
I've build a little internal DSL with python. I'm using assert vor validation. If the end user types a wrong parameter the dsl should report whats wrong. At the moment this looks like this:
Traceback (most recent call last):
File "tests/maskedtimefilter_test/FilterDSL_test.py", line 63, in test_dsl_validation
input(0): self.regular
File "/Users/sh/reetz/pythonpath/maskedtimedata/maskedtimefilter.py", line 885, in __lshift__
kwargs = self.dsl_validation(kwargs)
File "/Users/sh/reetz/pythonpath/maskedtimedata/maskedtimefilter.py", line 1483, in dsl_validation
check_if_valid(parameter)
File "/Users/sh/reetz/pythonpath/maskedtimedata/dsl.py", line 47, in kernel_a
def kernel_a (x): assert isinstance(x, (list, tuple, np.ndarray)), "kernel must be a list."
AssertionError: kernel must be a list.
But the end users are engineers and no computer scientists. Therefore a minimal Traceback is handy. Is it possible to shrink the Traceback to the essential information (where is the failure and what's the cause) like this?:
Traceback (most recent call last):
File "tests/maskedtimefilter_test/FilterDSL_test.py", line 63, in test_dsl_validation
input(0): self.regular
AssertionError: kernel must be a list.
Reluctantly I'd like to use normal prints!
Why not return the traceback data as an array and just work back from that?
import traceback
try:
codethatwillthrowanexception()
except:
exceptiondata = traceback.format_exc().splitlines()
exceptionarray = [exceptiondata[-1]] + exceptiondata[1:-1]
Somewhere in your call stack you can do this:
try:
my_function_that_can_raise_exception()
except Exception: #or AssertionError
import traceback
traceback.print_exc(limit=1)
limit is the depth of how many stack trace entries you want to show. (in your case, 1)
demo:
In [21]: def f():
...: assert False == True, 'Assertion!'
...:
In [22]: try:
...: f()
...: except Exception:
...: import traceback
...: traceback.print_exc(limit=1)
...:
Traceback (most recent call last):
File "<ipython-input-22-78f9db8d5188>", line 2, in <module>
f()
AssertionError: Assertion!
read more at traceback docs.
As the first part of the traceback is just what you called, you could so something simple, like:
try:
input(0): self.regular # base call
except AssertionError as ae: # catch the exception
print(ae) # print out the message
You can print out whatever you think would be useful in the except block.
Alternatively, you could do something more complex with the traceback module.