Related
I want to catch and log exceptions without exiting, e.g.,
try:
do_stuff()
except Exception as err:
print(Exception, err)
# I want to print the entire traceback here,
# not just the exception name and details
I want to print the exact same output that is printed when the exception is raised without the try/except intercepting the exception, and I do not want it to exit my program.
traceback.format_exc() or sys.exc_info() will yield more info if that's what you want.
import traceback
import sys
try:
do_stuff()
except Exception:
print(traceback.format_exc())
# or
print(sys.exc_info()[2])
Some other answer have already pointed out the traceback module.
Please notice that with print_exc, in some corner cases, you will not obtain what you would expect. In Python 2.x:
import traceback
try:
raise TypeError("Oups!")
except Exception, err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_exc()
...will display the traceback of the last exception:
Traceback (most recent call last):
File "e.py", line 7, in <module>
raise TypeError("Again !?!")
TypeError: Again !?!
If you really need to access the original traceback one solution is to cache the exception infos as returned from exc_info in a local variable and display it using print_exception:
import traceback
import sys
try:
raise TypeError("Oups!")
except Exception, err:
try:
exc_info = sys.exc_info()
# do you usefull stuff here
# (potentially raising an exception)
try:
raise TypeError("Again !?!")
except:
pass
# end of useful stuff
finally:
# Display the *original* exception
traceback.print_exception(*exc_info)
del exc_info
Producing:
Traceback (most recent call last):
File "t.py", line 6, in <module>
raise TypeError("Oups!")
TypeError: Oups!
Few pitfalls with this though:
From the doc of sys_info:
Assigning the traceback return value to a local variable in a function that is handling an exception will cause a circular reference. This will prevent anything referenced by a local variable in the same function or by the traceback from being garbage collected. [...] If you do need the traceback, make sure to delete it after use (best done with a try ... finally statement)
but, from the same doc:
Beginning with Python 2.2, such cycles are automatically reclaimed when garbage collection is enabled and they become unreachable, but it remains more efficient to avoid creating cycles.
On the other hand, by allowing you to access the traceback associated with an exception, Python 3 produce a less surprising result:
import traceback
try:
raise TypeError("Oups!")
except Exception as err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_tb(err.__traceback__)
... will display:
File "e3.py", line 4, in <module>
raise TypeError("Oups!")
If you're debugging and just want to see the current stack trace, you can simply call:
traceback.print_stack()
There's no need to manually raise an exception just to catch it again.
How to print the full traceback without halting the program?
When you don't want to halt your program on an error, you need to handle that error with a try/except:
try:
do_something_that_might_error()
except Exception as error:
handle_the_error(error)
To extract the full traceback, we'll use the traceback module from the standard library:
import traceback
And to create a decently complicated stacktrace to demonstrate that we get the full stacktrace:
def raise_error():
raise RuntimeError('something bad happened!')
def do_something_that_might_error():
raise_error()
Printing
To print the full traceback, use the traceback.print_exc method:
try:
do_something_that_might_error()
except Exception as error:
traceback.print_exc()
Which prints:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Better than printing, logging:
However, a best practice is to have a logger set up for your module. It will know the name of the module and be able to change levels (among other attributes, such as handlers)
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
In which case, you'll want the logger.exception function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.exception(error)
Which logs:
ERROR:__main__:something bad happened!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Or perhaps you just want the string, in which case, you'll want the traceback.format_exc function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.debug(traceback.format_exc())
Which logs:
DEBUG:__main__:Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Conclusion
And for all three options, we see we get the same output as when we have an error:
>>> do_something_that_might_error()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Which to use
Performance concerns aren't important here as IO usually dominates. I'd prefer, since it does precisely what's being requested in a forward compatible way:
logger.exception(error)
Logging levels and outputs can be adjusted, making it easy to turn off without touching the code. And usually doing what's directly needed is the most efficient way to do it.
First, do not use prints for logging, there is a stable, proven and well-thought out stdlib module to do that: logging. You definitely should use it instead.
Second, do not be tempted to do a mess with unrelated tools when there is a native and simple approach. Here it is:
log = logging.getLogger(__name__)
try:
call_code_that_fails()
except MyError:
log.exception('Any extra info you want to see in your logs')
That's it. You are done now.
Explanation for anyone who is interested in how things work under the hood
What log.exception is actually doing is just a call to log.error (that is, log event with level ERROR) and print traceback then.
Why is it better?
Well, here are some considerations:
it is just right;
it is straightforward;
it is simple.
Why should nobody use traceback or call logger with exc_info=True or get their hands dirty with sys.exc_info?
Well, just because! They all exist for different purposes. For example, traceback.print_exc's output is a little bit different from tracebacks produced by the interpreter itself. If you use it, you will confuse anyone who reads your logs, they will be banging their heads against them.
Passing exc_info=True to log calls is just inappropriate. But, it is useful when catching recoverable errors and you want to log them (using, e.g INFO level) with tracebacks as well, because log.exception produces logs of only one level - ERROR.
And you definitely should avoid messing with sys.exc_info as much as you can. It's just not a public interface, it's an internal one - you can use it if you definitely know what you are doing. It is not intended for just printing exceptions.
traceback.format_exception(exception_object)
If you only have the exception object, you can get the traceback as a string from any point of the code in Python 3 with:
import traceback
''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
Full example:
#!/usr/bin/env python3
import traceback
def f():
g()
def g():
raise Exception('asdf')
try:
g()
except Exception as e:
exc_obj = e
tb_str = ''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
print(tb_str)
Output:
Traceback (most recent call last):
File "./main.py", line 12, in <module>
g()
File "./main.py", line 9, in g
raise Exception('asdf')
Exception: asdf
Documentation: https://docs.python.org/3.9/library/traceback.html#traceback.format_exception
See also: Extract traceback info from an exception object
Tested in Python 3.9
In addition to Aaron Hall's answer, if you are logging, but don't want to use logging.exception() (since it logs at the ERROR level), you can use a lower level and pass exc_info=True. e.g.
try:
do_something_that_might_error()
except Exception:
logging.info('General exception noted.', exc_info=True)
I don't see this mentioned in any of the other answers. If you're passing around an Exception object for whatever reason...
In Python 3.5+ you can get a trace from an Exception object using traceback.TracebackException.from_exception(). For example:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
try:
stack_lvl_3()
except Exception as e:
# raise
return e
def stack_lvl_1():
e = stack_lvl_2()
return e
e = stack_lvl_1()
tb1 = traceback.TracebackException.from_exception(e)
print(''.join(tb1.format()))
However, the above code results in:
Traceback (most recent call last):
File "exc.py", line 10, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Exception: ('a1', 'b2', 'c3')
This is just two levels of the stack, as opposed to what would have been printed on screen had the exception been raised in stack_lvl_2() and not intercepted (uncomment the # raise line).
As I understand it, that's because an exception records only the current level of the stack when it is raised, stack_lvl_3() in this case. As it's passed back up through the stack, more levels are being added to its __traceback__. But we intercepted it in stack_lvl_2(), meaning all it got to record was levels 3 and 2. To get the full trace as printed on stdout we'd have to catch it at the highest (lowest?) level:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
stack_lvl_3()
def stack_lvl_1():
stack_lvl_2()
try:
stack_lvl_1()
except Exception as exc:
tb = traceback.TracebackException.from_exception(exc)
print('Handled at stack lvl 0')
print(''.join(tb.stack.format()))
Which results in:
Handled at stack lvl 0
File "exc.py", line 17, in <module>
stack_lvl_1()
File "exc.py", line 13, in stack_lvl_1
stack_lvl_2()
File "exc.py", line 9, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Notice that the stack print is different, the first and last lines are missing. Because it's a different format().
Intercepting the exception as far away from the point where it was raised as possible makes for simpler code while also giving more information.
In python3 (works in 3.9) we can define a function and can use that where ever we want to print the details.
import traceback
def get_traceback(e):
lines = traceback.format_exception(type(e), e, e.__traceback__)
return ''.join(lines)
try:
1/0
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
try:
spam(1,2)
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
The output would be like:
bash-3.2$ python3 /Users/soumyabratakole/PycharmProjects/pythonProject/main.py
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 26, in <module>
1/0
ZeroDivisionError: division by zero
------End--------
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 33, in <module>
spam(1,2)
NameError: name 'spam' is not defined
------End--------
If you have an Error object already, and you want to print the whole thing, you need to make this slightly awkward call:
import traceback
traceback.print_exception(type(err), err, err.__traceback__)
That's right, print_exception takes three positional arguments: The type of the exception, the actual exception object, and the exception's own internal traceback property.
In python 3.5 or later, the type(err) is optional... but it's a positional argument, so you still have to explicitly pass None in its place.
traceback.print_exception(None, err, err.__traceback__)
I have no idea why all of this isn't just traceback.print_exception(err). Why you would ever want to print out an error, along with a traceback other than the one that belongs to that error, is beyond me.
To get the precise stack trace, as a string, that would have been raised if no try/except were there to step over it, simply place this in the except block that catches the offending exception.
desired_trace = traceback.format_exc(sys.exc_info())
Here's how to use it (assuming flaky_func is defined, and log calls your favorite logging system):
import traceback
import sys
try:
flaky_func()
except KeyboardInterrupt:
raise
except Exception:
desired_trace = traceback.format_exc(sys.exc_info())
log(desired_trace)
It's a good idea to catch and re-raise KeyboardInterrupts, so that you can still kill the program using Ctrl-C. Logging is outside the scope of the question, but a good option is logging. Documentation for the sys and traceback modules.
You will need to put the try/except inside the most innerloop where the error may occur, i.e.
for i in something:
for j in somethingelse:
for k in whatever:
try:
something_complex(i, j, k)
except Exception, e:
print e
try:
something_less_complex(i, j)
except Exception, e:
print e
... and so on
In other words, you will need to wrap statements that may fail in try/except as specific as possible, in the most inner-loop as possible.
A remark about this answer's comments: print(traceback.format_exc()) does a better job for me than traceback.print_exc(). With the latter, the hello is sometimes strangely "mixed" with the traceback text, like if both want to write to stdout or stderr at the same time, producing weird output (at least when building from inside a text editor and viewing the output in the "Build results" panel).
Traceback (most recent call last):
File "C:\Users\User\Desktop\test.py", line 7, in
hell do_stuff()
File "C:\Users\User\Desktop\test.py", line 4, in do_stuff
1/0
ZeroDivisionError: integer division or modulo by zero
o
[Finished in 0.1s]
So I use:
import traceback, sys
def do_stuff():
1/0
try:
do_stuff()
except Exception:
print(traceback.format_exc())
print('hello')
import io
import traceback
try:
call_code_that_fails()
except:
errors = io.StringIO()
traceback.print_exc(file=errors) # Instead of printing directly to stdout, the result can be further processed
contents = str(errors.getvalue())
print(contents)
errors.close()
You want the traceback module. It will let you print stack dumps like Python normally does. In particular, the print_last function will print the last exception and a stack trace.
python 3 solution
stacktrace_helper.py:
from linecache import getline
import sys
import traceback
def get_stack_trace():
exc_type, exc_value, exc_tb = sys.exc_info()
trace = traceback.format_stack()
trace = list(filter(lambda x: ("\\lib\\" not in x and "/lib/" not in x and "stacktrace_helper.py" not in x), trace))
ex_type = exc_type.__name__
ex_line = exc_tb.tb_lineno
ex_file = exc_tb.tb_frame.f_code.co_filename
ex_message = str(exc_value)
line_code = ""
try:
line_code = getline(ex_file, ex_line).strip()
except:
pass
trace.insert(
0, f'File "{ex_file}", line {ex_line}, line_code: {line_code} , ex: {ex_type} {ex_message}',
)
return trace
def get_stack_trace_str(msg: str = ""):
trace = list(get_stack_trace())
trace_str = "\n".join(list(map(str, trace)))
trace_str = msg + "\n" + trace_str
return trace_str
This is my solution to write the error in a log file and also on console:
import logging, sys
import traceback
logging.basicConfig(filename='error.log', level=logging.DEBUG)
def handle_exception(exc_type, exc_value, exc_traceback):
if issubclass(exc_type, KeyboardInterrupt):
sys.__excepthook__(exc_type, exc_value, exc_traceback)
return
exc_info=(exc_type, exc_value, exc_traceback)
logging.critical("\nDate:" + str(datetime.datetime.now()), exc_info=(exc_type, exc_value, exc_traceback))
print("An error occured, check error.log to see the error details")
traceback.print_exception(*exc_info)
sys.excepthook = handle_exception
You could do:
try:
do_stuff()
except Exception, err:
print(Exception, err)
raise err
I want to catch and log exceptions without exiting, e.g.,
try:
do_stuff()
except Exception as err:
print(Exception, err)
# I want to print the entire traceback here,
# not just the exception name and details
I want to print the exact same output that is printed when the exception is raised without the try/except intercepting the exception, and I do not want it to exit my program.
traceback.format_exc() or sys.exc_info() will yield more info if that's what you want.
import traceback
import sys
try:
do_stuff()
except Exception:
print(traceback.format_exc())
# or
print(sys.exc_info()[2])
Some other answer have already pointed out the traceback module.
Please notice that with print_exc, in some corner cases, you will not obtain what you would expect. In Python 2.x:
import traceback
try:
raise TypeError("Oups!")
except Exception, err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_exc()
...will display the traceback of the last exception:
Traceback (most recent call last):
File "e.py", line 7, in <module>
raise TypeError("Again !?!")
TypeError: Again !?!
If you really need to access the original traceback one solution is to cache the exception infos as returned from exc_info in a local variable and display it using print_exception:
import traceback
import sys
try:
raise TypeError("Oups!")
except Exception, err:
try:
exc_info = sys.exc_info()
# do you usefull stuff here
# (potentially raising an exception)
try:
raise TypeError("Again !?!")
except:
pass
# end of useful stuff
finally:
# Display the *original* exception
traceback.print_exception(*exc_info)
del exc_info
Producing:
Traceback (most recent call last):
File "t.py", line 6, in <module>
raise TypeError("Oups!")
TypeError: Oups!
Few pitfalls with this though:
From the doc of sys_info:
Assigning the traceback return value to a local variable in a function that is handling an exception will cause a circular reference. This will prevent anything referenced by a local variable in the same function or by the traceback from being garbage collected. [...] If you do need the traceback, make sure to delete it after use (best done with a try ... finally statement)
but, from the same doc:
Beginning with Python 2.2, such cycles are automatically reclaimed when garbage collection is enabled and they become unreachable, but it remains more efficient to avoid creating cycles.
On the other hand, by allowing you to access the traceback associated with an exception, Python 3 produce a less surprising result:
import traceback
try:
raise TypeError("Oups!")
except Exception as err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_tb(err.__traceback__)
... will display:
File "e3.py", line 4, in <module>
raise TypeError("Oups!")
If you're debugging and just want to see the current stack trace, you can simply call:
traceback.print_stack()
There's no need to manually raise an exception just to catch it again.
How to print the full traceback without halting the program?
When you don't want to halt your program on an error, you need to handle that error with a try/except:
try:
do_something_that_might_error()
except Exception as error:
handle_the_error(error)
To extract the full traceback, we'll use the traceback module from the standard library:
import traceback
And to create a decently complicated stacktrace to demonstrate that we get the full stacktrace:
def raise_error():
raise RuntimeError('something bad happened!')
def do_something_that_might_error():
raise_error()
Printing
To print the full traceback, use the traceback.print_exc method:
try:
do_something_that_might_error()
except Exception as error:
traceback.print_exc()
Which prints:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Better than printing, logging:
However, a best practice is to have a logger set up for your module. It will know the name of the module and be able to change levels (among other attributes, such as handlers)
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
In which case, you'll want the logger.exception function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.exception(error)
Which logs:
ERROR:__main__:something bad happened!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Or perhaps you just want the string, in which case, you'll want the traceback.format_exc function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.debug(traceback.format_exc())
Which logs:
DEBUG:__main__:Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Conclusion
And for all three options, we see we get the same output as when we have an error:
>>> do_something_that_might_error()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Which to use
Performance concerns aren't important here as IO usually dominates. I'd prefer, since it does precisely what's being requested in a forward compatible way:
logger.exception(error)
Logging levels and outputs can be adjusted, making it easy to turn off without touching the code. And usually doing what's directly needed is the most efficient way to do it.
First, do not use prints for logging, there is a stable, proven and well-thought out stdlib module to do that: logging. You definitely should use it instead.
Second, do not be tempted to do a mess with unrelated tools when there is a native and simple approach. Here it is:
log = logging.getLogger(__name__)
try:
call_code_that_fails()
except MyError:
log.exception('Any extra info you want to see in your logs')
That's it. You are done now.
Explanation for anyone who is interested in how things work under the hood
What log.exception is actually doing is just a call to log.error (that is, log event with level ERROR) and print traceback then.
Why is it better?
Well, here are some considerations:
it is just right;
it is straightforward;
it is simple.
Why should nobody use traceback or call logger with exc_info=True or get their hands dirty with sys.exc_info?
Well, just because! They all exist for different purposes. For example, traceback.print_exc's output is a little bit different from tracebacks produced by the interpreter itself. If you use it, you will confuse anyone who reads your logs, they will be banging their heads against them.
Passing exc_info=True to log calls is just inappropriate. But, it is useful when catching recoverable errors and you want to log them (using, e.g INFO level) with tracebacks as well, because log.exception produces logs of only one level - ERROR.
And you definitely should avoid messing with sys.exc_info as much as you can. It's just not a public interface, it's an internal one - you can use it if you definitely know what you are doing. It is not intended for just printing exceptions.
traceback.format_exception(exception_object)
If you only have the exception object, you can get the traceback as a string from any point of the code in Python 3 with:
import traceback
''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
Full example:
#!/usr/bin/env python3
import traceback
def f():
g()
def g():
raise Exception('asdf')
try:
g()
except Exception as e:
exc_obj = e
tb_str = ''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
print(tb_str)
Output:
Traceback (most recent call last):
File "./main.py", line 12, in <module>
g()
File "./main.py", line 9, in g
raise Exception('asdf')
Exception: asdf
Documentation: https://docs.python.org/3.9/library/traceback.html#traceback.format_exception
See also: Extract traceback info from an exception object
Tested in Python 3.9
In addition to Aaron Hall's answer, if you are logging, but don't want to use logging.exception() (since it logs at the ERROR level), you can use a lower level and pass exc_info=True. e.g.
try:
do_something_that_might_error()
except Exception:
logging.info('General exception noted.', exc_info=True)
I don't see this mentioned in any of the other answers. If you're passing around an Exception object for whatever reason...
In Python 3.5+ you can get a trace from an Exception object using traceback.TracebackException.from_exception(). For example:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
try:
stack_lvl_3()
except Exception as e:
# raise
return e
def stack_lvl_1():
e = stack_lvl_2()
return e
e = stack_lvl_1()
tb1 = traceback.TracebackException.from_exception(e)
print(''.join(tb1.format()))
However, the above code results in:
Traceback (most recent call last):
File "exc.py", line 10, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Exception: ('a1', 'b2', 'c3')
This is just two levels of the stack, as opposed to what would have been printed on screen had the exception been raised in stack_lvl_2() and not intercepted (uncomment the # raise line).
As I understand it, that's because an exception records only the current level of the stack when it is raised, stack_lvl_3() in this case. As it's passed back up through the stack, more levels are being added to its __traceback__. But we intercepted it in stack_lvl_2(), meaning all it got to record was levels 3 and 2. To get the full trace as printed on stdout we'd have to catch it at the highest (lowest?) level:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
stack_lvl_3()
def stack_lvl_1():
stack_lvl_2()
try:
stack_lvl_1()
except Exception as exc:
tb = traceback.TracebackException.from_exception(exc)
print('Handled at stack lvl 0')
print(''.join(tb.stack.format()))
Which results in:
Handled at stack lvl 0
File "exc.py", line 17, in <module>
stack_lvl_1()
File "exc.py", line 13, in stack_lvl_1
stack_lvl_2()
File "exc.py", line 9, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Notice that the stack print is different, the first and last lines are missing. Because it's a different format().
Intercepting the exception as far away from the point where it was raised as possible makes for simpler code while also giving more information.
In python3 (works in 3.9) we can define a function and can use that where ever we want to print the details.
import traceback
def get_traceback(e):
lines = traceback.format_exception(type(e), e, e.__traceback__)
return ''.join(lines)
try:
1/0
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
try:
spam(1,2)
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
The output would be like:
bash-3.2$ python3 /Users/soumyabratakole/PycharmProjects/pythonProject/main.py
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 26, in <module>
1/0
ZeroDivisionError: division by zero
------End--------
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 33, in <module>
spam(1,2)
NameError: name 'spam' is not defined
------End--------
If you have an Error object already, and you want to print the whole thing, you need to make this slightly awkward call:
import traceback
traceback.print_exception(type(err), err, err.__traceback__)
That's right, print_exception takes three positional arguments: The type of the exception, the actual exception object, and the exception's own internal traceback property.
In python 3.5 or later, the type(err) is optional... but it's a positional argument, so you still have to explicitly pass None in its place.
traceback.print_exception(None, err, err.__traceback__)
I have no idea why all of this isn't just traceback.print_exception(err). Why you would ever want to print out an error, along with a traceback other than the one that belongs to that error, is beyond me.
To get the precise stack trace, as a string, that would have been raised if no try/except were there to step over it, simply place this in the except block that catches the offending exception.
desired_trace = traceback.format_exc(sys.exc_info())
Here's how to use it (assuming flaky_func is defined, and log calls your favorite logging system):
import traceback
import sys
try:
flaky_func()
except KeyboardInterrupt:
raise
except Exception:
desired_trace = traceback.format_exc(sys.exc_info())
log(desired_trace)
It's a good idea to catch and re-raise KeyboardInterrupts, so that you can still kill the program using Ctrl-C. Logging is outside the scope of the question, but a good option is logging. Documentation for the sys and traceback modules.
You will need to put the try/except inside the most innerloop where the error may occur, i.e.
for i in something:
for j in somethingelse:
for k in whatever:
try:
something_complex(i, j, k)
except Exception, e:
print e
try:
something_less_complex(i, j)
except Exception, e:
print e
... and so on
In other words, you will need to wrap statements that may fail in try/except as specific as possible, in the most inner-loop as possible.
A remark about this answer's comments: print(traceback.format_exc()) does a better job for me than traceback.print_exc(). With the latter, the hello is sometimes strangely "mixed" with the traceback text, like if both want to write to stdout or stderr at the same time, producing weird output (at least when building from inside a text editor and viewing the output in the "Build results" panel).
Traceback (most recent call last):
File "C:\Users\User\Desktop\test.py", line 7, in
hell do_stuff()
File "C:\Users\User\Desktop\test.py", line 4, in do_stuff
1/0
ZeroDivisionError: integer division or modulo by zero
o
[Finished in 0.1s]
So I use:
import traceback, sys
def do_stuff():
1/0
try:
do_stuff()
except Exception:
print(traceback.format_exc())
print('hello')
import io
import traceback
try:
call_code_that_fails()
except:
errors = io.StringIO()
traceback.print_exc(file=errors) # Instead of printing directly to stdout, the result can be further processed
contents = str(errors.getvalue())
print(contents)
errors.close()
You want the traceback module. It will let you print stack dumps like Python normally does. In particular, the print_last function will print the last exception and a stack trace.
python 3 solution
stacktrace_helper.py:
from linecache import getline
import sys
import traceback
def get_stack_trace():
exc_type, exc_value, exc_tb = sys.exc_info()
trace = traceback.format_stack()
trace = list(filter(lambda x: ("\\lib\\" not in x and "/lib/" not in x and "stacktrace_helper.py" not in x), trace))
ex_type = exc_type.__name__
ex_line = exc_tb.tb_lineno
ex_file = exc_tb.tb_frame.f_code.co_filename
ex_message = str(exc_value)
line_code = ""
try:
line_code = getline(ex_file, ex_line).strip()
except:
pass
trace.insert(
0, f'File "{ex_file}", line {ex_line}, line_code: {line_code} , ex: {ex_type} {ex_message}',
)
return trace
def get_stack_trace_str(msg: str = ""):
trace = list(get_stack_trace())
trace_str = "\n".join(list(map(str, trace)))
trace_str = msg + "\n" + trace_str
return trace_str
This is my solution to write the error in a log file and also on console:
import logging, sys
import traceback
logging.basicConfig(filename='error.log', level=logging.DEBUG)
def handle_exception(exc_type, exc_value, exc_traceback):
if issubclass(exc_type, KeyboardInterrupt):
sys.__excepthook__(exc_type, exc_value, exc_traceback)
return
exc_info=(exc_type, exc_value, exc_traceback)
logging.critical("\nDate:" + str(datetime.datetime.now()), exc_info=(exc_type, exc_value, exc_traceback))
print("An error occured, check error.log to see the error details")
traceback.print_exception(*exc_info)
sys.excepthook = handle_exception
You could do:
try:
do_stuff()
except Exception, err:
print(Exception, err)
raise err
I am trying to test the error handling of a class and I need to simulate read() raising a MemoryError. This is a simplified example.
import mock
def memory_error_read():
raise MemoryError
def read_file_by_filename(filename):
try:
handle = open(filename)
content = handle.read()
except MemoryError:
raise Exception("%s is too big" % self.filename)
finally:
handle.close()
return content
#mock.patch("__builtin__.file.read", memory_error_read)
def testfunc():
try:
read_file_by_filename("/etc/passwd")
except Exception, e:
return
print("Should have received exception")
testfunc()
When I run this I get the following traceback.
# ./read_filename.py
Traceback (most recent call last):
File "./read_filename.py", line 34, in <module>
testfunc()
File "build/bdist.linux-i686/egg/mock.py", line 1214, in patched
File "build/bdist.linux-i686/egg/mock.py", line 1379, in __exit__
TypeError: can't set attributes of built-in/extension type 'file'
It appears that I can't patch the builtin read function. Is there a way to trick it?
Is it possible to do what I want?
ANSWER
Here's the updated code based on Ben's suggestion below.
from __future__ import with_statement
...
def test_func():
with mock.patch("__builtin__.open", new_callable=mock.mock_open) as mo:
mock_file = mo.return_value
mock_file.read.side_effect = MemoryError
try:
read_file_by_filename("/etc/passwd")
except Exception, e:
if "is too big" in str(e):
return
else:
raise
print("Should have caught an exception")
Have you looked at mock_open?
You should be able to have that return a mock object that has an Exception raising side effect on read().
I want to catch and log exceptions without exiting, e.g.,
try:
do_stuff()
except Exception as err:
print(Exception, err)
# I want to print the entire traceback here,
# not just the exception name and details
I want to print the exact same output that is printed when the exception is raised without the try/except intercepting the exception, and I do not want it to exit my program.
traceback.format_exc() or sys.exc_info() will yield more info if that's what you want.
import traceback
import sys
try:
do_stuff()
except Exception:
print(traceback.format_exc())
# or
print(sys.exc_info()[2])
Some other answer have already pointed out the traceback module.
Please notice that with print_exc, in some corner cases, you will not obtain what you would expect. In Python 2.x:
import traceback
try:
raise TypeError("Oups!")
except Exception, err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_exc()
...will display the traceback of the last exception:
Traceback (most recent call last):
File "e.py", line 7, in <module>
raise TypeError("Again !?!")
TypeError: Again !?!
If you really need to access the original traceback one solution is to cache the exception infos as returned from exc_info in a local variable and display it using print_exception:
import traceback
import sys
try:
raise TypeError("Oups!")
except Exception, err:
try:
exc_info = sys.exc_info()
# do you usefull stuff here
# (potentially raising an exception)
try:
raise TypeError("Again !?!")
except:
pass
# end of useful stuff
finally:
# Display the *original* exception
traceback.print_exception(*exc_info)
del exc_info
Producing:
Traceback (most recent call last):
File "t.py", line 6, in <module>
raise TypeError("Oups!")
TypeError: Oups!
Few pitfalls with this though:
From the doc of sys_info:
Assigning the traceback return value to a local variable in a function that is handling an exception will cause a circular reference. This will prevent anything referenced by a local variable in the same function or by the traceback from being garbage collected. [...] If you do need the traceback, make sure to delete it after use (best done with a try ... finally statement)
but, from the same doc:
Beginning with Python 2.2, such cycles are automatically reclaimed when garbage collection is enabled and they become unreachable, but it remains more efficient to avoid creating cycles.
On the other hand, by allowing you to access the traceback associated with an exception, Python 3 produce a less surprising result:
import traceback
try:
raise TypeError("Oups!")
except Exception as err:
try:
raise TypeError("Again !?!")
except:
pass
traceback.print_tb(err.__traceback__)
... will display:
File "e3.py", line 4, in <module>
raise TypeError("Oups!")
If you're debugging and just want to see the current stack trace, you can simply call:
traceback.print_stack()
There's no need to manually raise an exception just to catch it again.
How to print the full traceback without halting the program?
When you don't want to halt your program on an error, you need to handle that error with a try/except:
try:
do_something_that_might_error()
except Exception as error:
handle_the_error(error)
To extract the full traceback, we'll use the traceback module from the standard library:
import traceback
And to create a decently complicated stacktrace to demonstrate that we get the full stacktrace:
def raise_error():
raise RuntimeError('something bad happened!')
def do_something_that_might_error():
raise_error()
Printing
To print the full traceback, use the traceback.print_exc method:
try:
do_something_that_might_error()
except Exception as error:
traceback.print_exc()
Which prints:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Better than printing, logging:
However, a best practice is to have a logger set up for your module. It will know the name of the module and be able to change levels (among other attributes, such as handlers)
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
In which case, you'll want the logger.exception function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.exception(error)
Which logs:
ERROR:__main__:something bad happened!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Or perhaps you just want the string, in which case, you'll want the traceback.format_exc function instead:
try:
do_something_that_might_error()
except Exception as error:
logger.debug(traceback.format_exc())
Which logs:
DEBUG:__main__:Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Conclusion
And for all three options, we see we get the same output as when we have an error:
>>> do_something_that_might_error()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in do_something_that_might_error
File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!
Which to use
Performance concerns aren't important here as IO usually dominates. I'd prefer, since it does precisely what's being requested in a forward compatible way:
logger.exception(error)
Logging levels and outputs can be adjusted, making it easy to turn off without touching the code. And usually doing what's directly needed is the most efficient way to do it.
First, do not use prints for logging, there is a stable, proven and well-thought out stdlib module to do that: logging. You definitely should use it instead.
Second, do not be tempted to do a mess with unrelated tools when there is a native and simple approach. Here it is:
log = logging.getLogger(__name__)
try:
call_code_that_fails()
except MyError:
log.exception('Any extra info you want to see in your logs')
That's it. You are done now.
Explanation for anyone who is interested in how things work under the hood
What log.exception is actually doing is just a call to log.error (that is, log event with level ERROR) and print traceback then.
Why is it better?
Well, here are some considerations:
it is just right;
it is straightforward;
it is simple.
Why should nobody use traceback or call logger with exc_info=True or get their hands dirty with sys.exc_info?
Well, just because! They all exist for different purposes. For example, traceback.print_exc's output is a little bit different from tracebacks produced by the interpreter itself. If you use it, you will confuse anyone who reads your logs, they will be banging their heads against them.
Passing exc_info=True to log calls is just inappropriate. But, it is useful when catching recoverable errors and you want to log them (using, e.g INFO level) with tracebacks as well, because log.exception produces logs of only one level - ERROR.
And you definitely should avoid messing with sys.exc_info as much as you can. It's just not a public interface, it's an internal one - you can use it if you definitely know what you are doing. It is not intended for just printing exceptions.
traceback.format_exception(exception_object)
If you only have the exception object, you can get the traceback as a string from any point of the code in Python 3 with:
import traceback
''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
Full example:
#!/usr/bin/env python3
import traceback
def f():
g()
def g():
raise Exception('asdf')
try:
g()
except Exception as e:
exc_obj = e
tb_str = ''.join(traceback.format_exception(None, exc_obj, exc_obj.__traceback__))
print(tb_str)
Output:
Traceback (most recent call last):
File "./main.py", line 12, in <module>
g()
File "./main.py", line 9, in g
raise Exception('asdf')
Exception: asdf
Documentation: https://docs.python.org/3.9/library/traceback.html#traceback.format_exception
See also: Extract traceback info from an exception object
Tested in Python 3.9
In addition to Aaron Hall's answer, if you are logging, but don't want to use logging.exception() (since it logs at the ERROR level), you can use a lower level and pass exc_info=True. e.g.
try:
do_something_that_might_error()
except Exception:
logging.info('General exception noted.', exc_info=True)
I don't see this mentioned in any of the other answers. If you're passing around an Exception object for whatever reason...
In Python 3.5+ you can get a trace from an Exception object using traceback.TracebackException.from_exception(). For example:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
try:
stack_lvl_3()
except Exception as e:
# raise
return e
def stack_lvl_1():
e = stack_lvl_2()
return e
e = stack_lvl_1()
tb1 = traceback.TracebackException.from_exception(e)
print(''.join(tb1.format()))
However, the above code results in:
Traceback (most recent call last):
File "exc.py", line 10, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Exception: ('a1', 'b2', 'c3')
This is just two levels of the stack, as opposed to what would have been printed on screen had the exception been raised in stack_lvl_2() and not intercepted (uncomment the # raise line).
As I understand it, that's because an exception records only the current level of the stack when it is raised, stack_lvl_3() in this case. As it's passed back up through the stack, more levels are being added to its __traceback__. But we intercepted it in stack_lvl_2(), meaning all it got to record was levels 3 and 2. To get the full trace as printed on stdout we'd have to catch it at the highest (lowest?) level:
import traceback
def stack_lvl_3():
raise Exception('a1', 'b2', 'c3')
def stack_lvl_2():
stack_lvl_3()
def stack_lvl_1():
stack_lvl_2()
try:
stack_lvl_1()
except Exception as exc:
tb = traceback.TracebackException.from_exception(exc)
print('Handled at stack lvl 0')
print(''.join(tb.stack.format()))
Which results in:
Handled at stack lvl 0
File "exc.py", line 17, in <module>
stack_lvl_1()
File "exc.py", line 13, in stack_lvl_1
stack_lvl_2()
File "exc.py", line 9, in stack_lvl_2
stack_lvl_3()
File "exc.py", line 5, in stack_lvl_3
raise Exception('a1', 'b2', 'c3')
Notice that the stack print is different, the first and last lines are missing. Because it's a different format().
Intercepting the exception as far away from the point where it was raised as possible makes for simpler code while also giving more information.
In python3 (works in 3.9) we can define a function and can use that where ever we want to print the details.
import traceback
def get_traceback(e):
lines = traceback.format_exception(type(e), e, e.__traceback__)
return ''.join(lines)
try:
1/0
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
try:
spam(1,2)
except Exception as e:
print('------Start--------')
print(get_traceback(e))
print('------End--------')
The output would be like:
bash-3.2$ python3 /Users/soumyabratakole/PycharmProjects/pythonProject/main.py
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 26, in <module>
1/0
ZeroDivisionError: division by zero
------End--------
------Start--------
Traceback (most recent call last):
File "/Users/soumyabratakole/PycharmProjects/pythonProject/main.py", line 33, in <module>
spam(1,2)
NameError: name 'spam' is not defined
------End--------
If you have an Error object already, and you want to print the whole thing, you need to make this slightly awkward call:
import traceback
traceback.print_exception(type(err), err, err.__traceback__)
That's right, print_exception takes three positional arguments: The type of the exception, the actual exception object, and the exception's own internal traceback property.
In python 3.5 or later, the type(err) is optional... but it's a positional argument, so you still have to explicitly pass None in its place.
traceback.print_exception(None, err, err.__traceback__)
I have no idea why all of this isn't just traceback.print_exception(err). Why you would ever want to print out an error, along with a traceback other than the one that belongs to that error, is beyond me.
To get the precise stack trace, as a string, that would have been raised if no try/except were there to step over it, simply place this in the except block that catches the offending exception.
desired_trace = traceback.format_exc(sys.exc_info())
Here's how to use it (assuming flaky_func is defined, and log calls your favorite logging system):
import traceback
import sys
try:
flaky_func()
except KeyboardInterrupt:
raise
except Exception:
desired_trace = traceback.format_exc(sys.exc_info())
log(desired_trace)
It's a good idea to catch and re-raise KeyboardInterrupts, so that you can still kill the program using Ctrl-C. Logging is outside the scope of the question, but a good option is logging. Documentation for the sys and traceback modules.
You will need to put the try/except inside the most innerloop where the error may occur, i.e.
for i in something:
for j in somethingelse:
for k in whatever:
try:
something_complex(i, j, k)
except Exception, e:
print e
try:
something_less_complex(i, j)
except Exception, e:
print e
... and so on
In other words, you will need to wrap statements that may fail in try/except as specific as possible, in the most inner-loop as possible.
A remark about this answer's comments: print(traceback.format_exc()) does a better job for me than traceback.print_exc(). With the latter, the hello is sometimes strangely "mixed" with the traceback text, like if both want to write to stdout or stderr at the same time, producing weird output (at least when building from inside a text editor and viewing the output in the "Build results" panel).
Traceback (most recent call last):
File "C:\Users\User\Desktop\test.py", line 7, in
hell do_stuff()
File "C:\Users\User\Desktop\test.py", line 4, in do_stuff
1/0
ZeroDivisionError: integer division or modulo by zero
o
[Finished in 0.1s]
So I use:
import traceback, sys
def do_stuff():
1/0
try:
do_stuff()
except Exception:
print(traceback.format_exc())
print('hello')
import io
import traceback
try:
call_code_that_fails()
except:
errors = io.StringIO()
traceback.print_exc(file=errors) # Instead of printing directly to stdout, the result can be further processed
contents = str(errors.getvalue())
print(contents)
errors.close()
You want the traceback module. It will let you print stack dumps like Python normally does. In particular, the print_last function will print the last exception and a stack trace.
python 3 solution
stacktrace_helper.py:
from linecache import getline
import sys
import traceback
def get_stack_trace():
exc_type, exc_value, exc_tb = sys.exc_info()
trace = traceback.format_stack()
trace = list(filter(lambda x: ("\\lib\\" not in x and "/lib/" not in x and "stacktrace_helper.py" not in x), trace))
ex_type = exc_type.__name__
ex_line = exc_tb.tb_lineno
ex_file = exc_tb.tb_frame.f_code.co_filename
ex_message = str(exc_value)
line_code = ""
try:
line_code = getline(ex_file, ex_line).strip()
except:
pass
trace.insert(
0, f'File "{ex_file}", line {ex_line}, line_code: {line_code} , ex: {ex_type} {ex_message}',
)
return trace
def get_stack_trace_str(msg: str = ""):
trace = list(get_stack_trace())
trace_str = "\n".join(list(map(str, trace)))
trace_str = msg + "\n" + trace_str
return trace_str
This is my solution to write the error in a log file and also on console:
import logging, sys
import traceback
logging.basicConfig(filename='error.log', level=logging.DEBUG)
def handle_exception(exc_type, exc_value, exc_traceback):
if issubclass(exc_type, KeyboardInterrupt):
sys.__excepthook__(exc_type, exc_value, exc_traceback)
return
exc_info=(exc_type, exc_value, exc_traceback)
logging.critical("\nDate:" + str(datetime.datetime.now()), exc_info=(exc_type, exc_value, exc_traceback))
print("An error occured, check error.log to see the error details")
traceback.print_exception(*exc_info)
sys.excepthook = handle_exception
You could do:
try:
do_stuff()
except Exception, err:
print(Exception, err)
raise err
A common pattern in python is to catch an error in an upstream module and re-raise that error as something more useful.
try:
config_file = open('config.ini', 'r')
except IOError:
raise ConfigError('Give me my config, user!')
This will generate a stack trace of the form
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
__main__.ConfigError: Give me my config, user!
Is there any way to access the wrapped exception in order to generate a stack trace more like this?
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
__builtin__.IOError: File Does not exist.
Exception wrapped by:
File "<stdin>", line 4, in <module>
__main__.ConfigError: Give me my config, user!
EDIT:
The problem i'm trying to defeat is that some 3rd party code can wrap exceptions up to 3 times and I want to be able to determine the root cause, i.e. a generic way to inspect the exception stack and determine the root cause of an exception without having to add any extra code to 3rd party modules.
This is known as Exception Chaining and is suported in Python 3.
PEP 3134: http://www.python.org/dev/peps/pep-3134/
In Python 2, the old exception is lost when you raise a new one, unless you save it in the except block.
Use the traceback module. It will allow you to access the most recent traceback and store it in a string. For example,
import traceback
try:
config_file = open('config.ini', 'r')
except OSError:
tb = traceback.format_exc()
raise ConfigError('Give me my config, user!',tb)
The "nested" traceback will be stored in tb and passed to ConfigError, where you can work with it however you want.
Here is an example of how to unwind PEP-3134 exception chains.
Note that due to legacy reasons some Python frameworks may not use exception chaining, but instead of wrap exceptions in their own way. For example. SQLALchemy DBABIError uses orig attribute.
class Foobar(Exception):
pass
class Dummy(Exception):
pass
def func1():
raise Foobar("func1() argh")
def func2():
try:
func1()
except Exception as e:
raise Dummy("func2 vyaaarrg!") from e
try:
func2()
except Exception as e:
print(f"Current {e.__class__}: {e}")
print(f"Nested {e.__cause__.__class__}:{e.__cause__}")
Prints
Current <class '__main__.Dummy'>: func2 vyaaarrg!
Nested <class '__main__.Foobar'>:func1() argh