While stack traces are useful in Python, most often the data at the root of the problem are missing - is there a way of making sure that at least locals() (and possibly globals()) are added to printed stacktrace?
You can install your own exception hook and output what you need from there:
import sys, traceback
def excepthook(type, value, tb):
traceback.print_exception(type, value, tb)
while tb.tb_next:
tb = tb.tb_next
print >>sys.stderr, 'Locals:', tb.tb_frame.f_locals
print >>sys.stderr, 'Globals:', tb.tb_frame.f_globals
sys.excepthook = excepthook
def x():
y()
def y():
foo = 1
bar = 0
foo/bar
x()
To print vars from each frame in a traceback, change the above loop to
while tb:
print >>sys.stderr, 'Locals:', tb.tb_frame.f_locals
print >>sys.stderr, 'Globals:', tb.tb_frame.f_globals
tb = tb.tb_next
This is a Box of Pandora. Values can be very large in printed form; printing all locals in a stack trace can easily lead to new problems just due to error output. That's why this is not implemented in general in Python.
In small examples, though, i. e. if you know that your values aren't too large to be printed properly, you can step along the traceback yourself:
import sys
import traceback
def c():
clocal = 1001
raise Exception("foo")
def b():
blocal = 23
c()
def a():
alocal = 42
b()
try:
a()
except Exception:
frame = sys.exc_info()[2]
formattedTb = traceback.format_tb(frame)
frame = frame.tb_next
while frame:
print formattedTb.pop(0), '\t', frame.tb_frame.f_locals
frame = frame.tb_next
The output will be sth like this:
File "/home/alfe/tmp/stacktracelocals.py", line 19, in <module>
a()
{'alocal': 42}
File "/home/alfe/tmp/stacktracelocals.py", line 16, in a
b()
{'blocal': 23}
File "/home/alfe/tmp/stacktracelocals.py", line 12, in b
c()
{'clocal': 1001}
And you can, of course, install your own except hook as thg435 suggested in his answer.
if you didn't know about this already, use the pdb post-mortem feature:
x = 3.0
y = 0.0
print x/y
def div(a, b):
return a / b
print div(x,y)
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-3-d03977de5fc3> in div(a, b)
1 def div(a, b):
----> 2 return a / b
ZeroDivisionError: float division
import pdb
pdb.pm()
> <ipython-input-3-148da0dcdc9e>(2)div()
0 return a/b
ipdb> l
1 def div(a,b):
----> 2 return a/b
ipdb> a
3.0
ipdb> b
0.0
etc.
there are cases where you really need the prints though, of course. you're better off instrumenting the code (via try/except) to print out extra information around a specific weird exception you are debugging than putting this for everything though, imho.
Try traceback-with-variables package.
Usage:
from traceback_with_variables import traceback_with_variables
def main():
...
with traceback_with_variables():
...your code...
Exceptions with it:
Traceback with variables (most recent call last):
File "./temp.py", line 7, in main
return get_avg_ratio([h1, w1], [h2, w2])
sizes_str = '300 200 300 0'
h1 = 300
w1 = 200
h2 = 300
w2 = 0
File "./temp.py", line 10, in get_avg_ratio
return mean([get_ratio(h, w) for h, w in [size1, size2]])
size1 = [300, 200]
size2 = [300, 0]
File "./temp.py", line 10, in <listcomp>
return mean([get_ratio(h, w) for h, w in [size1, size2]])
.0 = <tuple_iterator object at 0x7ff61e35b820>
h = 300
w = 0
File "./temp.py", line 13, in get_ratio
return height / width
height = 300
width = 0
builtins.ZeroDivisionError: division by zero
Installation:
pip install traceback-with-variables
Related
I want to print an error's line number and error message in a nicely displayed way. The follow is my code, which uses linecache:
import linecache
def func():
if xx == 1:
print('ok')
try:
func()
except:
exc_type, exc_obj, tb = sys.exc_info()
f = tb.tb_frame
lineno = tb.tb_lineno
filename = f.f_code.co_filename
linecache.checkcache(filename)
line = linecache.getline(filename, lineno, f.f_globals)
print_('ERROR - (LINE {} "{}"): {}'.format(lineno, line.strip(), exc_obj))
However, this only gives where the func() is called:
ERROR - (LINE 8 ""): name 'xx' is not defined
Is there a way to print the line number where the error actually occured, which should be Line 4? Or even better, can I print Line 8 and then trace back to line 4? For example, if I do not use try - except, the code:
def func():
if xx == 1:
print('ok')
func()
will give me the following error message, which is much better to locate the error:
File "<input>", line 5, in <module>
File "<input>", line 2, in func
NameError: name 'xx' is not defined. Did you mean: 'xxx'?
You can use traceback and sys modules to get advanced traceback output like you are wishing for.
Here is an example:
import traceback
import sys
def func():
zeroDivide = 1 / 0
try:
func()
except Exception:
print(traceback.format_exc()) # This line is for getting traceback.
print(sys.exc_info()[2]) # This line is getting for the error type.
Output will be:
Traceback (most recent call last):
File "b:\abc\1234\pppp\main.py", line 10, in <module>
func()
File "b:\abc\1234\pppp\main.py", line 7, in func
zeroDivide = 1 / 0
ZeroDivisionError: division by zero
You can use the traceback module to get the line number of the error,
import traceback
def function():
try:
# code
except:
tb_list = traceback.extract_tb(sys.exc_info()[2])
line_number = tb_list[-1][1]
print("An error occurred on line:", line_number)
You can use the traceback.extract_tb() function. This function returns a list of traceback objects, each of which contain information about the stack trace. The last element of this list, tb_list[-1], holds information about the line where the exception occurred. To access the line number, you can use the second element of this tuple, tb_list[-1][1]. This value can then be printed using the print() function.
To get the line number as an int you can get the traceback as a list from traceback.extract_tb(). Looking at the last item gives you the line where the exception was raised:
#soPrintLineOfError2
import sys
import traceback
def func():
if xx == 1:
print('ok')
try:
func()
except Exception as e:
tb = sys.exc_info()[2]
ss = traceback.extract_tb(tb)
ss1 = ss[-1]
print(ss1.line)
print(ss1.lineno)
Output:
if xx == 1:
6
How can we print the values of arguments passed to the functions in the call stack when an error stack trace is printed? I would like the output to be exactly as in the example below.
Example:
Traceback (most recent call last):
File "./file.py", line 615, in func0 **(arg0) arg0 = 0 was passed**
result = func1(arg1, arg2)
File "./file.py", line 728, in func1 **(arg1, arg2) arg1 = 1 and arg2 = 2 was passed**
return int_value[25]
TypeError: 'int' object is not iterable
I'd like the information inside the ** **s above to also be printed in addition to the normal output in the stack trace. What I envision is that the debugger automatically prints the passed arguments as well. That would give a clear picture of the "functional pipeline" that the data was passed through and what happened to it in the pipeline and which function did not do what it was supposed to do. This would help debugging a lot.
I searched quite a bit and found these related questions:
How to print call stack with argument values?
How to print function arguments in sys.settrace?
but the answers to neither of them worked for me: The answer to the 1st one led to ModuleNotFoundError: No module named 'stackdump'. The answer to the 2nd one crashed my ipython interpreter with a very long stack trace.
I also looked up:
https://docs.python.org/3/library/sys.html#sys.settrace
https://docs.python.org/3/library/traceback.html
There seems to be a capture_locals variable for TracebackExceptions, but I didn't quite understand how to make it work.
Pure Python3
Talking about TracebackExceptions and it's capture_locals argument, we can use it as follows:
#!/usr/bin/env python3
import traceback
c = ['Help me!']
def bar(a = 3):
d = {1,2,3}
e = {}
foo(a)
def foo(a=4):
b = 4
if a != b:
raise Exception("a is not equal to 4")
try:
bar(3)
except Exception as ex:
tb = traceback.TracebackException.from_exception(ex, capture_locals=True)
print("".join(tb.format()))
Which prints all local variables for each frame:
$ ./test.py
Traceback (most recent call last):
File "./test.py", line 21, in <module>
bar(3)
__annotations__ = {}
__builtins__ = <module 'builtins' (built-in)>
__cached__ = None
__doc__ = None
__file__ = './test.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7f81073704c0>
__name__ = '__main__'
__package__ = None
__spec__ = None
bar = <function bar at 0x7f81073b11f0>
c = ['Help me!']
ex = Exception('a is not equal to 4')
foo = <function foo at 0x7f810728a160>
traceback = <module 'traceback' from '/usr/lib/python3.8/traceback.py'>
File "./test.py", line 11, in bar
foo(a)
a = 3
d = {1, 2, 3}
e = {}
File "./test.py", line 17, in foo
raise Exception("a is not equal to 4")
a = 3
b = 4
Exception: a is not equal to 4
Looks a bit too verbose, but sometimes this data could be vital in debugging some crash.
Loguru
There is also a package loguru that prints "Fully descriptive exceptions":
2018-07-17 01:38:43.975 | ERROR | __main__:nested:10 - What?!
Traceback (most recent call last):
File "test.py", line 12, in <module>
nested(0)
└ <function nested at 0x7f5c755322f0>
> File "test.py", line 8, in nested
func(5, c)
│ └ 0
└ <function func at 0x7f5c79fc2e18>
File "test.py", line 4, in func
return a / b
│ └ 0
└ 5
ZeroDivisionError: division by zero
Probably exist better alternatives, but you can use a decorator for this:
def print_stack_arguments(func):
def new_func(*original_args, **original_kwargs):
try:
return func(*original_args, **original_kwargs)
except Exception as e:
print('Function: ', func.__name__)
print('Args: ', original_args)
print('Kwargs: ', original_kwargs)
print(e)
raise
return new_func
#print_stack_arguments
def print_error(value):
a = []
print(a[1])
#print_stack_arguments
def print_noerror(value):
print('No exception raised')
print_noerror('testing no exception')
print_error('testing exception')
I am trying to suppress a error/warning in my log while calling a library. Assume i have this code
try:
kazoo_client.start()
except:
pass
This is calling a zookeeper client which throws some exception which bubble up, now i don't want the warn/error in my logs when i call kazoo_client.start() is there a way to get this suppressed when you call the client
Assuming python 2.7.17
Try this approach:
import sys, StringIO
def funky() :
"1" + 1 # This should raise an error
sys.stderr = StringIO.StringIO()
funky() # this should call the funky function
And your code should look something like this:
import sys, StringIO
# import kazoo somehere around here
sys.stderr = StringIO.StringIO()
kazoo_client.start()
And lastly the Python 3 example:
import sys
from io import StringIO
# import kazoo somehere around here
sys.stderr = StringIO()
kazoo_client.start()
If you know the exception, try contextlib.suppress:
>>> from contextlib import suppress
>>> x = (i for i in range(10))
>>> with suppress(StopIteration):
... for i in range(11):
... print(next(x))
0
1
2
3
4
5
6
7
8
9
Without suppress it throws StopIteration error at last iteration.
>>> x = (i for i in range(10))
>>> for i in range(11):
... print(next(x))
0
1
2
3
4
5
6
7
8
9
Traceback (most recent call last):
File "<ipython-input-10-562798e05ad5>", line 2, in <module>
print(next(x))
StopIteration
Suppress is Pythonic, safe and explicit.
So in your case:
with suppress(SomeError):
kazoo_client.start()
EDIT:
To suppress all exceptions:
with suppress(Exception):
kazoo_client.start()
I would like to suggest a more generic approach, which can be used in general.
I leave you an example of how to create an decorator who ignore errors.
import functools
# Use the Decorator Design Pattern
def ignore_error_decorator(function_reference):
#functools.wraps(function_reference) # the decorator inherits the original function signature
def wrapper(*args):
try:
result = function_reference(*args) # execute the function
return result # If the function executed correctly, return
except Exception as e:
pass # Default ignore; You can also log the error or do what ever you want
return wrapper # Return the wrapper reference
if __name__ == '__main__':
# First alternative to use. Compose the decorator with another function
def my_first_function(a, b):
return a + b
rez_valid = ignore_error_decorator(my_first_function)(1, 3)
rez_invalid = ignore_error_decorator(my_first_function)(1, 'a')
print("Alternative_1 valid: {result}".format(result=rez_valid))
print("Alternative_1 invalid: {result}".format(result=rez_invalid)) # None is return by the exception bloc
# Second alternative. Decorating a function
#ignore_error_decorator
def my_second_function(a, b):
return a + b
rez_valid = my_second_function(1, 5)
rez_invalid = my_second_function(1, 'a')
print("Alternative_2 valid: {result}".format(result=rez_valid))
print("Alternative_2 invalid: {result}".format(result=rez_invalid)) # None is return by the exception bloc
Getting back to your problem, using my alternative you have to run
ignore_error_decorator(kazoo_client.start)()
Suppose I have a function that raises unexpected exceptions, so I wrap it in ipdb:
def boom(x, y):
try:
x / y
except Exception as e:
import ipdb; ipdb.set_trace()
def main():
x = 2
y = 0
boom(x, y)
if __name__ == '__main__':
main()
I can move up the stack to find out what values x and y have:
$ python crash.py
> /tmp/crash.py(6)boom()
5 except Exception as e:
----> 6 import ipdb; ipdb.set_trace()
7
ipdb> u
> /tmp/crash.py(11)main()
10 y = 0
---> 11 boom(x, y)
12
ipdb> p y
0
However, when debugging, I want to just put a debugger at the top level:
def boom(x, y):
x / y
def main():
x = 2
y = 0
boom(x, y)
if __name__ == '__main__':
try:
main()
except Exception as e:
import ipdb; ipdb.set_trace()
I can display the traceback, but I can't view the variables inside the function called:
$ python crash.py
> /tmp/crash.py(14)<module>()
12 main()
13 except Exception as e:
---> 14 import ipdb; ipdb.set_trace()
ipdb> !import traceback; traceback.print_exc(e)
Traceback (most recent call last):
File "crash.py", line 12, in <module>
main()
File "crash.py", line 8, in main
boom(x, y)
File "crash.py", line 3, in boom
x / y
ZeroDivisionError: integer division or modulo by zero
ipdb> d # I want to see what value x and y had!
*** Newest frame
The exception object clearly still has references to the stack when the exception occurred. Can I access x and y here, even though the stack has unwound?
Turns out that it is possible to extract variables from a traceback object.
To manually extract values:
ipdb> !import sys
ipdb> !tb = sys.exc_info()[2]
ipdb> p tb.tb_next.tb_frame.f_locals
{'y': 0, 'x': 2}
Even better, you can use an exception to explicitly do post-mortem debugging on that stack:
import sys
def boom(x, y):
x / y
def main():
x = 2
y = 0
boom(x, y)
if __name__ == '__main__':
try:
main()
except Exception as e:
# Most debuggers allow you to just do .post_mortem()
# but see https://github.com/gotcha/ipdb/pull/94
tb = sys.exc_info()[2]
import ipdb; ipdb.post_mortem(tb)
Which gets us straight to the offending code:
> /tmp/crash.py(4)boom()
3 def boom(x, y):
----> 4 x / y
5
ipdb> p x
2
You can also use the context manager
with ipdb.launch_ipdb_on_exception():
main()
It's an easy-to-use wrapper using ipdb.post_mortem.
Depending on what you need, there are 2 general best practices.
Just print the variables with minimal code edits
Have a look at some related packages. For simple usage you might pick traceback-with-variables (pip install traceback-with-variables), here is it's postcard
Or try tbvaccine, or better-exceptions, or any other package
Programmatically access variables to use them in your code
Use inspect module
except ... as ...:
x = inspect.trace()[-1][0].f_locals['x']
What about debugger?
Debugger is made for step-by-step execution and breakpoints. Using it to inspect exception reasons is really inconvenient and should be avoided. You can automate your debug session using two mentioned best practices.
try:
left_break = signs_pos[dave - 1]
except IndexError:
left_error = True
try:
right_break = signs_pos[dave + 1]
except IndexError:
right_error = True
if left_error == True:
current_cal = user_input[:right_break]
elif right_error == True:
current_cal = user_input[left_break:]
else:
current_cal = user_input[left_break:right_break]
I've only started looking at try functions and I need some help with this. What I would like to happen is that if when it tries to find left_break and it gives an error it will set left_error to be true. But if it does not give an error left_break will be set properly.
When the code runs and either right or left does not give an error it does not set left_break or right_break properly.
Traceback (most recent call last):
File "C:\Users\Max\Desktop\MaxsCal.py", line 170, in <module>
current_cal = user_input[:right_break]
NameError: name 'right_break' is not defined
This is the error I get without the try function.
Traceback (most recent call last):
File "C:\Users\Max\Desktop\MaxsCal.py", line 157, in <module>
right_break = signs_pos[dave + 1]
IndexError: list index out of range
Both right_error and left_error will not be true.
The reason why this is happening, is that because you are trying to assign something to a variable inside a try/except, it will not actually exist if you raised an exception.
Here is a simple example to clarify this:
try:
x = 6 / 0
except ZeroDivisionError:
print('this failed')
print(x)
>>> print(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
Now, to remedy this and you are looking to actually use the variable even if it fails in the try/except, you want to declare it before you are calling the thing that might fail.
x = 0
try:
x = 6 / 0
except ZeroDivisionError:
print('this failed')
print(x)
>>> print(x)
>>> 0
Or even inside your try works, but before what you are calling:
try:
x = 0
x = 6 / 0
except ZeroDivisionError:
print('this failed')
print(x)
>>> print(x)
>>> 0
As mentioned to me in the comments, you could also set a default in your except:
try:
x = 6 / 0
except ZeroDivisionError:
print('this failed')
x = 0
print(x)
>>> print(x)
>>> 0
left_break and right_break are only available within the scope of the try block. You can either define them before the try/except block or add an else block to the exception, as discussed in this thread.
Your error is coming from the fact that the variable is out of scope. Right break is only in scope within the try block, change it to
right_break = None
try:
right_break = signs_pos[dave + 1]
except IndexError:
right_error = True