How can we print the values of arguments passed to the functions in the call stack when an error stack trace is printed? I would like the output to be exactly as in the example below.
Example:
Traceback (most recent call last):
File "./file.py", line 615, in func0 **(arg0) arg0 = 0 was passed**
result = func1(arg1, arg2)
File "./file.py", line 728, in func1 **(arg1, arg2) arg1 = 1 and arg2 = 2 was passed**
return int_value[25]
TypeError: 'int' object is not iterable
I'd like the information inside the ** **s above to also be printed in addition to the normal output in the stack trace. What I envision is that the debugger automatically prints the passed arguments as well. That would give a clear picture of the "functional pipeline" that the data was passed through and what happened to it in the pipeline and which function did not do what it was supposed to do. This would help debugging a lot.
I searched quite a bit and found these related questions:
How to print call stack with argument values?
How to print function arguments in sys.settrace?
but the answers to neither of them worked for me: The answer to the 1st one led to ModuleNotFoundError: No module named 'stackdump'. The answer to the 2nd one crashed my ipython interpreter with a very long stack trace.
I also looked up:
https://docs.python.org/3/library/sys.html#sys.settrace
https://docs.python.org/3/library/traceback.html
There seems to be a capture_locals variable for TracebackExceptions, but I didn't quite understand how to make it work.
Pure Python3
Talking about TracebackExceptions and it's capture_locals argument, we can use it as follows:
#!/usr/bin/env python3
import traceback
c = ['Help me!']
def bar(a = 3):
d = {1,2,3}
e = {}
foo(a)
def foo(a=4):
b = 4
if a != b:
raise Exception("a is not equal to 4")
try:
bar(3)
except Exception as ex:
tb = traceback.TracebackException.from_exception(ex, capture_locals=True)
print("".join(tb.format()))
Which prints all local variables for each frame:
$ ./test.py
Traceback (most recent call last):
File "./test.py", line 21, in <module>
bar(3)
__annotations__ = {}
__builtins__ = <module 'builtins' (built-in)>
__cached__ = None
__doc__ = None
__file__ = './test.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7f81073704c0>
__name__ = '__main__'
__package__ = None
__spec__ = None
bar = <function bar at 0x7f81073b11f0>
c = ['Help me!']
ex = Exception('a is not equal to 4')
foo = <function foo at 0x7f810728a160>
traceback = <module 'traceback' from '/usr/lib/python3.8/traceback.py'>
File "./test.py", line 11, in bar
foo(a)
a = 3
d = {1, 2, 3}
e = {}
File "./test.py", line 17, in foo
raise Exception("a is not equal to 4")
a = 3
b = 4
Exception: a is not equal to 4
Looks a bit too verbose, but sometimes this data could be vital in debugging some crash.
Loguru
There is also a package loguru that prints "Fully descriptive exceptions":
2018-07-17 01:38:43.975 | ERROR | __main__:nested:10 - What?!
Traceback (most recent call last):
File "test.py", line 12, in <module>
nested(0)
└ <function nested at 0x7f5c755322f0>
> File "test.py", line 8, in nested
func(5, c)
│ └ 0
└ <function func at 0x7f5c79fc2e18>
File "test.py", line 4, in func
return a / b
│ └ 0
└ 5
ZeroDivisionError: division by zero
Probably exist better alternatives, but you can use a decorator for this:
def print_stack_arguments(func):
def new_func(*original_args, **original_kwargs):
try:
return func(*original_args, **original_kwargs)
except Exception as e:
print('Function: ', func.__name__)
print('Args: ', original_args)
print('Kwargs: ', original_kwargs)
print(e)
raise
return new_func
#print_stack_arguments
def print_error(value):
a = []
print(a[1])
#print_stack_arguments
def print_noerror(value):
print('No exception raised')
print_noerror('testing no exception')
print_error('testing exception')
Related
I want to print an error's line number and error message in a nicely displayed way. The follow is my code, which uses linecache:
import linecache
def func():
if xx == 1:
print('ok')
try:
func()
except:
exc_type, exc_obj, tb = sys.exc_info()
f = tb.tb_frame
lineno = tb.tb_lineno
filename = f.f_code.co_filename
linecache.checkcache(filename)
line = linecache.getline(filename, lineno, f.f_globals)
print_('ERROR - (LINE {} "{}"): {}'.format(lineno, line.strip(), exc_obj))
However, this only gives where the func() is called:
ERROR - (LINE 8 ""): name 'xx' is not defined
Is there a way to print the line number where the error actually occured, which should be Line 4? Or even better, can I print Line 8 and then trace back to line 4? For example, if I do not use try - except, the code:
def func():
if xx == 1:
print('ok')
func()
will give me the following error message, which is much better to locate the error:
File "<input>", line 5, in <module>
File "<input>", line 2, in func
NameError: name 'xx' is not defined. Did you mean: 'xxx'?
You can use traceback and sys modules to get advanced traceback output like you are wishing for.
Here is an example:
import traceback
import sys
def func():
zeroDivide = 1 / 0
try:
func()
except Exception:
print(traceback.format_exc()) # This line is for getting traceback.
print(sys.exc_info()[2]) # This line is getting for the error type.
Output will be:
Traceback (most recent call last):
File "b:\abc\1234\pppp\main.py", line 10, in <module>
func()
File "b:\abc\1234\pppp\main.py", line 7, in func
zeroDivide = 1 / 0
ZeroDivisionError: division by zero
You can use the traceback module to get the line number of the error,
import traceback
def function():
try:
# code
except:
tb_list = traceback.extract_tb(sys.exc_info()[2])
line_number = tb_list[-1][1]
print("An error occurred on line:", line_number)
You can use the traceback.extract_tb() function. This function returns a list of traceback objects, each of which contain information about the stack trace. The last element of this list, tb_list[-1], holds information about the line where the exception occurred. To access the line number, you can use the second element of this tuple, tb_list[-1][1]. This value can then be printed using the print() function.
To get the line number as an int you can get the traceback as a list from traceback.extract_tb(). Looking at the last item gives you the line where the exception was raised:
#soPrintLineOfError2
import sys
import traceback
def func():
if xx == 1:
print('ok')
try:
func()
except Exception as e:
tb = sys.exc_info()[2]
ss = traceback.extract_tb(tb)
ss1 = ss[-1]
print(ss1.line)
print(ss1.lineno)
Output:
if xx == 1:
6
I want to check a string - is it an import command? I have tried
# Helper - analyses a string - is it an import string?
"""
fromlike - from foo import bar
classic - import foo
classic_as - import foo as baz
"""
def check_is_import(string):
importname = ''
fromlike = False
classic = False
classic_as = False
if string[0:4] is 'from':
fromlike = True
importname = ''
if not fromlike and (string[0:6] is 'import'):
classic = True
importname = string.split(' ')[1]
if classic:
commandlist = string.split(' ')
if commandlist[2] is 'as':
classic_as = True
importname = commandlist[3]
del commandlist
if fromlike:
return ('fromlike', importname)
elif classic and (not classic_as):
return ('classic', importname)
elif classic_as:
return ('classic_as', importname)
else:
return ('no_import', importname)
but it worked for "fromlike" imports. (Note: I'm not asking "why does this code don't work?", I'm just searching a solution) What code will sure detect all imports? Basically my code takes a slice of the string. If the [0:4] slice equals 'from', the string is a "fromlike import". Else: if the [0:6] slice equals 'import', the string is a "classic import". If it detects 'as', it will find the pseudo-name. This function must return a tuple which contains the import type under index 0 and imported module-name under index 1.
If you want to be sure to handle all Python import forms, have Python do the parsing. Use the ast.parse() function and use the resulting parse tree; you'll either get Import or ImportFrom objects:
| Import(alias* names)
| ImportFrom(identifier? module, alias* names, int? level)
Each alias consists of a name and optional identifier used to import the name as:
-- import name with optional 'as' alias.
alias = (identifier name, identifier? asname)
Note that there can be multiple imports! You either have classic or fromlike imports, and both can import multiple names. Your function needs to return a list of (type, name) tuples. For invalid inputs, raise an exception (ValueError is a good fit here):
import ast
def check_is_import(string):
try:
body = ast.parse(string).body
except SyntaxError:
# not valid Python
raise ValueError('No import found')
if len(body) > 1:
# not a single statement
raise ValueError('Multiple statements found')
if not isinstance(body[0], (ast.Import, ast.ImportFrom)):
raise ValueError('No import found')
type_ = 'classic' if isinstance(body[0], ast.Import) else 'fromlike'
results = []
for alias in body[0].names:
alias_type = type_
if alias.asname:
alias_type += '_as'
results.append((alias_type, alias.asname or alias.name))
return results
The method should probably be renamed to extract_import_names(), as that reflects what it does much better.
Demo:
>>> check_is_import('from foo import bar')
[('fromlike', 'bar')]
>>> check_is_import('import foo')
[('classic', 'foo')]
>>> check_is_import('import foo as baz')
[('classic_as', 'baz')]
>>> check_is_import('from foo import bar, baz as spam, monty as python')
[('fromlike', 'bar'), ('fromlike_as', 'spam'), ('fromlike_as', 'python')]
>>> check_is_import('import foo as baz, baz, spam as ham')
[('classic_as', 'baz'), ('classic', 'baz'), ('classic_as', 'ham')]
>>> check_is_import('invalid python')
Traceback (most recent call last):
File "<stdin>", line 3, in check_is_import
File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
invalid python
^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in check_is_import
ValueError: No import found
>>> check_is_import('import foo; import bar')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in check_is_import
ValueError: Multiple statements found
>>> check_is_import('1 + 1 == 2')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in check_is_import
ValueError: No import found
I am writing UT for one of my function where I have to patch 2 objects.
#patch('mypackage.models.db_models.MongoClient',
return_value={})
#patch('mypackage.models.db_models.GridFS')
def test_file_in_db(self, mock_mongoclient, mock_gridfs):
print "*"*80
print mock_gridfs
print mock_gridfs.return_value
print "*"*80
mock_gridfs.return_value.new_file.return_value = {}
This gives error:
----------------------------------------------------------------------
Traceback (most recent call last):
File "/venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/tests/models/test_db_models.py", line 29, in test_file_in_db
mock_gridfs.return_value.new_file.return_value = {}
AttributeError: 'dict' object has no attribute 'new_file'
-------------------- >> begin captured stdout << ---------------------
********************************************************************************
<MagicMock name='MongoClient' id='4385486992'>
{}
********************************************************************************
--------------------- >> end captured stdout << ----------------------
When I am accessing second argument, means mock_gridfs why it return Mock object for MongoClient?
You have them in the wrong order, put the params in the reverse order you define them.
#patch('mypackage.models.db_models.MongoClient',
return_value={})
#patch('mypackage.models.db_models.GridFS')
def test_file_in_db(self, mock_gridfs, mock_mongoclient):
While stack traces are useful in Python, most often the data at the root of the problem are missing - is there a way of making sure that at least locals() (and possibly globals()) are added to printed stacktrace?
You can install your own exception hook and output what you need from there:
import sys, traceback
def excepthook(type, value, tb):
traceback.print_exception(type, value, tb)
while tb.tb_next:
tb = tb.tb_next
print >>sys.stderr, 'Locals:', tb.tb_frame.f_locals
print >>sys.stderr, 'Globals:', tb.tb_frame.f_globals
sys.excepthook = excepthook
def x():
y()
def y():
foo = 1
bar = 0
foo/bar
x()
To print vars from each frame in a traceback, change the above loop to
while tb:
print >>sys.stderr, 'Locals:', tb.tb_frame.f_locals
print >>sys.stderr, 'Globals:', tb.tb_frame.f_globals
tb = tb.tb_next
This is a Box of Pandora. Values can be very large in printed form; printing all locals in a stack trace can easily lead to new problems just due to error output. That's why this is not implemented in general in Python.
In small examples, though, i. e. if you know that your values aren't too large to be printed properly, you can step along the traceback yourself:
import sys
import traceback
def c():
clocal = 1001
raise Exception("foo")
def b():
blocal = 23
c()
def a():
alocal = 42
b()
try:
a()
except Exception:
frame = sys.exc_info()[2]
formattedTb = traceback.format_tb(frame)
frame = frame.tb_next
while frame:
print formattedTb.pop(0), '\t', frame.tb_frame.f_locals
frame = frame.tb_next
The output will be sth like this:
File "/home/alfe/tmp/stacktracelocals.py", line 19, in <module>
a()
{'alocal': 42}
File "/home/alfe/tmp/stacktracelocals.py", line 16, in a
b()
{'blocal': 23}
File "/home/alfe/tmp/stacktracelocals.py", line 12, in b
c()
{'clocal': 1001}
And you can, of course, install your own except hook as thg435 suggested in his answer.
if you didn't know about this already, use the pdb post-mortem feature:
x = 3.0
y = 0.0
print x/y
def div(a, b):
return a / b
print div(x,y)
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-3-d03977de5fc3> in div(a, b)
1 def div(a, b):
----> 2 return a / b
ZeroDivisionError: float division
import pdb
pdb.pm()
> <ipython-input-3-148da0dcdc9e>(2)div()
0 return a/b
ipdb> l
1 def div(a,b):
----> 2 return a/b
ipdb> a
3.0
ipdb> b
0.0
etc.
there are cases where you really need the prints though, of course. you're better off instrumenting the code (via try/except) to print out extra information around a specific weird exception you are debugging than putting this for everything though, imho.
Try traceback-with-variables package.
Usage:
from traceback_with_variables import traceback_with_variables
def main():
...
with traceback_with_variables():
...your code...
Exceptions with it:
Traceback with variables (most recent call last):
File "./temp.py", line 7, in main
return get_avg_ratio([h1, w1], [h2, w2])
sizes_str = '300 200 300 0'
h1 = 300
w1 = 200
h2 = 300
w2 = 0
File "./temp.py", line 10, in get_avg_ratio
return mean([get_ratio(h, w) for h, w in [size1, size2]])
size1 = [300, 200]
size2 = [300, 0]
File "./temp.py", line 10, in <listcomp>
return mean([get_ratio(h, w) for h, w in [size1, size2]])
.0 = <tuple_iterator object at 0x7ff61e35b820>
h = 300
w = 0
File "./temp.py", line 13, in get_ratio
return height / width
height = 300
width = 0
builtins.ZeroDivisionError: division by zero
Installation:
pip install traceback-with-variables
I'm trying to implement an assert function. How can I get the text of the failing condition into the error message? If I have to parse it from the backtrace, can I portably rely on anything about the format of frames?
AssertionError is just like any other exception in python, and assert is a simple statement that is equivalent to
if __debug__:
if not expression: raise AssertionError
or
if __debug__:
if not expression1: raise AssertionError(expression2)
so you can add a second parameter to your assertion to have additional output
from sys import exc_info
from traceback import print_exception
# assertions are simply exceptions in Python
try:
assert False, "assert was false"
except AssertionError:
print_exception(*exc_info())
outputs
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
AssertionError: assert was false
If you're sure the expression to test is secure you could do something like this:
File my_assert.py:
import sys
def my_assert(condition):
caller = sys._getframe(1)
if not eval(condition, caller.f_globals, caller.f_locals):
raise AssertionError(repr(condition) + " on line " +
str(caller.f_lineno) + ' in ' +
caller.f_code.co_name)
File test_my_assert.py:
from my_assert import my_assert
global_var = 42
def test():
local_var = 17
my_assert('local_var*2 < global_var') # OK
my_assert('local_var > global_var')
test()
Output:
Traceback (most recent call last):
File "test_my_assert.py", line 10, in <module>
test()
File "test_my_assert.py", line 8, in test
my_assert('local_var > global_var')
File "my_assert.py", line 8, in my_assert
caller.f_code.co_name)
AssertionError: 'local_var > global_var' on line 8 in test
My very hackish solution:
def my_assert(condition):
if not eval(condition):
# error stuff
Then use it by placing the condition in quotation marks. It is then a string that can be printed in the error message.
Or, if you want it to actually raise an AssertionError:
def my_assert(condition):
if not eval(condition):
raise AssertionError(condition)