Python traceback broken: code out of sync with what is executed? - python

I'm running into a strange situation simply running a script from the command-line: when I run into errors, often the traceback given contains code that makes no sense. Every line in the traceback should be a method call, but sometimes they aren't. Also, the lines referenced in the traceback don't correspond to the given error. What's happening? As an example, here is a simple error where numpy wasn't imported, but the traceback makes little sense and refers to unrelated code lines:
Traceback (most recent call last):
File "bin/train_global_model.py", line 549, in <module>
if __name__ == '__main__':
File "bin/train_global_model.py", line 236, in main
def main():
File "bin/train_global_model.py", line 407, in do_training
tb_writer=train_writer,
File "bin/train_global_model.py", line 200, in run_iteration
print(accuracy)
NameError: global name 'np' is not defined
Pay special attention to the code lines referenced.
Is python caching code that is executed somewhere but then referring to the actual file when it's tracing an exception? Running Python 2.7.13.

Python saves line numbers not the actual source code, when running programs. For tracebacks it loads the source code and shows the corresponding lines to the numbers. When the source changes, while the program is running, you get lines out of sync.

Related

Error when running selected line of code in VS Code

I cannot run the selected block of the code in VS Code.
Given the code that works well if I run it as a whole
import numpy as np
x = np.arange(5)
print(x)
if I select the line print(x) and press Shift+Enter, it yields
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
It looks like the objects are erased from the memory as soon as the compilation is over.
Could somebody explain what is the reason and how to tackle this problem?
Thank you!
As you already know, the previous objects are erased with every execution of the code from the memory.
When you run just the print statement, it is like you would just run print(x) in a new file without defining it.
To my knowledge, this can't be changed, because the python interpreter works that way, and it creates a temporary file with the selected code and runs that. In that file are the objects not defined, and thus it raises an exception.

Spyder Trace Debugging - Python

I am having a weird situation where the trace for an AttributeError keeps referencing the exact same absolute line number in my code.
Traceback (most recent call last):
File "<ipython-input-59-513669e63f3e>", line 4, in <module>
"line of code being run"
File "modulepath", line 148, in "method that contains the line of code"
"content of the line causing the error"
AttributeError: 'list' object has no attribute 'attribute_name'
If I make line 148 empty (by putting in new lines, etc.) or if I put a comment on line 148, the trace keeps pointing to line 148 as the source of the error.
I am having trouble debugging this particular error in general since I am (to the best of my knowledge) not using that particular attribute on any list object in my code (but it seems to indicate that I'm doing so). I will figure out that issue on my own. My main question is: what causes Spyder to repeatedly reference the exact same absolute line number in a traceback? Thank you.
When in doubt, use a fresh new console - that worked for me.

python error Talentbuddy:EOFError: EOF when reading a line

I wrote some python code on Talentbuddy's code editor which lead to an EOFError: EOF when reading a line, but I ran the same code on python IDLE and everything is ok.
The code is meant to simply sum the number is given by Talenbuddy, but I don't know how to get the number.
def get_sum(a,b):
c=a+b
return c
a=raw_input()
b=raw_input()
print get_sum(a,b)
The error is:
Traceback (most recent call last):
File "eval_get_sum.py", line 3, in
from user_file import get_sum
File "/eval/user_file.py", line 4, in
a=raw_input()
EOFError: EOF when reading a line
Assuming this is "Simple Sum", you aren't supposed to be taking any raw_input. When you click "Check Solution", the system will call your function, providing appropriate arguments a and b; you don't have to call it yourself. Remove the last three lines and test again.

How can I modify a Python traceback object when raising an exception?

I'm working on a Python library used by third-party developers to write extensions for our core application.
I'd like to know if it's possible to modify the traceback when raising exceptions, so the last stack frame is the call to the library function in the developer's code, rather than the line in the library that raised the exception. There are also a few frames at the bottom of the stack containing references to functions used when first loading the code that I'd ideally like to remove too.
Thanks in advance for any advice!
You can remove the top of the traceback easily with by raising with the tb_next element of the traceback:
except:
ei = sys.exc_info()
raise ei[0], ei[1], ei[2].tb_next
tb_next is a read_only attribute, so I don't know of a way to remove stuff from the bottom. You might be able to screw with the properties mechanism to allow access to the property, but I don't know how to do that.
Take a look at what jinja2 does here:
https://github.com/mitsuhiko/jinja2/blob/5b498453b5898257b2287f14ef6c363799f1405a/jinja2/debug.py
It's ugly, but it seems to do what you need done. I won't copy-paste the example here because it's long.
Starting with Python 3.7, you can instantiate a new traceback object and use the .with_traceback() method when throwing. Here's some demo code using either sys._getframe(1) (or a more robust alternative) that raises an AssertionError while making your debugger believe the error occurred in myassert(False): sys._getframe(1) omits the top stack frame.
What I should add is that while this looks fine in the debugger, the console behavior unveils what this is really doing:
Traceback (most recent call last):
File ".\test.py", line 35, in <module>
myassert_false()
File ".\test.py", line 31, in myassert_false
myassert(False)
File ".\test.py", line 26, in myassert
raise AssertionError().with_traceback(back_tb)
File ".\test.py", line 31, in myassert_false
myassert(False)
AssertionError
Rather than removing the top of the stack, I have added a duplicate of the second-to-last frame.
Anyway, I focus on how the debugger behaves, and it seems this one works correctly:
"""Modify traceback on exception.
See also https://github.com/python/cpython/commit/e46a8a
"""
import sys
import types
def myassert(condition):
"""Throw AssertionError with modified traceback if condition is False."""
if condition:
return
# This function ... is not guaranteed to exist in all implementations of Python.
# https://docs.python.org/3/library/sys.html#sys._getframe
# back_frame = sys._getframe(1)
try:
raise AssertionError
except AssertionError:
traceback = sys.exc_info()[2]
back_frame = traceback.tb_frame.f_back
back_tb = types.TracebackType(tb_next=None,
tb_frame=back_frame,
tb_lasti=back_frame.f_lasti,
tb_lineno=back_frame.f_lineno)
raise AssertionError().with_traceback(back_tb)
def myassert_false():
"""Test myassert(). Debugger should point at the next line."""
myassert(False)
if __name__ == "__main__":
myassert_false()
You might also be interested in PEP-3134, which is implemented in python 3 and allows you to tack one exception/traceback onto an upstream exception.
This isn't quite the same thing as modifying the traceback, but it would probably be the ideal way to convey the "short version" to library users while still having the "long version" available.
What about not changing the traceback? The two things you request can both be done more easily in a different way.
If the exception from the library is caught in the developer's code and a new exception is raised instead, the original traceback will of course be tossed. This is how exceptions are generally handled... if you just allow the original exception to be raised but you munge it to remove all the "upper" frames, the actual exception won't make sense since the last line in the traceback would not itself be capable of raising the exception.
To strip out the last few frames, you can request that your tracebacks be shortened... things like traceback.print_exception() take a "limit" parameter which you could use to skip the last few entries.
That said, it should be quite possible to munge the tracebacks if you really need to... but where would you do it? If in some wrapper code at the very top level, then you could simply grab the traceback, take a slice to remove the parts you don't want, and then use functions in the "traceback" module to format/print as desired.
For python3, here's my answer. Please read the comments for an explanation:
def pop_exception_traceback(exception,n=1):
#Takes an exception, mutates it, then returns it
#Often when writing my repl, tracebacks will contain an annoying level of function calls (including the 'exec' that ran the code)
#This function pops 'n' levels off of the stack trace generated by exception
#For example, if print_stack_trace(exception) originally printed:
# Traceback (most recent call last):
# File "<string>", line 2, in <module>
# File "<string>", line 2, in f
# File "<string>", line 2, in g
# File "<string>", line 2, in h
# File "<string>", line 2, in j
# File "<string>", line 2, in k
#Then print_stack_trace(pop_exception_traceback(exception),3) would print:
# File "<string>", line 2, in <module>
# File "<string>", line 2, in j
# File "<string>", line 2, in k
#(It popped the first 3 levels, aka f g and h off the traceback)
for _ in range(n):
exception.__traceback__=exception.__traceback__.tb_next
return exception
This code might be of interest for you.
It takes a traceback and removes the first file, which should not be shown. Then it simulates the Python behavior:
Traceback (most recent call last):
will only be shown if the traceback contains more than one file.
This looks exactly as if my extra frame was not there.
Here my code, assuming there is a string text:
try:
exec(text)
except:
# we want to format the exception as if no frame was on top.
exp, val, tb = sys.exc_info()
listing = traceback.format_exception(exp, val, tb)
# remove the entry for the first frame
del listing[1]
files = [line for line in listing if line.startswith(" File")]
if len(files) == 1:
# only one file, remove the header.
del listing[0]
print("".join(listing), file=sys.stderr)
sys.exit(1)

Python exception backtrace tells me where line ends, where does it begin?

When A Python exception is thrown by code that spans multiple lines, e.g.:
myfoos = [foo("bar",
"baz",
"quux",
i) for i in range(10)]
Python will report the line number of the last line, and will show the code fragment from that line:
Traceback (most recent call last):
File "test.py", line 4, in <module>
i) for i in range(10)]
NameError: name 'foo' is not defined
Is there any way to determine what the first line is? Is there any way to catch the exception and manipulate the traceback object to be able to report something like this instead:
Traceback (most recent call last):
File "test.py", lines 1-4 in <module>
myfoos = [foo("bar",
"baz",
"quux",
i) for i in range(10)]
NameError: name 'foo' is not defined
Finding the beginning of the line will be really hard. You'll have to either parse the Python or maybe dig into the compiled byte code. There are modules in the standard library for parsing Python, but I can tell you from experience that interpreting their output is a black art. And I'm not sure the compiled byte code has the answer either...
In a try/except block you can except NameError and try setting NameError.lineno, though I'm not exactly sure if or how this works, but it's the best I've found thusfar.
try:
somecode
except NameError
NameError.lineno = [1,4]
You'll have to figure out where the statement begins and ends yourself somehow as well as which statement is raising the error.
Hope this helps

Categories

Resources