The documentation for the raise statement with no arguments says
If no expressions are present, raise re-raises the last exception that was active in the current scope.
I used to think that meant that the current function had to be executing an except clause. After reading this question and experimenting a little, I think it means that any function on the stack has to be executing an except clause, but I'm not sure. Also, I've realized I have no idea how the stack trace works with a no-arg raise:
def f():
try:
raise Exception
except:
g()
def g():
raise
f()
produces
Traceback (most recent call last):
File "foo", line 10, in <module>
f()
File "foo", line 5, in f
g()
File "foo", line 3, in f
raise Exception
Exception
That doesn't look like the stack at the time of the initial raise, or the stack at the time of the re-raise, or the concatenation of both stacks, or anything I can make sense of.
Am I right about a no-arg raise looking for any function on the stack executing an except clause? Also, how does the stack trace work on a reraise?
When you raise without arguments, the interpreter looks for the last exception raised and handled. It then acts the same as if you used raise with the most recent exception type, value and traceback.
This is stored in the interpreter state for the current thread, and the same information can be retrieved using sys.exc_info(). By 'handled' I mean that an except clause caught the exception. Quoting the try statement documentation:
Before an except clause’s suite is executed, details about the exception are assigned to three variables in the sys module: sys.exc_type receives the object identifying the exception; sys.exc_value receives the exception’s parameter; sys.exc_traceback receives a traceback object (see section The standard type hierarchy identifying the point in the program where the exception occurred. These details are also available through the sys.exc_info() function, which returns a tuple (exc_type, exc_value, exc_traceback).
See the implemenation notes in the Python evaluation loop (C code), specifically:
The second
bullet was for backwards compatibility: it was (and is) common to
have a function that is called when an exception is caught, and to
have that function access the caught exception via sys.exc_ZZZ.
(Example: traceback.print_exc()).
The traceback reflects how you came to the re-raise accurately. It is the current stack (line 10 calling f(), line 5 calling g()) plus the original location of the exception raised: line 3.
It turns out Python uses a surprising way of building tracebacks. Rather than building the whole stack trace on exception creation (like Java) or when an exception is raised (like I used to think), Python builds up a partial traceback one frame at a time as the exception bubbles up.
Every time an exception bubbles up to a new stack frame, as well as when an exception is raised with the one-argument form of raise (or the two-argument form, on Python 2), the Python bytecode interpreter loop executes PyTraceback_Here to add a new head to the linked list of traceback objects representing the stack trace. (0-argument raise, and 3-argument raise on Python 2, skip this step.)
Python maintains a per-thread stack of exceptions (and tracebacks) suspended by except and finally blocks that haven't finished executing. 0-argument raise restores the exception (and traceback) represented by the top entry on this stack, even if the except or finally is in a different function.
When f executes its raise:
raise Exception
Python builds a traceback corresponding to just that line:
File "foo", line 3, in f
raise Exception
When g executes 0-argument raise, this traceback is restored, but no entry is added for the 0-argument raise line.
Afterwards, as the exception bubbles up through the rest of the stack, entries for the g() and f() calls are added to the stack trace, resulting in the final stack trace that gets displayed:
Traceback (most recent call last):
File "foo", line 10, in <module>
f()
File "foo", line 5, in f
g()
File "foo", line 3, in f
raise Exception
Exception
The following piece of code might help you understand how the raise keyword works:
def fun(n):
try:
return 0 / n
except:
print('an exception raised:')
raise
try:
fun(0)
except:
print('cannot divide by zero')
try:
fun('0')
except:
print('cannot divide by a string')
Related
https://docs.python.org/3/library/traceback.html#module-traceback:
The module uses traceback objects — this is the object type that is stored in the sys.last_traceback variable and returned as the third item from sys.exc_info()
https://docs.python.org/3/library/traceback.html#traceback.extract_stack:
traceback.extract_stack(f=None, limit=None)
Extract the raw traceback from the current stack frame. The return value has the same format as for extract_tb(). The optional f and limit arguments have the same meaning as for print_stack().
https://docs.python.org/3/library/pdb.html#pdb.post_mortem pdb.post_mortem(traceback=None)
Enter post-mortem debugging of the given traceback object. If no traceback is given, it uses the one of the exception that is currently being handled (an exception must be being handled if the default is to be used).
yet
import traceback
import pdb
try:
1/0
except Exception:
t = traceback.extract_stack()
<buncha other stuff>
pdb.post_mortem(t)
doesn't work because traceback.extract_stack() returns a StackSummary object that pdb can't handle.
Use t = sys.exc_info()[2] instead.
Suppose I have the following custom exception.
class CustomException(TypeError):
def __init__(message, code):
super().__init__(f'{code}: message')
self.code = code
How does python know when to catch my exception in the following code?
try:
x = doSomething(a, b, c)
except CustomException:
raise
when I implement the doSomething() function, must it explicitly throw a CustomException in order for it to be caught? Like, for built-in exception classes, code can throw an exception like a KeyError and we don't have to explicitly say raise KeyError whenever we do something with a dictionary.
Any code that raises an exception has done so explicitly, including KeyError. No special handling is needed for custom exceptions versus the built-in types. A try...except can only catch an exception if one has been raised by code executed by any statement inside the try. This includes any further function calls, calls chain together into a callstack.
In the following example
>>> d = {}
>>> d['foo']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'foo'
the KeyError doesn't spring out of nothingness, the Python dict implementation raises that exception explicitly. This may not always be obvious because native code (code implemented in C here) doesn't show up in the Python traceback.
For the d[...] subscription operation the dict_subscript() function calls _PyErr_SetKeyError(key);, a helper function that uses PyErr_SetObject(), the C equivalent of raise to raise the KeyError(key) exception.
Catching exceptions works the same for all exception types, custom exceptions are not special here. When an exception is raised the normal code flow is interrupted, and the callstack is unwound until an active try statement is encountered, and then any exception handlers are tested, in order of definition in the source code, with isinstance(active_exception, ExceptionClassBeingHandled).
If nothing catches the exception, Python continues unwinding the callstack until it reaches the last stack frame, at which point Python would exit with a stack trace if nothing caught the exception.
I use the following code to call an arbitrary callable f() with appropriate number of parameters:
try:
res = f(arg1)
except TypeError:
res = f(arg1, arg2)
If f() is a two parameter function, calling it with just one parameter raises TypeError, so the function can be called properly in the except branch.
The problem is when f() is one-parameter function and the exception is raisen in the body of f() (possibly because of a bad call to some function), for example:
def f(arg):
map(arg) # raises TypeError
The control flow goes to the except branch because of the internal error of f(). Of course calling f() with two arguments raises a new TypeError. Then instead of traceback to the original error I get traceback to the call of f() with two parameters, which is much less helpful when debugging.
How can my code recognize exceptions raised not in the current scope to reraise them?
I want to write the code like this:
try:
res = f(arg1)
except TypeError:
if exceptionRaisedNotInTheTryBlockScope(): # <-- subject of the question
raise
res = f(arg1, arg2)
I know I can use a walkarround by adding exc_info = sys.exc_info() in the except block.
One of assumptions is I have no control over f() since it is given by user of my module. Also its __name__ attribute may be other than 'f'. Internal exception may be raised by bad recurrent call to f().
The walkarround is unsuitable since it complicates debugging by the author of f().
You can capture the exception object and examine it.
try:
res = f(arg1)
except TypeError as e:
if "f() missing 1 required positional argument" in e.args[0]:
res = f(arg1, arg2)
else:
raise
Frankly, though, not going the extra length to classify the exception should work fine as you should be getting both the original traceback and the secondary traceback if the error originated inside f() -- debugging should not be a problem.
Also, if you have control over f() you can make second argument optional and not have to second-guess it:
def f(a, b=None):
pass
Now you can call it either way.
How about this:
import inspect
if len(inspect.getargspec(f).args) == 1:
res = f(arg1)
else:
res = f(arg1, arg2)
Finally figured it out:
def exceptionRaisedNotInTheTryBlockScope():
return sys.exc_info()[2].tb_next is not None
sys.exc_info() returns a 3-element tuple. Its last element is the traceback of the last exception. If the traceback object is the only one in the traceback chain, then the exception has been raisen in the scope of the try block (https://docs.python.org/2/reference/datamodel.html).
According to https://docs.python.org/2/library/sys.html#sys.exc_info, it should be avoided to store the traceback value.
This question already has an answer here:
Why do I get a `NameError` (or `UnboundLocalError`) from using a named exception after the `except` block?
(1 answer)
Closed 2 years ago.
I have the following code:
def foo():
e = None
try:
raise Exception('I wish you would except me for who I am.')
except Exception as e:
print(e)
print(e)
foo()
In Python 2.7, this runs as expected and prints:
I wish you would except me for who I am.
I wish you would except me for who I am.
However in Python 3.x, the first line is printed, but the second line is not. It seems to delete the variable in the enclosing scope, giving me the following traceback from the last print statement:
Traceback (most recent call last):
File "python", line 9, in <module>
File "python", line 7, in foo
UnboundLocalError: local variable 'e' referenced before assignment
It is almost as if a del e statement is inserted after the except block. Is there any reasoning for this sort of behavior? I could understand it if the Python developers wanted except blocks to have their own local scope, and not leak into the surrounding scope, but why must it delete a variable in the outer scope that was previously assigned?
Quoting the documentation of try,
When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if
except E as N:
foo
was translated to
except E as N:
try:
foo
finally:
del N
This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
This is covered in these two PEPs.
PEP 3110 - Catching Exceptions in Python 3000
PEP 344 - Exception Chaining and Embedded Tracebacks
I am writing a class in Python and part of the code deals with a server. Therefore I need to deal with exceptions like ExpiredSession or ConnectionError.
Instead of writing exception handling code for every try/except block, I have a single function in the class to deal with the exceptions. something like this (inside the class definition)
def job_a(self):
try:
do something
except Exception as e:
#maybe print the error on screen.
self.exception_handling(e)
def job_b(self):
try:
do something else
except Exception as e:
#maybe print the error on screen.
self.exception_handling(e)
def exception_handling(self,e):
if isInstanceOf(e,ExpiredSession):
#deal with expired session.
self.reconnect()
if isInstanceOf(e,ConnectionError):
#deal with connection error
else:
#other exceptions
I am not sure if this kind of code would cause any problem because I haven't seen code do this. like, possible memory leak? (Now I notice the memory usage grows(though slowly) when I have more and more error/exception and eventually I have to restart the process before it eats all my memories). Not sure this is the cause.
Is it a good practice to pass exceptions to a single function?
This is a good use case for a context manager. You can see some examples of using context managers for error handling here. The contextmanager decorator allows you to write context managers concisely in the form of a single function. Here's a simple example:
class Foo(object):
def meth1(self):
with self.errorHandler():
1/0
def meth2(self):
with self.errorHandler():
2 + ""
def meth3(self):
with self.errorHandler():
# an unhandled ("unexpected") kind of exception
""[3]
#contextlib.contextmanager
def errorHandler(self):
try:
yield
except TypeError:
print "A TypeError occurred"
except ZeroDivisionError:
print "Divide by zero occurred"
Then:
>>> x = Foo()
>>> x.meth1()
Divide by zero occurred
>>> x.meth2()
A TypeError occurred
The with statement allows you to "offload" the error handling into a separate function where you catch the exceptions and do what you like with them. In your "real" functions (i.e., the functions that do the work but may raise the exceptions), you just need a with statement instead of an entire block of complicated try/except statements.
An additional advantage of this approach is that if an unforeseen exception is raised, it will propagate up normally with no extra effort:
>>> x.meth3()
Traceback (most recent call last):
File "<pyshell#394>", line 1, in <module>
x.meth3()
File "<pyshell#389>", line 12, in meth3
""[3]
IndexError: string index out of range
In your proposed solution, on the other hand, the exception is already caught in each function and the actual exception object is passed the handler. If the handler gets an unexpected error, it would have to manually reraise it (and can't even use a bare raise to do so). Using a context manager, unexpected exceptions have their ordinary behavior with no extra work required.