I have a situation where I want to do multiple things while handling an exception. Since I want to make this about the general case, I'll translate my specific case into some more general language.
When I have an exception in this piece of code, I want to:
Always perform a rollback-style operation
If it is an
application specific exception, I want to perform some logging and swallow the exception.
So I can think of two ways to solve it, both ugly:
# Method nested-try/except block
try:
try:
do_things()
except:
rollback()
raise
except SpecificException as err:
do_advanced_logging(err)
return
# Method Duplicate Code
try:
do_things()
except SpecificException as err:
rollback()
do_advanced_logging(err)
return
except:
rollback()
raise
Both will have the same behaviour.
I'm tending towards the nested try/except solution myself. While it might be slightly slower, I don't think the speed difference is relevant here - at the very least not for my specific case. Duplication of code is something I want to avoid also because my rollback() statement is slightly more involved that just a database rollback, even if it has the exact same purpose (it involves a web-API).
Is there a third option I haven't spotted that is better? Or is the duplicate code method better? Please note that the rollback() functionality is already factored out as much as possible, but still contains a function call and three arguments which includes a single hardcoded string. Since this string is unique, there's no reason to make it a named constant.
How about checking the exception instance type in code?
# Method .. No Duplicate Code
try:
do_things()
except Exception as e:
rollback()
if isinstance(e, SpecificException):
do_advanced_logging(e)
return
raise
how about putting the rollback in a finally clause? something like:
do_rollback = True
try:
do_things()
do_rollback = False
except SpecificException as err:
do_advanced_logging(err)
finally:
if do_rollback:
rollback()
an alternative is to use an else clause, which would let you do more in the non-exceptional case and not have exceptions all caught in the same place:
do_rollback = True
try:
do_things()
except SpecificException as err:
do_advanced_logging(err)
else:
record_success()
do_rollback = False
finally:
if do_rollback:
rollback()
is useful when record_success can raise a SpecificException, but you don't want to do_advanced_logging
You could write a context manager:
import random
class SpecificException(Exception):
pass
def do_things(wot=None):
print("in do_things, wot = {}".format(wot))
if wot:
raise wot("test")
def rollback():
print("rollback")
def do_advance_logging(exc_type, exc_val, traceback):
print("logging got {} ('{}')".format(exc_type, exc_val))
class rollback_on_error(object):
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, traceback):
# always rollback
rollback()
# log and swallow specific exceptions
if exc_type and issubclass(exc_type, SpecificException):
do_advance_logging(exc_type, exc_val, traceback)
return True
# propagate other exceptions
return False
def test():
try:
with rollback_on_error():
do_things(ValueError)
except Exception as e:
print("expected ValueError, got '{}'".format(type(e)))
else:
print("oops, should have caught a ValueError")
try:
with rollback_on_error():
do_things(SpecificException)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
try:
with rollback_on_error():
do_things(None)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
if __name__ == "__main__":
test()
But unless you have dozen occurrences of this pattern, I'd rather stick to the very obvious and perfectly pythonic solutions - either nested exceptions handlers or explicit typecheck (isinstance) in the except clause.
I have a try:finally block that must execute always (exception or not) unless a specific exception occurs. For the sake of argument let's say it's a ValueError, so I'm asking if I can implement:
try:
stuff()
except Exception as e:
if type(e) is ValueError: raise
#do important stuff
raise
#do important stuff
in a more elegant fashion to skip copy-pasting #importantstuff. If I ruled Python it would look something like:
try:
stuff()
finally except ValueError:
#do important stuff
Putting #importantstuff in a function is not an answer, but not possible is.
If you need finally to skip things in specific conditions, you'll need to use an explicit flag:
do_final_stuff = True
try:
# ...
except ValueError:
do_final_stuff = False
raise
finally:
if do_final_stuff:
# ...
You could also use a context manager here, to clean up afterwards. A context manager is passed the current active exception if there is one:
class MyContextManager:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not ValueError:
# do cleanup
with MyContextManager():
# ...
I want a more detailed error log.
Specifically, I'd like to access the f_locals of an exception raising frame from inside an except clause.
def dumb_raiser(string: str):
if not isinstance(string, str):
raise ValueError("Yup")
try:
dumb_raiser(1)
except ValueError as ex:
# grab f_locals from frame and log here
pass
How would you access the necessary frame to do this?
One approach is to simply tack on the function's locals() to the exception when you raise it:
def dumb_raiser(string: str):
if not isinstance(string, str):
raise ValueError("Yup", locals())
try:
dumb_raiser(1)
except ValueError as ex:
print(ex.args[1])
This prints: {'string': 1}, as you'd expect.
I don't know if this is the best way (you could also examine the stackframes here) but the idea is to just use a decorator.
def derp(f):
def wrapped(*args, **kwargs):
try:
f(*args, **kwargs)
except Exception as ex:
raise ValueError("Failed args %s" % args)
return wrapped
def dumb_raiser(string):
if not isinstance(string, str):
raise ValueError("Yup")
try:
dumb_raiser = derp(dumb_raiser)
dumb_raiser(1)
except ValueError as ex:
print ex.message
Obviously you could do whatever you want to in the decorated exception handling. I just re-raised a ValueError with a simple string to show the passed arguments.
You can pull from Sentry's Python client, which captures stack locals with f_locals.
Is there a way I can catch exceptions in the __enter__ method of a context manager without wrapping the whole with block inside a try?
class TstContx(object):
def __enter__(self):
raise Exception("I'd like to catch this exception")
def __exit__(self, e_typ, e_val, trcbak):
pass
with TstContx():
raise Exception("I don't want to catch this exception")
pass
I know that I can catch the exception within __enter__() itself, but can I access that error from the function that contains the with statement?
On the surface the question Catching exception in context manager __enter__() seems to be the same thing but that question is actually about making sure that __exit__ gets called, not with treating the __enter__ code differently from the block that the with statement encloses.
...evidently the motivation should be clearer. The with statement is setting up some logging for a fully automated process. If the program fails before the logging is set up, then I can't rely on the logging to notify me, so I have to do something special. And I'd rather achieve the effect without having to add more indentation, like this:
try:
with TstContx():
try:
print "Do something"
except Exception:
print "Here's where I would handle exception generated within the body of the with statement"
except Exception:
print "Here's where I'd handle an exception that occurs in __enter__ (and I suppose also __exit__)"
Another downside to using two try blocks is that the code that handles the exception in __enter__ comes after the code that handles exception in the subsequent body of the with block.
You can catch the exception using try/except inside of __enter__, then save the exception instance as an instance variable of the TstContx class, allowing you to access it inside of the with block:
class TstContx(object):
def __enter__(self):
self.exc = None
try:
raise Exception("I'd like to catch this exception")
except Exception as e:
self.exc = e
return self
def __exit__(self, e_typ, e_val, trcbak):
pass
with TstContx() as tst:
if tst.exc:
print("We caught an exception: '%s'" % tst.exc)
raise Exception("I don't want to catch this exception")
Output:
We caught an exception: 'I'd like to catch this exception'.
Traceback (most recent call last):
File "./torn.py", line 20, in <module>
raise Exception("I don't want to catch this exception")
Exception: I don't want to catch this exception
Not sure why you'd want to do this, though....
You can use contextlib.ExitStack as outlined in this doc example in order to check for __enter__ errors separately:
from contextlib import ExitStack
stack = ExitStack()
try:
stack.enter_context(TstContx())
except Exception: # `__enter__` produced an exception.
pass
else:
with stack:
... # Here goes the body of the `with`.
My background is in C# and I've just recently started programming in Python. When an exception is thrown I typically want to wrap it in another exception that adds more information, while still showing the full stack trace. It's quite easy in C#, but how do I do it in Python?
Eg. in C# I would do something like this:
try
{
ProcessFile(filePath);
}
catch (Exception ex)
{
throw new ApplicationException("Failed to process file " + filePath, ex);
}
In Python I can do something similar:
try:
ProcessFile(filePath)
except Exception as e:
raise Exception('Failed to process file ' + filePath, e)
...but this loses the traceback of the inner exception!
Edit: I'd like to see both exception messages and both stack traces and correlate the two. That is, I want to see in the output that exception X occurred here and then exception Y there - same as I would in C#. Is this possible in Python 2.6? Looks like the best I can do so far (based on Glenn Maynard's answer) is:
try:
ProcessFile(filePath)
except Exception as e:
raise Exception('Failed to process file' + filePath, e), None, sys.exc_info()[2]
This includes both the messages and both the tracebacks, but it doesn't show which exception occurred where in the traceback.
Python 3
In python 3 you can do the following:
try:
raise MyExceptionToBeWrapped("I have twisted my ankle")
except MyExceptionToBeWrapped as e:
raise MyWrapperException("I'm not in a good shape") from e
This will produce something like this:
Traceback (most recent call last):
...
MyExceptionToBeWrapped: ("I have twisted my ankle")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
...
MyWrapperException: ("I'm not in a good shape")
Python 2
It's simple; pass the traceback as the third argument to raise.
import sys
class MyException(Exception): pass
try:
raise TypeError("test")
except TypeError, e:
raise MyException(), None, sys.exc_info()[2]
Always do this when catching one exception and re-raising another.
Python 3 has the raise ... from clause to chain exceptions. Glenn's answer is great for Python 2.7, but it only uses the original exception's traceback and throws away the error message and other details. Here are some examples in Python 2.7 that add context information from the current scope into the original exception's error message, but keep other details intact.
Known Exception Type
try:
sock_common = xmlrpclib.ServerProxy(rpc_url+'/common')
self.user_id = sock_common.login(self.dbname, username, self.pwd)
except IOError:
_, ex, traceback = sys.exc_info()
message = "Connecting to '%s': %s." % (config['connection'],
ex.strerror)
raise IOError, (ex.errno, message), traceback
That flavour of raise statement takes the exception type as the first expression, the exception class constructor arguments in a tuple as the second expression, and the traceback as the third expression. If you're running earlier than Python 2.2, see the warnings on sys.exc_info().
Any Exception Type
Here's another example that's more general purpose if you don't know what kind of exceptions your code might have to catch. The downside is that it loses the exception type and just raises a RuntimeError. You have to import the traceback module.
except Exception:
extype, ex, tb = sys.exc_info()
formatted = traceback.format_exception_only(extype, ex)[-1]
message = "Importing row %d, %s" % (rownum, formatted)
raise RuntimeError, message, tb
Modify the Message
Here's another option if the exception type will let you add context to it. You can modify the exception's message and then reraise it.
import subprocess
try:
final_args = ['lsx', '/home']
s = subprocess.check_output(final_args)
except OSError as ex:
ex.strerror += ' for command {}'.format(final_args)
raise
That generates the following stack trace:
Traceback (most recent call last):
File "/mnt/data/don/workspace/scratch/scratch.py", line 5, in <module>
s = subprocess.check_output(final_args)
File "/usr/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory for command ['lsx', '/home']
You can see that it shows the line where check_output() was called, but the exception message now includes the command line.
In Python 3.x:
raise Exception('Failed to process file ' + filePath).with_traceback(e.__traceback__)
or simply
except Exception:
raise MyException()
which will propagate MyException but print both exceptions if it will not be handled.
In Python 2.x:
raise Exception, 'Failed to process file ' + filePath, e
You can prevent printing both exceptions by killing the __context__ attribute. Here I write a context manager using that to catch and change your exception on the fly:
(see http://docs.python.org/3.1/library/stdtypes.html for expanation of how they work)
try: # Wrap the whole program into the block that will kill __context__.
class Catcher(Exception):
'''This context manager reraises an exception under a different name.'''
def __init__(self, name):
super().__init__('Failed to process code in {!r}'.format(name))
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None:
self.__traceback__ = exc_tb
raise self
...
with Catcher('class definition'):
class a:
def spam(self):
# not really pass, but you get the idea
pass
lut = [1,
3,
17,
[12,34],
5,
_spam]
assert a().lut[-1] == a.spam
...
except Catcher as e:
e.__context__ = None
raise
I don't think you can do this in Python 2.x, but something similar to this functionality is part of Python 3. From PEP 3134:
In today's Python implementation, exceptions are composed of three
parts: the type, the value, and the traceback. The 'sys' module,
exposes the current exception in three parallel variables, exc_type,
exc_value, and exc_traceback, the sys.exc_info() function returns a
tuple of these three parts, and the 'raise' statement has a
three-argument form accepting these three parts. Manipulating
exceptions often requires passing these three things in parallel,
which can be tedious and error-prone. Additionally, the 'except'
statement can only provide access to the value, not the traceback.
Adding the 'traceback' attribute to exception values makes all
the exception information accessible from a single place.
Comparison to C#:
Exceptions in C# contain a read-only 'InnerException' property that
may point to another exception. Its documentation [10] says that
"When an exception X is thrown as a direct result of a previous
exception Y, the InnerException property of X should contain a
reference to Y." This property is not set by the VM automatically;
rather, all exception constructors take an optional 'innerException'
argument to set it explicitly. The 'cause' attribute fulfills
the same purpose as InnerException, but this PEP proposes a new form
of 'raise' rather than extending the constructors of all exceptions.
C# also provides a GetBaseException method that jumps directly to
the end of the InnerException chain; this PEP proposes no analog.
Note also that Java, Ruby and Perl 5 don't support this type of thing either. Quoting again:
As for other languages, Java and Ruby both discard the original
exception when another exception occurs in a 'catch'/'rescue' or
'finally'/'ensure' clause. Perl 5 lacks built-in structured
exception handling. For Perl 6, RFC number 88 [9] proposes an exception
mechanism that implicitly retains chained exceptions in an array
named ##.
For maximum compatibility between Python 2 and 3, you can use raise_from in the six library. https://six.readthedocs.io/#six.raise_from . Here is your example (slightly modified for clarity):
import six
try:
ProcessFile(filePath)
except Exception as e:
six.raise_from(IOError('Failed to process file ' + repr(filePath)), e)
You could use my CausedException class to chain exceptions in Python 2.x (and even in Python 3 it can be useful in case you want to give more than one caught exception as cause to a newly raised exception). Maybe it can help you.
Maybe you could grab the relevant information and pass it up? I'm thinking something like:
import traceback
import sys
import StringIO
class ApplicationError:
def __init__(self, value, e):
s = StringIO.StringIO()
traceback.print_exc(file=s)
self.value = (value, s.getvalue())
def __str__(self):
return repr(self.value)
try:
try:
a = 1/0
except Exception, e:
raise ApplicationError("Failed to process file", e)
except Exception, e:
print e
Assuming:
you need a solution, which works for Python 2 (for pure Python 3 see raise ... from solution)
just want to enrich the error message, e.g. providing some additional context
need the full stack trace
you can use a simple solution from the docs https://docs.python.org/3/tutorial/errors.html#raising-exceptions:
try:
raise NameError('HiThere')
except NameError:
print 'An exception flew by!' # print or log, provide details about context
raise # reraise the original exception, keeping full stack trace
The output:
An exception flew by!
Traceback (most recent call last):
File "<stdin>", line 2, in ?
NameError: HiThere
It looks like the key piece is the simplified 'raise' keyword that stands alone. That will re-raise the Exception in the except block.