Develop Reference

How can I add context to an exception in Python

I would like to add context to an exception like this:
def process(vals):
for key in vals:
try:
do_something(vals[key])
except Exception as ex: # base class. Not sure what to expect.
raise # with context regarding the key that was being processed.
I found a way that is uncharacteristically long winded for Python. Is there a better way than this?
try:
do_something(vals[key])
except Exception as ex:
args = list(ex.args)
if len(args) > 1:
args[0] = "{}: {}".format(key, args[0])
ex.args = tuple(args)
raise # Will re-trhow ValueError with new args[0]

The first item in ex.args is always the message -- if there is any. (Note for some exceptions, such as the one raised by assert False, ex.args is an empty tuple.)
I don't know of a cleaner way to modify the message than reassigning a new tuple to ex.args. (We can't modify the tuple since tuples are immutable).
The code below is similar to yours, except it constructs the tuple without using an intermediate list, it handles the case when ex.args is empty, and to make the code more readable, it hides the boilerplate inside a context manager:
import contextlib
def process(val):
with context(val):
do_something(val)
def do_something(val):
# assert False
return 1/val
#contextlib.contextmanager
def context(msg):
try:
yield
except Exception as ex:
msg = '{}: {}'.format(msg, ex.args[0]) if ex.args else str(msg)
ex.args = (msg,) + ex.args[1:]
raise
process(0)
yields a stack trace with this as the final message:
ZeroDivisionError: 0: division by zero

You could just raise a new exception:
def process(vals):
for key in vals:
try:
do_something(vals[key])
except Exception as ex:
raise Error(key, context=ex)
On Python 3 you don't need to provide the old exception explicitly, it will be available as __context__ attribute on the new exception object and the default exception handler will report it automatically:
def process(vals):
for key in vals:
try:
do_something(vals[key])
except Exception:
raise Error(key)
In you case, you should probably use the explicit raise Error(key) from ex syntax that sets __cause__ attribute on the new exception, see Exception Chaining and Embedded Tracebacks.
If the only issue is the verbosity of the message-amending code in your question; you could encapsulate it in a function:
try:
do_something(vals[key])
except Exception:
reraise_with_context(key=key) # reraise with extra info
where:
import inspect
import sys
def reraise_with_context(**context):
ex = sys.exc_info()[1]
if not context: # use locals from the caller scope
context = inspect.currentframe().f_back.f_locals
extra_info = ", ".join("%s=%s" % item for item in context.items())
amend_message(ex, extra_info)
raise
def amend_message(ex, extra):
msg = '{} with context: {}'.format(ex.args[0], extra) if ex.args else extra
ex.args = (msg,) + ex.args[1:]

Try...Except on multiple, independent statements, executing as many as possible

I want to execute several functions, gather their exceptions (if there are any), and raise a compound exception, calling as many of the functions as possible without breaking after one exception. For example, say I have
def f():
do_one()
do_two()
do_three()
The do_i functions don't depend on each other's status. The most obvious way to do what I want is this:
def f():
errors = []
for do_i in [do_one, do_two, do_three]:
try:
do_i()
except Exception as e:
errors.append(e)
if errors:
raise Exception(';'.join(errors))
or slightly better:
def catch_error(arr, f, *args, **kwargs):
try:
return f(*args, **kwargs)
except Exception as e:
arr.append(e)
return None
def f():
errors = []
for do_i in [do_one, do_two, do_three]:
catch_error(errors, do_i)
if errors:
raise Exception(';'.join(errors))
but this is still ugly. Is there a Pythonic way to do this that I'm missing, maybe with clever use of a with statement?
Edit: In a dream world Python would have this:
errors = []
awesome_block(errors):
do_one()
do_two()
do_three()
return 'yes!' if not errors else ';'.join(map(str, errors))

You could rewrite your function into a contextmanager, which does simplify your code a bit. I've maintained your convention of passing a list, though this yields the internal list, so you can use it later.
from contextlib import contextmanager
#contextmanager
def catch_errors(error_list=None):
error_list = error_list if error_list is not None else []
try:
yield error_list
except Exception as e:
error_list.append(e)
error_list = []
with catch_errors(error_list):
raise Exception("First exception")
with catch_errors(error_list):
raise ValueError("Second exception")
if error_list:
raise Exception(";".join(map(repr, error_list)))
I think repr is more useful than str here. #contextmanager allows usage in a with statement while you only have to write the function as a generator.
If you don't pass a list to the generator, then you need to keep track of the returned list.
with catch_errors() as errors1:
raise Exception("First exception")
print errors1 # Exception("First exception",)

How to write multiple try statements in one block in python?

I want to do:
try:
do()
except:
do2()
except:
do3()
except:
do4()
If do() fails, execute do2(), if do2() fails too, exceute do3() and so on.
best Regards

If you really don't care about the exceptions, you could loop over cases until you succeed:
for fn in (do, do2, do3, do4):
try:
fn()
break
except:
continue
This at least avoids having to indent once for every case. If the different functions need different arguments you can use functools.partial to 'prime' them before the loop.

I'd write a quick wrapper function first() for this.
usage: value = first([f1, f2, f3, ..., fn], default='All failed')
#!/usr/bin/env
def first(flist, default=None):
""" Try each function in `flist` until one does not throw an exception, and
return the return value of that function. If all functions throw exceptions,
return `default`
Args:
flist - list of functions to try
default - value to return if all functions fail
Returns:
return value of first function that does not throw exception, or
`default` if all throw exceptions.
TODO: Also accept a list of (f, (exceptions)) tuples, where f is the
function as above and (exceptions) is a tuple of exceptions that f should
expect. This allows you to still re-raise unexpected exceptions.
"""
for f in flist:
try:
return f()
except:
continue
else:
return default
# Testing.
def f():
raise TypeError
def g():
raise IndexError
def h():
return 1
# We skip two exception-throwing functions and return value of the last.
assert first([f, g, h]) == 1
assert first([f, g, f], default='monty') == 'monty'

It seems like a really odd thing to want to do, but I would probably loop over the functions and break out when there were no exception raised:
for func in [do, do2, do3]:
try:
func()
except Exception:
pass
else:
break

Here is the simplest way I found, just embed the try under the previous except.
try:
do()
except:
try:
do2()
except:
do3()

You should specify the type of the exception you are trying to catch each time.
try:
do()
except TypeError: #for example first one - TypeError
do_2()
except KeyError: #for example second one - KeyError
do_3()
and so on.

if you want multiple try statments you can do it like this, including the except statement. Extract (refactor) your statements. And use the magic of and and or to decide when to short-circuit.
def a():
try: # a code
except: pass # or raise
else: return True
def b():
try: # b code
except: pass # or raise
else: return True
def c():
try: # c code
except: pass # or raise
else: return True
def d():
try: # d code
except: pass # or raise
else: return True
def main():
try:
a() and b() or c() or d()
except:
pass

import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except OSError as err:
print("OS error: {0}".format(err))
except ValueError:
print("Could not convert data to an integer.")
except:
print("Unexpected error:", sys.exc_info()[0])
raise

Can I get the local variables of a Python function from which an exception was thrown?

I'm writing a custom logging system for a project. If a function throws an exception, I want to log its local variables. Is it possible to access the raising function's local variables from the except block that caught the exception? For example:
def myfunction():
v1 = get_a_value()
raise Exception()
try:
myfunction()
except:
# can I access v1 from here?

It's generally cleaner design to pass the value to the exception, if you know that your exception handling code is going to need it. However, if you're writing a debugger or something like that, where you will need to access variables without knowing which ones they are in advance, you can access an arbitrary variable in the context where the exception was thrown:
def myfunction():
v1 = get_a_value()
raise Exception()
try:
myfunction()
except:
# can I access v1 from here?
v1 = inspect.trace()[-1][0].f_locals['v1']
The functionality of the trace function, and the format of the traceback objects it deals with, are described in the inspect module documentation.

You can look up the local variables in the frame object, which you can get from sys.exc_info.
>>> import sys
>>> def f(a):
... b = a - 1
... print 1.0 / b
...
>>> try:
... f(1)
... except Exception, e:
... print sys.exc_info()[2].tb_next.tb_frame.f_locals
...
{'a': 1, 'b': 0}
You'll have to include the appropriate number of tb_nexts depending on from how deep in the stack the exception was thrown.

def myFunction()
v1 = get_a_value()
raise Exception(v1)
try:
myFunction()
except Exception, e:
v1 = e.args[0]

Yes, if it's your own exception type. Like this:
>>> class MyExn(Exception):
... def __init__(self, val):
... self.val = val
... def __str__(self):
... print "something wrong with:", str(self.val)
...
>>> def foo():
... val = 42
... raise MyExn(val)
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in foo
__main__.MyExnsomething wrong with: 42
>>> # Or in a try block:
>>> try:
... foo()
... except MyExn as e:
... print e.val
...
42
>>>

try:
myfunction()
except:
import sys
type, value, tb = sys.exc_info()
while tb.tb_next:
tb = tb.tb_next
frame = tb.tb_frame
print frame.f_locals['v1']

Useful for "handling" exceptions without using raise Exception(some_def_var) and also without using a library such as "inspect", for example.
However, I think if is to handle the exception "in real", might be better to use something like raise Exception(some_def_var). See #capfredf answer.
class MyClass():
def __init__(self):
self.my_attr_I = ""
self.my_attr_II = ""
def my_def_I(self):
try:
self.my_def_II()
except Exception as e:
print(self.my_attr_I)
print(self.my_attr_II)
def my_def_II(self):
self.my_attr_I = "TO EXCEPT I"
self.my_attr_II = "TO except II"
# NOTE: We will treat it as a general exception! By Questor
zero_division = 1/0

Calling nonexistent functions of None

If I name an exception as a variable, what properties are held by that variable? For example,
try:
None.nonexistent_function()
#return an AttributeError.
except Exception as ex:
self.assertEqual(__, ex.__class__.__name__)
In this instance, what is necessary to make the assertion True? How can the Name and Class of the Exception be determined?
This question is part of Python Koans, a recent port of Ruby Koans.

Try it at the REPL:
>>> try: None.foo()
... except Exception as ex: pass
...
>>> # ex is still in scope, so we can play around with it and find out for ourselves:
... ex.__class__.__name__
'AttributeError'
>>>

Hmmm... using Python's command line, I get:
>>> try:
... None.nonexistent_function()
... #return an AttributeError.
... except Exception as ex:
... print ex.__class__.__name__
...
AttributeError
>>>
So let's try:
>>> try:
... None.nonexistent_function()
... #return an AttributeError.
... except Exception as ex:
... print 'AttributeError' == ex.__class__.__name__
...
True
I don't have whatever your self object is handy, so you'll have to test the rest. Does that work for you?