How can I keep the try block as small as possible when I have to catch an exception which can occur in a generator?
A typical situation looks like this:
for i in g():
process(i)
If g() can raise an exception I need to catch, the first approach is this:
try:
for i in g():
process(i)
except SomeException as e:
pass # handle exception ...
But this will also catch the SomeException if it occurs in process(i) (this is what I do not want).
Is there a standard approach to handle this situation? Some kind of pattern?
What I am looking for would be something like this:
try:
for i in g():
except SomeException as e:
pass # handle exception ...
process(i)
(But this is syntactic nonsense of course.)
You could convert exceptions occurring in the inner block:
class InnerException(Exception):
pass
try:
for i in g():
try:
process(i)
except Exception as ex:
raise InnerException(ex)
except InnerException as ex:
raise ex.args[0]
except SomeException as e:
pass # handle exception ...
Another option is to write a local generator that wraps g:
def safe_g():
try:
for i in g():
yield i
except SomeException as e:
pass # handle exception ...
for i in safe_g():
process(i)
The straight-forward approach for this seems to be unwrap the for construct (which makes it impossible to catch exceptions just in the generator, due to its syntax) into its components.
gen = g()
while True:
try:
i = gen.next()
except StopIteration:
break
process(i)
Now we can just add our expected exception to the try block:
gen = g()
while True:
try:
i = gen.next()
except StopIteration:
break
except SomeException as e:
pass # handle exception ...
break
process(i)
Does that (besides it being ugly as hell) have disadvantages? And more: Is there a nicer solution?
(I won't accept my own answer because it being ugly, but maybe others like and upvote it.)
In your generator raise a different kind of exception, that you will be able to distinguish.
class GeneratorError(Exception):
pass
def g():
try:
yield <smth>
except:
raise GeneratorError
try:
for i in g():
process(i)
except GeneratorError:
pass # handle generator error
except SomeException as e:
pass # handle exception .
I don't know, if it works. You could evaluate g() into a list. I can't test it, because I don't have an iterator that throws an exception at hand.
try:
glist = list(g())
except SomeException as e:
pass # handle exception ...
for i in glist:
process(i)
Related
I have a situation where I want to do multiple things while handling an exception. Since I want to make this about the general case, I'll translate my specific case into some more general language.
When I have an exception in this piece of code, I want to:
Always perform a rollback-style operation
If it is an
application specific exception, I want to perform some logging and swallow the exception.
So I can think of two ways to solve it, both ugly:
# Method nested-try/except block
try:
try:
do_things()
except:
rollback()
raise
except SpecificException as err:
do_advanced_logging(err)
return
# Method Duplicate Code
try:
do_things()
except SpecificException as err:
rollback()
do_advanced_logging(err)
return
except:
rollback()
raise
Both will have the same behaviour.
I'm tending towards the nested try/except solution myself. While it might be slightly slower, I don't think the speed difference is relevant here - at the very least not for my specific case. Duplication of code is something I want to avoid also because my rollback() statement is slightly more involved that just a database rollback, even if it has the exact same purpose (it involves a web-API).
Is there a third option I haven't spotted that is better? Or is the duplicate code method better? Please note that the rollback() functionality is already factored out as much as possible, but still contains a function call and three arguments which includes a single hardcoded string. Since this string is unique, there's no reason to make it a named constant.
How about checking the exception instance type in code?
# Method .. No Duplicate Code
try:
do_things()
except Exception as e:
rollback()
if isinstance(e, SpecificException):
do_advanced_logging(e)
return
raise
how about putting the rollback in a finally clause? something like:
do_rollback = True
try:
do_things()
do_rollback = False
except SpecificException as err:
do_advanced_logging(err)
finally:
if do_rollback:
rollback()
an alternative is to use an else clause, which would let you do more in the non-exceptional case and not have exceptions all caught in the same place:
do_rollback = True
try:
do_things()
except SpecificException as err:
do_advanced_logging(err)
else:
record_success()
do_rollback = False
finally:
if do_rollback:
rollback()
is useful when record_success can raise a SpecificException, but you don't want to do_advanced_logging
You could write a context manager:
import random
class SpecificException(Exception):
pass
def do_things(wot=None):
print("in do_things, wot = {}".format(wot))
if wot:
raise wot("test")
def rollback():
print("rollback")
def do_advance_logging(exc_type, exc_val, traceback):
print("logging got {} ('{}')".format(exc_type, exc_val))
class rollback_on_error(object):
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, traceback):
# always rollback
rollback()
# log and swallow specific exceptions
if exc_type and issubclass(exc_type, SpecificException):
do_advance_logging(exc_type, exc_val, traceback)
return True
# propagate other exceptions
return False
def test():
try:
with rollback_on_error():
do_things(ValueError)
except Exception as e:
print("expected ValueError, got '{}'".format(type(e)))
else:
print("oops, should have caught a ValueError")
try:
with rollback_on_error():
do_things(SpecificException)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
try:
with rollback_on_error():
do_things(None)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
if __name__ == "__main__":
test()
But unless you have dozen occurrences of this pattern, I'd rather stick to the very obvious and perfectly pythonic solutions - either nested exceptions handlers or explicit typecheck (isinstance) in the except clause.
I am curious if there is a way in python to continue on within try/catch block, after you catch an exception, look at its properties, and if not relevant, then continue down the stack.
try:
# Code
except AppleError as apple_ex:
# look at 'apple_ex.error_code' error body, and if not relevant,
# continue on to next down the catch block...
# In other words, proceed to except BananaError and so on down.
except BananaError as banana_ex:
# ...
except Exception as ex:
# ...
That is not how exceptions are handled in Python. When you raise an exception in a try block, if you handle catching it in the except, it will fall inside that block, but will not continue to the next except at that same level. Observe this functional example:
try:
raise AttributeError()
except AttributeError:
raise TypeError()
except TypeError:
print("it got caught") # will not catch the TypeError raised above
So, in your try, we raise an AttributeError, we catch it, and then raise a TypeError inside catching the AttributeError.
The except TypeError will not catch that TypeError.
Based on how you are explaining your problem, you need to rethink how you are handling your exceptions and see if you can determine the handling of errors somewhere else, and raise the error there.
For example:
def some_func():
try:
thing()
except SomeException:
# analyze the exception here and raise the error you *should* raise
if apple_error_thing:
raise AppleError
elif banana_error_thing:
raise BananaError
else:
raise UnknownException
def your_func():
try:
some_func()
except AppleError as e:
print('Apple')
except BananaError as e:
print('Banana')
except UnknownException as e:
print('Unknown')
An AppleError is still an AppleError and not a BananaError, even if error_code is not relevant, so it makes no sense to fall through to BananaError.
You could instead define specific errors for your different error codes:
GRANNY_SMITH_ERROR = 1
MACINTOSH_ERROR = 2
class AppleError(Exception):
def __init__(self, error_code, *args):
super(AppleError, self).__init__(*args)
self.error_code = error_code
class GrannySmithError(AppleError):
def __init__(self, *args):
super(GrannySmithError, self).__init__(GRANNY_SMITH_ERROR, *args)
class MacintoshError(AppleError):
def __init__(self, *args):
super(MacintoshError, self).__init__(MACINTOSH_ERROR, *args)
Then you can try to match the specific error:
try: raise MacintoshError()
except MacintoshError as exc: print("mac")
except GrannySmithError as exc: print("granny smith")
If you do not care to distinguish between different types of apple errors, you can still trap all apple errors:
try: raise MacintoshError()
except AppleError as exc: print("generic apple")
You can combine these, for example, only doing special processing for GrannySmith, not for Macintosh:
try: raise MacintoshError()
except GrannySmithError as exc: print("granny smith")
except AppleError as exc: print("generic apple")
The important thing is to list the errors from most specific to least specific. If you test for AppleError before GrannySmithError, then it will never enter the GrannySmith block.
No, that isn't possible. After the exception is handled by the inner except it doesn't have the ability to get handled by the outer except:
From the docs on the try statement:
When the end of this block is reached, execution continues normally after the entire try statement. (This means that if two nested handlers exist for the same exception, and the exception occurs in the try clause of the inner handler, the outer handler will not handle the exception.)
In short your only solution might be to have another handler at an outer level and re-raise the exception in the inner handler, that is:
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as e:
print("caught")
raise ZeroDivisionError
except ZeroDivisionError as f:
print("caught")
Now the nested except raises an exception which is consequently caught by a similar handler.
I have a try/except where I repeat the except portion frequently in my code. This led me to believe that it would be better to separate the except portion into a function.
Below is my use case:
try:
...
except api.error.ReadError as e:
...
except api.error.APIConnectionError as e:
...
except Exception as e:
...
How would I go about separating this logic into a function so I can do something as simple as:
try:
...
except:
my_error_handling_function(e)
Just define the function:
def my_error_handling(e):
#Do stuff
...and pass in the exception object as the parameter:
try:
#some code
except Exception as e:
my_error_handling(e)
Using just a generic Exception type will allow you to have a single except clause and handle and test for different error types within your handling function.
In order to check for the name of the caught exception, you can get it by doing:
type(e).__name__
Which will print the name, such as ValueError, IOError, etc.
I would suggest refactoring your code so the try/except block is only present in a single location.
For instance, an API class with a send() method, seems like a reasonable candidate for containing the error handling logic you have described in your question.
Define your function:
def my_error_handling(e):
#Handle exception
And do what you're proposing:
try:
...
except Exception as e:
my_error_handling_function(e)
You can handle logic by getting the type of the exception 'e' within your function. See: python: How do I know what type of exception occurred?
If you don't like try-catch statement, you can use exception-decouple package and decorator.
from exception_decouple import redirect_exceptions
def my_error_handling(arg, e):
#Do stuff
#redirect_exceptions(my_error_handling, api.error.ReadError, api.error.APIConnectionError)
def foo(arg):
...
At below, I have a try-except block that I want to refactor it. As you see, it is not pythonic and not-maintainable.
try:
try:
foo()
except xError:
doSth()
raise
except:
exc_type = sys.exc_info()[0]
if exc_type == FooError:
doBlaBla()
else:
doFlaFla()
blablabla() # whatever the exceptions is, run this code block
foobar()
zoo()
...
I change that block as the below code;
try:
try:
foo()
except xError:
doSth()
raise
except FooError:
doBlaBla()
raise
except:
doFlaFla()
raise
blablabla() # This is where the problem happens for me.
foobar()
zoo()
...
As you can see, I need a except-finally like operation. It will not run when no exception raises but any exceptions. What do you advice ? How should I change this clode block ?
Why you've used innder try exempt block?
In my opinion, it is a good practice to put finally except statement, that will catch unexpected error.
So I suggest:
try:
fn()
except Exception1:
do stuff you want
except Exception 2:
do another stuff
except Exception as e:
# Here you will catch an unexpected error, and you can log it
raise UnknownError -> do stuff you want
Why not something like:
def others():
"""Other stuff to do when non-xError occurs."""
blablabla()
foobar()
zoo()
and then factor into a single try:
try:
foo()
except xError:
doSth()
raise
except FooError:
doBlaBla()
others()
except Exception:
doFlaFla()
others()
A bare except is usually a bad idea.
You can wrap your exception-code in a try-finally block, for example:
try:
try:
foo()
except xError:
doSth()
raise
# catch Exception to avoid catching system errors
except Exception as e:
try:
if isinstance(e, FooError):
doBlaBla()
else:
doFlaFla()
raise
finally:
# this will always run regardless of what you raise
Another approach could be something like this:
e = None
try:
try:
foo()
except xError:
doSth()
raise
except FooError as e:
doBlaBla()
except Exception as e:
doFlaFla()
raise
finally:
# this will always run regardless of what you raise
if e:
# exception took place...
I often find myself wanting to do something like this, I have something wrapped in try excepts like this
item= get_item()
try:
do_work(item)
except SomeError as err:
if err.code == 123:
do_something(item)
else:
# Actually I don't want to do something with this error code... I want to handle in 'except'
except:
put_back(item)
raise
Is there a way to raise into the except block below from the else? (a continue would be nice) I end up doing something like the following which isn't as clean
item= get_item()
try:
try:
do_work(item)
except SomeError as err:
if err.code == 123:
do_something(item)
else:
raise
except:
put_back(item)
raise
Is there anyway to do that?
If you are using a recent enough python version (2.5 and up), you should switch to using a context manager instead:
class WorkItemContextManager(object):
def __enter__(self):
self.item = get_item()
return self.item
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not None:
if exc_type is SomeError and exc_value.code == 123:
do_something(self.item)
return True # Exception handled
put_back(self.item)
Then:
with WorkItemContextManager() as item:
do_work(item)
The __exit__ method can return True if an exception has been handled; returning None will instead re-raise any exceptions raised in the with block.
If not, you are looking for a finally block instead:
item = get_item()
try:
do_work(item)
item = None
except SomeError as err:
if err.code == 123:
do_something(item)
item = None
finally:
if item is not None:
put_back(item)
The finally suite is guaranteed to be executed when the try: suite completes, or an exception has occurred. By setting item to None you basically tell the finally suite everything completed just fine, no need to put it back.
The finally handler takes over from your blanket except handler. If there has been an exception in do_work, item will not be set to None. If the SomeError handler doesn't catch the exception, or err.code is not 123, item will also not be set to None, and thus the put_back(item) method is executed.
My suggestion would be to create a function (or series of functions) that wraps the method throwing errors which you'd like to control. Something like...
def wrapper(arg):
try:
do_work(arg)
except SomeError as e:
if e.code == 123:
do_something(item)
# Other possible cleanup code
else:
raise
...then, when you want to call it...
try:
wrapper(arg)
except SomeError as e:
put_back(arg)
Context managers are excellent, but for some simple cases where you won't be reusing the logic anywhere else, they can be a bit heavy.
Instead of trying to have multiple except blocks, you can just test the exception inside a single except block:
item= get_item()
try:
do_work(item)
except Exception as err:
if isinstance(err, SomeError) and err.code == 123:
do_something(item)
else:
put_back(item)
raise
Note that this is pretty much what a context manager's __exit__ method ends up looking like, anyway.
Be wary that code that really belongs in finally doesn't end up here.
It's good to keep in mind what try-except flow is for, and one of their advantages is that they remove the need for status variables and status checks like
if not foo:
# do something
Also, an Exception class should represent a specific kind of error. If you need to make further decisions about the kind of error in an except block, it's a good sign that the class isn't specific enough to represent the program state. Your best bet is to subclass SomeError and only catch the subclass in the first except. Then other instances of SomeError will fall through to the second except block.