I had a function that returned a random member of several groups in order of preference. It went something like this:
def get_random_foo_or_bar():
"I'd rather have a foo than a bar."
if there_are_foos():
return get_random_foo()
if there_are_bars():
return get_random_bar()
raise IndexError, "No foos, no bars"
However, the first thing get_random_foo does is verify there are foos and raise an IndexError if not, so there_are_foos is redundant. Moreover, a database is involved and using separate functions creates a concurrency issue. Accordingly, I rewrote it something like this:
def get_random_foo_or_bar():
"Still prefer foos."
try:
return get_random_foo()
except IndexError:
pass
try:
return get_random_bar()
except IndexError:
pass
raise IndexError, "No foos, no bars"
But I find this much less readable, and as I've never had reason to use pass before it feels instictively wrong.
Is there a neater efficient pattern, or should I learn to accept pass?
Note: I'd like to avoid any nesting since other types may be added later.
Edit
Thanks everyone who said that pass is fine - that's reassuring!
Also thanks to those who suggested replacing the exception with a return value of None. I can see how this is a useful pattern, but I would argue it's semantically wrong in this situation: the functions have been asked to perform an impossible task so they should raise an exception. I prefer to follow the behaviour of the random module (eg. random.choice([])).
That is exactly how I would write it. It's simple and it makes sense. I see no problem with the pass statements.
If you want to reduce the repetition and you anticipate adding future types, you could roll this up into a loop. Then you could change the pass to a functionally-equivalent continue statement, if that's more pleasing to your eyes:
for getter in (get_random_foo, get_random_bar):
try:
return getter()
except IndexError:
continue # Ignore the exception and try the next type.
raise IndexError, "No foos, no bars"
Using try, except, pass is acceptable, but there is a cleaner way to write this using contextlib.suppress() available for python 3.4+.
from contextlib import suppress
def get_random_foo_or_bar():
"Still prefer foos."
with suppress(IndexError):
return get_random_foo()
with suppress(IndexError):
return get_random_bar()
raise IndexError("No foos, no bars")
pass is fine (there's a reason it's in the language!-), but a pass-free alternative just takes a bit more nesting:
try: return get_random_foo()
except IndexError:
try: return get_random_bar()
except IndexError:
raise IndexError "no foos, no bars"
Python's Zen (import this from the interactive interpreter prompt) says "flat is better than nested", but nesting is also in the language, for you to use when you decide (presumably being enlightened) that you can do better than that wise koan!-) (As in, "if you meet the Buddha on the road"...).
It looks a little weird to me that get_random_foo() is raising an IndexError when it doesn't take an index as a param (but it might make more sense in context). Why not have get_random_foo(), or a wrapper, catch the error and return None instead?
def get_random_foo_wrapper():
try:
return get_random_foo()
except IndexError:
return None
def get_random_foo_or_bar():
"I'd rather have a foo than a bar."
return get_random_foo_wrapper() or get_random_bar_wrapper() or None
Edit: I should mention that if foo & bar are objects that may evaluate to False (0 or '' say) then the or comparison will skip over them which is BAD
If it's just those two, could always just...
try:
return get_random_foo()
except IndexError:
try:
return get_random_bar()
except IndexError:
raise IndexError, "No foos, no bars"
If it's more than two, what you have written seems perfectly acceptable.
Building on Peter Gibson's suggestion, you could create a generic wrapper function that swallows a given exception. And then you could write a function that returns such a generic wrapper for a provided exception. Or heck, for a provided list of exceptions.
def maketrap(*exceptions):
def trap(func, *args, **kwargs):
try:
return func(*args, **kwargs)
except exceptions:
return None
return trap
def get_random_foo_or_bar():
mytrap = maketrap(IndexError)
return mytrap(get_random_foo) or mytrap(get_random_bar) or None
If you don't really need the exception message (just the type):
def get_random_foo_or_bar():
try:
return get_random_foo()
except IndexError:
return get_random_bar() # if failing at this point,
# the whole function will raise IndexError
Is it necessary that get_random_foo/bar() raise an IndexError if it's unable to succeed?
If they returned None, you could do:
def get_random_foo_or_bar():
return get_random_foo() or get_random_bar()
Related
I have this:
try:
if session.var:
otherVar = session.var
else:
util = db.utility[1]
otherVar = session.var = util.freshOutTheBank
except AttributeError:
util = db.utility[1]
otherVar = session.var = util.freshOutTheBank
...do stuff with otherVar
The case is that the session.var might not exist or could be None. This code is also run more than once by a user during a session.
How do I avoid repeating the code. I basically want to do an 'except and else' or am I looking at this incorrectly?
Assuming this is a web2py session object, note that it is an instance of gluon.Storage, which is like a dictionary with two exceptions: (1) keys can be accessed like properties, and (2) accessing a non-existent key/property returns None rather than raising an exception. So, you can simply do something like:
otherVar = session.var = session.var if session.var else db.utility[1].freshOutTheBank
Note, if you want to distinguish between non-existent keys and keys that have an explicit value of None, you cannot use hasattr(session, 'var'), as that will return True even if there is no 'var' key. Instead, you can check session.has_key('var'), which will return False if there is no 'var' key.
You can avoid using session.var if it doesn't exist by checking for it first, using hasattr. This avoids the need for the try/except block all together.
if hasattr(session, 'var') and session.var is not None:
...
else:
...
An alternative might be to have the else in your original code just raise an exception to get to the except block, but it's sort of ugly:
try:
if session.var:
...
else:
raise AttributeError
except AttributeError:
...
In this situation, I think the "Look Before you Leap" style of programming (using hasattr) is nicer than the usually more Pythonic style of "Easier to Ask Forgiveness than Permission" (which uses exceptions as part of flow control). But either one can work.
If your code was compartmentalized into smaller functions, it might be even easier to deal with the issue. For instance, if you wrote a get_session_var function, it could return from the successful case (inside the try and if blocks), and the two error cases could be resolved later:
def get_session_var(session):
try:
if session.var:
return session.var
except AttributeError:
pass
util = db.utility[1]
session.var = util.freshOutTheBank
return session.var
I have a class function in Python that either returns a success or a failure, but in case of a failure I want it to send a specific error string back. I have 3 approaches in mind:
Pass in an variable error_msg to the function originally set to None and in case of an error, it gets set to the error string. eg:
if !(foo(self, input, error_msg)):
print "no error"
else:
print error_msg
Return a tuple containing a bool and error_msg from the function.
I raise an exception in case of an error and catch it in the calling code. But since I don't see exceptions being used often in the codebase I am working on, so was not too sure about taking this approach.
What is the Pythonic way of doing this?
Create your own exception and raise that instead:
class MyValidationError(Exception):
pass
def my_function():
if not foo():
raise MyValidationError("Error message")
return 4
You can then call your function as:
try:
result = my_function()
except MyValidationError as exception:
# handle exception here and get error message
print exception.message
This style is called EAFP ("Easier to ask for forgiveness than permission") which means that you write the code as normal, raise exceptions when something goes wrong and handle that later:
This common Python
coding style assumes the existence of valid keys or attributes and
catches exceptions if the assumption proves false. This clean and fast
style is characterized by the presence of many try and except
statements. The technique contrasts with the LBYL style common to many
other languages such as C.
Raise an error:
if foo(self, input, error_msg):
raise SomethingError("You broke it")
And handle it:
try:
something()
except SomethingError as e:
print str(e)
It's the Pythonic approach and the most readable.
Returning a tuple like (12, None) may seem like a good solution, but it's hard to keep track of what each method returns if you're not consistent. Returning two different data types is even worse, as it will probably break code that assumes a constant data type.
What is the pythonic way for checking a user provided vector index?
def get_value(vector, index):
try:
return vector[index]
except IndexError:
raise ValueError('bad index')
or
def get_value(vector, index):
if -1 < index < len(vector):
return vector[index]
else:
raise ValueError('bad index')
Trying first is more Pythonic. From the glossary:
EAFP
Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.
Although I wouldn't even put that around a try...except since you're just throwing an IndexError, which it already does.
I think its reasonable to just let the IndexError propagate up to the user. The user is passing you an index into something. If its out of range IndexError is reasonably thrown from your interface. So I would just do:
def get_value(vector, index):
""" gets value... throws IndexError() if index out of range"""
return vector[index]
This also lets you use valid negative indices in your interface.
One way is to let the IndexError be thrown -- it's an invalid request. Another is to do vector.get(index) which gives you a None type on failure (same as try: return vector[index] except: return None). It really depends on what you intend to do upon failure. If you want to request the user try again, then .get() or try except is probably best. Otherwise raising the exception up to the top level indicates the program can't handle the request.
Note that the "Pythonic" way would normally be a try except -- but I reiterate that it's much more dependent on your use case.
I got many lines in a row which may throw an exception, but no matter what, it should still continue the next line. How to do this without individually try catching every single statement that may throw an exception?
try:
this_may_cause_an_exception()
but_I_still_wanna_run_this()
and_this()
and_also_this()
except Exception, e:
logging.exception('An error maybe occured in one of first occuring functions causing the others not to be executed. Locals: {locals}'.format(locals=locals()))
Let's see above code, all functions may throw exceptions, but it should still execute the next functions no matter if it threw an exception or not. Is there a nice way of doing that?
I dont wanna do this:
try:
this_may_cause_an_exception()
except:
pass
try:
but_I_still_wanna_run_this()
except:
pass
try:
and_this()
except:
pass
try:
and_also_this()
except:
pass
I think code should still continue to run after an exception only if the exception is critical (The computer will burn or the whole system will get messed up, it should stop the whole program, but for many small things also exceptions are thrown such as connection failed etc.)
I normally don't have any problems with exception handling, but in this case I'm using a 3rd party library which easily throws exceptions for small things.
After looking at m4spy's answer, i thought wouldn't it be possible, to have a decorator which will let every line in the function execute even if one of them raises an exception.
Something like this would be cool:
def silent_log_exceptions(func):
#wraps(func)
def _wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception:
logging.exception('...')
some_special_python_keyword # which causes it to continue executing the next line
return _wrapper
Or something like this:
def silent_log_exceptions(func):
#wraps(func)
def _wrapper(*args, **kwargs):
for line in func(*args, **kwargs):
try:
exec line
except Exception:
logging.exception('...')
return _wrapper
#silent_log_exceptions
def save_tweets():
a = requests.get('http://twitter.com)
x = parse(a)
bla = x * x
for func in [this_may_cause_an_exception,
but_I_still_wanna_run_this,
and_this,
and_also_this]:
try:
func()
except:
pass
There are two things to notice here:
All actions you want to perform have to represented by callables with the same signature (in the example, callables that take no arguments). If they aren't already, wrap them in small functions, lambda expressions, callable classes, etc.
Bare except clauses are a bad idea, but you probably already knew that.
An alternative approach, that is more flexible, is to use a higher-order function like
def logging_exceptions(f, *args, **kwargs):
try:
f(*args, **kwargs)
except Exception as e:
print("Houston, we have a problem: {0}".format(e))
I ran into something similar, and asked a question on SO here. The accepted answer handles logging, and watching for only a specific exception. I ended up with a modified version:
class Suppressor:
def __init__(self, exception_type, l=None):
self._exception_type = exception_type
self.logger = logging.getLogger('Suppressor')
if l:
self.l = l
else:
self.l = {}
def __call__(self, expression):
try:
exec expression in self.l
except self._exception_type as e:
self.logger.debug('Suppressor: suppressed exception %s with content \'%s\'' % (type(self._exception_type), e))
Usable like so:
s = Suppressor(yourError, locals())
s(cmdString)
So you could set up a list of commands and use map with the suppressor to run across all of them.
You can handle such a task with a decorator:
import logging
from functools import wraps
def log_ex(func):
#wraps(func)
def _wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception:
logging.exception('...')
return _wrapper
#log_ex
def this_may_cause_an_exception():
print 'this_may_cause_an_exception'
raise RuntimeError()
#log_ex
def but_i_wanna_run_this():
print 'but_i_wanna_run_this'
def test():
this_may_cause_an_exception()
but_i_wanna_run_this()
Calling the test function will look like (which will show that both functions were executed):
>>> test()
this_may_cause_an_exception
ERROR:root:...
Traceback (most recent call last):
File "<stdin>", line 5, in _wrapper
File "<stdin>", line 4, in my_func
RuntimeError
but_i_wanna_run_this
Sometimes, when language misses to support your elegant way of expressing an idea because language development literally failed the last decades, you can only rely on the fact that Python is still a dynamical language which supports the exec statement, which makes the following possible:
code="""
for i in range(Square_Size):
Square[i,i] #= 1
Square[i+1,i] #= 2
#dowhatever()
"""
This new operator makes code more pythonic and elegant since you don't need to specify additional if-statemens that guarantee that the index stays in bound or the function does succeed which is totally irrelevant to what we want to express (it just shouldn't stop) here (note: while safe indexing would be possible by creating a class based on the list class, this operator works whenever there should be a try catch) , in Lisp it would be easy to define it in a Lispy way, but it seams to be impossible to define it in an elegant way in Python, but still, here is the little preparser which will make it possible:
exec "\n".join([o+"try: "+z.replace("#","")+"\n"+o+"except: pass" if "#" in z else z for z in code.split("\n") for o in ["".join([h for h in z if h==" "])]]) #new <- hackish operator which wraps try catch into line
The result, assuming that Square was 4x4 and contained only zeros:
[1 0 0 0]
[2 1 0 0]
[0 2 1 0]
[0 0 2 1]
Relevant: The Sage / Sagemath CAS uses a preparse-function
which transforms code before it reaches the Python interpreter.
A monkey-patch for that function would be:
def new_preparse(code,*args, **kwargs):
code="\n".join([o+"try: "+z.replace("#","")+"\n"+o+"except: pass" if "#" in z else z for z in code.split("\n") for o in ["".join([h for h in z if h==" "])]])
return preparse(code)
sage.misc.preparser.preparse=new_preparse
Apart from the answers provided, I think its worth to note that one-line try-except statements have been proposed - see the related PEP 463 with the unfortunate rejection notice:
""" I want to reject this PEP. I think the proposed syntax is acceptable given the
desired semantics, although it's still a bit jarring. It's probably no worse than the
colon used with lambda (which echoes the colon used in a def just like the colon here
echoes the one in a try/except) and definitely better than the alternatives listed.
But the thing I can't get behind are the motivation and rationale. I don't think that
e.g. dict.get() would be unnecessary once we have except expressions, and I disagree
with the position that EAFP is better than LBYL, or "generally recommended" by Python.
(Where do you get that? From the same sources that are so obsessed with DRY they'd rather
introduce a higher-order-function than repeat one line of code? :-)
This is probably the most you can get out of me as far as a pronouncement. Given that
the language summit is coming up I'd be happy to dive deeper in my reasons for rejecting
it there (if there's demand).
I do think that (apart from never explaining those dreadful acronyms :-) this was a
well-written and well-researched PEP, and I think you've done a great job moderating the
discussion, collecting objections, reviewing alternatives, and everything else that is
required to turn a heated debate into a PEP. Well done Chris (and everyone who
helped), and good luck with your next PEP! """
try:
this_may_cause_an_exception()
except:
logging.exception('An error occured')
finally:
but_I_still_wanna_run_this()
and_this()
and_also_this()
You can use the finally block of exception handling. It is actually meant for cleanup code though.
EDIT:
I see you said all of the functions can throw exceptions, in which case larsmans' answer is about the cleanest I can think of to catch exception for each function call.
I have some test cases. The test cases rely on data which takes time to compute. To speed up testing, I've cached the data so that it doesn't have to be recomputed.
I now have foo(), which looks at the cached data. I can't tell ahead of time what it will look at, as that depends a lot on the test case.
If a test case fails cause it doesn't find the right cached data, I don't want it to fail - I want it to compute the data and then try again. I also don't know what exception in particular it will throw cause of missing data.
My code right now looks like this:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
try:
foo()
except:
if not dataComputed:
calculateData()
dataComputed = True
try:
foo()
except:
#error handling code
else:
#the same error handling code
What's the best way to re-structure this code?
I disagree with the key suggestion in the existing answers, which basically boils down to treating exceptions in Python as you would in, say, C++ or Java -- that's NOT the preferred style in Python, where often the good old idea that "it's better to ask forgiveness than permission" (attempt an operation and deal with the exception, if any, rather than obscuring your code's main flow and incurring overhead by thorough preliminary checks). I do agree with Gabriel that a bare except is hardly ever a good idea (unless all it does is some form of logging followed by a raise to let the exception propagate). So, say you have a tuple with all the exception types that you do expect and want to handle the same way, say:
expected_exceptions = KeyError, AttributeError, TypeError
and always use except expected_exceptions: rather than bare except:.
So, with that out of the way, one slightly less-repetitious approach to your needs is:
try:
foo1()
except expected_exceptions:
try:
if condition:
foobetter()
else:
raise
except expected_exceptions:
handleError()
A different approach is to use an auxiliary function to wrap the try/except logic:
def may_raise(expected_exceptions, somefunction, *a, **k):
try:
return False, somefunction(*a, **k)
except expected_exceptions:
return True, None
Such a helper may often come in useful in several different situations, so it's pretty common to have something like this somewhere in a project's "utilities" modules. Now, for your case (no arguments, no results) you could use:
failed, _ = may_raise(expected_exceptions, foo1)
if failed and condition:
failed, _ = may_raise(expected_exceptions, foobetter)
if failed:
handleError()
which I would argue is more linear and therefore simpler. The only issue with this general approach is that an auxiliary function such as may_raise does not FORCE you to deal in some way or other with exceptions, so you might just forget to do so (just like the use of return codes, instead of exceptions, to indicate errors, is prone to those return values mistakenly being ignored); so, use it sparingly...!-)
Using blanket exceptions isn't usually a great idea. What kind of Exception are you expecting there? Is it a KeyError, AttributeError, TypeError...
Once you've identified what type of error you're looking for you can use something like hasattr() or the in operator or many other things that will test for your condition before you have to deal with exceptions.
That way you can clean up your logic flow and save your exception handling for things that are really broken!
Sometimes there's no nice way to express a flow, it's just complicated. But here's a way to call foo() in only one place, and have the error handling in only one place:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
while True:
try:
foo()
break
except:
if not dataComputed:
calculateData()
dataComputed = True
continue
else:
#the error handling code
break
You may not like the loop, YMMV...
Or:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
done = False
while !done:
try:
foo()
done = True
except:
if not dataComputed:
calculateData()
dataComputed = True
continue
else:
#the error handling code
done = True
I like the alternative approach proposed by Alex Martelli.
What do you think about using a list of functions as argument of the may_raise. The functions would be executed until one succeed!
Here is the code
def foo(x):
raise Exception("Arrrgh!")
return 0
def foobetter(x):
print "Hello", x
return 1
def try_many(functions, expected_exceptions, *a, **k):
ret = None
for f in functions:
try:
ret = f(*a, **k)
except expected_exceptions, e:
print e
else:
break
return ret
print try_many((foo, foobetter), Exception, "World")
result is
Arrrgh!
Hello World
1
Is there a way to tell if you want to do foobetter() before making the call? If you get an exception it should be because something unexpected (exceptional!) happened. Don't use exceptions for flow control.