I got many lines in a row which may throw an exception, but no matter what, it should still continue the next line. How to do this without individually try catching every single statement that may throw an exception?
try:
this_may_cause_an_exception()
but_I_still_wanna_run_this()
and_this()
and_also_this()
except Exception, e:
logging.exception('An error maybe occured in one of first occuring functions causing the others not to be executed. Locals: {locals}'.format(locals=locals()))
Let's see above code, all functions may throw exceptions, but it should still execute the next functions no matter if it threw an exception or not. Is there a nice way of doing that?
I dont wanna do this:
try:
this_may_cause_an_exception()
except:
pass
try:
but_I_still_wanna_run_this()
except:
pass
try:
and_this()
except:
pass
try:
and_also_this()
except:
pass
I think code should still continue to run after an exception only if the exception is critical (The computer will burn or the whole system will get messed up, it should stop the whole program, but for many small things also exceptions are thrown such as connection failed etc.)
I normally don't have any problems with exception handling, but in this case I'm using a 3rd party library which easily throws exceptions for small things.
After looking at m4spy's answer, i thought wouldn't it be possible, to have a decorator which will let every line in the function execute even if one of them raises an exception.
Something like this would be cool:
def silent_log_exceptions(func):
#wraps(func)
def _wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception:
logging.exception('...')
some_special_python_keyword # which causes it to continue executing the next line
return _wrapper
Or something like this:
def silent_log_exceptions(func):
#wraps(func)
def _wrapper(*args, **kwargs):
for line in func(*args, **kwargs):
try:
exec line
except Exception:
logging.exception('...')
return _wrapper
#silent_log_exceptions
def save_tweets():
a = requests.get('http://twitter.com)
x = parse(a)
bla = x * x
for func in [this_may_cause_an_exception,
but_I_still_wanna_run_this,
and_this,
and_also_this]:
try:
func()
except:
pass
There are two things to notice here:
All actions you want to perform have to represented by callables with the same signature (in the example, callables that take no arguments). If they aren't already, wrap them in small functions, lambda expressions, callable classes, etc.
Bare except clauses are a bad idea, but you probably already knew that.
An alternative approach, that is more flexible, is to use a higher-order function like
def logging_exceptions(f, *args, **kwargs):
try:
f(*args, **kwargs)
except Exception as e:
print("Houston, we have a problem: {0}".format(e))
I ran into something similar, and asked a question on SO here. The accepted answer handles logging, and watching for only a specific exception. I ended up with a modified version:
class Suppressor:
def __init__(self, exception_type, l=None):
self._exception_type = exception_type
self.logger = logging.getLogger('Suppressor')
if l:
self.l = l
else:
self.l = {}
def __call__(self, expression):
try:
exec expression in self.l
except self._exception_type as e:
self.logger.debug('Suppressor: suppressed exception %s with content \'%s\'' % (type(self._exception_type), e))
Usable like so:
s = Suppressor(yourError, locals())
s(cmdString)
So you could set up a list of commands and use map with the suppressor to run across all of them.
You can handle such a task with a decorator:
import logging
from functools import wraps
def log_ex(func):
#wraps(func)
def _wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception:
logging.exception('...')
return _wrapper
#log_ex
def this_may_cause_an_exception():
print 'this_may_cause_an_exception'
raise RuntimeError()
#log_ex
def but_i_wanna_run_this():
print 'but_i_wanna_run_this'
def test():
this_may_cause_an_exception()
but_i_wanna_run_this()
Calling the test function will look like (which will show that both functions were executed):
>>> test()
this_may_cause_an_exception
ERROR:root:...
Traceback (most recent call last):
File "<stdin>", line 5, in _wrapper
File "<stdin>", line 4, in my_func
RuntimeError
but_i_wanna_run_this
Sometimes, when language misses to support your elegant way of expressing an idea because language development literally failed the last decades, you can only rely on the fact that Python is still a dynamical language which supports the exec statement, which makes the following possible:
code="""
for i in range(Square_Size):
Square[i,i] #= 1
Square[i+1,i] #= 2
#dowhatever()
"""
This new operator makes code more pythonic and elegant since you don't need to specify additional if-statemens that guarantee that the index stays in bound or the function does succeed which is totally irrelevant to what we want to express (it just shouldn't stop) here (note: while safe indexing would be possible by creating a class based on the list class, this operator works whenever there should be a try catch) , in Lisp it would be easy to define it in a Lispy way, but it seams to be impossible to define it in an elegant way in Python, but still, here is the little preparser which will make it possible:
exec "\n".join([o+"try: "+z.replace("#","")+"\n"+o+"except: pass" if "#" in z else z for z in code.split("\n") for o in ["".join([h for h in z if h==" "])]]) #new <- hackish operator which wraps try catch into line
The result, assuming that Square was 4x4 and contained only zeros:
[1 0 0 0]
[2 1 0 0]
[0 2 1 0]
[0 0 2 1]
Relevant: The Sage / Sagemath CAS uses a preparse-function
which transforms code before it reaches the Python interpreter.
A monkey-patch for that function would be:
def new_preparse(code,*args, **kwargs):
code="\n".join([o+"try: "+z.replace("#","")+"\n"+o+"except: pass" if "#" in z else z for z in code.split("\n") for o in ["".join([h for h in z if h==" "])]])
return preparse(code)
sage.misc.preparser.preparse=new_preparse
Apart from the answers provided, I think its worth to note that one-line try-except statements have been proposed - see the related PEP 463 with the unfortunate rejection notice:
""" I want to reject this PEP. I think the proposed syntax is acceptable given the
desired semantics, although it's still a bit jarring. It's probably no worse than the
colon used with lambda (which echoes the colon used in a def just like the colon here
echoes the one in a try/except) and definitely better than the alternatives listed.
But the thing I can't get behind are the motivation and rationale. I don't think that
e.g. dict.get() would be unnecessary once we have except expressions, and I disagree
with the position that EAFP is better than LBYL, or "generally recommended" by Python.
(Where do you get that? From the same sources that are so obsessed with DRY they'd rather
introduce a higher-order-function than repeat one line of code? :-)
This is probably the most you can get out of me as far as a pronouncement. Given that
the language summit is coming up I'd be happy to dive deeper in my reasons for rejecting
it there (if there's demand).
I do think that (apart from never explaining those dreadful acronyms :-) this was a
well-written and well-researched PEP, and I think you've done a great job moderating the
discussion, collecting objections, reviewing alternatives, and everything else that is
required to turn a heated debate into a PEP. Well done Chris (and everyone who
helped), and good luck with your next PEP! """
try:
this_may_cause_an_exception()
except:
logging.exception('An error occured')
finally:
but_I_still_wanna_run_this()
and_this()
and_also_this()
You can use the finally block of exception handling. It is actually meant for cleanup code though.
EDIT:
I see you said all of the functions can throw exceptions, in which case larsmans' answer is about the cleanest I can think of to catch exception for each function call.
Related
I want to implement a way to repeat a section of code as many times as it's needed using a context manager only, because of its pretty syntax. Like this:
with try_until_success(attempts=10):
command1()
command2()
command3()
The commands must be executed once if no errors happen. And they should be executed again if an error occurred, until 10 attempts has passed, if so the error must be raised. For example, it can be useful to reconnect to a data base. The syntax I represented is literal, I do not want to modify it (so do not suggest me to replace it with a kind of for of while statements).
Is there a way to implement try_until_success in Python to do what I want?
What I tried is:
from contextlib import contextmanager
#contextmanager
def try_until_success(attempts=None):
counter = 0
while True:
try:
yield
except Exception as exc:
pass
else:
break
counter += 1
if attempts is not None and counter >= attempts:
raise exc
And this gives me the error:
RuntimeError: generator didn't stop after throw()
I know, there are many ways to reach what I need using a loop instead of with-statement or with the help of a decorator. But both have syntax disadvantages. For example, in case of a loop I have to insert try-except block, and in case of a decorator I have to define a new function.
I have already looked at the questions:
How do I make a contextmanager with a loop inside?
Conditionally skipping the body of Python With statement
They did not help in my question.
The problem is that the body of the with statement does not run within the call to try_until_success. That function returns an object with a __enter__ method; that __enter__ method calls and returns, then the body of the with statement is executed. There is no provision for wrapping the body in any kind of loop that would allow it to be repeated once the end of the with statement is reached.
This goes against how context managers were designed to work, you'd likely have to resort to non-standard tricks like patching the bytecode to do this.
See the official docs on the with statement and the original PEP 343 for how they are expanded. It might help you understand why this isn't going to be officially supported, and maybe why other commenters are generally saying this is a bad thing to try and do.
As an example of something that might work, maybe try:
class try_until_success:
def __init__(self, attempts):
self.attempts = attempts
self.attempt = 0
self.done = False
self.failures = []
def __iter__(self):
while not self.done and self.attempt < self.attempts:
i = self.attempt
yield self
assert i != self.attempt, "attempt not attempted"
if self.done:
return
if self.failures:
raise Exception("failures occurred", self.failures)
def __enter__(self):
self.attempt += 1
def __exit__(self, _ext, exc, _tb):
if exc:
self.failures.append(exc)
return True
self.done = True
for attempt in try_until_success(attempts=10):
with attempt:
command1()
command2()
command3()
you'd probably want to separate out the context manager from the iterator (to help prevent incorrect usage) but it sort of does something similar to what you were after
Is there a way to implement try_until_success in Python to do what I
want?
Yes. You don't need to make it a context manager. Just make it a function accepting a function:
def try_until_success(command, attempts=1):
for _ in range(attempts):
try:
return command()
except Exception as exc:
err = exc
raise err
And then the syntax is still pretty clear, no for or while statements - not even with:
attempts = 10
try_until_success(command1, attempts)
try_until_success(command2, attempts)
try_until_success(command3, attempts)
In my Python code, I'm using the PyQt5 module to display a GUI. Sometimes, I encounter a Runtime Error if I delete an element then attempt to use a function on the element's instance. This error is only displayed in the console, and it does not actually interfere with the GUI or terminate it.
Regardless, I would like to remove it (the Runtime Errors). My first thought was to use a try/except block on the code, and except the Runtime Error that I was talking about. The problem with this, is that if I encase my whole code with the try/except block, then if the error is caught, it will skip over to the end of my program and terminate:
try:
# If any errors occur here...
<code>
except RuntimeError:
# The GUI will stop entirely, and the Python interpreter will skip to this line
pass
Another solution to my problem is to encase any instance which could throw a RuntimeError with a try/catch block, like so:
try:
# If any errors occur here...
<code that may print an error>
except RuntimeError:
# Python wont display the error
pass
However given the mass amount of times I would need to do this in my code, I was wondering if there was a more efficient way of fixing this problem.
As per my comment, I would definitely go with the catch in the specific line calls that might throw the runtime error. This avoids you accidentally suppressing another error you were not anticipating.
Depending on what the calls are that might give the runtime error, I would prefer a decorator pattern to hide the try-except logic. Something like:
from functools import wraps
def catch_runtime_error(func):
#wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except RuntimeError:
pass # or whatever handle you fancy
return wrapper
which you would then use like:
#catch_runtime_error
def the_function_that_raises(...):
# whatever the body is
the_function_that_raises(...)
Alternatively you can use it more directly in your code:
def the_function_that_raises(...):
# whatever the body is
catch_runtime_error(the_function_that_raises)(...)
You can use customs decorators to to this (I don't know your python level but it's not the most beginner friendly thing). For example, this code here do not raise any error:
from functools import wraps
def IgnoreError(f):
#wraps(f)
def wrapper():
try:
f()
except ZeroDivisionError:
pass
return wrapper
#IgnoreError
def func1():
x = 5/0
func1()
In your case, you will have to define this function:
def IgnoreError(f):
def wrapper(*args, **kwargs):
try:
f(*args, **kwargs)
except RuntimeError:
pass
return wrapper
and then anytime you create a function that may raise the RuntimeError, just put the decorator #IgnoreError before your definition like this:
#IgnoreError
def func():
<your code here>
(if you want here's a video from TechWithTim explaining the decorators)
Lets say I have a function myFunc defined as
def myFunc(value):
return value if isinstance(value, int) else None
Now wherever in my project I use myFunc the enclosing funciton should return automatically if the value returned from myFunc is None and should continue if some integer value is returned
For example:
def dumbFunc():
# some code
# goes here..
result = myFunc('foo')
# some code
# goes here..
This funciton should automatically behave like..
def dumbFunc():
# some code
# goes here..
result = myFunc('foo')
if not result:
return
# some code
# goes here..
PS - I don't know whether this thing even possible or not.
This is simply not possible.
Apart from exceptions, you cannot give a called function the ability to impact the control flow of the calling scope. So a function call foo() can never interrupt the control flow without throwing an exception. As a consumer of the function, you (the calling function) always have the responsibility yourself to handle such cases and decide about your own control flow.
And it is a very good idea to do it like that. Just the possibility that a function call might interrupt my control flow without having a possibility to react on it first sounds like a pure nightmare. Just alone for the ability to release and cleanup resources, it is very important that the control flow is not taken from me.
Exceptions are the notable exception from this, but of course this is a deeply rooted language feature which also still gives me the ability to act upon it (by catching exceptions, and even by having finally blocks to perform clean up tasks). Exceptions are deliberately not silent but very loud, so that interruptions from the deterministic control flow are clearly visible and have a minimum impact when properly handled.
But having a silent feature that does neither give any control nor feedback would be just a terrible idea.
If myFunc is used at 100 places in my project, everywhere I need to put an if condition after it.
If your code is like that that you could just return nothing from any function that calls myFunc without having to do anything, then either you are building an unrealistic fantasy project, or you simply are not aware of the implications this can have to the calling code of the functions that would be returned that way.
ok, I'll bite.
on the one hand, this isn't really possible. if you want to check something you have to have a line in your code that checks it.
there are a few ways you could achieve something like this, but i think you may have already found the best one.
you already have this function:
def myFunc(value):
return value if isinstance(value, int) else None
I would probably have done:
def myFunc(value):
return isinstance(value, int)
but either way you could use it:
def dumb_func():
value = do_something()
if myFunc(value):
return
do_more()
return value
alternately you could use try and except
I would raise a TypeError, seeing as that seems to be what you are checking:
def myFunc(value):
if not isinstance(value, int):
raise TypeError('myFunc found that {} is not an int'.format(value))
then you can use this as such
def dumb_func():
value = do_something()
try:
myFunc(value):
Except TypeError as e:
print e # some feedback that this has happened, but no error raised
return
do_more()
return value
for bonus points you could define a custom exception (which is safer because then when you catch that specific error you know it wasn't raised by anything else in your code, also if you did that you could be lazier eg:)
Class CustomTypeError(TypeError):
pass
def dumb_func():
try:
value = do_something()
myFunc(value):
do_more()
return value
Except CustomTypeError as e:
print e # some feedback that this has happened, but no error raised
return
but none of this gets around the fact that if you want to act based on the result of a test, you have to check that result.
Python has a ternary conditional operator, and the syntax you used is right, so this will work:
def myFunc(value):
return value if isinstance(value, int) else None
def dumbFunc():
print("Works?")
result = myFunc(5)
print(result)
dumbFunc()
Result:
Works?
5
I want the function to return automatically in that case
This is not possible. To do that, you have to check the return value of myFunc() and act upon it.
PS: You could do that with a goto statement, but Python, fortunately, doesn't support this functionality.
I'm looking for a standard function (operator, decorator) that would be equivalent to the following hand-written function:
def defaulted(func, defaultVal):
try:
result = func()
except:
result = defaultVal
return result
Thanks!
There's nothing like that in the stdlib (to the best of my knowledge). For one thing, it's bad practice: you should never use a bare except. (Instead, specify the exceptions you want to catch; that way, you won't catch everything!)
Here's a decorator:
>>> def defaultval(error, value):
... def decorator(func):
... def defaulted(*args, **kwargs):
... try:
... return func(*args, **kwargs)
... except error:
... return value
... return defaulted
... return decorator
...
>>> #defaultval(NameError, "undefined")
... def get_var():
... return name
...
>>> get_var()
'undefined'
No. Python's philosophy is that explicit is better than implicit. Most Python functions that are expected to throw exceptions regularly, like dict.__getitem__, provide equivalent versions that return a default value, like dict.get.
I've used the new with contexts for something like this before; the code looked like:
with ignoring(IOError, OSError):
# some non-critical file operations
But I didn't need to return anything either. Generally, the whole-function level is a bad place for this, and by using decorators you'd block any chance at actually getting the error should you want to handle it more gracefully elsewhere.
Also, except: is extremely dangerous; you probably mean (at least) except Exception:, and probably something even more tightly scoped, like IOError or KeyError. Otherwise you will also catch things like Ctrl-C, SystemExit, and misspelled variable names.
I have some test cases. The test cases rely on data which takes time to compute. To speed up testing, I've cached the data so that it doesn't have to be recomputed.
I now have foo(), which looks at the cached data. I can't tell ahead of time what it will look at, as that depends a lot on the test case.
If a test case fails cause it doesn't find the right cached data, I don't want it to fail - I want it to compute the data and then try again. I also don't know what exception in particular it will throw cause of missing data.
My code right now looks like this:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
try:
foo()
except:
if not dataComputed:
calculateData()
dataComputed = True
try:
foo()
except:
#error handling code
else:
#the same error handling code
What's the best way to re-structure this code?
I disagree with the key suggestion in the existing answers, which basically boils down to treating exceptions in Python as you would in, say, C++ or Java -- that's NOT the preferred style in Python, where often the good old idea that "it's better to ask forgiveness than permission" (attempt an operation and deal with the exception, if any, rather than obscuring your code's main flow and incurring overhead by thorough preliminary checks). I do agree with Gabriel that a bare except is hardly ever a good idea (unless all it does is some form of logging followed by a raise to let the exception propagate). So, say you have a tuple with all the exception types that you do expect and want to handle the same way, say:
expected_exceptions = KeyError, AttributeError, TypeError
and always use except expected_exceptions: rather than bare except:.
So, with that out of the way, one slightly less-repetitious approach to your needs is:
try:
foo1()
except expected_exceptions:
try:
if condition:
foobetter()
else:
raise
except expected_exceptions:
handleError()
A different approach is to use an auxiliary function to wrap the try/except logic:
def may_raise(expected_exceptions, somefunction, *a, **k):
try:
return False, somefunction(*a, **k)
except expected_exceptions:
return True, None
Such a helper may often come in useful in several different situations, so it's pretty common to have something like this somewhere in a project's "utilities" modules. Now, for your case (no arguments, no results) you could use:
failed, _ = may_raise(expected_exceptions, foo1)
if failed and condition:
failed, _ = may_raise(expected_exceptions, foobetter)
if failed:
handleError()
which I would argue is more linear and therefore simpler. The only issue with this general approach is that an auxiliary function such as may_raise does not FORCE you to deal in some way or other with exceptions, so you might just forget to do so (just like the use of return codes, instead of exceptions, to indicate errors, is prone to those return values mistakenly being ignored); so, use it sparingly...!-)
Using blanket exceptions isn't usually a great idea. What kind of Exception are you expecting there? Is it a KeyError, AttributeError, TypeError...
Once you've identified what type of error you're looking for you can use something like hasattr() or the in operator or many other things that will test for your condition before you have to deal with exceptions.
That way you can clean up your logic flow and save your exception handling for things that are really broken!
Sometimes there's no nice way to express a flow, it's just complicated. But here's a way to call foo() in only one place, and have the error handling in only one place:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
while True:
try:
foo()
break
except:
if not dataComputed:
calculateData()
dataComputed = True
continue
else:
#the error handling code
break
You may not like the loop, YMMV...
Or:
if cacheExists:
loadCache()
dataComputed = False
else:
calculateData()
dataComputed = True
done = False
while !done:
try:
foo()
done = True
except:
if not dataComputed:
calculateData()
dataComputed = True
continue
else:
#the error handling code
done = True
I like the alternative approach proposed by Alex Martelli.
What do you think about using a list of functions as argument of the may_raise. The functions would be executed until one succeed!
Here is the code
def foo(x):
raise Exception("Arrrgh!")
return 0
def foobetter(x):
print "Hello", x
return 1
def try_many(functions, expected_exceptions, *a, **k):
ret = None
for f in functions:
try:
ret = f(*a, **k)
except expected_exceptions, e:
print e
else:
break
return ret
print try_many((foo, foobetter), Exception, "World")
result is
Arrrgh!
Hello World
1
Is there a way to tell if you want to do foobetter() before making the call? If you get an exception it should be because something unexpected (exceptional!) happened. Don't use exceptions for flow control.