This question already has answers here:
is there a pythonic way to try something up to a maximum number of times?
(10 answers)
Closed 7 months ago.
I am writing in Python 2.7 and encounter the following situation. I would like to try calling a function three times. If all three times raise errors, I will raise the last error I get. If any one of the calls succeed, I will quit trying and continue immediately.
Here is what I have right now:
output = None
error = None
for _e in range(3):
error = None
try:
print 'trial %d!' % (_e + 1)
output = trial_function()
except Exception as e:
error = e
if error is None:
break
if error is not None:
raise error
Is there a better snippet that achieve the same use case?
use decorator
from functools import wraps
def retry(times):
def wrapper_fn(f):
#wraps(f)
def new_wrapper(*args,**kwargs):
for i in range(times):
try:
print 'try %s' % (i + 1)
return f(*args,**kwargs)
except Exception as e:
error = e
raise error
return new_wrapper
return wrapper_fn
#retry(3)
def foo():
return 1/0;
print foo()
Here is one possible approach:
def attempt(func, times=3):
for _ in range(times):
try:
return func()
except Exception as err:
pass
raise err
A demo with a print statement in:
>>> attempt(lambda: 1/0)
Attempt 1
Attempt 2
Attempt 3
Traceback (most recent call last):
File "<pyshell#18>", line 1, in <module>
attempt(lambda: 1/0)
File "<pyshell#17>", line 8, in attempt
raise err
ZeroDivisionError: integer division or modulo by zero
If you're using Python 3.x and get an UnboundLocalError, you can adapt as follows:
def attempt(func, times=3):
to_raise = None
for _ in range(times):
try:
return func()
except Exception as err:
to_raise = err
raise to_raise
This is because the err is cleared at the end of the try statement; per the docs:
When an exception has been assigned using as target, it is cleared
at the end of the except clause.
Ignoring the debug output and the ancient Python dialect, this looks good. The only thing I would change is to put it into a function, you could then simply return the result of trial_function(). Also, the error = None then becomes unnecessary, including the associated checks. If the loop terminates, error must have been set, so you can just throw it. If you don't want a function, consider using else in combination with the for loop and breaking after the first result.
for i in range(3):
try:
result = foo()
break
except Exception as error:
pass
else:
raise error
use_somehow(result)
Of course, the suggestion to use a decorator for the function still holds. You can also apply this locally, the decorator syntax is only syntactic sugar after all:
# retry from powerfj's answer below
rfoo = retry(3)(foo)
result = rfoo()
Came across a clean way of doing the retries. There is a module called retry.
First install the module using
pip install retry
Then import the module in the code.
from retry import retry
Use #retry decorator above the method, We can pass the parameters to the decorator. Some of the parameters are tries , delay , Exception.
Example
from retry import retry
#retry(AssertionError, tries=3, delay=2)
def retryfunc():
try:
ret = False
assert ret, "Failed"
except Exception as ex:
print(ex)
raise ex
The above code asserts and fails everytime, but the retry decorator retries for 3 times with a delay of 2 seconds between retries. Also this only retries on Assertion failures since we have specified the error type as AssertionError on any other error the function wont retry.
Related
What is the best practice to run functions in sequence?
I have 4 functions, if the previous one failed, do not run the following ones. In each function, I set global error and error = 1 when exception occurs. Then in main, I just use if statement to check the value of error. I think there should be a better way to do it.
def main():
engine = conn_engine()
if error == 0:
process_sql()
if error == 0:
append_new_rows_to_prod()
if error == 0:
send_email_log()
The canonical way is to raise an exception within the function. For example:
def process_sql():
# Do stuff
if stuff_failed:
raise ProcessSQLException("Error while processing SQL")
def append_new_rows_to_prod():
# Do other stuff
if other_stuff_failed:
raise AppendRowsException("Error while appending rows")
def main():
engine = conn_engine()
try:
process_sql()
append_new_rows_to_prod()
send_email_log()
except ProcessSQLException, AppendRowsException as e:
# Handle exception, or gracefully exit
I am trying to implement the retry decorator on a serial query. A general idea of my code is shown below. I am struggling to get it to retry when the decorator is one method up in the hierarchy. How can I have the method be retried when it's one method up from the method that throws the exception?
One complication that is frustrating is my increment time per retry depends on the actual command. Some commands require more time than others. That's why I have the extra_time_per_retry passed in, and couldn't implement the retry decorator using the traditional #retry style.
FYI the _serial is created in the class on init via pySerial.
I got it to work with the retry decorator directly above the method that throws the exception. I would like it to be two above to keep my code clean.
I have tried feeding the retry decorator the exact exception type, but couldn't get it to work.
def _query_with_retries(self, cmd, extra_time_per_retry):
_retriable_query = retry(stop_max_attempt_number=5,
wait_incrementing_start=self._serial.timeout + extra_time_per_retry,
wait_incrementing_increment=10)(self._query)
return _retriable_query(cmd)
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
return self._readlines()
def _readlines(self):
response_str = self._serial.read_until('\r', 256) # Max 256 bytes
# Parse response here, if a bad one set bad_response = true
if bad_response:
raise ResponseError("Response had custom error xyz")
I guess you could pack your call to _readlines() into an exception handling block, reraising the error:
python 3.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception as e:
raise e
return answer
python 2.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception:
t, v, tb = sys.exc_info()
raise t, v, tb
return answer
This way, you catch the exception directly when it occurs, and raise it inside the method which will be retried. I am not sure whether this declutters enough for you, however it should work.
Some might complain about using a blank except Exception, however since I am reraising it always immediately, I do not see any harm.
I am trying to run a function whenever an error occurs. I don't want to use try or except because my code is very large and there are so much chances of an error to occur , so I can't use try.. everywhere. This is what I am expecting:
>>> if ValueError: #don't works , just assuming.
print("Hey! You are done.")
>>> int("abc")
Hey! You are done.
>>> int("1b")
Hey! You are done.
>>>
Is there any way to do this?
If your code is a whole block, I recommend splitting it into functions. From here, you can wrap each function in a decorator that takes an error and a function to be run on error:
def handle_error(error, error_func):
def decorator(func):
def wrapper(*args, **kwargs):
r = None
try:
r = func(*args, **kwargs)
except error as e:
r = error_func()
finally:
return r
return wrapper
return decorator
Then use on functions like so:
def bad_value():
print('bad value given!')
#handle_error(ValueError, bad_value)
def multiply(a, b):
return a * b
Of course, you can be more 'broad', and catch all exceptions...
#handle_error(Exception, error_func)
def func():
# ...
A ValueError is triggered by a particular piece of code, and will raise itself immediately after the offending statement. int("abc") will raise a ValueError on its own, and the program execution will stop, before it reaches any if ValueError statement.
You need a try/except block to allow python to catch the error and continue executing. I can't see any way to achieve what you want without one.
Consider the following code:
def f(x):
if x < 10:
return Exception("error")
else:
raise Exception("error2")
if __name__ == "__main__":
try:
f(5) # f(20)
except Exception:
print str(Exception)
Is there any difference?
When should I use return Exception and When should I use raise?
raise and return are two inherently different keywords.
raise, commonly known as throw in other languages, produces an error in the current level of the call-stack. You can catch a raised error by covering the area where the error might be raised in a try and handling that error in an except.
try:
if something_bad:
raise generate_exception()
except CertainException, e:
do_something_to_handle_exception(e)
return on the other hand, returns a value to where the function was called from, so returning an exception usually is not the functionality you are looking for in a situation like this, since the exception itself is not the thing triggering the except it is instead the raiseing of the exception that triggers it.
I'm practicing TDD in Python and came across a problem in testing whether an exception is raised.
Here is my test_phonebook.py with test_add_empty_name_raises_exception which fails.
import unittest
import phonebook
class Test(unittest.TestCase):
def test_add_empty_name_raises_exception(self):
self.assertRaises(ValueError, phonebook.add, "", "1111111111")
if __name__ == "__main__":
# import sys;sys.argv = ['', 'Test.testName']
unittest.main()
Below is my phonebook.py with the method add which adds the data into the dictionary.
import re
_phonebook = {}
file_name = "phonebook.txt"
def is_valid_name(name):
return re.search(r"([A-Z][a-z]*)([\\s\\\'-][A-Z][a-z]*)*", name) is not None
def is_valid_number(number):
return re.search(r"\+?[\d ]+$", number) is not None
def add(name, number):
try:
if is_valid_name(name) and is_valid_number(number):
_phonebook[name] = number
else:
raise ValueError("Invalid arguments.", name, number)
except ValueError as err:
print err.args
if __name__ == '__main__':
pass
My problem is that the test fails even though it is seen in the console log that there was a ValueError raised within the add method.
Finding files... done.
Importing test modules ... done.
('Invalid arguments.', '', '1111111111')
======================================================================
FAIL: test_add_empty_name_raises_exception (path.to.phonebook.test_phonebook.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "path\to\phonebook\test_phonebook.py", line 13, in test_add_empty_name_raises_exception
self.assertRaises(ValueError, phonebook.add, "", "1111111111")
AssertionError: ValueError not raised
----------------------------------------------------------------------
Ran 1 test in 0.002s
How do I solve this? I there something I forgot?
I also tried using the new format for handling exceptions in tests in Python 2.7 but it still hasn't caught the ValueError raising.
def test_add_empty_name_raises_exception(self):
with self.assertRaises(ValueError):
self.phonebook.add("", "1111111111)
I also changed the form of the test case into using lambdas but still no changes.
def test_add_empty_name_raises_exception(self):
self.assertRaises(ValueError, lambda: phonebook.add("", "1111111111"))
I also cleaned my directory and restarted Eclipse Luna and problem still persists.
Possible solution
I was reading the 8.Errors and Exceptions documentation and got to the "Raising Exceptions" part which states that:
If you need to determine whether an exception was raised but don’t intend to handle it,
a simpler form of the raise statement allows you to re-raise the exception:
I added this to the existing add method as such:
def add(name, number):
try:
if is_valid_name(name) and is_valid_number(number):
_phonebook[name] = number
print "Added %s:\t%s" % (name, number)
else:
raise ValueError("Invalid arguments.", name, number)
except ValueError as err:
print err.args
raise
Which caused the test case to pass.
Is this the correct way? To call raise again in the except block?
When you catch an exception (in your except ValueError as err: block), you prevent it from continuing back up the call stack to eventually terminate the program. Essentially, you're saying "I know how to handle this, so no need to panic anyone else."
Re-raising an exception is the proper thing to do if you caught the exception but didn't do so to actually fix anything, for instance, to log that it occurred. Typically, though, one catches an exception in order to correct it.
In your case, you're catching the exception almost immediately after you yourself raised it. Why not put your logging statement in the same else block as the raise? No need for a try: ... except: indent at all.
def add(name, number):
if is_valid_name(name) and is_valid_number(number):
_phonebook[name] = number
print "Added %s:\t%s" % (name, number)
else:
print "Invalid arguments.", name, number
raise ValueError("Invalid arguments.", name, number)
return