I was reading this article and it was showing this interesting bit of code:
class Car(object):
def _factory_error_handler(self):
try:
yield
except FactoryColorError, err:
stacktrace = sys.exc_info()[2]
raise ValidationError(err.message), None, stacktrace
def _create_customizer_error_handler(self, vin):
try:
yield
except CustomizerError, err:
self._factory.remove_car(vin)
stacktrace = sys.exc_info()[2]
raise ValidationError(err.message), None, stacktrace
def _update_customizer_error_handler(self, vin):
try:
yield
except CustomizerError, err:
stacktrace = sys.exc_info()[2]
raise ValidationError(err.message), None, stacktrace
def create(self, color, stereo):
with self._factory_error_handler():
vin = self._factory.make_car(color)
with self._create_customizer_error_handler(vin):
self._customizer.update_car(vin, stereo)
return vin
def update(self, vin, color, stereo):
with self._factory_error_handler():
self._factory.update_car_color(vin, color)
with self._update_customizer_error_handler(vin):
self._customizer.update_car(vin, stereo)
return
Here, they are yielding with no argument in a try block. And then using it within a with block. Can someone please explain what is going on here?
The code in the post appears to be wrong. It would make sense if the various error handlers were decorated with contextmanager. Note that, in the post, the code imports contextmanager but doesn't use it. This makes me think the person just made a mistake in creating the post and left contextmanager out of that example. (The later examples in the post do use contextmanager.) I think the code as posted will cause an AttributeError, because the various _error_handler functions aren't context managers and don't have the right __enter__ and __exit__ methods.
With contextmanager, the code makes sense based on the documentation:
At the point where the generator yields, the block nested in the with statement is executed. The generator is then resumed after the block is exited. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred.
Related
For example, my function call make look like the following:
def parse(self, text):
...
return self.parse_helper(text)
#staticmethod
def parser_helper(text):
...
return normalize(text)
#staticmethod
def normalize(text):
...
try:
...
except:
raise ValueError('normalize failed.')
If the 'parse' is the function to be provided to users to call, if an exception occurs in normalize(), the whole program terminates. To avoid this, to let users decide what to do when exception occurs, do I have to and try ... except blocks into both 'parser_helper' and 'parse', and let use to use try...except when 'parse' is called?
What's the normal practice of handling this? If there are a few more layers of function calls embedded other than 3 as shown below, do I have to use try ... except block in each layer of function, in order to transfer the handling of exception to the end users at the very top?
The practice to pass a parameter to decide wether to raise or be silent regarding an exception, is something that exists, for example in os.makedirs
You could do
def parse(self, text, error_silent=False):
...
try:
return self.parse_helper(text)
except ValueError as e:
if error_silent:
return None
raise e
I am trying to implement the retry decorator on a serial query. A general idea of my code is shown below. I am struggling to get it to retry when the decorator is one method up in the hierarchy. How can I have the method be retried when it's one method up from the method that throws the exception?
One complication that is frustrating is my increment time per retry depends on the actual command. Some commands require more time than others. That's why I have the extra_time_per_retry passed in, and couldn't implement the retry decorator using the traditional #retry style.
FYI the _serial is created in the class on init via pySerial.
I got it to work with the retry decorator directly above the method that throws the exception. I would like it to be two above to keep my code clean.
I have tried feeding the retry decorator the exact exception type, but couldn't get it to work.
def _query_with_retries(self, cmd, extra_time_per_retry):
_retriable_query = retry(stop_max_attempt_number=5,
wait_incrementing_start=self._serial.timeout + extra_time_per_retry,
wait_incrementing_increment=10)(self._query)
return _retriable_query(cmd)
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
return self._readlines()
def _readlines(self):
response_str = self._serial.read_until('\r', 256) # Max 256 bytes
# Parse response here, if a bad one set bad_response = true
if bad_response:
raise ResponseError("Response had custom error xyz")
I guess you could pack your call to _readlines() into an exception handling block, reraising the error:
python 3.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception as e:
raise e
return answer
python 2.x
#retry
def _query(self, cmd):
cmd_msg = cmd + '\r'
self._serial.reset_input_buffer()
self._serial.reset_output_buffer()
self._serial.write(cmd_msg)
try:
answer = self._readlines()
except Exception:
t, v, tb = sys.exc_info()
raise t, v, tb
return answer
This way, you catch the exception directly when it occurs, and raise it inside the method which will be retried. I am not sure whether this declutters enough for you, however it should work.
Some might complain about using a blank except Exception, however since I am reraising it always immediately, I do not see any harm.
Is it possible to tell if there was an exception once you're in the finally clause? Something like:
try:
funky code
finally:
if ???:
print('the funky code raised')
I'm looking to make something like this more DRY:
try:
funky code
except HandleThis:
# handle it
raised = True
except DontHandleThis:
raised = True
raise
else:
raised = False
finally:
logger.info('funky code raised %s', raised)
I don't like that it requires to catch an exception, which you don't intend to handle, just to set a flag.
Since some comments are asking for less "M" in the MCVE, here is some more background on the use-case. The actual problem is about escalation of logging levels.
The funky code is third party and can't be changed.
The failure exception and stack trace does not contain any useful diagnostic information, so using logger.exception in an except block is not helpful here.
If the funky code raised then some information which I need to see has already been logged, at level DEBUG. We do not and can not handle the error, but want to escalate the DEBUG logging because the information needed is in there.
The funky code does not raise, most of the time. I don't want to escalate logging levels for the general case, because it is too verbose.
Hence, the code runs under a log capture context (which sets up custom handlers to intercept log records) and some debug info gets re-logged retrospectively:
try:
with LogCapture() as log:
funky_code() # <-- third party badness
finally:
# log events are buffered in memory. if there was an exception,
# emit everything that was captured at a WARNING level
for record in log.captured:
if <there was an exception>:
log_fn = mylogger.warning
else:
log_fn = getattr(mylogger, record.levelname.lower())
log_fn(record.msg, record.args)
Using a contextmanager
You could use a custom contextmanager, for example:
class DidWeRaise:
__slots__ = ('exception_happened', ) # instances will take less memory
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# If no exception happened the `exc_type` is None
self.exception_happened = exc_type is not None
And then use that inside the try:
try:
with DidWeRaise() as error_state:
# funky code
finally:
if error_state.exception_happened:
print('the funky code raised')
It's still an additional variable but it's probably a lot easier to reuse if you want to use it in multiple places. And you don't need to toggle it yourself.
Using a variable
In case you don't want the contextmanager I would reverse the logic of the trigger and toggle it only in case no exception has happened. That way you don't need an except case for exceptions that you don't want to handle. The most appropriate place would be the else clause that is entered in case the try didn't threw an exception:
exception_happened = True
try:
# funky code
except HandleThis:
# handle this kind of exception
else:
exception_happened = False
finally:
if exception_happened:
print('the funky code raised')
And as already pointed out instead of having a "toggle" variable you could replace it (in this case) with the desired logging function:
mylog = mylogger.WARNING
try:
with LogCapture() as log:
funky_code()
except HandleThis:
# handle this kind of exception
else:
# In case absolutely no exception was thrown in the try we can log on debug level
mylog = mylogger.DEBUG
finally:
for record in log.captured:
mylog(record.msg, record.args)
Of course it would also work if you put it at the end of your try (as other answers here suggested) but I prefer the else clause because it has more meaning ("that code is meant to be executed only if there was no exception in the try block") and may be easier to maintain in the long run. Although it's still more to maintain than the context manager because the variable is set and toggled in different places.
Using sys.exc_info (works only for unhandled exceptions)
The last approach I want to mention is probably not useful for you but maybe useful for future readers who only want to know if there's an unhandled exception (an exception that was not caught in any except block or has been raised inside an except block). In that case you can use sys.exc_info:
import sys
try:
# funky code
except HandleThis:
pass
finally:
if sys.exc_info()[0] is not None:
# only entered if there's an *unhandled* exception, e.g. NOT a HandleThis exception
print('funky code raised')
raised = True
try:
funky code
raised = False
except HandleThis:
# handle it
finally:
logger.info('funky code raised %s', raised)
Given the additional background information added to the question about selecting a log level, this seems very easily adapted to the intended use-case:
mylog = WARNING
try:
funky code
mylog = DEBUG
except HandleThis:
# handle it
finally:
mylog(...)
You can easily assign your caught exception to a variable and use it in the finally block, eg:
>>> x = 1
>>> error = None
>>> try:
... x.foo()
... except Exception as e:
... error = e
... finally:
... if error is not None:
... print(error)
...
'int' object has no attribute 'foo'
Okay, so what it sounds like you actually just want to either modify your existing context manager, or use a similar approach: logbook actually has something called a FingersCrossedHandler that would do exactly what you want. But you could do it yourself, like:
#contextmanager
def LogCapture():
# your existing buffer code here
level = logging.WARN
try:
yield
except UselessException:
level = logging.DEBUG
raise # Or don't, if you just want it to go away
finally:
# emit logs here
Original Response
You're thinking about this a bit sideways.
You do intend to handle the exception - you're handling it by setting a flag. Maybe you don't care about anything else (which seems like a bad idea), but if you care about doing something when an exception is raised, then you want to be explicit about it.
The fact that you're setting a variable, but you want the exception to continue on means that what you really want is to raise your own specific exception, from the exception that was raised:
class MyPkgException(Exception): pass
class MyError(PyPkgException): pass # If there's another exception type, you can also inherit from that
def do_the_badness():
try:
raise FileNotFoundError('Or some other code that raises an error')
except FileNotFoundError as e:
raise MyError('File was not found, doh!') from e
finally:
do_some_cleanup()
try:
do_the_badness()
except MyError as e:
print('The error? Yeah, it happened')
This solves:
Explicitly handling the exception(s) that you're looking to handle
Making the stack traces and original exceptions available
Allowing your code that's going to handle the original exception somewhere else to handle your exception that's thrown
Allowing some top-level exception handling code to just catch MyPkgException to catch all of your exceptions so it can log something and exit with a nice status instead of an ugly stack trace
If it was me, I'd do a little re-ordering of your code.
raised = False
try:
# funky code
except HandleThis:
# handle it
raised = True
except Exception as ex:
# Don't Handle This
raise ex
finally:
if raised:
logger.info('funky code was raised')
I've placed the raised boolean assignment outside of the try statement to ensure scope and made the final except statement a general exception handler for exceptions that you don't want to handle.
This style determines if your code failed. Another approach might me to determine when your code succeeds.
success = False
try:
# funky code
success = True
except HandleThis:
# handle it
pass
except Exception as ex:
# Don't Handle This
raise ex
finally:
if success:
logger.info('funky code was successful')
else:
logger.info('funky code was raised')
If exception happened --> Put this logic in the exception block(s).
If exception did not happen --> Put this logic in the try block after the point in code where the exception can occur.
Finally blocks should be reserved for "cleanup actions," according to the Python language reference. When finally is specified the interpreter proceeds in the except case as follows: Exception is saved, then the finally block is executed first, then lastly the Exception is raised.
I'm trying to catch an exception thrown in the caller of a generator:
class MyException(Exception):
pass
def gen():
for i in range(3):
try:
yield i
except MyException:
print 'handled exception'
for i in gen():
print i
raise MyException
This outputs
$ python x.py
0
Traceback (most recent call last):
File "x.py", line 14, in <module>
raise MyException
__main__.MyException
when I was intending for it to output
$ python x.py
0
handled exception
1
handled exception
2
handled exception
In retrospect, I think this is because the caller has a different stack from the generator, so the exception isn't bubbled up to the generator. Is that correct? Is there some other way to catch exceptions raised in the caller?
Aside: I can make it work using generator.throw(), but that requires modifying the caller:
def gen():
for i in range(3):
try:
yield i
except MyException:
print 'handled exception'
yield
import sys
g = gen()
for i in g:
try:
print i
raise MyException
except:
g.throw(*sys.exc_info())
You may be thinking that when execution hits yield in the generator, the generator executes the body of the for loop, sort of like a Ruby function with yield and a block. That's not how things work in Python.
When execution hits yield, the generator's stack frame is suspended and removed from the stack, and control returns to the code that (implicitly) called the generator's next method. That code then enters the loop body. At the time the exception is raised, the generator's stack frame is not on the stack, and the exception does not go through the generator as it bubbles up.
The generator has no way to respond to this exception.
You might also have gotten confused - as I was just now and arriving here - by the yield in the context managers (contextlib.contextmanager).
Their usage can be:
from contextlib import contextmanager
#contextmanager
def mycontext():
try:
yield
except MyException:
print 'handled exception'
So my solution to a similar case to what you described above is:
def gen():
for i in range(3):
yield i
for ii in gen():
with mycontext():
print ii
raise MyException
Which gives the expected output and uses all of the yields.
A bit late to the party here, but maybe someone with similar knots in their minds will find it helpful.
Note: putting the context INSIDE the generator will be the same mistake as using try ... except there!!
I'm trying to add SimPy simulation to a project I'm working on and I have some confusion about version 3's release/request.
I was able to implement resources using a 'with'-block without trouble but in my situation I want to request/release a resource without using a 'with'-block.
However, I cannot find an example of this using SimPy 3. I read the documentation/source regarding resources but still can't get it quite right. Could someone explain how to properly:
...
Request a Resource with the method: 'request()'
...
Release that Resource with the method: 'release()'
...
Thanks, and sorry for the bother.
PS: I'm intending to use Resources.resource
If you want to use a resource without a with block (and you know you won’t get interrupted), its just:
req = resource.request()
yield req
# do stuff
resource.release(req)
Using with on an object calls __enter__ when you enter the with block, and __exit__ when you leave. So when you do
res = resource.Resource()
with res.request() as req:
# stuff
You're really calling __enter__ on a Request object, doing #stuff then calling __exit__:
class Request(base.Put):
def __exit__(self, exc_type, value, traceback):
super(Request, self).__exit__(exc_type, value, traceback)
self.resource.release(self)
class Put(Event): # base.Put
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
# If the request has been interrupted, remove it from the queue:
if not self.triggered:
self.resource.put_queue.remove(self)
So, the with block is equivalent to this:
res = resource.Resource(...)
req = res.request()
#stuff
if not req.triggered:
res.put_queue.remove(req)
res.release(req)
However, the with block is also making sure that the cleanup code is called no matter what exceptions are thrown during #stuff. You'll lose that with the above code.
It's all outlined in PEP343;
with EXPR as VAR:
BLOCK
becomes:
mgr = (EXPR)
exit = type(mgr).__exit__ # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
try:
VAR = value # Only if "as VAR" is present
BLOCK
except:
# The exceptional case is handled here
exc = False
if not exit(mgr, *sys.exc_info()):
raise
# The exception is swallowed if exit() returns true
finally:
# The normal and non-local-goto cases are handled here
if exc:
exit(mgr, None, None, None)
This is exactly how python uses with... as... blocks, but I'm presuming there's some reason you don't want to use these. If that's the case, then you just need the __enter__ and __exit__ functions. The way i think of it, is __enter__ sets everything up, and __exit__ does all the cleanup.