Develop Reference

Python: remember status in sequence of procedures

First of all, sorry for the wording of the question, I can't express it in a more compact form.
Let's say I have a code like this in Python:
something_happened = False
def main():
# 'procs' is a list of procedures
for proc in procs:
try:
# Any of these can set the 'something_happened'
# global var to True
proc()
except as e:
handle_unexpected_exception(e)
continue
# If some procedure found some problem,
# print a remainder to check the logging files
if something_happened:
print('Check the logfile, just in case.')
Any of the involved procedures may encounter some problem but execution MUST continue, the problem is properly logged and that's the ONLY handling needed, really, because the problems that may arise while running the procedures shouldn't stop the program, this shouldn't involve raising an exception and stopping the execution.
The reason why the logfile should be checked is that some of the problems may need further human action, but the program can't do anything about them, other than logging them and keep running (long story).
Right now the only way of achieving this that I can think about is to make each procedure to set something_happened == True after logging a potential problem, but using a global variable which may be set from any of the procedures, or returning a status code from the procedures.
And yes, I know I can raise an exception from the procedures instead of setting a global or returning an error code, but that would only work because I'm running them in a loop, and this may change in the future (and then raising an exception will jump out the try-block), so that's my last resort.
Can anyone suggest a better way of dealing with this situation? Yes, I know, this is a very particular use case, but that's the reason why I'm not raising an exception in the first place, and I'm just curious because I didn't find anything after googling for hours...
Thanks in advance :)

You have a variable that may be set to True by any of the procs. It looks like a common OOP schema:
class A():
"""Don't do that"""
def __init__(self, logger):
self._logger = logger
self._something_happened = False
def proc1(self):
try:
...
except KeyError as e:
self._something_happened = True
self._logger.log(...)
def proc2(self):
...
def execute(self):
for proc in [self.proc1, self.proc2, ...]:
try:
proc()
except as e:
self._handle_unexpected_exception(e)
continue
if self._something_happened:
print('Check the logfile, just in case.')
But that's a very bad idea, because you're violating the Single Responsibility Principle: your classs has to know about proc1, proc2, ... You have to reverse the idea:
class Context:
def __init__(self):
self.something_happened = False
def main():
ctx = Context()
for proc in procs:
try:
proc(ctx) # proc may set ctx.something_happened to True
except as e:
handle_unexpected_exception(e)
continue
if ctx.something_happened:
print('Check the logfile, just in case.')
Creating a void class like that is not attracting. You can take the idea further:
class Context:
def __init__(self, logger):
self._logger = logger
self._something_happened = False
def handle_err(self, e):
self._something_happened = True
self._logger.log(...)
def handle_unexpected_exception(self, e):
...
self._logger.log(...)
def after(self):
if self._something_happened:
print('Check the logfile, just in case.')
def proc1(ctx):
try:
...
except KeyError as e:
ctx.handle_err(e) # you delegate the error handling to ctx
def proc2(ctx):
...
def main():
ctx = Context(logging.gerLogger("main"))
for proc in procs:
try:
proc(ctx)
except as e:
ctx.handle_unexpected_exception(e)
ctx.after()
The main benefit here is you that can use another Context if you want:
def StrictContext():
def handle_err(self, e):
raise e
def handle_unexpected_exception(self, e):
raise e
def after(self):
pass
Or
class LooseContext:
def handle_err(self, e):
pass
def handle_unexpected_exception(self, e):
pass
def after(self):
pass
Or whatever you need.

Looks like the cleaner solution is to raise an exception, and I will change the code accordingly. They only problem is what will happen if in the future the loop goes away, but I suppose I'll cross that bridge when I arrive to it ;) and then I'll use another solution or I'll try to change the main code miself.
#cglacet, #Phydeaux, thanks for your help and suggestions.

Catching exception in decorator that wraps generator

I have a decorator that wraps a generator that yields from inside a nose test case. For every iteration, I'm looking to catch and run a specific teardown if an exception occurs, however it does not seem to behave as expected.
def print_log(test_case):
#wraps(test_case)
def run_test(self):
try:
for _ in test_case(self): pass
except:
Test_Loop.failure_teardown(self)
raise
return run_test
Is there something I am doing wrong?

I'm not sure exactly what the unexpected behavior is, but maybe it is happening because you are not trying each loop iteration individually.
Maybe this will work?
def print_log(test_case):
#wraps(test_case)
def run_test(self):
from six.moves import next
test_iter = iter(test_case(self))
while True:
try:
next(test_iter)
except StopIteration:
break
except Exception:
Test_Loop.failure_teardown(self)
raise
return run_test

Function returns None instead raising Exception

I have the next code and it works well.
class AggregateException(Exception):
pass
class Class(object):
def some_method(self):
...
try:
if aggregate(count):
status = Checked
except AggregateException:
status = Rejected
...
def aggregate(count):
if <condition>:
raise AggregateException('Invalid count')
But when I try to simplify logic of some_method() I get an unexpected result. Below you can see code
class Class(object):
def some_method(self):
...
self.aggregate_count(count)
...
def aggregate_count(count):
try:
if aggregate(count):
status = Checked
except AggregateException:
status = Rejected
When an error occurs in aggregate_count(), some_method() continues to run instead finish.

I'm not entirely sure what do you want to do. But as far as I understood you'd like to keep the program running if an exception occur. And that's what your script is doing. The exceptstatement catches your exception and preventing your program from crashing (and change status). Thereafter the rest of the some_method-function is executed. If you want to see it on the command line simply print it in the except block.
If you'd like to crash your program and print the exception to the command line, remove the try and except statement.
Hope that helps.

How do I make a contextmanager with a loop inside?

I want something like this:
from contextlib import contextmanager
#contextmanager
def loop(seq):
for i in seq:
try:
do_setup(i)
yield # with body executes here
do_cleanup(i)
except CustomError as e:
print(e)
with loop([1,2,3]):
do_something_else()
do_whatever()
But contextmanager doesn't work because it expects the generator to yield exactly once.
The reason why I want this is because I basically want to make my own custom for loop. I have a modified IPython that is used to control test equipment. It's obviously a full Python REPL, but most of the time the user is just calling predefined functions (similar to Bash prompt), and the user is not expected to be a programmer or familiar with Python. There needs to be a way to loop over some arbitrary code with setup/cleanup and exception handling for each iteration, and it should be about as simple to type as the above with statement.

I think a generator works better here:
def loop(seq):
for i in seq:
try:
print('before')
yield i # with body executes here
print('after')
except CustomError as e:
print(e)
for i in loop([1,2,3]):
print(i)
print('code')
will give:
before
1
code
after
before
2
code
after
before
3
code
after
Python enters and exits a with block only once so you can't have logic int the enter / exit steps that would be done repeatedly.

A more complete answer, for if the exception might happen outside the generator:
from contextlib import contextmanager
class CustomError(RuntimeError):
pass
#contextmanager
def handle_custom_error():
try:
yield
except CustomError as e:
print(f"handled: {e}")
def loop(seq):
for i in seq:
try:
print('before')
if i == 0:
raise CustomError("inside generator")
yield i # for body executes here
print('after')
except CustomError as e:
print(f"handled: {e}")
#handle_custom_error()
def do_stuff(i):
if i == 1:
raise CustomError("inside do_stuff")
print(f"i = {i}")
for i in loop(range(3)):
do_stuff(i)
Output:
before
handled: inside generator
before
handled: inside do_stuff
after
before
i = 2
after

Exceptions for the whole class

I'm writing a program in Python, and nearly every method im my class is written like this:
def someMethod(self):
try:
#...
except someException:
#in case of exception, do something here
#e.g display a dialog box to inform the user
#that he has done something wrong
As the class grows, it is a little bit annoying to write the same try-except block over and over. Is it possible to create some sort of 'global' exception for the whole class? What's the recommended way in Python to deal with this?

Write one or more exception handler functions that, given a function and the exception raised in it, does what you want to do (e.g. displays an alert). If you need more than one, write them.
def message(func, e):
print "Exception", type(e).__name__, "in", func.__name__
print str(e)
Now write a decorator that applies a given handler to a called function:
import functools
def handle_with(handler, *exceptions):
try:
handler, cleanup = handler
except TypeError:
cleanup = lambda f, e: None
def decorator(func):
#functools.wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except exceptions or Exception as e:
return handler(func, e)
else:
e = None
finally:
cleanup(func, e)
return wrapper
return decorator
This only captures the exceptions you specify. If you don't specify any, Exception is caught. Additionally, the first argument can be a tuple (or other sequence) of two handler functions; the second handler, if given, is called in a finally clause. The value returned from the primary handler is returned as the value of the function call.
Now, given the above, you can write:
#handle_with(message, TypeError, ValueError)
def add(x, y):
return x + y
You could also do this with a context manager:
from contextlib import contextmanager
#contextmanager
def handler(handler, *exceptions):
try:
handler, cleanup = handler
except TypeError:
cleanup = lambda e: None
try:
yield
except exceptions or Exception as e:
handler(e)
else:
e = None
finally:
cleanup(e)
Now you can write:
def message(e):
print "Exception", type(e).__name__
print str(e)
def add(x, y):
with handler(message, TypeError, ValueError):
return x + y
Note that the context manager doesn't know what function it's in (you can find this out, sorta, using inspect, though this is "magic" so I didn't do it) so it gives you a little less useful information. Also, the context manager doesn't give you the opportunity to return anything in your handler.

I can think of two options:
Write a decorator that can wrap each method in the try block.
Write a "dispatcher" method that calls the appropriate method inside a try block, then call that method instead of the individual ones. That is, instead of calling obj.someMethod(), obj.otherMethod, you call obj.dispatch('someMethod') or obj.dispatch('otherMethod'), where dispatch is a wrapper that contains the try block.
Your approach seems like a bit of a strange design, though. It might make more sense to have the dialog-box stuff in some other part of the code, some higher-level event loop that catches errors and displays messages about them.