I was curious if this causes any bad behaviors. I ran a test case and got no errors so I assume its OK (although probably not good practice). Just wanted to know how python deals with the issue I assumed should have existed?
with open("somefile.txt","r") as fileinfo:
fileinfo = fileinfo.readlines()
print fileinfo
I thought overwritting "fileinfo" would cause issues exiting the with statement (raise some error about not being able to .close() a list). Does the with statement retain a local copy of the file reference? Thanks!
Of course Python retains an internal reference to the object used in the with statement. Otherwise how would it work when you don't use the as clause?
A with statement does indeed store a local reference to the file object (although I am not positive exactly what is stored in self.gen)
Looked into the topic, specifically researching the context manager and found this which gives slightly more detail for those interested.
class GeneratorContextManager(object):
def __init__(self, gen):
# Store local copy of "file reference"
self.gen = gen
def __enter__(self):
try:
return self.gen.next()
except StopIteration:
raise RuntimeError("generator didn't yield")
def __exit__(self, type, value, traceback):
if type is None:
try:
self.gen.next()
except StopIteration:
return
else:
raise RuntimeError("generator didn't stop")
else:
try:
self.gen.throw(type, value, traceback)
raise RuntimeError("generator didn't stop after throw()")
except StopIteration:
return True
except:
# only re-raise if it's *not* the exception that was
# passed to throw(), because __exit__() must not raise
# an exception unless __exit__() itself failed. But
# throw() has to raise the exception to signal
# propagation, so this fixes the impedance mismatch
# between the throw() protocol and the __exit__()
# protocol.
#
if sys.exc_info()[1] is not value:
raise
def contextmanager(func):
def helper(*args, **kwds):
return GeneratorContextManager(func(*args, **kwds))
return helper
Related
I have a situation where I want to do multiple things while handling an exception. Since I want to make this about the general case, I'll translate my specific case into some more general language.
When I have an exception in this piece of code, I want to:
Always perform a rollback-style operation
If it is an
application specific exception, I want to perform some logging and swallow the exception.
So I can think of two ways to solve it, both ugly:
# Method nested-try/except block
try:
try:
do_things()
except:
rollback()
raise
except SpecificException as err:
do_advanced_logging(err)
return
# Method Duplicate Code
try:
do_things()
except SpecificException as err:
rollback()
do_advanced_logging(err)
return
except:
rollback()
raise
Both will have the same behaviour.
I'm tending towards the nested try/except solution myself. While it might be slightly slower, I don't think the speed difference is relevant here - at the very least not for my specific case. Duplication of code is something I want to avoid also because my rollback() statement is slightly more involved that just a database rollback, even if it has the exact same purpose (it involves a web-API).
Is there a third option I haven't spotted that is better? Or is the duplicate code method better? Please note that the rollback() functionality is already factored out as much as possible, but still contains a function call and three arguments which includes a single hardcoded string. Since this string is unique, there's no reason to make it a named constant.
How about checking the exception instance type in code?
# Method .. No Duplicate Code
try:
do_things()
except Exception as e:
rollback()
if isinstance(e, SpecificException):
do_advanced_logging(e)
return
raise
how about putting the rollback in a finally clause? something like:
do_rollback = True
try:
do_things()
do_rollback = False
except SpecificException as err:
do_advanced_logging(err)
finally:
if do_rollback:
rollback()
an alternative is to use an else clause, which would let you do more in the non-exceptional case and not have exceptions all caught in the same place:
do_rollback = True
try:
do_things()
except SpecificException as err:
do_advanced_logging(err)
else:
record_success()
do_rollback = False
finally:
if do_rollback:
rollback()
is useful when record_success can raise a SpecificException, but you don't want to do_advanced_logging
You could write a context manager:
import random
class SpecificException(Exception):
pass
def do_things(wot=None):
print("in do_things, wot = {}".format(wot))
if wot:
raise wot("test")
def rollback():
print("rollback")
def do_advance_logging(exc_type, exc_val, traceback):
print("logging got {} ('{}')".format(exc_type, exc_val))
class rollback_on_error(object):
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, traceback):
# always rollback
rollback()
# log and swallow specific exceptions
if exc_type and issubclass(exc_type, SpecificException):
do_advance_logging(exc_type, exc_val, traceback)
return True
# propagate other exceptions
return False
def test():
try:
with rollback_on_error():
do_things(ValueError)
except Exception as e:
print("expected ValueError, got '{}'".format(type(e)))
else:
print("oops, should have caught a ValueError")
try:
with rollback_on_error():
do_things(SpecificException)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
try:
with rollback_on_error():
do_things(None)
except Exception as e:
print("oops, didn't expect exception '{}' here".format(e))
else:
print("ok, no exception")
if __name__ == "__main__":
test()
But unless you have dozen occurrences of this pattern, I'd rather stick to the very obvious and perfectly pythonic solutions - either nested exceptions handlers or explicit typecheck (isinstance) in the except clause.
I have a try:finally block that must execute always (exception or not) unless a specific exception occurs. For the sake of argument let's say it's a ValueError, so I'm asking if I can implement:
try:
stuff()
except Exception as e:
if type(e) is ValueError: raise
#do important stuff
raise
#do important stuff
in a more elegant fashion to skip copy-pasting #importantstuff. If I ruled Python it would look something like:
try:
stuff()
finally except ValueError:
#do important stuff
Putting #importantstuff in a function is not an answer, but not possible is.
If you need finally to skip things in specific conditions, you'll need to use an explicit flag:
do_final_stuff = True
try:
# ...
except ValueError:
do_final_stuff = False
raise
finally:
if do_final_stuff:
# ...
You could also use a context manager here, to clean up afterwards. A context manager is passed the current active exception if there is one:
class MyContextManager:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not ValueError:
# do cleanup
with MyContextManager():
# ...
I'm trying to add SimPy simulation to a project I'm working on and I have some confusion about version 3's release/request.
I was able to implement resources using a 'with'-block without trouble but in my situation I want to request/release a resource without using a 'with'-block.
However, I cannot find an example of this using SimPy 3. I read the documentation/source regarding resources but still can't get it quite right. Could someone explain how to properly:
...
Request a Resource with the method: 'request()'
...
Release that Resource with the method: 'release()'
...
Thanks, and sorry for the bother.
PS: I'm intending to use Resources.resource
If you want to use a resource without a with block (and you know you won’t get interrupted), its just:
req = resource.request()
yield req
# do stuff
resource.release(req)
Using with on an object calls __enter__ when you enter the with block, and __exit__ when you leave. So when you do
res = resource.Resource()
with res.request() as req:
# stuff
You're really calling __enter__ on a Request object, doing #stuff then calling __exit__:
class Request(base.Put):
def __exit__(self, exc_type, value, traceback):
super(Request, self).__exit__(exc_type, value, traceback)
self.resource.release(self)
class Put(Event): # base.Put
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
# If the request has been interrupted, remove it from the queue:
if not self.triggered:
self.resource.put_queue.remove(self)
So, the with block is equivalent to this:
res = resource.Resource(...)
req = res.request()
#stuff
if not req.triggered:
res.put_queue.remove(req)
res.release(req)
However, the with block is also making sure that the cleanup code is called no matter what exceptions are thrown during #stuff. You'll lose that with the above code.
It's all outlined in PEP343;
with EXPR as VAR:
BLOCK
becomes:
mgr = (EXPR)
exit = type(mgr).__exit__ # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
try:
VAR = value # Only if "as VAR" is present
BLOCK
except:
# The exceptional case is handled here
exc = False
if not exit(mgr, *sys.exc_info()):
raise
# The exception is swallowed if exit() returns true
finally:
# The normal and non-local-goto cases are handled here
if exc:
exit(mgr, None, None, None)
This is exactly how python uses with... as... blocks, but I'm presuming there's some reason you don't want to use these. If that's the case, then you just need the __enter__ and __exit__ functions. The way i think of it, is __enter__ sets everything up, and __exit__ does all the cleanup.
I got a lot of code like this :
try:
# do a lot of stuff
except StuffError as e:
log.exception(e):
send_mail_to_admin()
raise e
For DRY, I wanted to refactor that into :
def post_mortem(log, e):
log.exception(e):
send_mail_to_admin()
# some other stuff
raise e
Then do :
try:
# do a lot of stuff
except StuffError as e:
post_mortem(log, e)
The trouble is now I don't get the proper stack trace, since the exception is raised from another file.
How can I get the same stack trace I would have had with the first code ?
Pass the exc_info() information also, as one of the parameters like this
post_mortem(log, e, sys.exc_info()[2])
And in the post_mortem
def post_mortem(log, e, traceBack):
traceback.print_tb(traceBack)
To get the entire stacktrace, you can do like shown in this example
import traceback, sys
def post_mortem(log, e, tb):
print "".join(traceback.format_list(traceback.extract_stack()[:-2]) + [traceback.format_tb(tb)[0]])
def throwError():
try:
raise NameError("I don't like your name")
except NameError as e:
post_mortem("", e, sys.exc_info()[2])
def callThrowError():
throwError()
callThrowError()
Output
File "/home/thefourtheye/Desktop/Test.py", line 15, in <module>
callThrowError()
File "/home/thefourtheye/Desktop/Test.py", line 13, in callThrowError
throwError()
File "/home/thefourtheye/Desktop/Test.py", line 8, in throwError
raise NameError("I don't like your name")
..you wanted it DRY? Try this then :)
class ReportExceptions(object):
def __enter__(self):
pass
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not None:
print("Sending mail to admin about {0!r}".format(exc_value))
And use it like this:
with ReportExceptions():
# your code here..
With a context manager, you don't have to care about re-raising exceptions, saving tracebacks or other things: the __exit__() method will be executed no matter what, passing you the necessary information. If you do nothing about it (eg. return True) the exception will just continue its way out..
Side note: you can instantiate the context manager only once, for example:
class ReportExceptions(object):
def __init__(self, email):
self.email = email
def __enter__(self):
pass
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not None:
print("Sending mail to {0!r} about {1!r}"
"".format(self.email, exc_value))
report_exceptions = ReportExceptions('foo#bar.com')
then:
with report_exceptions:
# ..your code here..
FYI: raising an exception with custom traceback
In case you really need to re-raise an exception, you can save the traceback and keep it for later..
try:
100 / 0
except ZeroDivisionError, e:
exc_info = sys.exc_info()
...later on...
raise exc_info[0], exc_info[1], exc_info[2]
(The syntax is actually raise expression, expression, expression, I couln't figure out a nicer way to use the tuple directly...)
Inspecting frames along the path
You can access tb attributes to inspect the execution frames along the path, for example the locals from the "most external" point would be tb.tb_frame.f_locals, the inner frame is at tb.tb_next.tb_frame, etc...
(you can also use the inspect module, see https://stackoverflow.com/a/10115462/148845)
From within an __exit__ block in a custom cursor class I want to catch an exception so I can in turn throw a more specific exception. What is the proper way to do this?
class Cursor:
def __enter__(self):
...
def __exit__(self, ex_type, ex_val, tb):
if ex_type == VagueThirdPartyError:
# get new more specific error based on error code in ex_val and
# return that one in its place.
return False # ?
else:
return False
Raising the specific exception within the __exit__ block seems like a hack, but maybe I'm over thinking it.
The proper procedure is to raise the new exception inside of the __exit__ handler.
You should not raise the exception that was passed in though; to allow for context manager chaining, in that case you should just return a falsey value from the handler. Raising your own exceptions is however perfectly fine.
Note that it is better to use the identity test is to verify the type of the passed-in exception:
def __exit__(self, ex_type, ex_val, tb):
if ex_type is VagueThirdPartyError:
if ex_val.args[0] == 'foobar':
raise SpecificException('Foobarred!')
# Not raising a new exception, but surpressing the current one:
if ex_val.args[0] == 'eggs-and-ham':
# ignore this exception
return True
if ex_val.args[0] == 'baz':
# re-raise this exception
return False
# No else required, the function exits and `None` is returned
You could also use issubclass(ex_type, VagueThirdPartyError) to allow for subclasses of the specific exception.