I have a program which request info from the twitter API, and from time to time I get an error:
IOError: [Errno socket error] [Errno 54] Connection reset by peer
I want to know how can I keep my script running (looping). I know that this has something to do with:
try:
except IOError:
but I can not figure it out.
The simpler structure is like this:
my_while_or_for_loop:
some_code_here_maybe
try:
my_code_or_function_that_sometimes_fails()
except IOError:
pass # or some code to clean things that went wrong or to log the failure
some_more_code_here_maybe
You want to read the docs
The full construction can be more complex and includes try/except/else/finally.
From an example in docs:
>>> def divide(x, y):
... try:
... result = x / y
... except ZeroDivisionError:
... print "division by zero!"
... else:
... print "result is", result
... finally:
... print "executing finally clause"
Here is is documentationabout exceptions...
Simply, if a code block have possibility to cause some known errors (like input output error) in some conditions, you define an try-exceptblock to handle such errors. That will make your script keep runnung and let you execute diffrent code blocks according to diffrent error status.... Like:
try:
<do something>
except IOError:
<an input-output error occured, do this...>
except ValueError:
<we got something diffrent then we expected, do something diffrent>
except LookupError:
pass # we do not need to handle this one, so just kkeep going...
except:
<some diffrent error occured, do somethnig more diffrent>
If you simply do nothing and continue, you can use pass, like:
try:
<do something>
except:
pass
The part you're missing is pass. That is a simple no-op expression, which exists because Python can have no empty blocks.
The longer explanation:
What you need to do is to catch the IOError exception being thrown, and ignore it (possibly logging it and such) using pass.
To do that you need to wrap the code that can fail in a try and except block, like this:
try:
<code that can fail>
except IOError:
pass
What this does is explicitly ignore IOErrors, while not ignoring others. If you want to ignore all exceptions, simply remove the IOError part so the line says except:.
You should really read the Python tutorial, specifically the section about error handling.
Try this:
try:
<do something - you code here>
except IOError: pass
Or why not:
with ignored(IOError):
<code that can fail>
Related
Is it possible to tell if there was an exception once you're in the finally clause? Something like:
try:
funky code
finally:
if ???:
print('the funky code raised')
I'm looking to make something like this more DRY:
try:
funky code
except HandleThis:
# handle it
raised = True
except DontHandleThis:
raised = True
raise
else:
raised = False
finally:
logger.info('funky code raised %s', raised)
I don't like that it requires to catch an exception, which you don't intend to handle, just to set a flag.
Since some comments are asking for less "M" in the MCVE, here is some more background on the use-case. The actual problem is about escalation of logging levels.
The funky code is third party and can't be changed.
The failure exception and stack trace does not contain any useful diagnostic information, so using logger.exception in an except block is not helpful here.
If the funky code raised then some information which I need to see has already been logged, at level DEBUG. We do not and can not handle the error, but want to escalate the DEBUG logging because the information needed is in there.
The funky code does not raise, most of the time. I don't want to escalate logging levels for the general case, because it is too verbose.
Hence, the code runs under a log capture context (which sets up custom handlers to intercept log records) and some debug info gets re-logged retrospectively:
try:
with LogCapture() as log:
funky_code() # <-- third party badness
finally:
# log events are buffered in memory. if there was an exception,
# emit everything that was captured at a WARNING level
for record in log.captured:
if <there was an exception>:
log_fn = mylogger.warning
else:
log_fn = getattr(mylogger, record.levelname.lower())
log_fn(record.msg, record.args)
Using a contextmanager
You could use a custom contextmanager, for example:
class DidWeRaise:
__slots__ = ('exception_happened', ) # instances will take less memory
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# If no exception happened the `exc_type` is None
self.exception_happened = exc_type is not None
And then use that inside the try:
try:
with DidWeRaise() as error_state:
# funky code
finally:
if error_state.exception_happened:
print('the funky code raised')
It's still an additional variable but it's probably a lot easier to reuse if you want to use it in multiple places. And you don't need to toggle it yourself.
Using a variable
In case you don't want the contextmanager I would reverse the logic of the trigger and toggle it only in case no exception has happened. That way you don't need an except case for exceptions that you don't want to handle. The most appropriate place would be the else clause that is entered in case the try didn't threw an exception:
exception_happened = True
try:
# funky code
except HandleThis:
# handle this kind of exception
else:
exception_happened = False
finally:
if exception_happened:
print('the funky code raised')
And as already pointed out instead of having a "toggle" variable you could replace it (in this case) with the desired logging function:
mylog = mylogger.WARNING
try:
with LogCapture() as log:
funky_code()
except HandleThis:
# handle this kind of exception
else:
# In case absolutely no exception was thrown in the try we can log on debug level
mylog = mylogger.DEBUG
finally:
for record in log.captured:
mylog(record.msg, record.args)
Of course it would also work if you put it at the end of your try (as other answers here suggested) but I prefer the else clause because it has more meaning ("that code is meant to be executed only if there was no exception in the try block") and may be easier to maintain in the long run. Although it's still more to maintain than the context manager because the variable is set and toggled in different places.
Using sys.exc_info (works only for unhandled exceptions)
The last approach I want to mention is probably not useful for you but maybe useful for future readers who only want to know if there's an unhandled exception (an exception that was not caught in any except block or has been raised inside an except block). In that case you can use sys.exc_info:
import sys
try:
# funky code
except HandleThis:
pass
finally:
if sys.exc_info()[0] is not None:
# only entered if there's an *unhandled* exception, e.g. NOT a HandleThis exception
print('funky code raised')
raised = True
try:
funky code
raised = False
except HandleThis:
# handle it
finally:
logger.info('funky code raised %s', raised)
Given the additional background information added to the question about selecting a log level, this seems very easily adapted to the intended use-case:
mylog = WARNING
try:
funky code
mylog = DEBUG
except HandleThis:
# handle it
finally:
mylog(...)
You can easily assign your caught exception to a variable and use it in the finally block, eg:
>>> x = 1
>>> error = None
>>> try:
... x.foo()
... except Exception as e:
... error = e
... finally:
... if error is not None:
... print(error)
...
'int' object has no attribute 'foo'
Okay, so what it sounds like you actually just want to either modify your existing context manager, or use a similar approach: logbook actually has something called a FingersCrossedHandler that would do exactly what you want. But you could do it yourself, like:
#contextmanager
def LogCapture():
# your existing buffer code here
level = logging.WARN
try:
yield
except UselessException:
level = logging.DEBUG
raise # Or don't, if you just want it to go away
finally:
# emit logs here
Original Response
You're thinking about this a bit sideways.
You do intend to handle the exception - you're handling it by setting a flag. Maybe you don't care about anything else (which seems like a bad idea), but if you care about doing something when an exception is raised, then you want to be explicit about it.
The fact that you're setting a variable, but you want the exception to continue on means that what you really want is to raise your own specific exception, from the exception that was raised:
class MyPkgException(Exception): pass
class MyError(PyPkgException): pass # If there's another exception type, you can also inherit from that
def do_the_badness():
try:
raise FileNotFoundError('Or some other code that raises an error')
except FileNotFoundError as e:
raise MyError('File was not found, doh!') from e
finally:
do_some_cleanup()
try:
do_the_badness()
except MyError as e:
print('The error? Yeah, it happened')
This solves:
Explicitly handling the exception(s) that you're looking to handle
Making the stack traces and original exceptions available
Allowing your code that's going to handle the original exception somewhere else to handle your exception that's thrown
Allowing some top-level exception handling code to just catch MyPkgException to catch all of your exceptions so it can log something and exit with a nice status instead of an ugly stack trace
If it was me, I'd do a little re-ordering of your code.
raised = False
try:
# funky code
except HandleThis:
# handle it
raised = True
except Exception as ex:
# Don't Handle This
raise ex
finally:
if raised:
logger.info('funky code was raised')
I've placed the raised boolean assignment outside of the try statement to ensure scope and made the final except statement a general exception handler for exceptions that you don't want to handle.
This style determines if your code failed. Another approach might me to determine when your code succeeds.
success = False
try:
# funky code
success = True
except HandleThis:
# handle it
pass
except Exception as ex:
# Don't Handle This
raise ex
finally:
if success:
logger.info('funky code was successful')
else:
logger.info('funky code was raised')
If exception happened --> Put this logic in the exception block(s).
If exception did not happen --> Put this logic in the try block after the point in code where the exception can occur.
Finally blocks should be reserved for "cleanup actions," according to the Python language reference. When finally is specified the interpreter proceeds in the except case as follows: Exception is saved, then the finally block is executed first, then lastly the Exception is raised.
I have a try/except where I repeat the except portion frequently in my code. This led me to believe that it would be better to separate the except portion into a function.
Below is my use case:
try:
...
except api.error.ReadError as e:
...
except api.error.APIConnectionError as e:
...
except Exception as e:
...
How would I go about separating this logic into a function so I can do something as simple as:
try:
...
except:
my_error_handling_function(e)
Just define the function:
def my_error_handling(e):
#Do stuff
...and pass in the exception object as the parameter:
try:
#some code
except Exception as e:
my_error_handling(e)
Using just a generic Exception type will allow you to have a single except clause and handle and test for different error types within your handling function.
In order to check for the name of the caught exception, you can get it by doing:
type(e).__name__
Which will print the name, such as ValueError, IOError, etc.
I would suggest refactoring your code so the try/except block is only present in a single location.
For instance, an API class with a send() method, seems like a reasonable candidate for containing the error handling logic you have described in your question.
Define your function:
def my_error_handling(e):
#Handle exception
And do what you're proposing:
try:
...
except Exception as e:
my_error_handling_function(e)
You can handle logic by getting the type of the exception 'e' within your function. See: python: How do I know what type of exception occurred?
If you don't like try-catch statement, you can use exception-decouple package and decorator.
from exception_decouple import redirect_exceptions
def my_error_handling(arg, e):
#Do stuff
#redirect_exceptions(my_error_handling, api.error.ReadError, api.error.APIConnectionError)
def foo(arg):
...
Let's say I have a program that runs continuously, waiting for order from a program with standard input. The method that keeps waiting for order is called "run" using while.
As you see, when run() gets certain order, they pass the order to certain function.
When I run the program, every time I give a command that can cause an error (say: Index error), it breaks and shut down (obviously)
I decided to try to catch the error with try/except
def a(order):
try:
<some algorithm>
return something
except Exception, error:
stderr.write(error)
stderr.flush()
def b(order):
try:
<some algorithm>
return something
except Exception, error:
stderr.write(error)
stderr.flush()
def run(order)
while stdin.notclosed:
try:
read stdin
if stdin==specific order :
x=a(stdin order)
else:
x=b(stdin order)
except Exception,error:
stderr.write(error)
stderr.flush()
run()
However, it seems the program that gives the order can't read the error. From my analyst, it seems the program that gives order only start reading stderr after the program that reads the order ends. However, due to try/catch, the program never ends. Is there anyway that to catch the error, write it, then end it. (The error can came from any function)
PS: Let's assume you can't modify or read the program that gives order. (This is competition, the reason I said this, is since that when I access the stderr, it's empty.)
Not sure if this does what you need, but you could re-raise the exception being handled by adding an emptyraisestatement at the end of theexceptblock as shown below. This will either cause the exception to be handled by the next higher-uptry/exceptblock, if there is one, or terminate the program if there isn't.
Example:
def a(order):
try:
<some algorithm>
return something
except Exception, error:
stderr.write(error)
stderr.flush()
raise # re-raise exception
def a(order):
try:
<some algorithm>
return something
except Exception, error:
import traceback
trace = traceback.format_exc()
return trace
def b(order):
try:
<some algorithm>
return something
except Exception, error:
import traceback
trace = traceback.format_exc()
return trace
def run(order)
while stdin.notclosed:
try:
read stdin
if stdin==specific order :
x=a(stdin order)
else:
x=b(stdin order)
#check if x == trace then sys.exit()
except Exception,error:
stderr.write(error)
stderr.flush()
run()
Dive into Python -
This is a small snippet from fileinfo.py used in the book. This is opening an MP3 file and reading the last 128 bytes to fetch and later parse the metadata.
try:
fsock = open(filename, "rb", 0)
try:
fsock.seek(-128, 2)
tagdata = fsock.read(128)
finally:
fsock.close()
.
. # process tagdata: will NEVER raise IOError though
.
except IOError:
pass
This can be refactored as:
try:
fsock = open(filename, "rb", 0)
try:
fsock.seek(-128, 2)
tagdata = fsock.read(128)
except IOError:
pass
finally:
fsock.close()
.
. # process tagdata
.
I even used to have this question when I was learning Java. Should we just keep the logic that can actually raise an exception inside the try..except block or for the sake of keeping a code that does one particular job in ONE place; keep the other code that will NEVER raise an exception also within a try...except?
The try/finally clause's primary purpose is to close the file regardless of what happens, it doesn't make sense to move it the outer try/except as I assume you are trying to do:
try:
fsock = open(filename, "rb", 0)
try:
fsock.seek(-128, 2)
tagdata = fsock.read(128)
except:
pass
except IOError:
pass
finally:
fsock.close()
The reason being, if IOError is actually raised, calling fsock.close() would raise another exception, since fsock would not have been assigned. Instead of either, it'd be preferable to use the with statement which will automatically close the file for you:
try:
with open(filename, 'rb') as fsock:
fsock.seek(-128, 2)
tagdata = fsock.read(128)
except IOError:
pass
The most accepted standard is to put as little code in the try..except as possible. Reasoning is that you don't know what the other code will raise if there's a ton of code in a try.. then it becomes really messy.
You can see lots of good styling information in PEP 8, amongst which is:
- Additionally, for all try/except clauses, limit the 'try' clause
to the absolute minimum amount of code necessary. Again, this
avoids masking bugs.
Yes:
try:
value = collection[key]
except KeyError:
return key_not_found(key)
else:
return handle_value(value)
No:
try:
# Too broad!
return handle_value(collection[key])
except KeyError:
# Will also catch KeyError raised by handle_value()
return key_not_found(key)
The second piece of code is syntactically invalid, so you should prefer the first form.
If you'd make it syntactically valid by adding an except or finally clause, it would be semantically invalid: if the open fails, you'd still be trying to close fsock, which would not be assigned.
If the open fails then you can't assign to tagdata, so you should not allow the code to reach the point where you process tagdata. Often the best way to handle this is to process the IOError at a higher level (i.e. wrap this in a function and handle it in the calling context).
BTW, in modern Python we don't need to use finally for this sort of thing - we have a more powerful idiom. We also have an else clause that can be attached to try/except blocks that is executed only if the exception handlers are not invoked.
So we get something like:
def get_data():
with open(filename, "rb", 0) as fsock:
fsock.seek(-128, 2)
return fsock.read(128)
def do_processing():
try: tagdata = get_data()
except IOError: handle_error()
else: process(tagdata)
I have code:
try:
print test.qwerq]
try:
print test.sdqwe]
except:
pass
except:
pass
How to print debug info for all errors in nested try ?
Re-raise exceptions.
try:
print test[qwerq]
try:
print test[qwe]
except:
# Do something with the exception.
raise
except:
# Do something here too, just for fun.
raise
It should be noted that in general you don't want to do this. You're better off not catching the exception if you're not going to do anything about it.
If you want to just print the call stack and not crash, look into the traceback module.