My django view goes through a list , uses regex to detect specific elements in the list and finally returns a dict of the contents.
Both IndexError and ValueError can occur while parsing the list.
I need to handle the exceptions in this case.I tried like this
def parse_list(oldlist):
try:
newlist=create_newlist(oldlist)
except Exception as e:
logger.debug(str(e))
else:
mydict = make_dict(newlist)
def create_newlist(oldlist):
mydict = {}
for elem in oldlist:
if re.match('somepattern',elem[0]):
mydict['somekey']=int(elem[0].strip())
else:
raise ValueError(elem[0],' expects an integer')
...
return mydict
Is using the Exception class in except Exception as e: the correct way to handle any exception originating from the above function?
when I wrote a unit test method
def test_me(self):
dl = parse_list(self.somelist)
print 'dl=\n',dl
self.assertTrue(len(dl),0)
I got the console output as
ERROR: test_me (myapp.tests.ParseTest)
..
IndexError: list index out of range
Why is the exception not logged by logger?
Is using the Exception class in except Exception as e: the correct way
to handle any exception originating from the above function?
When handling exceptions you should try to be as specific as possible. In your case, you should catch IndexError and ValueError instead of a general Exception:
try:
...
except (ValueError, IndexError) as e:
...
Your other question:
Why is the exception not logged by logger?
It depends on the configuration of the logger. You're printing a 'debug' message but it could be set to only log/display messages with level 'error' or higher. See logging documentation in Python for more information.
Related
May be a little confused on how flask handles errors. Working on adding error handling to a DB backend. I essentially have a worker script, a router, and error handling script.
The error handling is pretty basic. I have a separate module with the following classes. They inherit from exception so there is not much added to them
class EmptyQueryError(Exception):
pass
class BadURLError(Exception):
pass
My main route method is as follows, and calls a method which takes care of post process/etc.
def queryJobData(projectId, jobId):
try:
dbv.jobData(projectId, jobId)
except EmptyQueryError as e:
abort(400, "An unexpected Exception Occurred: {}".format(e))
except Exception as e:
abort(400, "An unexpected Exception Occurred: {}".format(e))
Lastly, the main driver function is contained in the dbv object from about. The flow of this works properly, as for when I pass in valued project and job ids, the query is successful and it returns a document as I aim to. However, when I purposely insert errors to get an exception to raise, that's where issues occur. Here is the handler function:
def jobData(self, projectId, jobId):
try:
print("[Querying the job data]")
if (request.method == "GET"):
qData = [x for x in self.collection.find({"projectId": projectId, "jobId": jobId})]
# input(qData)
if (len(qData) == 0):
raise EmptyQueryError("EmptyQueryError: The url you attempted to query did not return any data")
pp.data = qData
unrolled_data = pp.unrollData()
df = pd.DataFrame(unrolled_data)
pps = PostProcessSummary(df)
table_test = pps.pivot_summary()
return dumps(table_test)
except Exception as e:
print(e)
finally:
pass
I purposely did not import the "request" module so it raises an error. I can see it gets caught by "jobData":
However, it never enters one of the exception blocks in "queryJobData", where the handle "jobData" is called.
To say the least this has thrown me a for a loop. Almost all other pieces of software I've built would handle this exception accordingly. (ie it follows the pattern where if one exception is raised elsewhere, it should be handled by the parent calling the child generating the exception). First time using Flask so I imagine I'm missing something obvious I can't find in documentation.
edit:
The exception in jobData() gets caught and it exists back into queryJobData() as if nothing happens. For instance in this block it goes directly to the return and not to handle the raised exception
try:
dbv_ret = dbv.jobData(projectId, jobId)
return dbv_ret
except TypeError:
abort(400, "TypeError Occurred")
except EmptyQueryError:
print("[ARE WE HERE?!]")
abort(404, "Empty Query")
except BadURLError:
abort(404, "Bad URL")
except Exception as e:
abort(404, "An unexptec exception has occurred: {}".format(e))
You a catching all exceptions in you function, so anything that happens inside the try/except block will be caught inside your except block, and then your finally block will be executed.
If you want to pass the EmptyQueryError and BadURLError exceptions to the calling function, raise it outside the try/except block. Or, if you want, re-raise it inside your except block
class EmptyQueryError(Exception):
pass
class BadURLError(Exception):
pass
def jobData():
try:
print("[Querying the job data]")
# this will be caught by the except block
raise EmptyQueryError("EmptyQueryError: The url you attempted to query did not return any data")
except Exception as e:
# catching all exceptions. Including the ones you are raising.
# if you don't re-raise the exception here, no error will be passed
print("Wow, exception")
print(e)
finally:
print("I'm in finally block ;)")
# not returning anything
pass
if __name__ == "__main__":
try:
returned_value = jobData()
# will print None
print(returned_value)
except EmptyQueryError as query_err:
# will never be caught
print("Got a EmptyQueryError")
except BadURLError as url_err:
# will never be caught
print("Got a BadURLError")
The sequence of prints will be:
[Querying the job data]
Wow, exception
EmptyQueryError: The url you attempted to query did not return any data
I'm in finally block ;)
None
You can do something like:
def jobData(data):
if len(data) == 0:
raise EmptyQueryError("EmptyQueryError: The url you attempted to query did not return any data")
try:
# do your thing here
pass
except Exception as e:
print(e)
finally:
print("I'm in finally block ;)")
# not returning anything
pass
Now the EmptyQueryError will be caught by the calling function.
I have a try/except where I repeat the except portion frequently in my code. This led me to believe that it would be better to separate the except portion into a function.
Below is my use case:
try:
...
except api.error.ReadError as e:
...
except api.error.APIConnectionError as e:
...
except Exception as e:
...
How would I go about separating this logic into a function so I can do something as simple as:
try:
...
except:
my_error_handling_function(e)
Just define the function:
def my_error_handling(e):
#Do stuff
...and pass in the exception object as the parameter:
try:
#some code
except Exception as e:
my_error_handling(e)
Using just a generic Exception type will allow you to have a single except clause and handle and test for different error types within your handling function.
In order to check for the name of the caught exception, you can get it by doing:
type(e).__name__
Which will print the name, such as ValueError, IOError, etc.
I would suggest refactoring your code so the try/except block is only present in a single location.
For instance, an API class with a send() method, seems like a reasonable candidate for containing the error handling logic you have described in your question.
Define your function:
def my_error_handling(e):
#Handle exception
And do what you're proposing:
try:
...
except Exception as e:
my_error_handling_function(e)
You can handle logic by getting the type of the exception 'e' within your function. See: python: How do I know what type of exception occurred?
If you don't like try-catch statement, you can use exception-decouple package and decorator.
from exception_decouple import redirect_exceptions
def my_error_handling(arg, e):
#Do stuff
#redirect_exceptions(my_error_handling, api.error.ReadError, api.error.APIConnectionError)
def foo(arg):
...
I have:
MY_PATH_DIR = 'path/to/my/json/file.json'
try:
with open(MY_PATH_DIR, 'r') as f:
MY_PATH_DIR = json.load(f)
except IOError, RuntimeError, ValueError:
pass
except PermissionDenied:
pass
And I want to catch all possible errors. With
IOError - I am catching errors when the file doesn't exist or has a
syntax error (non valid JSON).
RuntimeError - couldn't test it but I think that makes sense from the
documentation in case of an unexpected error
ValueError - I got from here in case nothing got returned
PermissionDenied - is a specific Django error
Are there any other Exceptions that would make sense? I'm not sure if OSError makes sense here. I think that would be raised earlier, right?
The purpose of capturing exceptions is to control the program's behavior when something bad happened, but in an expected way. If you are not even sure what would cause that exception happen, capturing it would only swallow the underlying programming errors you might have.
I wouldn't add as many kinds of exception as possible to that single block of code, you should only add what you care about. To take it to extreme, each line of code would yield certain exceptions but for obvious reason you couldn't do try except for all of them.
Edit:
For the sake of correctness, since you mentioned I don't want my code to break in any case, you could simply do:
try:
# json.load
except Exception as e:
print "Let's just ignore all exceptions, like this one: %s" % str(e)
This is would give you what exception happens as output.
import random
import sys
def main():
"""Demonstrate the handling of various kinds of exceptions."""
# This is like what you are doing in your code.
exceptions = IOError, RuntimeError, ValueError
try:
raise random.choice(exceptions)()
except exceptions as error:
print('Currently handling:', repr(error))
# The following is not much different from Shang Wang's answer.
try:
raise random.choice(exceptions)()
except Exception as error:
print('Currently handling:', repr(error))
# However, the following code will sometimes not handle the exception.
exceptions += SystemExit, KeyboardInterrupt, GeneratorExit
try:
raise random.choice(exceptions)()
except Exception as error:
print('Currently handling:', repr(error))
# The code can be slightly altered to take the new errors into account.
try:
raise random.choice(exceptions)()
except BaseException as error:
print('Currently handling:', repr(error))
# This does not take into account classes not in the exception hierarchy.
class Death:
pass
try:
raise Death()
except BaseException as error:
print('Currently handling:', repr(error))
# If your version of Python does not consider raising an exception from an
# instance of a class not derived from the BaseException class, the way to
# get around this problem would be with the following code instead.
try:
raise Death()
except:
error = sys.exc_info()[1]
print('Currently handling:', repr(error))
if __name__ == '__main__':
main()
I'm adding logging to some python code that deals with exceptions, in the example below what's the correct syntax for wanting to log exception details (e.g. via logger.exception()) when a TypeError or AttributeError occurs?
try:
...
except (TypeError, AttributeError):
# want to do a logger.exception(x) here but not sure what to use for x
...
raise CustomError("Unable to parse column status)
exception(...) is just a convenience method which takes a message just like the other methods:
def exception(self, msg, *args):
"""
Convenience method for logging an ERROR with exception information.
"""
self.error(msg, exc_info=1, *args)
So you would just use it like
logger.exception("Some error message")
and the logging handler will automatically add the exception information from the current exception. Only use this in an exception handler (i.e. in a except: block)!
If you want the exception details, you need to bind the exception itself to a local variable, like this:
except (TypeError, AttributeError), e:
# e is the Exception object
logger.exception(e)
If you need to do different things based on the type of the exception, then you can catch them separately:
except TypeError, e:
logger.exception('There was a Type Error; details are %s' % e)
# Do something, or raise another exception
except AttributeError, e:
logger.exception('There was an Attribute Error; details are %s' % e)
# Do something, or raise another exception
And if you need more information about the context of the exception itself, look into the sys.exc_info() function; it can get you the traceback, and the details about exactly where the exception occurred.
This question already has answers here:
How can I write a `try`/`except` block that catches all exceptions?
(10 answers)
Closed 1 year ago.
Is it possible to catch any error in Python? I don't care what the specific exceptions will be, because all of them will have the same fallback.
Using except by itself will catch any exception short of a segfault.
try:
something()
except:
fallback()
You might want to handle KeyboardInterrupt separately in case you need to use it to exit your script:
try:
something()
except KeyboardInterrupt:
return
except:
fallback()
There's a nice list of basic exceptions you can catch here. I also quite like the traceback module for retrieving a call stack from the exception. Try traceback.format_exc() or traceback.print_exc() in an exception handler.
try:
# do something
except Exception, e:
# handle it
For Python 3.x:
try:
# do something
except Exception as e:
# handle it
You might want also to look at sys.excepthook:
When an exception is raised and uncaught, the interpreter calls
sys.excepthook with three arguments, the exception class, exception
instance, and a traceback object. In an interactive session this
happens just before control is returned to the prompt; in a Python
program this happens just before the program exits. The handling of
such top-level exceptions can be customized by assigning another
three-argument function to sys.excepthook.
Example:
def except_hook(type, value, tback):
# manage unhandled exception here
sys.__excepthook__(type, value, tback) # then call the default handler
sys.excepthook = except_hook
Quoting the bounty text:
I want to be able to capture ANY exception even weird ones like
keyboard interrupt or even system exit (e.g. if my HPC manger throws
an error) and get a handle to the exception object e, whatever it
might be. I want to process e and custom print it or even send it by
email
Look at the exception hierarchy, you need to catch BaseException:
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
This will capture KeyboardInterrupt, SystemExit, and GeneratorExit, which all inherit from BaseException but not from Exception, e.g.
try:
raise SystemExit
except BaseException as e:
print(e.with_traceback)
Not mentioning the type of exception you want to handle itself does the job.
Try this:
try:
#code in which you expect an exception
except:
#prints the exception occured
if you want to know the type of exception that occurred:
try:
# code in which you expect an exception
except Exception as e:
print(e)
# for any exception to be catched
print(type(e))
# to know the type of exception.
for detailed explanation go trough this
https://www.tutorialspoint.com/python/python_exceptions.htm
# in python 3
# if you want the error
try:
func()
except Exception as e:
exceptionFunc(e)
# if you simply want to know an error occurs
try:
func()
except:
exceptionFunc()
# if you don't even wanna do anything
try:
func()
except:
pass
The following only worked for me (both in PY2 and PY3):
try:
# (Anything that produces any kind of error)
except:
ertype = sys.exc_info()[0] # E.g. <class 'PermissionError'>
description = sys.exc_info()[1] # E.g. [Errno 13] Permission denied: ...
# (Handle as needed )
Built-In Exceptions in Python
Built-In exception classes are divided into Base error classes from which the error classes are defined and Concrete error classes which define exceptions which you are more likely to see time to time.
The more detailed document about the buit-In exception can be found in [https://docs.python.org/3/library/exceptions.html]
Custom Exceptions
It is used to fit your specific application situation. For example, you can create your own exception as RecipeNotValidError as the recipe is not valid in your class for developing a cooking app.
Implementation
class RecipeNotValidError(Exception):
def __init__(self):
self.message = "Your recipe is not valid"
try:
raise RecipeNotValidError
except RecipeNotValidError as e:
print(e.message)
These are custom exceptions that are not defined in the standard library. The steps you can follow to create custom classes are :
Subclass the Exception class.
Create a new Exception class of your choice.
Write your code and use the try...except flow to capture and handle your custom exception.