according to werkzeug docs, InternalServerError has original_exception parameter. That looks cool and I'would like to use it in following way to handle exceptions:
#app.errorhandler(Exception)
def handle_errors(err):
return err if isinstance(err, HTTPException) else InternalServerError(original_exception=err)
and log it in #app.after_request method to pack all additional info from request and response, where standard request logging occurs. Unfortunatelly I'm not able to find out how to use stored original_exception. Where is it stored?
When I check sys.exc_info() in #app.after_request, it's also empty (or (None, None, None)).
One way is probably log exceptions directly in #app.errorhandler, request info should be accessible there and response is not ready (because of error) anyways, but what happens with original_exception after storing in InternalServerError???
Thanks for answer, Michal
I found myself here looking for the same thing - how to access the original_exception, but I eventually gave up and settled on traceback to give me the info I was seeking.
This is the key component, capturing the traceback stack as a list:
formatted_lines = traceback.format_exc().splitlines()
Here's my final code in which I extracted the line number of the error and the specific trigger, then passed them to the page:
from werkzeug.exceptions import InternalServerError
import traceback
#app.errorhandler(InternalServerError)
def handle_500(e):
templateData = {
'errorAt' : 'Internal Server Error',
'errorIs' : 'Check ~/www/gunicorn.error for details'
}
try:
formatted_lines = traceback.format_exc().splitlines()
for error_location in formatted_lines:
if 'intvlm8r.py' in error_location:
templateData['errorAt'] = error_location
templateData['errorIs'] = formatted_lines[-1]
except Exception as e:
app.logger.debug(f'handle_500: Unhandled exception: {e}')
return render_template('500_generic.html', **templateData)
Related
I am just getting into aws lambda functions and have written a function that fetches some data from a dynamodb table.
This is the function:
import boto3
from boto3.dynamodb.conditions import Key, Attr
import botocore.exceptions
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
dynamodb=boto3.resource('dynamodb')
appointmentsTable = dynamodb.Table('Appointments')
class NotFoundError(Exception):
pass
def lambda_handler(event, context):
try:
logger.info(f'event: {event}')
bookedAppointments = fetchAppointments(event)
logger.info(f'Response: {bookedAppointments}')
return sendResponse(True , 200 , 'Appointments found' , bookedAppointments)
except NotFoundError:
return sendResponse(True , 400, 'No booked appointments Found' , [])
except Exception as error:
return sendResponse(False , 500 , 'Error in fetch booked appointments' , str(error))
def sendResponse(success , statusCode , message , responseData):
return {
'success' : success,
'statusCode' : statusCode,
'message': message,
'responseData' : responseData
}
def fetchAppointments(event):
consId = event.get('consId')
try:
bookedAppointments = appointmentsTable.query(
IndexName = 'consId-index',
KeyConditionExpression = Key('consId').eq(consId),
FilterExpression = 'booked=:b',
ExpressionAttributeValues = {
':b' : True
}
)
except botocore.exceptions.ClientError as error:
logger.exception(f'Error in fetchAppointments function: {error}')
raise error
if bookedAppointments.get('Items') == []:
raise NotFoundError
sortedResult = sortResult(bookedAppointments.get('Items'))
return sortedResult
def sortResult(listTobeSorted):
return sorted(listTobeSorted , key=lambda k: (k['appointmentDate'] , k['appointmentSlot']))
I know wrapping all of your code logic in a try-catch block is bad practise , so is there a better way to handle any exceptions that may occur in the fetchAppointments function?
There are different ways to solve issues like this, but the underlying truth is: it always depends on what you want to do. So the following advice might be ok for this specific case, but might not be ok for other uses cases. So take the following with a grain of salt.
That said, the first thing to analyse is the different outputs of your fetchAppointments() function:
One or more appointments found
No appointment found
Some application error/exception
Coming to your question: should you use exception handling for this? Yes, but I would not use it as much as you did.
I would only use it for the "proper" exception in this case and the rest should be covered by "normal" application logic.
Consider the following simplified version of your handler:
def lambda_handler(event, context):
try:
appointments = fetchAppointments(event)
except Exception as error:
return sendResponse(False, 500, 'Error in fetch booked appointments', str(error))
if not appointments:
return sendResponse(True, 400, 'No booked appointments Found', [])
return sendResponse(True, 200, 'Appointments found', appointments)
As you can see, only the "real" application exception is handled with exception handling and the rest just uses "normal" logic to figure out, if there are appointments or not.
Code quality is certainly a good thing we don't have to start from scratch IMO, In my experience what I learned is
it should work first and then it should look beautiful
But in Cloud, there is more to it, it should be optimized in terms of cost as well.
So there you already started on the right path already by handling the exception
If you don't handle the exception and let the function fail, by default lambda would try to execute it 3 times
So you would be charged for those executions. You can customize this behaviour
As far as handling the exception goes #Jens explained it real good.
I have written a view that decrypts a GPG encrypted file and returns it as plain text. This works fine in general. The problem is, if the file is empty or otherwise contains invalid GPG data, gnupg returns an empty result rather than throw an exception.
I need to be able to do something like this inside decrypt_file to check to see if the decryption failed and raise an error:
if data.ok:
return str(data)
else:
raise APIException(data.status)
If I do this, I see the APIException raised in the Django debug output, but it's not translating to a 500 response to the client. Instead the client gets a 200 response with an empty body. I can raise the APIException in my get method and it sends a 500 response, but there I don't have access to the gnupg error message.
Here is a very simplified version of my view:
from rest_framework.views import APIView
from django.http import FileResponse
from django.core import files
from gnupg import GPG
class FileDownload(APIView):
def decrypt_file(self, file):
gpg = GPG()
data = gpg.decrypt(file.read())
return data
def get(self, request, id, format=None):
f = open('/tmp/foo', 'rb')
file = files.File(f)
return FileResponse(self.decrypt_file(file))
I have read the docs on DRF exception handling here, but it doesn't seem to provide a solution to this problem. I am fairly new to Django and python in general, so it's entirely possible I'm missing something obvious. Helpful advice would be appreciated. Thanks.
If you raise an error in python, every function in the trace re-raise it until someone catch it. You can first declare a exception :
from rest_framework.exceptions import APIException
class ServiceUnavailable(APIException):
status_code = 503
default_detail = 'Service temporarily unavailable, try again later.'
Then in your decrypt_file function, raise this exception if decryption is not successful. You can pass an argument to modify the message. Then, in the get method, you should call decrypt_file function and pass your file as an argument. If any things goes wrong, your function raise that exception and then, get method re-raise it until Django Rest Framework exception handler catch it.
EDIT:
In your decrypt function, do something like this:
from rest_framework.response import Response
def decrypt_file(self, file):
... # your codes
if data.ok:
return str(data)
else:
raise ServiceUnavailable
def get(self, request, id, format=None):
f = open('/tmp/foo', 'rb')
result = decrypt_file(f)
f.close()
return Response({'data': result})
If i have some bad authorization data (for example wrong password) SUDS rises exception (400, u'Bad Request') from which i cant get anything, but in teh log is response, which contains data that password is wrong, but how to get this response? I tried like this:
except Exception as e:
print str(e)
print self._client.last_received()
It prints:
(400, u'Bad Request')
None
But in log there is long xml which contains <SOAP-ENV:Reason><SOAP-ENV:Text xml:lang="en">Sender not authorized</SOAP-ENV:Text></SOAP-ENV:Reason>
I am pulling this out of a comment and into an answer because of the code block.
import suds.client
try:
auth_url = "https://url.to.my.service/authenticator?wsdl"
auth_client = suds.client.Client(auth_url)
cookie = auth_client.service.authenticate(user,password)
except Exception as e:
print str(e)
print auth_client.last_received()
Using this code, I receive the appropriate response from my service if I pass an invalid password:
Server raised fault: 'error.pwd.incorrect'
None
And an appropriate response if I pass an invalid user id:
Server raised fault: 'error.uid.missing'
None
Something you may want to consider doing, is changing your except statement to catch suds.WebFault instead of the generic exception. There may be something else that is occurring and triggering your exception block.
One other thing that may help with your issue, is to pass faults=True in your Client() call.
The Client can be configured to throw web faults as WebFault or to
return a tuple (, )
The code I posted above would look like this:
auth_client = suds.client.Client(auth_url, faults=True)
I use CherryPy to run a very simple web server. It is intended to process the GET parameters and, if they are correct, do something with them.
import cherrypy
class MainServer(object):
def index(self, **params):
# do things with correct parameters
if 'a' in params:
print params['a']
index.exposed = True
cherrypy.quickstart(MainServer())
For example,
http://127.0.0.1:8080/abcde:
404 Not Found
The path '/abcde' was not found.
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 656, in respond
response.body = self.handler()
File "C:\Python27\lib\site-packages\cherrypy\lib\encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 386, in __call__
raise self
NotFound: (404, "The path '/abcde' was not found.")
Powered by CherryPy 3.2.4
I am trying to catch this exception and show a blank page because the clients do not care about it. Specifically, the result would be an empty body, no matter the url or query string that resulted in an exception.
I had a look at documentation on error handling cherrypy._cperror, but I did not find a way to actually use it.
Note: I gave up using CherryPy and found a simple solution using BaseHTTPServer (see my answer below)
Docs somehow seem to miss this section. This is what I found while looking for detailed explanation for custom error handling from the source code.
Custom Error Handling
Anticipated HTTP responses
The 'error_page' config namespace can be used to provide custom HTML output for
expected responses (like 404 Not Found). Supply a filename from which the
output will be read. The contents will be interpolated with the values
%(status)s, %(message)s, %(traceback)s, and %(version)s using plain old Python
string formatting.
_cp_config = {
'error_page.404': os.path.join(localDir, "static/index.html")
}
Beginning in version 3.1, you may also provide a function or other callable as
an error_page entry. It will be passed the same status, message, traceback and
version arguments that are interpolated into templates
def error_page_402(status, message, traceback, version):
return "Error %s - Well, I'm very sorry but you haven't paid!" % status
cherrypy.config.update({'error_page.402': error_page_402})
Also in 3.1, in addition to the numbered error codes, you may also supply
error_page.default to handle all codes which do not have their own error_page
entry.
Unanticipated errors
CherryPy also has a generic error handling mechanism: whenever an unanticipated
error occurs in your code, it will call
Request.error_response to
set the response status, headers, and body. By default, this is the same
output as
HTTPError(500). If you want to provide
some other behavior, you generally replace "request.error_response".
Here is some sample code that shows how to display a custom error message and
send an e-mail containing the error
from cherrypy import _cperror
def handle_error():
cherrypy.response.status = 500
cherrypy.response.body = [
"<html><body>Sorry, an error occurred</body></html>"
]
sendMail('error#domain.com',
'Error in your web app',
_cperror.format_exc())
#cherrypy.config(**{'request.error_response': handle_error})
class Root:
pass
Note that you have to explicitly set
response.body
and not simply return an error message as a result.
Choose what's most suitable for you: Default Methods, Custom Error Handling.
I don't think you should use BaseHTTPServer. If your app is that simple, just get a lightweight framework (e. g. Flask), even though it might be a bit overkill, OR stay low level but still within the WSGI standard and use a WSGI-compliant server.
CherryPy IS catching your exception. That's how it returns a valid page to the browser with the caught exception.
I suggest you read through all the documentation. I realize it isn't the best documentation or organized well, but if you at least skim through it the framework will make more sense. It is a small framework, but does almost everything you'd expect from a application server.
import cherrypy
def show_blank_page_on_error():
"""Instead of showing something useful to developers but
disturbing to clients we will show a blank page.
"""
cherrypy.response.status = 500
cherrypy.response.body = ''
class Root():
"""Root of the application"""
_cp_config = {'request.error_response': show_blank_page_on_error}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception
See this for the example in the documentation on the page mentioned above for further reference.
You can simply use a try/except clause:
try:
cherrypy.quickstart(MainServer())
except: #catches all errors, including basic python errors
print("Error!")
This will catch every single error. But if you want to catch only cherrypy._cperror:
from cherrypy import _cperror
try:
cherrypy.quickstart(MainServer())
except _cperror.CherryPyException: #catches only CherryPy errors.
print("CherryPy error!")
Hope this helps!
import cherrypy
from cherrypy import HTTPError
def handle_an_exception():
cherrypy.response.status = 500
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
cherrypy.response.body = b'Internal Server Error'
def handle_a_404(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Error page for 404'.encode('UTF-8')
def handle_default(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Default error page: {status}'.encode('UTF-8')
class Root:
"""Root of the application"""
_cp_config = {
# handler for an unhandled exception
'request.error_response': handle_an_exception,
# specific handler for HTTP 404 error
'error_page.404': handle_a_404,
# default handler for any other HTTP error
'error_page.default': handle_default
}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception("an exception")
#cherrypy.expose
def simulate400(self):
raise HTTPError(status=400, message="Bad Things Happened")
cherrypy.quickstart(Root())
Test with:
http://127.0.0.1:8080/
http://127.0.0.1:8080/simulate400
http://127.0.0.1:8080/missing
Though this was the one of the top results when I searched for cherrypy exception handling, accepted answer did not fully answered the question. Following is a working code against cherrypy 14.0.0
# Implement handler method
def exception_handler(status, message, traceback, version)
# Your logic goes here
class MyClass()
# Update configurations
_cp_config = {"error_page.default": exception_handler}
Note the method signature. Without this signature your method will not get invoked.Following are the contents of method parameters,
status : HTTP status and a description
message : Message attached to the exception
traceback : Formatted stack trace
version : Cherrypy version
Maybe you could use a 'before_error_response' handler from cherrypy.tools
#cherrypy.tools.register('before_error_response', priority=90)
def handleexception():
cherrypy.response.status = 500
cherrypy.response.body = ''
And don't forget to enable it:
tools.handleexception.on = True
I gave up using CherryPy and ended up using the follwing code, which solves the issue in a few lines with the standard BaseHTTPServer:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from urlparse import urlparse, parse_qs
class GetHandler(BaseHTTPRequestHandler):
def do_GET(self):
url = urlparse(self.path)
d = parse_qs(url[4])
if 'c' in d:
print d['c'][0]
self.send_response(200)
self.end_headers()
return
server = HTTPServer(('localhost', 8080), GetHandler)
server.serve_forever()
In my GAE application, I have several request handlers that return JSON-formated response. When one of these is called, if an error occurs (exception, or programming error), the output is not JSON: it is the stack trace.
What I need is:
Output without error:
{
"foo" : 1
"bar" : 2
"status" : "OK"
}
Output when an error occurs:
{
"status" : "ERR"
"errorMessage" : "An error occurred!"
}
My question is: What is the best practice to make sure that, in any case, the output will be a JSON-formated response? Of course, a common solution for every request handlers would be great.
Any help would be appreciated.
Sure - use the ereporter class (described here: https://stackoverflow.com/a/4296664/336505), but create a custom BaseHandler that formats your uncaught exceptions as JSON output:
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(etc, etc) # format the exception
If an error occurs, to avoid getting a stack trace or other ugly output, you will need to employ a try ... except: http://docs.python.org/tutorial/errors.html
e.g.
try:
# ... your code ...
except TypeError as e:
# ... do something with this error type
except:
# ... generic exception catchall
# output that JSON response