I have written a view that decrypts a GPG encrypted file and returns it as plain text. This works fine in general. The problem is, if the file is empty or otherwise contains invalid GPG data, gnupg returns an empty result rather than throw an exception.
I need to be able to do something like this inside decrypt_file to check to see if the decryption failed and raise an error:
if data.ok:
return str(data)
else:
raise APIException(data.status)
If I do this, I see the APIException raised in the Django debug output, but it's not translating to a 500 response to the client. Instead the client gets a 200 response with an empty body. I can raise the APIException in my get method and it sends a 500 response, but there I don't have access to the gnupg error message.
Here is a very simplified version of my view:
from rest_framework.views import APIView
from django.http import FileResponse
from django.core import files
from gnupg import GPG
class FileDownload(APIView):
def decrypt_file(self, file):
gpg = GPG()
data = gpg.decrypt(file.read())
return data
def get(self, request, id, format=None):
f = open('/tmp/foo', 'rb')
file = files.File(f)
return FileResponse(self.decrypt_file(file))
I have read the docs on DRF exception handling here, but it doesn't seem to provide a solution to this problem. I am fairly new to Django and python in general, so it's entirely possible I'm missing something obvious. Helpful advice would be appreciated. Thanks.
If you raise an error in python, every function in the trace re-raise it until someone catch it. You can first declare a exception :
from rest_framework.exceptions import APIException
class ServiceUnavailable(APIException):
status_code = 503
default_detail = 'Service temporarily unavailable, try again later.'
Then in your decrypt_file function, raise this exception if decryption is not successful. You can pass an argument to modify the message. Then, in the get method, you should call decrypt_file function and pass your file as an argument. If any things goes wrong, your function raise that exception and then, get method re-raise it until Django Rest Framework exception handler catch it.
EDIT:
In your decrypt function, do something like this:
from rest_framework.response import Response
def decrypt_file(self, file):
... # your codes
if data.ok:
return str(data)
else:
raise ServiceUnavailable
def get(self, request, id, format=None):
f = open('/tmp/foo', 'rb')
result = decrypt_file(f)
f.close()
return Response({'data': result})
Related
according to werkzeug docs, InternalServerError has original_exception parameter. That looks cool and I'would like to use it in following way to handle exceptions:
#app.errorhandler(Exception)
def handle_errors(err):
return err if isinstance(err, HTTPException) else InternalServerError(original_exception=err)
and log it in #app.after_request method to pack all additional info from request and response, where standard request logging occurs. Unfortunatelly I'm not able to find out how to use stored original_exception. Where is it stored?
When I check sys.exc_info() in #app.after_request, it's also empty (or (None, None, None)).
One way is probably log exceptions directly in #app.errorhandler, request info should be accessible there and response is not ready (because of error) anyways, but what happens with original_exception after storing in InternalServerError???
Thanks for answer, Michal
I found myself here looking for the same thing - how to access the original_exception, but I eventually gave up and settled on traceback to give me the info I was seeking.
This is the key component, capturing the traceback stack as a list:
formatted_lines = traceback.format_exc().splitlines()
Here's my final code in which I extracted the line number of the error and the specific trigger, then passed them to the page:
from werkzeug.exceptions import InternalServerError
import traceback
#app.errorhandler(InternalServerError)
def handle_500(e):
templateData = {
'errorAt' : 'Internal Server Error',
'errorIs' : 'Check ~/www/gunicorn.error for details'
}
try:
formatted_lines = traceback.format_exc().splitlines()
for error_location in formatted_lines:
if 'intvlm8r.py' in error_location:
templateData['errorAt'] = error_location
templateData['errorIs'] = formatted_lines[-1]
except Exception as e:
app.logger.debug(f'handle_500: Unhandled exception: {e}')
return render_template('500_generic.html', **templateData)
I'm using the swagger editor (OpenApi 2) for creating flask apis in python. When you define a model in swagger and use it as a schema for the body of a request, swagger validates the body before handing the control to you in the X_controller.py files.
I want to add some code before that validation happens (for printing logs for debugging purposes). Swagger just prints to stdout errors like the following and they are not useful when you have a lot of fields (I need the key that isn't valid).
https://host/path validation error: False is not of type 'string'
10.255.0.2 - - [20/May/2020:20:20:20 +0000] "POST /path HTTP/1.1" 400 116 "-" "GuzzleHttp/7"
I know tecnically you can remove the validations in swagger and do them manually in your code but I want to keep using this feature, when it works it's awesome.
Any ideas on how to do this or any alternative to be able to log the requests are welcome.
After some time studying the matter this is what I learnt.
First let's take a look at how a python-flask server made with Swagger Editor works.
Swagger Editor generates the server stub through Swagger Codegen using the definition written in Swagger Editor. This server stub returned by codegen uses the framework Connexion on top of flask to handle all the HTTP requests and responses, including the validation against the swagger definition (swagger.yaml).
Connexion is a framework that makes it easy to develop python-flask servers because it has a lot of functionality you'd have to make yourself already built in, like parameter validation. All we need to do is replace (in this case modify) these connexion validators.
There are three validators:
ParameterValidator
RequestBodyValidator
ResponseValidator
They get mapped to flask by default but we can replace them easily in the __main__.py file as we'll see.
Our goal is to replace the default logs and default error response to some custom ones. I'm using a custom Error model and a function called error_response() for preparing error responses, and Loguru for logging the errors (not mandatory, you can keep the original one).
To make the changes needed, looking at the connexion validators code, we can see that most of it can be reused, we only need to modify:
RequestBodyValidator: __call__() and validate_schema()
ParameterValidator: __call__()
So we only need to create two new classes that extend the original ones, and copy and modify those functions.
Be careful when copying and pasting. This code is based on connexion==1.1.15. If your are on a different version you should base your classes on it.
In a new file custom_validators.py we need:
import json
import functools
from flask import Flask
from loguru import logger
from requests import Response
from jsonschema import ValidationError
from connexion.utils import all_json, is_null
from connexion.exceptions import ExtraParameterProblem
from swagger_server.models import Error
from connexion.decorators.validation import ParameterValidator, RequestBodyValidator
app = Flask(__name__)
def error_response(response: Error) -> Response:
return app.response_class(
response=json.dumps(response.to_dict(), default=str),
status=response.status,
mimetype='application/json')
class CustomParameterValidator(ParameterValidator):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __call__(self, function):
"""
:type function: types.FunctionType
:rtype: types.FunctionType
"""
#functools.wraps(function)
def wrapper(request):
if self.strict_validation:
query_errors = self.validate_query_parameter_list(request)
formdata_errors = self.validate_formdata_parameter_list(request)
if formdata_errors or query_errors:
raise ExtraParameterProblem(formdata_errors, query_errors)
for param in self.parameters.get('query', []):
error = self.validate_query_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('path', []):
error = self.validate_path_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('header', []):
error = self.validate_header_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('formData', []):
error = self.validate_formdata_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
return function(request)
return wrapper
class CustomRequestBodyValidator(RequestBodyValidator):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __call__(self, function):
"""
:type function: types.FunctionType
:rtype: types.FunctionType
"""
#functools.wraps(function)
def wrapper(request):
if all_json(self.consumes):
data = request.json
if data is None and len(request.body) > 0 and not self.is_null_value_valid:
# the body has contents that were not parsed as JSON
return error_response(Error(
status=415,
description="Invalid Content-type ({content_type}), JSON data was expected".format(content_type=request.headers.get("Content-Type", ""))
))
error = self.validate_schema(data, request.url)
if error and not self.has_default:
return error
response = function(request)
return response
return wrapper
def validate_schema(self, data, url):
if self.is_null_value_valid and is_null(data):
return None
try:
self.validator.validate(data)
except ValidationError as exception:
description = f'Validation error. Attribute "{exception.validator_value}" return this error: "{exception.message}"'
logger.error(description)
return error_response(Error(
status=400,
description=description
))
return None
Once we have our validators, we have to map them to the flask app (__main__.py) using validator_map:
validator_map = {
'parameter': CustomParameterValidator,
'body': CustomRequestBodyValidator,
'response': ResponseValidator,
}
app = connexion.App(__name__, specification_dir='./swagger/', validator_map=validator_map)
app.app.json_encoder = encoder.JSONEncoder
app.add_api(Path('swagger.yaml'), arguments={'title': 'MyApp'})
If you also need to replace the validator I didn't use in this example, just create a custom child class of ResponseValidator and replace it on the validator_map dictionary in __main__.py.
Connexion docs:
https://connexion.readthedocs.io/en/latest/request.html
Forgive me for repeating an answer first posted at https://stackoverflow.com/a/73051652/1630244
Have you tried the Connexion before_request feature? Here's an example that logs the headers and content before Connexion validates the body:
import connexion
import logging
from flask import request
logger = logging.getLogger(__name__)
conn_app = connexion.FlaskApp(__name__)
#conn_app.app.before_request
def before_request():
for h in request.headers:
logger.debug('header %s', h)
logger.debug('data %s', request.get_data())
For our requirements, I made the following changes
I updated the input validation exception code from 422 to 400.
I also modified the default Json error output.
My issue
My FastAPI generated automatic documentation is sill showing default error code and error message format.
My Question
Is it possible to update the API documentation to reflect my change like the correct error code and the right error output format?
Currently there is no simple way. You have to modify the OpenAPI file as described here. Meaning you have to load the dictionary and remove the references to the error 422. Here is a minimal example:
from fastapi import FastAPI
from fastapi.openapi.utils import get_openapi
from fastapi.exceptions import RequestValidationError
app = FastAPI()
#app.get("/items/")
async def read_items():
return [{"name": "Foo"}]
#app.exception_handler(RequestValidationError)
async def validation_exception_handler(request, exc):
"""Your custom validation exception."""
message =
return JSONResponse(
status_code=400,
content={"message": f"Validation error: {exc}"}
)
def custom_openapi():
if app.openapi_schema:
return app.openapi_schema
openapi_schema = get_openapi()
# look for the error 422 and removes it
for method in openapi_schema["paths"]:
try:
del openapi_schema["paths"][method]["post"]["responses"]["422"]
except KeyError:
pass
app.openapi_schema = openapi_schema
return app.openapi_schema
app.openapi = custom_openapi
The current way to do it is modifying the generated OpenAPI https://fastapi.tiangolo.com/advanced/extending-openapi/
I have the following error handlers:
#api.errorhandler(DatabaseException)
def handle_database_exception(database_error):
return database_error.query_error, 400
#api.errorhandler(ValidationException)
def handle_validation_exception(validation_error):
return {'fields': validation_error.body_error}, 400
These are simple classes:
class ValidationException(Exception):
def __init__(self, body_error=None, missing_files=None):
Exception.__init__(self, "Validation failed")
self.body_error = body_error
if missing_files is not None:
for missing_file in missing_files:
self.body_error[missing_file] = ['required file']
class DatabaseException(Exception):
def __init__(self, details):
Exception.__init__(self, "Database Error")
self.query_error = details
Here's my problem:
If I call raise the DatabaseException in any of my routes, it falls through and I get a 500 template from flask.
The really funny thing is that the ValidationException, which was implemented earlier works just fine.
I went through in detail what's going on, when the ValidationException is raised, it goes through response.py and ends up in the error handler. I'm unfortunately can't comprehend everything that's going on deep inside flask, but in debug, the DatabaseException is certainly went on a different route.
I expect that the error handlers get called. If I raise a DatabaseException in one of my routes, it should get called.
Sorry, my answer was a bit strange. If you want to return a JSON response, you could do it like this.
class ValidationException(Exception):
def __init__(self, body_error=None, missing_files=None):
Exception.__init__(self, "Validation failed")
self.body_error = body_error
if missing_files is not None:
for missing_file in missing_files:
self.body_error[missing_file] = ['required file']
#api.errorhandler(ValidationException)
def handle_validation_exception(validation_error):
from flask import jsonify, make_response
return make_response(jsonify({'fields': validation_error.body_error}), 400)
This way is also possible.
#api.errorhandler(ValidationException)
def handle_validation_exception(validation_error):
return "{'fields': validation_error.body_error}", 400
The DatabaseException works fine and returns 400 Bad Request with a plaintext body.
Have fun ^2.
DatabaseException worked perfectly for me.
In the second solution, you try to return a Python dictionary. I assume that the missing quotes cause the error. Maybe you could use jsonify and make_response.
#api.errorhandler(DatabaseException)
def handle_validation_exception(validation_error):
from flask import jsonify, make_response
return make_response(jsonify({ 'fields': validation_error.query_error }), 400)
Have fun!
I use CherryPy to run a very simple web server. It is intended to process the GET parameters and, if they are correct, do something with them.
import cherrypy
class MainServer(object):
def index(self, **params):
# do things with correct parameters
if 'a' in params:
print params['a']
index.exposed = True
cherrypy.quickstart(MainServer())
For example,
http://127.0.0.1:8080/abcde:
404 Not Found
The path '/abcde' was not found.
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 656, in respond
response.body = self.handler()
File "C:\Python27\lib\site-packages\cherrypy\lib\encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 386, in __call__
raise self
NotFound: (404, "The path '/abcde' was not found.")
Powered by CherryPy 3.2.4
I am trying to catch this exception and show a blank page because the clients do not care about it. Specifically, the result would be an empty body, no matter the url or query string that resulted in an exception.
I had a look at documentation on error handling cherrypy._cperror, but I did not find a way to actually use it.
Note: I gave up using CherryPy and found a simple solution using BaseHTTPServer (see my answer below)
Docs somehow seem to miss this section. This is what I found while looking for detailed explanation for custom error handling from the source code.
Custom Error Handling
Anticipated HTTP responses
The 'error_page' config namespace can be used to provide custom HTML output for
expected responses (like 404 Not Found). Supply a filename from which the
output will be read. The contents will be interpolated with the values
%(status)s, %(message)s, %(traceback)s, and %(version)s using plain old Python
string formatting.
_cp_config = {
'error_page.404': os.path.join(localDir, "static/index.html")
}
Beginning in version 3.1, you may also provide a function or other callable as
an error_page entry. It will be passed the same status, message, traceback and
version arguments that are interpolated into templates
def error_page_402(status, message, traceback, version):
return "Error %s - Well, I'm very sorry but you haven't paid!" % status
cherrypy.config.update({'error_page.402': error_page_402})
Also in 3.1, in addition to the numbered error codes, you may also supply
error_page.default to handle all codes which do not have their own error_page
entry.
Unanticipated errors
CherryPy also has a generic error handling mechanism: whenever an unanticipated
error occurs in your code, it will call
Request.error_response to
set the response status, headers, and body. By default, this is the same
output as
HTTPError(500). If you want to provide
some other behavior, you generally replace "request.error_response".
Here is some sample code that shows how to display a custom error message and
send an e-mail containing the error
from cherrypy import _cperror
def handle_error():
cherrypy.response.status = 500
cherrypy.response.body = [
"<html><body>Sorry, an error occurred</body></html>"
]
sendMail('error#domain.com',
'Error in your web app',
_cperror.format_exc())
#cherrypy.config(**{'request.error_response': handle_error})
class Root:
pass
Note that you have to explicitly set
response.body
and not simply return an error message as a result.
Choose what's most suitable for you: Default Methods, Custom Error Handling.
I don't think you should use BaseHTTPServer. If your app is that simple, just get a lightweight framework (e. g. Flask), even though it might be a bit overkill, OR stay low level but still within the WSGI standard and use a WSGI-compliant server.
CherryPy IS catching your exception. That's how it returns a valid page to the browser with the caught exception.
I suggest you read through all the documentation. I realize it isn't the best documentation or organized well, but if you at least skim through it the framework will make more sense. It is a small framework, but does almost everything you'd expect from a application server.
import cherrypy
def show_blank_page_on_error():
"""Instead of showing something useful to developers but
disturbing to clients we will show a blank page.
"""
cherrypy.response.status = 500
cherrypy.response.body = ''
class Root():
"""Root of the application"""
_cp_config = {'request.error_response': show_blank_page_on_error}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception
See this for the example in the documentation on the page mentioned above for further reference.
You can simply use a try/except clause:
try:
cherrypy.quickstart(MainServer())
except: #catches all errors, including basic python errors
print("Error!")
This will catch every single error. But if you want to catch only cherrypy._cperror:
from cherrypy import _cperror
try:
cherrypy.quickstart(MainServer())
except _cperror.CherryPyException: #catches only CherryPy errors.
print("CherryPy error!")
Hope this helps!
import cherrypy
from cherrypy import HTTPError
def handle_an_exception():
cherrypy.response.status = 500
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
cherrypy.response.body = b'Internal Server Error'
def handle_a_404(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Error page for 404'.encode('UTF-8')
def handle_default(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Default error page: {status}'.encode('UTF-8')
class Root:
"""Root of the application"""
_cp_config = {
# handler for an unhandled exception
'request.error_response': handle_an_exception,
# specific handler for HTTP 404 error
'error_page.404': handle_a_404,
# default handler for any other HTTP error
'error_page.default': handle_default
}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception("an exception")
#cherrypy.expose
def simulate400(self):
raise HTTPError(status=400, message="Bad Things Happened")
cherrypy.quickstart(Root())
Test with:
http://127.0.0.1:8080/
http://127.0.0.1:8080/simulate400
http://127.0.0.1:8080/missing
Though this was the one of the top results when I searched for cherrypy exception handling, accepted answer did not fully answered the question. Following is a working code against cherrypy 14.0.0
# Implement handler method
def exception_handler(status, message, traceback, version)
# Your logic goes here
class MyClass()
# Update configurations
_cp_config = {"error_page.default": exception_handler}
Note the method signature. Without this signature your method will not get invoked.Following are the contents of method parameters,
status : HTTP status and a description
message : Message attached to the exception
traceback : Formatted stack trace
version : Cherrypy version
Maybe you could use a 'before_error_response' handler from cherrypy.tools
#cherrypy.tools.register('before_error_response', priority=90)
def handleexception():
cherrypy.response.status = 500
cherrypy.response.body = ''
And don't forget to enable it:
tools.handleexception.on = True
I gave up using CherryPy and ended up using the follwing code, which solves the issue in a few lines with the standard BaseHTTPServer:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from urlparse import urlparse, parse_qs
class GetHandler(BaseHTTPRequestHandler):
def do_GET(self):
url = urlparse(self.path)
d = parse_qs(url[4])
if 'c' in d:
print d['c'][0]
self.send_response(200)
self.end_headers()
return
server = HTTPServer(('localhost', 8080), GetHandler)
server.serve_forever()