Is it possible to use an after_request for every route in my api?
My main goal would be to do something like this:
#blueprint.route('/endpoint', methods=['GET'])
#after_request_handler(valid_status_codes=[200, 404])
def endpoint():
return {"hello": "world"}, 200
Then in the after_request_handler I would check if the status_code of the response I receive is in the valid_status_codes list I passed as argument and therefore know if the request was successful or not.
Something like this:
def after_request_handler(valid_status_codes, response):
if response.status_code in valid_status_codes:
# Send request was successful metric event
else:
# Send request was unsuccessful metric event
return response
With this I want to have metrics of every endpoint in my api and know if they start to fail for some reason. Maybe there is some library or an easier way to do this. Thanks.
just use errorhandler + after_request:
from flask import Flask
from werkzeug.exceptions import HTTPException
app = Flask(__name__)
#app.route('/good')
def good():
return 'ok'
#app.route('/bad')
def bad():
raise Exception('bad route')
#app.errorhandler(Exception)
def error(e):
# do here all what you need with errors
if isinstance(e, HTTPException):
print('response code %s' % e.code)
else:
print('undefined error %s' % str(e))
raise e
#app.after_request
def log(response):
# do here all what you need with success responses
print('good response %s' % response.status_code)
return response
run server and open /, /bad, /good:
response code 404
undefined error bad route
good response 200
Related
Details of application:
UI: Angular
Backend: Python Flask (using Swagger)
Database: MongoDB
We have a few backend python methods which will be called from the UI side to do CURD operations on the database.
Each of the methods has a decorator which will check the header information to ensure that only a genuine person can call the methods.
From the UI side when these API's are called, this authorization decorator is not creating any problem and a proper response is returned to the UI (as we are passing the header information also to the request)
But now we are writing unit test cases for the API's. Here each test case will call the backend method and because of the authorization decorator, I am getting errors and not able to proceed. How can I handle this issue?
backend_api.py
--------------
from commonlib.auth import require_auth
#require_auth
def get_records(record_id):
try:
record_details = records_coll.find_one({"_id": ObjectId(str(record_id))})
if record_details is not None:
resp = jsonify({"msg": "Found Record", "data": str(record_details)})
resp.status_code = 200
return resp
else:
resp = jsonify({"msg": "Record not found"})
resp.status_code = 404
return resp
except Exception as ex:
resp = jsonify({"msg": "Exception Occured",'Exception Details': ex}))
resp.status_code = 500
return resp
commonlib/auth.py
-----------------
### some lines of code here
def require_auth(func):
"""
Decorator that can be added to a function to check for authorization
"""
def wrapper(*args, **kwargs):
print(*args,**kwargs)
username = get_username()
security_log = {
'loginId': username,
'securityProtocol': _get_auth_type(),
}
try:
if username is None:
raise SecurityException('Authorization header or cookie not found')
if not is_auth_valid():
raise SecurityException('Authorization header or cookie is invalid')
except SecurityException as ex:
log_security(result='DENIED', message=str(ex))
unauthorized(str(ex))
return func(*args, **kwargs)
return wrapper
test_backend_api.py
-------------------
class TestBackendApi(unittest.TestCase):
### some lines of code here
#mock.patch("pymongo.collection.Collection.find_one", side_effect=[projects_json])
def test_get_records(self, mock_call):
from backend_api import get_records
ret_resp = get_records('61729c18afe7a83268c6c9b8')
final_response = ret_resp.get_json()
message1 = "return response status code is not 200"
self.assertEqual(ret_resp.status_code, 200, message1)
Error snippet :
---------------
E RuntimeError: Working outside of request context.
E
E This typically means that you attempted to use functionality that needed
E an active HTTP request. Consult the documentation on testing for
E information about how to avoid this problem.
I'd like to handle errors using AioHttp and Connexion in my python web apis in the same way Flask does through #app.errorhandler(Exception)
In another words, let's say my services raises SomethingAlreadyExists and I want to return 409 Conflict, rather than add the code below in all my apis:
try:
myservice.create_something(..)
except SomethingAlreadyExists as error: # Repeated code -> DRY
return json_response({"message": str(error)}, status=409)
I'd like to just call the myservice.create_something(..) in the API layer and the error handle would return the 409 for SomethingAlreadyExists exceptions or 404 for SomethingNotFound.
Note:
In Flask land it would be something like below:
import connexion
def create_api_app(version='api'):
connexion_app = connexion.FlaskApp(__name__, specification_dir='../api/')
connexion_app.add_api('openapi.yaml', validate_responses=True)
app = connexion_app.app
# It intercepts the specific exception and returns the respective status_code
#app.errorhandler(InvalidValueException)
def bad_request_handler(error):
return 'Bad Request: {}'.format(str(error)), 400
#app.errorhandler(NotFoundException)
def not_found_handler(error):
return 'Not found: {}'.format(str(error)), 404
#app.errorhandler(AlreadyExistsException)
def conflict_handler(error):
return 'Conflict: {}'.format(str(error)), 409
# my_service.py
def get_model(i):
model = get_model_or_none(id)
if btask is None:
raise NotFoundException(f"Model id:{id} not found.")
...
# api.py
def get_model(id):
model = my_service.get_model(id)
# handle errors not required ;)
return btask.to_dict()
I'd like to do the same in my AioHttp connexion app:
from connexion import AioHttpApp
def create_app():
connexion_app = AioHttpApp(__name__, port=8000, specification_dir="../", only_one_api=True)
connexion_app.add_api("openapi.yaml", pass_context_arg_name="request")
# Do something here.. like
# web.Application(middlewares=[handle_already_exists_errors]) --> doesn't work
# OR
# connex_app.add_error_handler(
# AlreadyExistsException, handle_already_exists_errors) --> doesn't work too
return connexion_app
Cheers and I'll appreciate any help!
Roger
I was digging into the connexion and aiohttp code and figured out a way to do it using middlewares:
import json
from aiohttp.web import middleware
from connexion.lifecycle import ConnexionResponse
from connexion import AioHttpApp
from .exceptions import NotFoundException, AlreadyExistsException
def create_app():
connexion_app = AioHttpApp(
__name__, port=8000, specification_dir="../", only_one_api=True
)
connexion_app.add_api("openapi.yaml", pass_context_arg_name="request")
connexion_app.app.middlewares.append(errors_handler_middleware)
return connexion_app
#middleware
async def errors_handler_middleware(request, handler):
""" Handle standard errors returning response following the connexion style messages"""
try:
response = await handler(request)
return response
except NotFoundException as error:
return json_error(message=str(error), title='Not Found', status_code=404)
except AlreadyExistsException as error:
return json_error(message=str(error), title='Conflict', status_code=409)
def json_error(message, title, status_code):
return ConnexionResponse(
body=json.dumps({
'title': title,
'detail': message,
'status': status_code,
}).encode('utf-8'),
status_code=status_code,
content_type='application/json'
)
I have created a simple flask app that and I'm reading the response from python as:
response = requests.post(url,data=json.dumps(data), headers=headers )
data = json.loads(response.text)
Now my issue is that under certain conditions I want to return a 400 or 500 message response. So far I'm doing it like this:
abort(400, 'Record not found')
#or
abort(500, 'Some error...')
This does print the message on the terminal:
But in the API response I kept getting a 500 error response:
The structure of the code is as follows:
|--my_app
|--server.py
|--main.py
|--swagger.yml
Where server.py has this code:
from flask import render_template
import connexion
# Create the application instance
app = connexion.App(__name__, specification_dir="./")
# read the swagger.yml file to configure the endpoints
app.add_api("swagger.yml")
# Create a URL route in our application for "/"
#app.route("/")
def home():
"""
This function just responds to the browser URL
localhost:5000/
:return: the rendered template "home.html"
"""
return render_template("home.html")
if __name__ == "__main__":
app.run(host="0.0.0.0", port="33")
And main.py has all the function I'm using for the API endpoints.
E.G:
def my_funct():
abort(400, 'Record not found')
When my_funct is called, I get the Record not found printed on the terminal, but not in the response from the API itself, where I always get the 500 message error.
You have a variety of options:
The most basic:
#app.route('/')
def index():
return "Record not found", 400
If you want to access the headers, you can grab the response object:
#app.route('/')
def index():
resp = make_response("Record not found", 400)
resp.headers['X-Something'] = 'A value'
return resp
Or you can make it more explicit, and not just return a number, but return a status code object
from flask_api import status
#app.route('/')
def index():
return "Record not found", status.HTTP_400_BAD_REQUEST
Further reading:
You can read more about the first two here: About Responses (Flask quickstart)
And the third here: Status codes (Flask API Guide)
I like to use the flask.Response class:
from flask import Response
#app.route("/")
def index():
return Response(
"The response body goes here",
status=400,
)
flask.abort is a wrapper around werkzeug.exceptions.abort which is really just a helper method to make it easier to raise HTTP exceptions. That's fine in most cases, but for restful APIs, I think it may be better to be explicit with return responses.
Here's some snippets from a Flask app I wrote years ago. It has an example of a 400 response
import werkzeug
from flask import Flask, Response, json
from flask_restplus import reqparse, Api, Resource, abort
from flask_restful import request
from flask_cors import CORS
app = Flask(__name__)
CORS(app)
api = Api(app)
parser = reqparse.RequestParser()
parser.add_argument('address_to_score', type=werkzeug.datastructures.FileStorage, location='files')
class MissingColumnException(Exception):
pass
class InvalidDateFormatException(Exception):
pass
#api.route('/project')
class Project(Resource):
#api.expect(parser)
#api.response(200, 'Success')
#api.response(400, 'Validation Error')
def post(self):
"""
Takes in an excel file of addresses and outputs a JSON with scores and rankings.
"""
try:
df, input_trees, needed_zones = data.parse_incoming_file(request)
except MissingColumnException as e:
abort(400, 'Excel File Missing Mandatory Column(s):', columns=str(e))
except Exception as e:
abort(400, str(e))
project_trees = data.load_needed_trees(needed_zones, settings['directories']['current_tree_folder'])
df = data.multiprocess_query(df, input_trees, project_trees)
df = data.score_locations(df)
df = data.rank_locations(df)
df = data.replace_null(df)
output_file = df.to_dict('index')
resp = Response(json.dumps(output_file), mimetype='application/json')
resp.status_code = 200
return resp
#api.route('/project/health')
class ProjectHealth(Resource):
#api.response(200, 'Success')
def get(self):
"""
Returns the status of the server if it's still running.
"""
resp = Response(json.dumps('OK'), mimetype='application/json')
resp.status_code = 200
return resp
You can return a tuple with the second element being the status (either 400 or 500).
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello():
return "Record not found", 400
if __name__ == '__main__':
app.run()
Example of calling the API from python:
import requests
response = requests.get('http://127.0.0.1:5000/')
response.text
# 'This is a bad request!'
response.status_code
# 400
I think you're using the abort() function correctly. I suspect the issue here is that an error handler is that is catching the 400 error and then erroring out which causes the 500 error. See here for more info on flask error handling.
As an example, the following would change a 400 into a 500 error:
#app.errorhandler(400)
def handle_400_error(e):
raise Exception("Unhandled Exception")
If you're not doing any error handling, it could be coming from the connexion framework, although I'm not familiar with this framework.
You can simply use #app.errorhandler decorator.
example:
#app.errorhandler(400)
def your_function():
return 'your custom text', 400
I lately started using Flask in one of my projects to provide data via a simple route. So far I return a json file containing the data and some other information. When running my Flask app I see the status code of this request in terminal. I would like to return the status code as a part of my final json file. Is it possible to catch the same code I see in terminal?
Some simple might look like this
from flask import Flask
from flask import jsonify
app = Flask(__name__)
#app.route('/test/<int1>/<int2>/')
def test(int1,int2):
int_sum = int1 + int2
return jsonify({"result":int_sum})
if __name__ == '__main__':
app.run(port=8082)
And in terminal I get:
You are who set the response code (by default 200 on success response), you can't catch this value before the response is emited. But if you know the result of your operation you can put it on the final json.
#app.route('/test/<int1>/<int2>/')
def test(int1, int2):
int_sum = int1 + int2
response_data = {
"result": int_sum,
"sucess": True,
"status_code": 200
}
# make sure the status_code on your json and on the return match.
return jsonify(response_data), 200 # <- the status_code displayed code on console
By the way if you access this endpoint from a request library, on the response object you can find the status_code and all the http refered data plus the json you need.
Python requests library example
import requests
req = requests.get('your.domain/test/3/3')
print req.url # your.domain/test/3/3
print req.status_code # 200
print req.json() # {u'result': 6, u'status_code: 200, u'success': True}
You can send HTTP status code as follow:
#app.route('/test')
def test():
status_code = 200
return jsonify({'name': 'Nabin Khadka'}, status_code) # Notice second element of the return tuple(return)
This way you can control what status code to return to the client (typically to web browser.)
this is a test script to request data from Rovi API, provided by the API itself.
test.py
import requests
import time
import hashlib
import urllib
class AllMusicGuide(object):
api_url = 'http://api.rovicorp.com/data/v1.1/descriptor/musicmoods'
key = 'my key'
secret = 'secret'
def _sig(self):
timestamp = int(time.time())
m = hashlib.md5()
m.update(self.key)
m.update(self.secret)
m.update(str(timestamp))
return m.hexdigest()
def get(self, resource, params=None):
"""Take a dict of params, and return what we get from the api"""
if not params:
params = {}
params = urllib.urlencode(params)
sig = self._sig()
url = "%s/%s?apikey=%s&sig=%s&%s" % (self.api_url, resource, self.key, sig, params)
resp = requests.get(url)
if resp.status_code != 200:
# THROW APPROPRIATE ERROR
print ('unknown err')
return resp.content
from another script I import the module:
from roviclient.test import AllMusicGuide
and create an instance of the class inside a mood function:
def mood():
test = AllMusicGuide()
print (test.get('[moodids=moodids]'))
according to documentation, the following is the syntax for requests:
descriptor/musicmoods?apikey=apikey&sig=sig [&moodids=moodids] [&format=format] [&country=country] [&language=language]
but running the script I get the following error:
unknown err
<h1>Gateway Timeout</h1>:
what is wrong?
"504, try once more. 502, it went through."
Your code is fine, this is a network issue. "Gateway Timeout" is a 504. The intermediate host handling your request was unable to complete it. It made its own request to another server on your behalf in order to handle yours, but this request took too long and timed out. Usually this is because of network congestion in the backend; if you try a few more times, does it sometimes work?
In any case, I would talk to your network administrator. There could be any number of reasons for this and they should be able to help fix it for you.