I have a complex service that runs flask queries asynchronously. So the flask app accepts requests and submits them to a queue and returns a handle to the caller. Then an async service picks up these requests and runs them and then submits the response to a data-store. The caller would continuously poll the flask endpoint to check if the data is available. Currently, this asynchronous feature is only available for a single flask endpoint. But I want to extend this to multiple flask endpoints. As such, I am putting in the code that submits the request to the queue in a python decorator. So that this decorator can be applied to any flask endpoint and then it would support this asynchronous feature.
But to achieve this seamlessly, I have the need to setup a custom request context for flask. This is because the flask endpoints use request.args, request.json, jsonify from flask. And the async service just calls the functions associated with the flask endpoints.
I tried using app.test_request_context() but this doesn't allow me to assign to request.json.
with app.test_request_context() as req:
req.request.json = json.dump(args)
The above doesn't work and throws the below error
AttributeError: can't set attribute
How can I achieve this?
Answer is
builder = EnvironBuilder(path='/',
query_string=urllib.urlencode(query_options), method='POST', data=json.dumps(post_payload),
content_type="application/json")
env = builder.get_environ()
with app.request_context(env):
func_to_call(*args, **kwargs)
Related
I have a FastAPI app that serves an ML model. It is deployed on Kubernetes. For best practices Kubernetes recommends implementing a liveliness endpoint in your API that it can probe to see when the application has completed startup, as well as a readiness endpoint to probe to see when the application is ready to start receiving requests.
Currently, I have implemented both the liveliness and readiness endpoints as a single endpoint, which returns a status code of 200 once the ML model has been loaded and the endpoints are available for requests.
This is ok, but ideally, I would like a liveliness endpoint to return 200 once FastAPI's startup has completed, and a readiness endpoint to return 200 once the models have been loaded (takes much longer than the application startup).
FastAPI allows for startup event triggers where I could initiate the loading of the model, but no endpoints become available until the application startup is complete, which will not be complete until the startup events are also complete.
Is there anyway to implement and "post-startup" event in FastAPI where I could initiate the loading of the model?
Here is some simple example code for what I would like to achieve:
from fastapi import FastAPI, Response
from request import PredictionRequest
import model
app = FastAPI()
#app.on_event("post-startup") # not possible
def load_model():
model.load()
#app.get("/live")
def is_live():
return Response(status_code=200)
#app.get("/ready")
def is_ready():
if model.is_loaded():
return Response(status_code=200)
else:
return Response(status_code=409)
#app.post('/predict')
def predict(request: PredictionRequest):
return model.predict(request)
At the moment there are only 2 events: "shutdown" and "startup"
These are a subsection of the ASGI protocol and are implemented by Starlette and available in FastAPI.
In Flask, the request is available to any function that's down the call-path, so it doesn't have to be passed explicitly.
Is there anything similar in FastAPI?
Basically, I want to allow, within the same app, a request to be "real" or "dummy", where the dummy will not actually carry out some actions, just emit them (yes, I know that checking it down the stack is not nice, but I don't have control over all the code).
let's say I have
#app.post("/someurl")
def my_response():
func1()
def func1():
func2()
def func2():
# access some part of the request
I.e. where I don't need to pass the request as a param all the way down to func2.
In Flask, I'd just access the request directly, but I don't know how to do it in FastAPI.
For example, in Flask I could do
def func2():
x = request.my_variable
# do somethinh with x
Request here is local to the specific URL call, so if there are two concurrent execution of the func2 (with whatever URL), they will get the correct request.
FastAPI uses Starlette for this.
(Code from docs)
from fastapi import FastAPI, Request
app = FastAPI()
#app.get("/items/{item_id}")
def read_root(item_id: str, request: Request):
client_host = request.client.host
return {"client_host": client_host, "item_id": item_id}
Reference:
Using Request Directly
Edit:
I do not think this is something FastAPI provides out of the box. Starlette also doesn't seem to, but there is this project which adds context middleware to Starlette which should be easy to integrate into a FastAPI app.
Starlette Context
I provided an answer that may be of help, here. It leverages ContextVars and Starlette middleware (used in FastAPI) to make request object information globally available. It doesn't make the entire request object globally available -- but if you have some specific data you need from the request object, this solution may help!
I am working on an application which is based on GAE with python 2.7.13. What I want to do is that to make a bunch of async API calls inside a handler. Something like that:
class MakeRequests(webapp2.RequestHandler):
def post(self, *v, **kv):
*do an async api call#1*
*do an async api call#2*
*do an async api call#3*
*wait for response from all of above api requests*
*make response in a way like if call#1 failes, make it's expected*
*attributes in response as None, if call#2 succeeds add it's*
*attributes in response etc. This is just an example.*
For that purpose, I have tried libraries like asyncio, grequests, requests and simple-requests, they don't seems to be working because either they are not compatible with with GAE or with python 2.7.13.
Can anyone help me here?
Urlfetch, which is bundled by default with GAE has a way of making asynchronous calls:
from google.appengine.api import urlfetch
def post(self, *v, **kv):
rpcs = []
for url in urls:
rpc = urlfetch.create_rpc()
urlfetch.make_fetch_call(rpc, url)
rpcs.append(rpc)
results = [rpc.get_result() for rpc in rpcs]
# do stuff with results
If, for some reason you don't want to use urlfetch you can parallelize the requests manually by using threading and a synchronized Queue to read the results.
I'm trying to write an HTTP server in python 2.7. I'm trying to use ready-made classes to simplify the job (such as SimpleHTTPServer, BaseHTTPRequestHandler, etc.).
The server should listen for GET requests, and once it gets one - parse the request (the path and the arguments), and interact with an already initialized object (which accesses a DB, counts number of requests, etc.) - let's call it the 'handler', and return a response.
I understand that the RequestHandler class (e.g. BaseHTTPRequestHandler) will be constructed for each request. How can I pass the 'handler' to the handling routines, so that they could call its methods?
Thanks!
Use a framework to further simplify your job. Here is an example in flask:
from flask import Flask
from flask import request
app = Flask(__name__)
your_handler = SomeHandlerClass()
#app.route("/")
def index():
return your_handler.do_something_with(request)
if __name__ == "__main__":
app.run()
request is a proxy object that holds all the incoming request data.
I am implementing an endpoint in my Flask application that receives a collection of HTTP requests, and returns a collection of the corresponding HTTP responses. In order to accomplish this, I need my endpoint to call other endpoints in order to construct the result. However, because Flask is blocking while processing the original request, it cannot process the nested requests and the application gets deadlocked.
Is there any way to issue a request within a request in flask in a way that doesn't result in a deadlock?
I included a segment of my code which I believe should be enough to illustrate the problem without overwhelming you. If you would like to see more of it please let me know and I'll share.
from requests import Session, Request
def split(request):
multipart = request.stream.read()
boundary = request.content_type.split(';')[1]
prefix = ' boundary"'
suffix = '"'
delimiter = '--%s' % boundary[len(prefix)+1:-len(suffix)]
subrequests = [s.lstrip() for s in multipart.split(delimiter)]
for sub in subrequests:
status_line, _, more_lines = sub.partition('\n')
method, path, version = status_line.split()
headers, _, body = more_lines.partition('\n\n')
url = 'http://localhost:3000' + path
return Request(method, url, headers=headers, data=body)
#app.route('/batch', methods=["GET", "POST"])
def batch():
subrequests = split(request)
session = Session()
responses = []
for sub in subrequests:
response.append(s.send(sub.prepare())) # Deadlock!
There are two candidate solutions that I considered which I found to be unsatisfactory:
Don't issue a full request. Instead, just call the function that is mapped to the endpoint of interest (url_for). I am unsatisfied by this approach because the nested requests have their own headers and cookies which are neglected by this approach. Furthermore, code in the 'before_request' and 'after_request' handlers won't get called automatically
Run multiple instances of the application. This will solve the problem, but expose my service to a pretty simple DoS attack. If I have X instances running, All an attacker would need to do is to hit my service with X different requests to cause a deadlock.
Thank you.
Knowing that the internal flask server is not production-ready, when using only for development, pass the threaded=true parameter to app.run.
app.run(debug=True, threaded=True)
This happens cause you're using the flask devserver. It's not for production use.
In production environment you would use an application server (uWSGI, GUnicorn, Tornado, ...) with or without a webserver layer (NGINX, Apache,...) to proxy/balance connections to the workers protecting (not completely but in a lot of environments it's acceptable) from DoS attacks.