I am trying to write a file sharing application that exposes a REST interface.
The library I am using, Flask-RESTful only supports returning JSON by default. Obviously attempting to serve binary data over JSON is not a good idea at all.
What is the most "RESTful" way of serving up binary data through a GET method? It appears possible to extend Flask-RESTful to support returning different data representations besides JSON but the documentation is scarce and I'm not sure if it's even the best approach.
The approach suggested in the Flask-RESTful documentation is to declare our supported representations on the Api object so that it can support other mediatypes. The mediatype we are looking for is application/octet-stream.
First, we need to write a representation function:
from flask import Flask, send_file, safe_join
from flask_restful import Api
app = Flask(__name__)
api = Api(app)
#api.representation('application/octet-stream')
def output_file(data, code, headers):
filepath = safe_join(data["directory"], data["filename"])
response = send_file(
filename_or_fp=filepath,
mimetype="application/octet-stream",
as_attachment=True,
attachment_filename=data["filename"]
)
return response
What this representation function does is to convert the data, code, headers our method returns into a Response object with mimetype application/octet-stream. Here we use send_file function to construct this Response object.
Our GET method can be something like:
from flask_restful import Resource
class GetFile(Resource):
def get(self, filename):
return {
"directory": <Our file directory>,
"filename": filename
}
And that's all the coding we need. When sending this GET request, we need to change the Accept mimetype to Application/octet-stream so that our API will call the representation function. Otherwise it will return the JSON data as by default.
There's an xml example on github
I know this question was asked 7 years ago so it probably doesn't matter any more to #Ayrx. Hope it helps to whoever drops by.
As long as you're setting the Content-Type header accordingly and respecting the Accept header sent by the client, you're free to return any format you want. You can just have a view that returns your binary data with the application/octet-stream content type.
After lot of trials and experiments, including hours of browsing to make the Response class as Single Responsible down loader
class DownloadResource(Resource):
def get(self):
item_list = dbmodel.query.all()
item_list = [item.image for item in item_list]
data = json.dumps({'items':item_list})
response = make_response(data)
response.headers['Content-Type'] = 'text/json'
response.headers['Content-Disposition'] = 'attachment; filename=selected_items.json'
return response
Change your filename and content type to support the format you want.
Related
I'm quite new to apis and i have a problem. I want to use it to pass an image and process it. I thought just changing it to base64 will help and i will just pass a string to endpoint. But problem is that in base64 string there are '/' signs which break my url. Any ideas how i can fix it or there are any better ideas out there?
Code is simple:
app = FastAPI()
#app.get("/get_predictions/{base64_str}")
def get_predictions(base64_str: str):
return get_model_predictions(base64_str)
get_model_predictions just handles the image and return what it has to return
you can use UploadFile to pass files to your endpoint.
Here is an example:
from fastapi import FastAPI, File, UploadFile
#app.post("/uploadfile/")
async def create_upload_file(file: UploadFile = File(...)):
return {"filename": file.filename}
Be careful in my example I use the POST method and not GET.
I allow myself to offer you a solution with POST since you say you want to 'pass' an image, so this is done with a POST route and not GET
You can learn more by looking at this section of the documentation:
https://fastapi.tiangolo.com/tutorial/request-files/
Using get request for file transfer is a bad idea. Firstly you will have a giant url, secondly information you pass in get request may be exposed to other people.
The right solution is to use post request that will hide all the information in the request body. There you can transfer your images not only by using base_64 but also like multipart-formdata.
Here is the documentation to the Fast API post requests: post request or answer how to post an image using Fast API
I have seen Tornado documentations and examples where self.write method is widely used to render some value on HTML, where the POST request was run in a handler. But I could not find much clarity on how to return the response back to client.
For example, I am calling a POST request on a Tornado server from my client. The code that accepts post request is:
class strest(tornado.web.RequestHandler):
def post(self):
value = self.get_argument('key')
cbtp = cbt.main(value)
With this, I can find the value of cbtp and with self.write(cbtp), I can get it printed in HTML. But instead, I want to return this value to the client in JSON format, like {'cbtp':cbtp}
I want to know how to modify my code so that this response is sent to the client, or give me some documentation where this this is fluently explained.
Doing something like
res = {cbtp: cbtp}
return cbtp
throws a BadYieldError: yielded unknown object
You just need to set the output type as JSON and json.dumps your output.
Normally I have the set_default_headers in a parent class called RESTRequestHandler. If you want just one request that is returning JSON you can set the headers in the post call.
class strest(tornado.web.RequestHandler):
def set_default_headers(self):
self.set_header("Content-Type", 'application/json')
def post(self):
value = self.get_argument('key')
cbtp = cbt.main(value)
r = json.dumps({'cbtp': cbtp})
self.write(r)
If the given chunk is a dictionary, we write it as JSON and set the Content-Type of the response to be application/json. (if you want to send JSON as a different Content-Type, call set_header after calling write()).
Using it should give you exactly what you want:
self.write(json.dumps({'cbtp': cbtp}))
I looked many questions similar to my title but I have not found any that had same problem as me yet.
I did requests.post to post JSON to API restful server. Below is the snippet
import requests
def upload_data():
url = "http://127.0.0.1:8088/"
data = {"value":"abc123"}
response = requests.post(url, data=data)
print response.status_code, response.reason, response.text
upload_data()
And for the server side
from flask_restful import Api, Resource
from flask import request
class MyAPI(Resource):
def get():
pass
def post(self):
value = request.data['value']
response_object = {
'value':value
}
return response_object, 201
I was hoping to get the POST function to work by showing the result of 201 Created with
{
'value':'abc123'
}
But whenever I run the script, it gives me error saying that
value = request.data["value"]
TypeError: string indices must be integers, not str
I am sorry if this is a bad question but if anyone could show me what I have been missing in this script, I really appreciate it. Thank you.
That's because request data hasn't been parsed into a python dictionary. Were you perhaps thinking of
data = json.loads(request.data)
However please note that you are not actually posting a JSON body to your flask server. You are posting multipart formdata. So you may probably be looking for the answer posted by luoluo.
One the other hand if you really wanted to deal with json, The correct way to send json looks something like this:
requests.post(url, json=data)
And then the loads as suggested.
The request.data is a string, while request.values is a MultiDict.
You need update your code to :
value = request.values.get('value')
instead of
value = request.data['value']
According to the doc
args
A MultiDict with the parsed contents of the query string. (The part in the URL after the question mark).
form
A MultiDict with the parsed form data from POST or PUT requests. Please keep in mind that file uploads will not end up here, but instead in the files attribute.
values
A CombinedMultiDict with the contents of both form and args.
data
Contains the incoming request data as string in case it came with a mimetype Flask does not handle.
I am speaking to the gmail api and would like to batch the requests. They have a friendly guide for this here, https://developers.google.com/gmail/api/guides/batch, which suggests that I should be able to use multipart/mixed and include different urls.
I am using Python and the Requests library, but am unsure how to issue different urls. Answers like this one How to send a "multipart/form-data" with requests in python? don't mention an option for changing that part.
How do I do this?
Unfortunately, requests does not support multipart/mixed in their API. This has been suggested in several GitHub issues (#935 and #1081), but there are no updates on this for now. This also becomes quite clear if you search for "mixed" in the requests sources and get zero results :(
Now you have several options, depending on how much you want to make use of Python and 3rd-party libraries.
Google API Client
Now, the most obvious answer to this problem is to use the official Python API that Google is providing here. It comes with a HttpBatchRequest class that can handle the batch requests that you need. This is documented in detail in this guide.
Essentially, you create an HttpBatchRequest object and add all your requests to it. The library will then put everything together (taken from the guide above):
batch = BatchHttpRequest()
batch.add(service.animals().list(), callback=list_animals)
batch.add(service.farmers().list(), callback=list_farmers)
batch.execute(http=http)
Now, if for whatever reason you cannot or will not use the official Google libraries you will have to build parts of the request body yourself.
requests + email.mime
As I already mentioned, requests does not officially support multipart/mixed. But that does not mean that we cannot "force" it. When creating a Request object, we can use the files parameter to provide multipart data.
files is a dictionary that accepts 4-tuple values of this format: (filename, file_object, content_type, headers). The filename can be empty. Now we need to convert a Request object into a file(-like) object. I wrote a small method that covers the basic examples from the Google example. It is partly inspired by the internal methods that Google uses in their Python library:
import requests
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
BASE_URL = 'http://www.googleapis.com/batch'
def serialize_request(request):
'''Returns the string representation of the request'''
mime_body = ''
prepared = request.prepare()
# write first line (method + uri)
if request.url.startswith(BASE_URL):
mime_body = '%s %s\r\n' % (request.method, request.url[len(BASE_URL):])
else:
mime_body = '%s %s\r\n' % (request.method, request.url)
part = MIMENonMultipart('application', 'http')
# write headers (if possible)
for key, value in prepared.headers.iteritems():
mime_body += '%s: %s\r\n' % (key, value)
if getattr(prepared, 'body', None) is not None:
mime_body += '\r\n' + prepared.body + '\r\n'
return mime_body.encode('utf-8').lstrip()
This method will transform a requests.Request object into a UTF-8 encoded string that can later be used a a payload for a MIMENonMultipart object, i.e. the different multiparts.
Now in order to generate the actual batch request, we first need to squeeze a list of (Google API) requests into a files dictionary for the requests lib. The following method will take a list of requests.Request objects, transform each into a MIMENonMultipart and then return a dictionary that complies to the structure of the files dictionary:
import uuid
def prepare_requests(request_list):
message = MIMEMultipart('mixed')
output = {}
# thanks, Google. (Prevents the writing of MIME headers we dont need)
setattr(message, '_write_headers', lambda self: None)
for request in request_list:
message_id = new_id()
sub_message = MIMENonMultipart('application', 'http')
sub_message['Content-ID'] = message_id
del sub_message['MIME-Version']
sub_message.set_payload(serialize_request(request))
# remove first line (from ...)
sub_message = str(sub_message)
sub_message = sub_message[sub_message.find('\n'):]
output[message_id] = ('', str(sub_message), 'application/http', {})
return output
def new_id():
# I am not sure how these work exactly, so you will have to adapt this code
return '<item%s:12930812#barnyard.example.com>' % str(uuid.uuid4())[-4:]
Finally, we need to change the Content-Type from multipart/form-data to multipart/mixed and also remove the Content-Disposition and Content-Type headers from each request part. These we generated by requests and cannot be overwritten by the files dictionary.
import re
def finalize_request(prepared):
# change to multipart/mixed
old = prepared.headers['Content-Type']
prepared.headers['Content-Type'] = old.replace('multipart/form-data', 'multipart/mixed')
# remove headers at the start of each boundary
prepared.body = re.sub(r'\r\nContent-Disposition: form-data; name=.+\r\nContent-Type: application/http\r\n', '', prepared.body)
I have tried my best to test this with the Google Example from the Batching guide:
sheep = {
"animalName": "sheep",
"animalAge": "5",
"peltColor": "green"
}
commands = []
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals/pony'))
commands.append(requests.Request('PUT', 'http://www.googleapis.com/batch/farm/v1/animals/sheep', json=sheep, headers={'If-Match': '"etag/sheep"'}))
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals', headers={'If-None-Match': '"etag/animals"'}))
files = prepare_requests(commands)
r = requests.Request('POST', 'http://www.googleapis.com/batch', files=files)
prepared = r.prepare()
finalize_request(prepared)
s = requests.Session()
s.send(prepared)
And the resulting request should be close enough to what Google is providing in their Batching guide:
POST http://www.googleapis.com/batch
Content-Length: 1006
Content-Type: multipart/mixed; boundary=a21beebd15b74be89539b137bbbc7293
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item8065:12930812#barnyard.example.com>
GET /farm/v1/animals
If-None-Match: "etag/animals"
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item5158:12930812#barnyard.example.com>
GET /farm/v1/animals/pony
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item0ec9:12930812#barnyard.example.com>
PUT /farm/v1/animals/sheep
Content-Length: 63
Content-Type: application/json
If-Match: "etag/sheep"
{"animalAge": "5", "animalName": "sheep", "peltColor": "green"}
--a21beebd15b74be89539b137bbbc7293--
In the end, I highly recommend the official Google library but if you cannot use it, you will have to improvise a bit :)
Disclaimer: I havent actually tried to send this request to the Google API Endpoints because the authentication procedure is too much of a hassle. I was just trying to get as close as possible to the HTTP request that is described in the Batching guide. There might be some problems with \r and \n line endings, depending on how strict the Google Endpoints are.
Sources:
requests github (especially issues #935 and #1081)
requests API documentation
Google APIs for Python
What is the correct way to stream JSON response from Django? E.g. use 'StreamingHttpResponse` for the following:
def sample_json_view(self):
data = { ... }
resp = http.HttpResponse(content_type="application/json")
json.dump(data, resp)
return resp
My goal is to use StreamingHttpResponse to minimize the latency in the view. Does StreamingHttpResponse offer benefit over HttpResponse where we are writing?
Or: Does writing to file-like HttpResponse object (like with json.dump) make Django / mod_wsgi to buffer the whole response on the server before starting to stream the response to the client? (increasing latency, all JSON response must be generated first).
This depends on how your data is being generated, and if you need the content rendered before all of the data is generated. The Django docs seem to discourage this, saying "StreamingHttpResponse should only be used in situations where it is absolutely required that the whole content isn’t iterated before transferring the data to the client."
For an example of how to correctly use StreamingHttpResponse, see Django 1.5 - using the new StreamingHttpResponse