I'm creating an app using FastAPI that is supposed to generate resized version of uploaded images. The upload should be done through POST/images and after calling a path /images/800x400 it should show a random image from the database with 800x400 size.
I'm getting an error while trying to display an image.
from fastapi.responses import FileResponse
import uuid
app = FastAPI()
db = []
#app.post("/images/")
async def create_upload_file(file: UploadFile = File(...)):
contents = await file.read()
db.append(file)
with open(file.filename, "wb") as f:
f.write(contents)
return {"filename": file.filename}
#app.get("/images/")
async def show_image():
return db[0]
As a response I get:
{
"filename": "70188bdc-923c-4bd3-be15-8e71966cab31.jpg",
"content_type": "image/jpeg",
"file": {}
}
I would like to use: return FileResponse(some_file_path)
and in the file path put the filename from above. Is it right way of thinking?
First of all, you are adding the File object to your db list, that explains the response you get.
You want to write the content of the file to your db.
You also do not need to write it to the file system if you are using that as your "persistence" (of course all the files will go away if you shutdown or reload your app).
from fastapi import FastAPI, File, UploadFile
from fastapi.responses import Response
import os
from random import randint
import uuid
app = FastAPI()
db = []
#app.post("/images/")
async def create_upload_file(file: UploadFile = File(...)):
file.filename = f"{uuid.uuid4()}.jpg"
contents = await file.read() # <-- Important!
db.append(contents)
return {"filename": file.filename}
#app.get("/images/")
async def read_random_file():
# get a random file from the image db
random_index = randint(0, len(db) - 1)
# return a response object directly as FileResponse expects a file-like object
# and StreamingResponse expects an iterator/generator
response = Response(content=db[random_index])
return response
If you want to actually save the files to disk this is the method I would use (a real db is still preferred for a full application)
from fastapi import FastAPI, File, UploadFile
from fastapi.responses import FileResponse
import os
from random import randint
import uuid
IMAGEDIR = "fastapi-images/"
app = FastAPI()
#app.post("/images/")
async def create_upload_file(file: UploadFile = File(...)):
file.filename = f"{uuid.uuid4()}.jpg"
contents = await file.read() # <-- Important!
# example of how you can save the file
with open(f"{IMAGEDIR}{file.filename}", "wb") as f:
f.write(contents)
return {"filename": file.filename}
#app.get("/images/")
async def read_random_file():
# get a random file from the image directory
files = os.listdir(IMAGEDIR)
random_index = randint(0, len(files) - 1)
path = f"{IMAGEDIR}{files[random_index]}"
# notice you can use FileResponse now because it expects a path
return FileResponse(path)
Reference:
(FastAPI inherits responses from Starlette)
Starlette Response
Starlette StreamingResponse
Starlette FileResponse
(Tiangolo's documentation is still very good to have)
FastAPI Response
FastAPI StreamingResponse
FastAPI FileResponse
Related
I am trying to upload an mp4 video file using UploadFile in FastAPI.
However, the uploaded format is not readable by OpencCV (cv2).
This is my endpoint:
from fastapi import FastAPI, File, UploadFile
from fastapi.responses import PlainTextResponse
#app.post("/video/test", response_class=PlainTextResponse)
async def detect_faces_in_video(video_file: UploadFile):
contents = await video_file.read()
print(type(video_file)) # <class 'starlette.datastructures.UploadFile'>
print(type(contents)) # <class 'bytes'>
return ""
and the two file formats (i.e., bytes and UploadFile) are not readable by OpenCV.
You are trying to pass either the file contents (bytes) or UploadFile object; however, VideoCapture() accepts either a video filename, capturing device or or an IP video stream.
UploadFile is basically a SpooledTemporaryFile (a file-like object) that operates similar to a TemporaryFile. However, it does not have a visible name in the file system. As you mentioned that you wouldn't be keeping the files on the server after processing them, you could copy the file contents to a NamedTemporaryFile that "has a visible name in the file system, which can be used to open the file" (using the name attribute), as described here and here. As per the documentation:
Whether the name can be used to open the file a second time, while the
named temporary file is still open, varies across platforms (it can be
so used on Unix; it cannot on Windows). If delete is true (the
default), the file is deleted as soon as it is closed.
Hence, on Windows you need to set the delete argument to False when instantiating a NamedTemporaryFile, and once you are done with it, you can manually delete it, using the os.remove() or os.unlink() method.
Below are given two options on how to do that. Option 1 implements a solution using a def endpoint, while Option 2 uses an async def endpoint (utilising the aiofiles library). For the difference between def and async def, please have a look at this answer. If you are expecting users to upload rather large files in size that wouldn't fit into memory, have a look at this and this answer on how to read the uploaded video file in chunks instead.
Option 1 - Using def endpoint
from fastapi import FastAPI, File, UploadFile
from tempfile import NamedTemporaryFile
import os
#app.post("/video/detect-faces")
def detect_faces(file: UploadFile = File(...)):
temp = NamedTemporaryFile(delete=False)
try:
try:
contents = file.file.read()
with temp as f:
f.write(contents);
except Exception:
return {"message": "There was an error uploading the file"}
finally:
file.file.close()
res = process_video(temp.name) # Pass temp.name to VideoCapture()
except Exception:
return {"message": "There was an error processing the file"}
finally:
#temp.close() # the `with` statement above takes care of closing the file
os.remove(temp.name)
return res
Option 2 - Using async def endpoint
from fastapi import FastAPI, File, UploadFile
from tempfile import NamedTemporaryFile
from fastapi.concurrency import run_in_threadpool
import aiofiles
import asyncio
import os
#app.post("/video/detect-faces")
async def detect_faces(file: UploadFile = File(...)):
try:
async with aiofiles.tempfile.NamedTemporaryFile("wb", delete=False) as temp:
try:
contents = await file.read()
await temp.write(contents)
except Exception:
return {"message": "There was an error uploading the file"}
finally:
await file.close()
res = await run_in_threadpool(process_video, temp.name) # Pass temp.name to VideoCapture()
except Exception:
return {"message": "There was an error processing the file"}
finally:
os.remove(temp.name)
return res
I have uploaded a .mp4 file in postman and now i need to read the file contents into a variable in python
This is my postman upload
This is my python file where i my request will be catched in this function
now how will be decode the file uploaded in this function
from starlette.requests import Request
from starlette.routing import Route, Mount
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse, FileResponse
class Post:
async def PostValuesToDb(requests: Request):
return PlainTextResponse("true")
routes = [
Route("/post", Post.PostValuesToDb,
name="PostValuesToDb", methods=["POST"]),
]
app = Starlette(
routes=routes
)
PostValuesToDb(requests: Request) captures the postman request.
as per the documentation
for the time being i am choosing to save the contents in a local directory, you can do as you wish from the contents variable
from starlette.requests import Request
from starlette.routing import Route, Mount
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse, FileResponse
class Post:
async def PostValuesToDb(requests: Request):
form = await requests.form()
# print(dir(form)) # check what methods and attrs are available
# print(form.multi_items) # check all form fields submitted
filename = form["image"].filename # my particular form field
contents = await form["image"].read()
with open(f'./data/{filename}', 'wb') as uploaded_file:
uploaded_file.write(contents)
return PlainTextResponse("true")
routes = [
Route("/post", Post.PostValuesToDb,
name="PostValuesToDb", methods=["POST"]),
]
app = Starlette(
routes=routes
)
I'm trying to create a POST route where the media type would be set to "image/*" but when my route requires an argument, it's media type become "application/json" (even when I specify the response_class value).
Do you have an idea why?
Here is my source code:
from io import BytesIO
from fastapi.responses import ORJSONResponse
from fastapi import FastAPI
from fastapi.responses import StreamingResponse, JSONResponse, FileResponse
app = FastAPI(default_response_class=ORJSONResponse)
#app.post("/working", response_class=FileResponse)
async def send_w():
with open("/tmp/test/b.jpg", "rb") as f:
image = BytesIO(f.read())
image.seek(0)
return StreamingResponse(image, media_type="image/jpeg")
#app.post("/a", response_class=FileResponse)
async def send_a(a):
with open("/tmp/test/b.jpg", "rb") as f:
image = BytesIO(f.read())
image.seek(0)
return StreamingResponse(image, media_type="image/jpeg")
#app.post("/b", response_class=FileResponse)
async def send_b(b):
return "/tmp/test/b.jpg"
/a route:
You need to create function args to handle file type.
e.g.
from fastapi import FastAPI, File, UploadFile
#app.post("/image")
async def upload(data : UploadFile = File(...)):
image = await data.read()
# do your process here
read more:
https://fastapi.tiangolo.com/tutorial/request-files/
Is there a way to download a file through FastAPI? The files we want are located in an Azure Datalake and retrieving them from the lake is not an issue, the problem occurs when we try to get the bytes we get from the datalake down to a local machine.
We have tried using different modules in FastAPI such as starlette.responses.FileResponse and fastapi.Response with no luck.
In Flask this is not an issue and can be done in the following manner:
from io import BytesIO
from flask import Flask
from werkzeug import FileWrapper
flask_app = Flask(__name__)
#flask_app.route('/downloadfile/<file_name>', methods=['GET'])
def get_the_file(file_name: str):
the_file = FileWrapper(BytesIO(download_file_from_directory(file_name)))
if the_file:
return Response(the_file, mimetype=file_name, direct_passthrough=True)
When running this with a valid file name the file automatically downloads. Is there equivalent way to this in FastAPI?
Solved
After some more troubleshooting I found a way to do this.
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
if the_file:
return Response(the_file)
So after a lot of troubleshooting and hours of looking through documentation, this was all it took, simply returning the bytes as Response(the_file).
After some more troubleshooting I found a way to do this.
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
if the_file:
return Response(the_file)
So after a lot of troubleshooting and hours of looking through documentation, this was all it took, simply returning the bytes as Response(the_file) with no extra parameters and no extra formatting for the raw bytes object.
As far as I know, you need to set media_type to the adequate type. I did that with some code a year ago and it worked fine.
#app.get("/img/{name}")
def read(name: str, access_token_cookie: str=Cookie(None)):
r = internal.get_data(name)
if r is None:
return RedirectResponse(url="/static/default.png")
else:
return Response(content=r["data"], media_type=r["mime"])
r is a dictionary with the data as raw bytes and mime the type of the data as given by PythonMagick.
To add a custom filename to #Markus's answer, in case your api's path doesn't end with a neat filename or you want to determine a custom filename from server side and give to the user:
from fastapi import APIRouter, Response
router = APIRouter()
#router.get('/downloadfile/{file_name}', tags=['getSkynetDL'])
async def get_the_file(file_name: str):
# the_file object is raw bytes
the_file = download_file_from_directory(file_name)
filename1 = make_filename(file_name) # a custom filename
headers1 = {'Content-Disposition': f'attachment; filename="{filename1}"'}
if the_file:
return Response(the_file, headers=headers1)
Example
Here's my code:
from typing import List
from fastapi import FastAPI, File, UploadFile
import asyncio
import concurrent.futures
app = FastAPI()
#app.post("/send_images")
async def update_item(
files: List[UploadFile] = File(...),
):
return {"res": len(files)}
And I send requests to this server with (these specific urls are just for example here):
import requests
import os
import json
import numpy as np
import time
import io
import httpx
import asyncio
from datetime import datetime
import pandas as pd
from random import shuffle
import pandas as pd
import urllib
import io
from PIL import Image
from matplotlib import pyplot as plt
import concurrent.futures
urls = ['https://sun9-63.userapi.com/c638920/v638920705/1a54d/xSREwpakJD4.jpg',
'https://sun9-28.userapi.com/c854024/v854024084/1160d8/LDMVHYgguAw.jpg',
'https://sun9-54.userapi.com/c854220/v854220084/111f66/LdcbEpPR6tg.jpg',
'https://sun9-40.userapi.com/c841420/v841420283/4c8bb/Mii6GSCrmpo.jpg',
'https://sun6-16.userapi.com/CPQpllJ0KtaArvQKkPsHTZDCupqjRJ_8l07ejA/iyg2hRR_kM4.jpg',
'https://sun9-1.userapi.com/c638920/v638920705/1a53b/SMta6Bv-k7s.jpg',
'https://sun9-36.userapi.com/c857332/v857332580/56ad/rJCGKFw03FQ.jpg',
'https://sun6-14.userapi.com/iPsfmW0ibE8RsMh0k2lUFdRxHZ4Q41yctB7L3A/ajJHY3WN6Xg.jpg',
'https://sun9-28.userapi.com/c854324/v854324383/1c1dc3/UuFigBF7WDI.jpg',
'https://sun6-16.userapi.com/UVXVAT-tYudG5_24FMaBWTB9vyW8daSrO2WPFQ/RMjv7JZvowA.jpg']
os.environ['NO_PROXY'] = '127.0.0.1'
async def request_get_4(list_urls):
async with httpx.AsyncClient() as client:
r = httpx.post("http://127.0.0.1:8001/send_images", files={f'num_{ind}': el for ind, el in enumerate(list_urls)})
print(r.text)
return r
async def request_get_3(url):
async with httpx.AsyncClient() as client:
return await client.get(url)
from collections import defaultdict
async def main():
start = datetime.now()
tasks = [asyncio.create_task(request_get_3(url)) for url in urls[0:10]]
result = await asyncio.gather(*tasks)
data_to_send = []
for ind, resp in enumerate(result):
if resp.status_code == 200:
image_bytes = io.BytesIO(resp.content)
image_bytes.seek(0)
data_to_send.append(image_bytes)
end = datetime.now()
print(result)
print(len(data_to_send))
batch_size = 2
batch_num = len(data_to_send) // batch_size
tasks = [asyncio.create_task(request_get_4(data_to_send[i * batch_size: (i+1) * batch_size])) for i in range(batch_num)]
result = await asyncio.gather(*tasks)
left_data = data_to_send[batch_size*(batch_num):]
print(len(left_data))
print(result)
asyncio.run(main())
I am trying to load image which are contained in urls, then form batches of them and send them to FastAPI server. But it doesn't work.
I get the following error:
{"detail":[{"loc":["body","files"],"msg":"field required","type":"value_error.missing"}]}
How can I fix my problem and be able to send multiple files through httpx to FastAPI?
The problem is that HTTPX 0.13.3 doesn't support multiple file uploading as this arg wants dictionary and dictionary can't have same key values.
We can use this pull request https://github.com/encode/httpx/pull/1032/files to fix this problem (now it can also accept List[Tuple[str, FileTypes]]])!
UPDATE:
this problem is solved now!
Update2:
Here I show how I use httpx for different requests:
async def request_post_batch(fastapi_url: str, url_content_batch: List[BinaryIO]) -> httpx.Response:
"""
Send batch to FastAPI server.
"""
async with httpx.AsyncClient(timeout=httpx.Timeout(100.0)) as client:
r = await client.post(
fastapi_url,
files=[('bytes_image', url_content) for url_content in url_content_batch]
)
return r
async def request_post_logs(logstash_url: str, logs: List[Dict]) -> httpx.Response:
"""
Send logs to logstash
"""
async with httpx.AsyncClient(timeout=httpx.Timeout(100.0)) as client:
r = await client.post(
logstash_url,
json=logs
)
return r
This example will pass multiple files under the files field.
async with httpx.AsyncClient(timeout=httpx.Timeout(100.0)) as client:
response = await client.post(f"/api/v1/upload-files",
files=[("files", ("image.png", b"{}", "image/png")), ("files", ("image2.png", b"{}", "image/png"))],
)