Related
Specifically, I want the below example to work:
from typing import List
from pydantic import BaseModel
from fastapi import FastAPI, UploadFile, File
app = FastAPI()
class DataConfiguration(BaseModel):
textColumnNames: List[str]
idColumn: str
#app.post("/data")
async def data(dataConfiguration: DataConfiguration,
csvFile: UploadFile = File(...)):
pass
# read requested id and text columns from csvFile
If this is not the proper way for a POST request, please let me know how to select the required columns from an uploaded CSV file in FastAPI.
As per FastAPI documentation:
You can declare multiple Form parameters in a path operation, but you
can't also declare Body fields that you expect to receive as JSON, as
the request will have the body encoded using
application/x-www-form-urlencoded instead of application/json (when the form includes files, it is encoded as multipart/form-data).
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Note that you need to have python-multipart installed first—if you haven't already—since uploaded files are sent as "form data". For instance:
pip install python-multipart
Method 1
As described here, one can define files and form data at the same time using File and Form fields. Below is a working example. If you have a large number of parameters and would like to define them separately from the endpoint, please have a look at this answer on how to create a custom dependency class.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI
from typing import List
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.post("/submit")
def submit(
name: str = Form(...),
point: float = Form(...),
is_accepted: bool = Form(...),
files: List[UploadFile] = File(...),
):
return {
"JSON Payload ": {"name": name, "point": point, "is_accepted": is_accepted},
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can test it by accessing the template below at http://127.0.0.1:8000. If your template does not include any Jinja code, you could alternatively return a simple HTMLResponse.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" action="http://127.0.0.1:8000/submit" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
</body>
</html>
You can also test it using the interactive OpenAPI docs (provided by Swagger UI) at http://127.0.0.1:8000/docs, or Python requests, as shown below:
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, data=payload, files=files)
print(resp.json())
Method 2
One can use Pydantic models, along with Dependencies to inform the "submit" route (in the case below) that the parameterised variable base depends on the Base class. Please note, this method expects the base data as query (not body) parameters (which are then converted into an equivalent JSON payload using .dict() method) and the Files as multipart/form-data in the body.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI, Depends
from typing import List
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from typing import Optional
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#app.post("/submit")
def submit(base: Base = Depends(), files: List[UploadFile] = File(...)):
return {
"JSON Payload ": base.dict(),
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
Again, you can test it using the template below, which, this time, uses Javascript to modify the action attribute of the form, in order to pass the form data as query params to the URL.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" id="myForm" onclick="transformFormData();" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
<script>
function transformFormData(){
var myForm = document.getElementById('myForm');
var qs = new URLSearchParams(new FormData(myForm)).toString();
myForm.action = 'http://127.0.0.1:8000/submit?' + qs;
}
</script>
</body>
</html>
As mentioned earlier you can also use Swagger UI, or Python requests, as shown in the example below. Note that this time, the payload is passed to the params parameter of requests.post(), as you submit query parameters, not form-data (body) params, which was the case in the previous method.
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, params=payload, files=files)
print(resp.json())
Method 3
Another option would be to pass the body data as a single parameter (of type Form) in the form of a JSON string. On server side, you can create a dependency function, where you parse the data using parse_raw method and validate the data against the corresponding model. If ValidationError is raised, an HTTP_422_UNPROCESSABLE_ENTITY error is sent back to the client, including the error message. Example is given below:
app.py
from fastapi import FastAPI, status, Form, UploadFile, File, Depends, Request
from pydantic import BaseModel, ValidationError
from fastapi.exceptions import HTTPException
from fastapi.encoders import jsonable_encoder
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
def checker(data: str = Form(...)):
try:
model = Base.parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
#app.post("/submit")
def submit(model: Base = Depends(checker), files: List[UploadFile] = File(...)):
return {"JSON Payload ": model, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
In case you had multiple models and would like to avoid creating a checker function for each model, you could instead create a checker class, as described in the documentation, and have a dictionary of your models that you can use to look up for a model to parse. Example:
# ...
models = {"base": Base, "other": SomeOtherModel}
class DataChecker:
def __init__(self, name: str):
self.name = name
def __call__(self, data: str = Form(...)):
try:
model = models[self.name].parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
base_checker = DataChecker("base")
other_checker = DataChecker("other")
#app.post("/submit")
def submit(model: Base = Depends(base_checker), files: List[UploadFile] = File(...)):
# ...
test.py
Note that in JSON, boolean values are represented using the true or false literals in lower case, whereas in Python they must be capitalised as either True or False.
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': '{"name": "foo", "point": 0.13, "is_accepted": false}'}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Or, if you prefer:
import requests
import json
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': json.dumps({"name": "foo", "point": 0.13, "is_accepted": False})}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Test using Fetch API or Axios
templates/index.html
<!DOCTYPE html>
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.27.2/axios.min.js"></script>
</head>
<body>
<input type="file" id="fileInput" name="file" multiple><br>
<input type="button" value="Submit using fetch" onclick="submitUsingFetch()">
<input type="button" value="Submit using axios" onclick="submitUsingAxios()">
<script>
function submitUsingFetch() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
fetch('/submit', {
method: 'POST',
body: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
function submitUsingAxios() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
axios({
method: 'POST',
url: '/submit',
data: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
</script>
</body>
</html>
Method 4
A further method comes from the discussion here, and incorporates a custom class with a classmethod used to transform a given JSON string into a Python dictionary, which is then used for validation against the Pydantic model. Similar to Method 3 above, the input data should be passed as a single Form parameter in the form of JSON string (defining the parameter with Body type would also work and still expect the JSON string as form data, as in this case the data comes encoded as multipart/form-data). Thus, the same test.py file(s) and index.html template from the previous method can be used for testing the below.
app.py
from fastapi import FastAPI, File, Body, UploadFile, Request
from pydantic import BaseModel
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
import json
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#classmethod
def __get_validators__(cls):
yield cls.validate_to_json
#classmethod
def validate_to_json(cls, value):
if isinstance(value, str):
return cls(**json.loads(value))
return value
#app.post("/submit")
def submit(data: Base = Body(...), files: List[UploadFile] = File(...)):
return {"JSON Payload ": data, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can't mix form-data with json.
Per FastAPI documentation:
Warning:
You can declare multiple File and Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using multipart/form-data instead of application/json.
This is not a limitation of FastAPI, it's part of the HTTP protocol.
You can, however, use Form(...) as a workaround to attach extra string as form-data:
from typing import List
from fastapi import FastAPI, UploadFile, File, Form
app = FastAPI()
#app.post("/data")
async def data(textColumnNames: List[str] = Form(...),
idColumn: str = Form(...),
csvFile: UploadFile = File(...)):
pass
I went with the very elegant Method3 from #Chris (originally proposed from #M.Winkwns). However, I modified it slightly to work with any Pydantic model:
from typing import Type, TypeVar
from pydantic import BaseModel, ValidationError
from fastapi import Form
Serialized = TypeVar("Serialized", bound=BaseModel)
def form_json_deserializer(schema: Type[Serialized], data: str = Form(...)) -> Serialized:
"""
Helper to serialize request data not automatically included in an application/json body but
within somewhere else like a form parameter. This makes an assumption that the form parameter with JSON data is called 'data'
:param schema: Pydantic model to serialize into
:param data: raw str data representing the Pydantic model
:raises ValidationError: if there are errors parsing the given 'data' into the given 'schema'
"""
try:
return schema.parse_raw(data)
except ValidationError as e
raise HTTPException(detail=jsonable_encoder(e.errors()), status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)
When you use it in an endpoint you can then use functools.partial to bind the specific Pydantic model:
import functools
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/upload")
async def upload(
data: Base = Depends(functools.partial(form_json_deserializer, Base)),
files: Sequence[UploadFile] = File(...)
) -> Base:
return data
As stated by #Chris (and just for completeness):
As per FastAPI documentation,
You can declare multiple Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using application/x-www-form-urlencoded instead of application/json. (But when the form includes files, it is encoded as multipart/form-data)
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Since his Method1 wasn't an option and Method2 can't work for deeply nested datatypes I came up with a different solution:
Simply convert your datatype to a string/json and call pydantics parse_raw function
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/submit")
async def submit(base: str = Form(...), files: List[UploadFile] = File(...)):
try:
model = Base.parse_raw(base)
except pydantic.ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY
) from e
return {"JSON Payload ": received_data, "Uploaded Filenames": [file.filename for file in files]}
Example using pythantic models for cleaner documentation. The file is encoded to base64 any other logic can be applied.
class BaseTestUser(BaseModel):
name: str
image_1920: str
class UpdateUserEncodeFile(BaseTestUser):
def __init__(self, name: str = Form(...), image_1920: UploadFile = File(...)):
super().__init__(name=name, image_1920=base64.b64encode(image_1920.file.read()))
#routers
#router.put("/users/{id}/encoded", status_code=status.HTTP_200_OK)
def user_update_encode(id: int, user:UpdateUserEncodeFile=Depends()):
return user
Specifically, I want the below example to work:
from typing import List
from pydantic import BaseModel
from fastapi import FastAPI, UploadFile, File
app = FastAPI()
class DataConfiguration(BaseModel):
textColumnNames: List[str]
idColumn: str
#app.post("/data")
async def data(dataConfiguration: DataConfiguration,
csvFile: UploadFile = File(...)):
pass
# read requested id and text columns from csvFile
If this is not the proper way for a POST request, please let me know how to select the required columns from an uploaded CSV file in FastAPI.
As per FastAPI documentation:
You can declare multiple Form parameters in a path operation, but you
can't also declare Body fields that you expect to receive as JSON, as
the request will have the body encoded using
application/x-www-form-urlencoded instead of application/json (when the form includes files, it is encoded as multipart/form-data).
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Note that you need to have python-multipart installed first—if you haven't already—since uploaded files are sent as "form data". For instance:
pip install python-multipart
Method 1
As described here, one can define files and form data at the same time using File and Form fields. Below is a working example. If you have a large number of parameters and would like to define them separately from the endpoint, please have a look at this answer on how to create a custom dependency class.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI
from typing import List
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.post("/submit")
def submit(
name: str = Form(...),
point: float = Form(...),
is_accepted: bool = Form(...),
files: List[UploadFile] = File(...),
):
return {
"JSON Payload ": {"name": name, "point": point, "is_accepted": is_accepted},
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can test it by accessing the template below at http://127.0.0.1:8000. If your template does not include any Jinja code, you could alternatively return a simple HTMLResponse.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" action="http://127.0.0.1:8000/submit" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
</body>
</html>
You can also test it using the interactive OpenAPI docs (provided by Swagger UI) at http://127.0.0.1:8000/docs, or Python requests, as shown below:
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, data=payload, files=files)
print(resp.json())
Method 2
One can use Pydantic models, along with Dependencies to inform the "submit" route (in the case below) that the parameterised variable base depends on the Base class. Please note, this method expects the base data as query (not body) parameters (which are then converted into an equivalent JSON payload using .dict() method) and the Files as multipart/form-data in the body.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI, Depends
from typing import List
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from typing import Optional
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#app.post("/submit")
def submit(base: Base = Depends(), files: List[UploadFile] = File(...)):
return {
"JSON Payload ": base.dict(),
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
Again, you can test it using the template below, which, this time, uses Javascript to modify the action attribute of the form, in order to pass the form data as query params to the URL.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" id="myForm" onclick="transformFormData();" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
<script>
function transformFormData(){
var myForm = document.getElementById('myForm');
var qs = new URLSearchParams(new FormData(myForm)).toString();
myForm.action = 'http://127.0.0.1:8000/submit?' + qs;
}
</script>
</body>
</html>
As mentioned earlier you can also use Swagger UI, or Python requests, as shown in the example below. Note that this time, the payload is passed to the params parameter of requests.post(), as you submit query parameters, not form-data (body) params, which was the case in the previous method.
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, params=payload, files=files)
print(resp.json())
Method 3
Another option would be to pass the body data as a single parameter (of type Form) in the form of a JSON string. On server side, you can create a dependency function, where you parse the data using parse_raw method and validate the data against the corresponding model. If ValidationError is raised, an HTTP_422_UNPROCESSABLE_ENTITY error is sent back to the client, including the error message. Example is given below:
app.py
from fastapi import FastAPI, status, Form, UploadFile, File, Depends, Request
from pydantic import BaseModel, ValidationError
from fastapi.exceptions import HTTPException
from fastapi.encoders import jsonable_encoder
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
def checker(data: str = Form(...)):
try:
model = Base.parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
#app.post("/submit")
def submit(model: Base = Depends(checker), files: List[UploadFile] = File(...)):
return {"JSON Payload ": model, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
In case you had multiple models and would like to avoid creating a checker function for each model, you could instead create a checker class, as described in the documentation, and have a dictionary of your models that you can use to look up for a model to parse. Example:
# ...
models = {"base": Base, "other": SomeOtherModel}
class DataChecker:
def __init__(self, name: str):
self.name = name
def __call__(self, data: str = Form(...)):
try:
model = models[self.name].parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
base_checker = DataChecker("base")
other_checker = DataChecker("other")
#app.post("/submit")
def submit(model: Base = Depends(base_checker), files: List[UploadFile] = File(...)):
# ...
test.py
Note that in JSON, boolean values are represented using the true or false literals in lower case, whereas in Python they must be capitalised as either True or False.
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': '{"name": "foo", "point": 0.13, "is_accepted": false}'}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Or, if you prefer:
import requests
import json
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': json.dumps({"name": "foo", "point": 0.13, "is_accepted": False})}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Test using Fetch API or Axios
templates/index.html
<!DOCTYPE html>
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.27.2/axios.min.js"></script>
</head>
<body>
<input type="file" id="fileInput" name="file" multiple><br>
<input type="button" value="Submit using fetch" onclick="submitUsingFetch()">
<input type="button" value="Submit using axios" onclick="submitUsingAxios()">
<script>
function submitUsingFetch() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
fetch('/submit', {
method: 'POST',
body: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
function submitUsingAxios() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
axios({
method: 'POST',
url: '/submit',
data: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
</script>
</body>
</html>
Method 4
A further method comes from the discussion here, and incorporates a custom class with a classmethod used to transform a given JSON string into a Python dictionary, which is then used for validation against the Pydantic model. Similar to Method 3 above, the input data should be passed as a single Form parameter in the form of JSON string (defining the parameter with Body type would also work and still expect the JSON string as form data, as in this case the data comes encoded as multipart/form-data). Thus, the same test.py file(s) and index.html template from the previous method can be used for testing the below.
app.py
from fastapi import FastAPI, File, Body, UploadFile, Request
from pydantic import BaseModel
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
import json
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#classmethod
def __get_validators__(cls):
yield cls.validate_to_json
#classmethod
def validate_to_json(cls, value):
if isinstance(value, str):
return cls(**json.loads(value))
return value
#app.post("/submit")
def submit(data: Base = Body(...), files: List[UploadFile] = File(...)):
return {"JSON Payload ": data, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can't mix form-data with json.
Per FastAPI documentation:
Warning:
You can declare multiple File and Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using multipart/form-data instead of application/json.
This is not a limitation of FastAPI, it's part of the HTTP protocol.
You can, however, use Form(...) as a workaround to attach extra string as form-data:
from typing import List
from fastapi import FastAPI, UploadFile, File, Form
app = FastAPI()
#app.post("/data")
async def data(textColumnNames: List[str] = Form(...),
idColumn: str = Form(...),
csvFile: UploadFile = File(...)):
pass
I went with the very elegant Method3 from #Chris (originally proposed from #M.Winkwns). However, I modified it slightly to work with any Pydantic model:
from typing import Type, TypeVar
from pydantic import BaseModel, ValidationError
from fastapi import Form
Serialized = TypeVar("Serialized", bound=BaseModel)
def form_json_deserializer(schema: Type[Serialized], data: str = Form(...)) -> Serialized:
"""
Helper to serialize request data not automatically included in an application/json body but
within somewhere else like a form parameter. This makes an assumption that the form parameter with JSON data is called 'data'
:param schema: Pydantic model to serialize into
:param data: raw str data representing the Pydantic model
:raises ValidationError: if there are errors parsing the given 'data' into the given 'schema'
"""
try:
return schema.parse_raw(data)
except ValidationError as e
raise HTTPException(detail=jsonable_encoder(e.errors()), status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)
When you use it in an endpoint you can then use functools.partial to bind the specific Pydantic model:
import functools
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/upload")
async def upload(
data: Base = Depends(functools.partial(form_json_deserializer, Base)),
files: Sequence[UploadFile] = File(...)
) -> Base:
return data
As stated by #Chris (and just for completeness):
As per FastAPI documentation,
You can declare multiple Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using application/x-www-form-urlencoded instead of application/json. (But when the form includes files, it is encoded as multipart/form-data)
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Since his Method1 wasn't an option and Method2 can't work for deeply nested datatypes I came up with a different solution:
Simply convert your datatype to a string/json and call pydantics parse_raw function
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/submit")
async def submit(base: str = Form(...), files: List[UploadFile] = File(...)):
try:
model = Base.parse_raw(base)
except pydantic.ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY
) from e
return {"JSON Payload ": received_data, "Uploaded Filenames": [file.filename for file in files]}
Example using pythantic models for cleaner documentation. The file is encoded to base64 any other logic can be applied.
class BaseTestUser(BaseModel):
name: str
image_1920: str
class UpdateUserEncodeFile(BaseTestUser):
def __init__(self, name: str = Form(...), image_1920: UploadFile = File(...)):
super().__init__(name=name, image_1920=base64.b64encode(image_1920.file.read()))
#routers
#router.put("/users/{id}/encoded", status_code=status.HTTP_200_OK)
def user_update_encode(id: int, user:UpdateUserEncodeFile=Depends()):
return user
Specifically, I want the below example to work:
from typing import List
from pydantic import BaseModel
from fastapi import FastAPI, UploadFile, File
app = FastAPI()
class DataConfiguration(BaseModel):
textColumnNames: List[str]
idColumn: str
#app.post("/data")
async def data(dataConfiguration: DataConfiguration,
csvFile: UploadFile = File(...)):
pass
# read requested id and text columns from csvFile
If this is not the proper way for a POST request, please let me know how to select the required columns from an uploaded CSV file in FastAPI.
As per FastAPI documentation:
You can declare multiple Form parameters in a path operation, but you
can't also declare Body fields that you expect to receive as JSON, as
the request will have the body encoded using
application/x-www-form-urlencoded instead of application/json (when the form includes files, it is encoded as multipart/form-data).
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Note that you need to have python-multipart installed first—if you haven't already—since uploaded files are sent as "form data". For instance:
pip install python-multipart
Method 1
As described here, one can define files and form data at the same time using File and Form fields. Below is a working example. If you have a large number of parameters and would like to define them separately from the endpoint, please have a look at this answer on how to create a custom dependency class.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI
from typing import List
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.post("/submit")
def submit(
name: str = Form(...),
point: float = Form(...),
is_accepted: bool = Form(...),
files: List[UploadFile] = File(...),
):
return {
"JSON Payload ": {"name": name, "point": point, "is_accepted": is_accepted},
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can test it by accessing the template below at http://127.0.0.1:8000. If your template does not include any Jinja code, you could alternatively return a simple HTMLResponse.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" action="http://127.0.0.1:8000/submit" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
</body>
</html>
You can also test it using the interactive OpenAPI docs (provided by Swagger UI) at http://127.0.0.1:8000/docs, or Python requests, as shown below:
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, data=payload, files=files)
print(resp.json())
Method 2
One can use Pydantic models, along with Dependencies to inform the "submit" route (in the case below) that the parameterised variable base depends on the Base class. Please note, this method expects the base data as query (not body) parameters (which are then converted into an equivalent JSON payload using .dict() method) and the Files as multipart/form-data in the body.
app.py
from fastapi import Form, File, UploadFile, Request, FastAPI, Depends
from typing import List
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from typing import Optional
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#app.post("/submit")
def submit(base: Base = Depends(), files: List[UploadFile] = File(...)):
return {
"JSON Payload ": base.dict(),
"Filenames": [file.filename for file in files],
}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
Again, you can test it using the template below, which, this time, uses Javascript to modify the action attribute of the form, in order to pass the form data as query params to the URL.
templates/index.html
<!DOCTYPE html>
<html>
<body>
<form method="post" id="myForm" onclick="transformFormData();" enctype="multipart/form-data">
name : <input type="text" name="name" value="foo"><br>
point : <input type="text" name="point" value=0.134><br>
is_accepted : <input type="text" name="is_accepted" value=True><br>
<label for="file">Choose files to upload</label>
<input type="file" id="files" name="files" multiple>
<input type="submit" value="submit">
</form>
<script>
function transformFormData(){
var myForm = document.getElementById('myForm');
var qs = new URLSearchParams(new FormData(myForm)).toString();
myForm.action = 'http://127.0.0.1:8000/submit?' + qs;
}
</script>
</body>
</html>
As mentioned earlier you can also use Swagger UI, or Python requests, as shown in the example below. Note that this time, the payload is passed to the params parameter of requests.post(), as you submit query parameters, not form-data (body) params, which was the case in the previous method.
test.py
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
payload ={"name": "foo", "point": 0.13, "is_accepted": False}
resp = requests.post(url=url, params=payload, files=files)
print(resp.json())
Method 3
Another option would be to pass the body data as a single parameter (of type Form) in the form of a JSON string. On server side, you can create a dependency function, where you parse the data using parse_raw method and validate the data against the corresponding model. If ValidationError is raised, an HTTP_422_UNPROCESSABLE_ENTITY error is sent back to the client, including the error message. Example is given below:
app.py
from fastapi import FastAPI, status, Form, UploadFile, File, Depends, Request
from pydantic import BaseModel, ValidationError
from fastapi.exceptions import HTTPException
from fastapi.encoders import jsonable_encoder
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
def checker(data: str = Form(...)):
try:
model = Base.parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
#app.post("/submit")
def submit(model: Base = Depends(checker), files: List[UploadFile] = File(...)):
return {"JSON Payload ": model, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
In case you had multiple models and would like to avoid creating a checker function for each model, you could instead create a checker class, as described in the documentation, and have a dictionary of your models that you can use to look up for a model to parse. Example:
# ...
models = {"base": Base, "other": SomeOtherModel}
class DataChecker:
def __init__(self, name: str):
self.name = name
def __call__(self, data: str = Form(...)):
try:
model = models[self.name].parse_raw(data)
except ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
)
return model
base_checker = DataChecker("base")
other_checker = DataChecker("other")
#app.post("/submit")
def submit(model: Base = Depends(base_checker), files: List[UploadFile] = File(...)):
# ...
test.py
Note that in JSON, boolean values are represented using the true or false literals in lower case, whereas in Python they must be capitalised as either True or False.
import requests
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': '{"name": "foo", "point": 0.13, "is_accepted": false}'}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Or, if you prefer:
import requests
import json
url = 'http://127.0.0.1:8000/submit'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
data = {'data': json.dumps({"name": "foo", "point": 0.13, "is_accepted": False})}
resp = requests.post(url=url, data=data, files=files)
print(resp.json())
Test using Fetch API or Axios
templates/index.html
<!DOCTYPE html>
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.27.2/axios.min.js"></script>
</head>
<body>
<input type="file" id="fileInput" name="file" multiple><br>
<input type="button" value="Submit using fetch" onclick="submitUsingFetch()">
<input type="button" value="Submit using axios" onclick="submitUsingAxios()">
<script>
function submitUsingFetch() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
fetch('/submit', {
method: 'POST',
body: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
function submitUsingAxios() {
var fileInput = document.getElementById('fileInput');
if (fileInput.files[0]) {
var formData = new FormData();
formData.append("data", JSON.stringify({"name": "foo", "point": 0.13, "is_accepted": false}));
for (const file of fileInput.files)
formData.append('files', file);
axios({
method: 'POST',
url: '/submit',
data: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
</script>
</body>
</html>
Method 4
A further method comes from the discussion here, and incorporates a custom class with a classmethod used to transform a given JSON string into a Python dictionary, which is then used for validation against the Pydantic model. Similar to Method 3 above, the input data should be passed as a single Form parameter in the form of JSON string (defining the parameter with Body type would also work and still expect the JSON string as form data, as in this case the data comes encoded as multipart/form-data). Thus, the same test.py file(s) and index.html template from the previous method can be used for testing the below.
app.py
from fastapi import FastAPI, File, Body, UploadFile, Request
from pydantic import BaseModel
from typing import Optional, List
from fastapi.templating import Jinja2Templates
from fastapi.responses import HTMLResponse
import json
app = FastAPI()
templates = Jinja2Templates(directory="templates")
class Base(BaseModel):
name: str
point: Optional[float] = None
is_accepted: Optional[bool] = False
#classmethod
def __get_validators__(cls):
yield cls.validate_to_json
#classmethod
def validate_to_json(cls, value):
if isinstance(value, str):
return cls(**json.loads(value))
return value
#app.post("/submit")
def submit(data: Base = Body(...), files: List[UploadFile] = File(...)):
return {"JSON Payload ": data, "Filenames": [file.filename for file in files]}
#app.get("/", response_class=HTMLResponse)
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
You can't mix form-data with json.
Per FastAPI documentation:
Warning:
You can declare multiple File and Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using multipart/form-data instead of application/json.
This is not a limitation of FastAPI, it's part of the HTTP protocol.
You can, however, use Form(...) as a workaround to attach extra string as form-data:
from typing import List
from fastapi import FastAPI, UploadFile, File, Form
app = FastAPI()
#app.post("/data")
async def data(textColumnNames: List[str] = Form(...),
idColumn: str = Form(...),
csvFile: UploadFile = File(...)):
pass
I went with the very elegant Method3 from #Chris (originally proposed from #M.Winkwns). However, I modified it slightly to work with any Pydantic model:
from typing import Type, TypeVar
from pydantic import BaseModel, ValidationError
from fastapi import Form
Serialized = TypeVar("Serialized", bound=BaseModel)
def form_json_deserializer(schema: Type[Serialized], data: str = Form(...)) -> Serialized:
"""
Helper to serialize request data not automatically included in an application/json body but
within somewhere else like a form parameter. This makes an assumption that the form parameter with JSON data is called 'data'
:param schema: Pydantic model to serialize into
:param data: raw str data representing the Pydantic model
:raises ValidationError: if there are errors parsing the given 'data' into the given 'schema'
"""
try:
return schema.parse_raw(data)
except ValidationError as e
raise HTTPException(detail=jsonable_encoder(e.errors()), status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)
When you use it in an endpoint you can then use functools.partial to bind the specific Pydantic model:
import functools
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/upload")
async def upload(
data: Base = Depends(functools.partial(form_json_deserializer, Base)),
files: Sequence[UploadFile] = File(...)
) -> Base:
return data
As stated by #Chris (and just for completeness):
As per FastAPI documentation,
You can declare multiple Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using application/x-www-form-urlencoded instead of application/json. (But when the form includes files, it is encoded as multipart/form-data)
This is not a limitation of FastAPI, it's part of the HTTP protocol.
Since his Method1 wasn't an option and Method2 can't work for deeply nested datatypes I came up with a different solution:
Simply convert your datatype to a string/json and call pydantics parse_raw function
from pydantic import BaseModel
from fastapi import Form, File, UploadFile, FastAPI
class OtherStuff(BaseModel):
stuff: str
class Base(BaseModel):
name: str
stuff: OtherStuff
#app.post("/submit")
async def submit(base: str = Form(...), files: List[UploadFile] = File(...)):
try:
model = Base.parse_raw(base)
except pydantic.ValidationError as e:
raise HTTPException(
detail=jsonable_encoder(e.errors()),
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY
) from e
return {"JSON Payload ": received_data, "Uploaded Filenames": [file.filename for file in files]}
Example using pythantic models for cleaner documentation. The file is encoded to base64 any other logic can be applied.
class BaseTestUser(BaseModel):
name: str
image_1920: str
class UpdateUserEncodeFile(BaseTestUser):
def __init__(self, name: str = Form(...), image_1920: UploadFile = File(...)):
super().__init__(name=name, image_1920=base64.b64encode(image_1920.file.read()))
#routers
#router.put("/users/{id}/encoded", status_code=status.HTTP_200_OK)
def user_update_encode(id: int, user:UpdateUserEncodeFile=Depends()):
return user
I have a machine learning model deployed using FastAPI, but the issue is I need the model to take two-body parameters
app = FastAPI()
class Inputs(BaseModel):
industry: str = None
file: UploadFile = File(...)
#app.post("/predict")
async def predict(inputs: Inputs):
# params
industry = inputs.industry
file = inputs.file
### some code ###
return predicted value
When I tried to send the input parameters I am getting an error in postman, please see the pic below,
From the FastAPI discussion thread--(#657)
if you are receiving JSON data, with application/json, use normal Pydantic models.
This would be the most common way to communicate with an API.
If you are receiving a raw file, e.g. a picture or PDF file to store it in the server, then use UploadFile, it will be sent as form data (multipart/form-data).
If you need to receive some type of structured content that is not JSON but you want to validate in some way, for example, an Excel file, you would still have to upload it using UploadFile and do all the necessary validations in your code. You could use Pydantic in your own code for your validations, but there's no way for FastAPI to do it for you in that case.
So, in your case, the router should be as,
from fastapi import FastAPI, File, UploadFile, Form
app = FastAPI()
#app.post("/predict")
async def predict(
industry: str = Form(...),
file: UploadFile = File(...)
):
# rest of your logic
return {"industry": industry, "filename": file.filename}
I'm looking for some library or example of code to format FastAPI validation messages into human-readable format. E.g. this endpoint:
#app.get("/")
async def hello(name: str):
return {"hello": name}
Will produce the next json output if we miss name query parameter:
{
"detail":[
{
"loc":[
"query",
"name"
],
"msg":"field required",
"type":"value_error.missing"
}
]
}
So my questions is, how to:
Transform it into something like "name field is required" (for all kinds of possible errors) to show in toasts.
Use it to display form validation messages
Generate forms themselves from api description if it's possible
FastAPI has a great Exception Handling, so you can customize your exceptions in many ways.
You can raise an HTTPException, HTTPException is a normal Python exception with additional data relevant for APIs. But you can't return it you need to raise it because it's a Python exception
from fastapi import HTTPException
...
#app.get("/")
async def hello(name: str):
if not name:
raise HTTPException(status_code=404, detail="Name field is required")
return {"Hello": name}
By adding name: str as a query parameter it automatically becomes required so you need to add Optional
from typing import Optional
...
#app.get("/")
async def hello(name: Optional[str] = None):
error = {"Error": "Name field is required"}
if name:
return {"Hello": name}
return error
$ curl 127.0.0.1:8000/?name=imbolc
{"Hello":"imbolc"}
...
$ curl 127.0.0.1:8000
{"Error":"Name field is required"}
But in your case, and i think this is the best way to handling errors in FastAPI overriding the validation_exception_handler:
from fastapi import FastAPI, Request, status
from fastapi.encoders import jsonable_encoder
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
...
#app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
return JSONResponse(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
content=jsonable_encoder({"detail": exc.errors(), "Error": "Name field is missing"}),
)
...
#app.get("/")
async def hello(name: str):
return {"hello": name}
You will get a response like this:
$ curl 127.0.0.1:8000
{
"detail":[
{
"loc":[
"query",
"name"
],
"msg":"field required",
"type":"value_error.missing"
}
],
"Error":"Name field is missing"
}
You can customize your content however if you like:
{
"Error":"Name field is missing",
"Customize":{
"This":"content",
"Also you can":"make it simpler"
}
}
I reached here with a similar question - and I ended up handling the RequestValidationError to give back a response where every field is an array of the issues with that field.
The response to your request would become (with a status_code=400)
{
"detail": "Invalid request",
"errors": {"name": ["field required"]}
}
that's quite handy to manage on the frontend for snackbar notifications and flexible enough.
Here's the handler
from collections import defaultdict
from fastapi import status
from fastapi.encoders import jsonable_encoder
from fastapi.responses import JSONResponse
#app.exception_handler(RequestValidationError)
async def custom_form_validation_error(request, exc):
reformatted_message = defaultdict(list)
for pydantic_error in exc.errors():
loc, msg = pydantic_error["loc"], pydantic_error["msg"]
filtered_loc = loc[1:] if loc[0] in ("body", "query", "path") else loc
field_string = ".".join(filtered_loc) # nested fields with dot-notation
reformatted_message[field_string].append(msg)
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content=jsonable_encoder(
{"detail": "Invalid request", "errors": reformatted_message}
),
)
I think the best I can come up with is actually PlainTextResponse
Adding these:
from fastapi.exceptions import RequestValidationError
#app.exception_handler(RequestValidationError)
async def validation_exception_handler(request, exc):
return PlainTextResponse(str(exc), status_code=400)
You get a more human-friendly error message like these in plain text:
1 validation error
path -> item_id
value is not a valid integer (type=type_error.integer)
It's well documented in FastAPI docs here.