I am trying to learn FastAPI and Pydantic to put a list of object into MongoDB, but I got an error saying 422 Unprocessable Entity. I understand the error message indicated the server understands the JSON format, but is unable to handle it. I tried to wrap it with another model, but it looks like it doesn't work.
Let's say I have a list of object as:
[
{
"date": "2022-12-13",
"symbol": "nsht",
"price": "45.12"
},
{
"date": "2022-12-13",
"symbol": "asdf",
"price": "45.14442"
}
]
And I want to add it to the database by following object model as:
class EODCoinModel(BaseModel):
id: PyObjectId = Field(default_factory=PyObjectId, alias="_id")
date: str = Field(...)
symbol: str = Field(...)
price: float = Field(...)
class Config:
allow_population_by_field_name = True
arbitrary_types_allowed = True
json_encoders = {ObjectId: str}
schema_extra = {
"example": {
"date": "2022-12-13",
"symbol": "AAPL",
"price": "45.12"
}
}
class ItemList(BaseModel):
data: List[EODCoinModel]
And PUT method as:
#app.post("/", response_description="Add new coin", response_model=ItemList)
async def create_coin(coin: ItemList = Body(...)):
coin = jsonable_encoder(coin)
print(coin)
new_coin = await db["coin"].insert_one(coin)
created_coin = await db["coin"].find_one({"_id": new_coin.inserted_id})
return JSONResponse(status_code=status.HTTP_201_CREATED, content=created_coin)
the Post request expects a body like this:
{"data" : [
"date": "2022-12-13",
"symbol": "nsht",
"price": "45.12"]
}
So then I ended up with not using Pydantic model validation. Just take the whole list and do inset_many.
#app.post("/", response_description="Add new coin")
async def create_coin(coin: List):
coin = jsonable_encoder(coin)
print(coin)
new_ coin = await db["coin"].insert_many(coin)
return JSONResponse(status_code=status.HTTP_201_CREATED)
Related
I've been working with FastAPI for some time, it's a great framework.
However real life scenarios can be surprising, sometimes a non-standard approach is necessary. There's a one case I'd like to ask your help with.
There's a strange external requirement that a model response should be formatted as stated in example:
Desired behavior:
GET /object/1
{status: ‘success’, data: {object: {id:‘1’, category: ‘test’ …}}}
GET /objects
{status: ‘success’, data: {objects: [...]}}}
Current behavior:
GET /object/1 would respond:
{id: 1,field1:"content",... }
GET /objects/ would send a List of Object e.g.,:
{
[
{id: 1,field1:"content",... },
{id: 1,field1:"content",... },
...
]
}
You can substitute 'object' by any class, it's just for description purposes.
How to write a generic response model that will suit those reqs?
I know I can produce response model that would contain status:str and (depending on class) data structure e.g ticket:Ticket or tickets:List[Ticket].
The point is there's a number of classes so I hope there's a more pythonic way to do it.
Thanks for help.
Generic model with static field name
A generic model is a model where one field (or multiple) are annotated with a type variable. Thus the type of that field is unspecified by default and must be specified explicitly during subclassing and/or initialization. But that field is still just an attribute and an attribute must have a name. A fixed name.
To go from your example, say that is your model:
{
"status": "...",
"data": {
"object": {...} # type variable
}
}
Then we could define that model as generic in terms of the type of its object attribute.
This can be done using Pydantic's GenericModel like this:
from typing import Generic, TypeVar
from pydantic import BaseModel
from pydantic.generics import GenericModel
M = TypeVar("M", bound=BaseModel)
class GenericSingleObject(GenericModel, Generic[M]):
object: M
class GenericMultipleObjects(GenericModel, Generic[M]):
objects: list[M]
class BaseGenericResponse(GenericModel):
status: str
class GenericSingleResponse(BaseGenericResponse, Generic[M]):
data: GenericSingleObject[M]
class GenericMultipleResponse(BaseGenericResponse, Generic[M]):
data: GenericMultipleObjects[M]
class Foo(BaseModel):
a: str
b: int
class Bar(BaseModel):
x: float
As you can see, GenericSingleObject reflects the generic type we want for data, whereas GenericSingleResponse is generic in terms of the type parameter M of GenericSingleObject, which is the type of its data attribute.
If we now want to use one of our generic response models, we would need to specify it with a type argument (a concrete model) first, e.g. GenericSingleResponse[Foo].
FastAPI deals with this just fine and can generate the correct OpenAPI documentation. The JSON schema for GenericSingleResponse[Foo] looks like this:
{
"title": "GenericSingleResponse[Foo]",
"type": "object",
"properties": {
"status": {
"title": "Status",
"type": "string"
},
"data": {
"$ref": "#/definitions/GenericSingleObject_Foo_"
}
},
"required": [
"status",
"data"
],
"definitions": {
"Foo": {
"title": "Foo",
"type": "object",
"properties": {
"a": {
"title": "A",
"type": "string"
},
"b": {
"title": "B",
"type": "integer"
}
},
"required": [
"a",
"b"
]
},
"GenericSingleObject_Foo_": {
"title": "GenericSingleObject[Foo]",
"type": "object",
"properties": {
"object": {
"$ref": "#/definitions/Foo"
}
},
"required": [
"object"
]
}
}
}
To demonstrate it with FastAPI:
from fastapi import FastAPI
app = FastAPI()
#app.get("/foo/", response_model=GenericSingleResponse[Foo])
async def get_one_foo() -> dict[str, object]:
return {"status": "foo", "data": {"object": {"a": "spam", "b": 123}}}
Sending a request to that route returns the following:
{
"status": "foo",
"data": {
"object": {
"a": "spam",
"b": 123
}
}
}
Dynamically created model
If you actually want the attribute name to also be different every time, that is obviously no longer possible with static type annotations. In that case we would have to resort to actually creating the model type dynamically via pydantic.create_model.
In that case there is really no point in genericity anymore because type safety is out of the window anyway, at least for the data model. We still have the option to define a GenericResponse model, which we can specify via our dynamically generated models, but this will make every static type checker mad, since we'll be using variables for types. Still, it might make for otherwise concise code.
We just need to define an algorithm for deriving the model parameters:
from typing import Any, Generic, Optional, TypeVar
from pydantic import BaseModel, create_model
from pydantic.generics import GenericModel
M = TypeVar("M", bound=BaseModel)
def create_data_model(
model: type[BaseModel],
plural: bool = False,
custom_plural_name: Optional[str] = None,
**kwargs: Any,
) -> type[BaseModel]:
data_field_name = model.__name__.lower()
if plural:
model_name = f"Multiple{model.__name__}"
if custom_plural_name:
data_field_name = custom_plural_name
else:
data_field_name += "s"
kwargs[data_field_name] = (list[model], ...) # type: ignore[valid-type]
else:
model_name = f"Single{model.__name__}"
kwargs[data_field_name] = (model, ...)
return create_model(model_name, **kwargs)
class GenericResponse(GenericModel, Generic[M]):
status: str
data: M
Using the same Foo and Bar examples as before:
class Foo(BaseModel):
a: str
b: int
class Bar(BaseModel):
x: float
SingleFoo = create_data_model(Foo)
MultipleBar = create_data_model(Bar, plural=True)
This also works as expected with FastAPI including the automatically generated schemas/documentations:
from fastapi import FastAPI
app = FastAPI()
#app.get("/foo/", response_model=GenericResponse[SingleFoo]) # type: ignore[valid-type]
async def get_one_foo() -> dict[str, object]:
return {"status": "foo", "data": {"foo": {"a": "spam", "b": 123}}}
#app.get("/bars/", response_model=GenericResponse[MultipleBar]) # type: ignore[valid-type]
async def get_multiple_bars() -> dict[str, object]:
return {"status": "bars", "data": {"bars": [{"x": 3.14}, {"x": 0}]}}
Output is essentially the same as with the first approach.
You'll have to see, which one works better for you. I find the second option very strange because of the dynamic key/field name. But maybe that is what you need for some reason.
This question already has answers here:
FastApi 422 Unprocessable Entity, on authentication, how to fix?
(2 answers)
Closed last month.
I'm getting this error while trying to accept a pedantic model. After debugging for quite some time I believe the problem is with accepting CodeCreate
Pydantic model
class BaseCode(BaseModel):
index: Optional[int] = Field(None)
email: EmailStr
gen_time: datetime
expire_time: datetime
class CodeCreate(BaseCode):
code: int
used_time: Optional[datetime] = Field(None)
class Config:
orm_mode = True
class Message(BaseModel):
message: str
Code ORM
class Code(Base):
__tablename__ = 'code'
index = Column(Integer, primary_key=True, autoincrement=True)
code = Column(Integer)
email = Column(String, ForeignKey('user.email'))
gen_time = Column(DateTime)
expire_time = Column(DateTime)
used_time = Column(DateTime, nullable=True)
Handler
#app.post('/verify-code', response_model=schemas.Message, responses={404: {"model": schemas.Message}, 406: {"model": schemas.Message}})
async def verify_code(code: schemas.CodeCreate, response: Response, device_name: str = Body(..., embed=True), db=Depends(get_db)):
result = crud.verify_and_insert_code(db=db, code=code)
if result == 'matched':
response.status_code = status.HTTP_202_ACCEPTED
return crud.start_new_session(db=db, session=schemas.Session(session_id='1234', start_time=datetime.now(), email=code.email, device_name=device_name))
elif result == 'not-matched':
response.status_code = status.HTTP_200_OK
elif result == 'expire':
response.status_code = status.HTTP_406_NOT_ACCEPTABLE
elif result == 'invalid':
response.status_code = status.HTTP_404_NOT_FOUND
return schemas.Message(message="Item not found")
Body of request from the client
{
"code": {
"index": 0,
"email": "user#example.com",
"gen_time": "2022-01-24T16:38:12.612Z",
"expire_time": "2022-01-24T16:38:12.612Z",
"code": 0,
"used_time": "2022-01-24T16:38:12.612Z"
},
"device_name": "string"
}
Response body for 422
{
"detail": [
{
"loc": [
"body",
"code",
"email"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body",
"code",
"gen_time"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body",
"code",
"expire_time"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"body",
"code",
"code"
],
"msg": "field required",
"type": "value_error.missing"
}
]
}
Temporary Solution
Removing this ORM confirmation code from Pydantic model solves the issue. I believe there might be a clash between CodeCreate pydantic model and Code ORM model, but I don't know how to resolve it.
class Config:
orm_mode = True
The 422 Unprocessable Entity error because of ContentType is incorrect. The FastAPI/Pydantic need ContentType = application/json to parse request body.
Are you sure your POST request has ContentType header is application/json?
If not add it!
According to MDN
here,
a 422 Unprocessable Entity means that the information of the request could not be processed.
In this case, the most likely problem is that the data of the POST request that is sent does not match with the Pydantic model.
Make sure the data that is sent is in the correct format.
Just as stated above by Brian Law, your request body is in the form of Code, which is not a pydantic model, but a database one.
When you send the POST request, the body should match up with CodeCreate, not Code.
Found the solution after debugging for quite a while.
The ORM configured pedantic model can't be used to receive requests from the client.
In my case, I had to create another class that extends the CodeCreate class add ORM configuration to that class and use CodeCreate for body from the client.
class BaseCode(BaseModel):
index: Optional[int] = Field(None)
email: EmailStr
gen_time: datetime
expire_time: datetime
class CodeCreate(BaseCode):
code: int
used_time: Optional[datetime] = Field(None)
class Code(CodeCreate):
class Config:
orm_mode = True
Post request
#app.post('/verify-code')
async def verify_code(code: schemas.CodeCreate):
return 'success'
I'm struggling to build a multi-nested dictionary that I'll be using to add to my collection in mongodb. I'm questioning the approach and my attempt at the the solution.
Here is the problem:
I built a function that identifies the delta's between the local collection data and updates I'm getting from my golden source.
The function produces a dictionary of all the delta's. The dictionary contains the tag as the key, and the new delta update as the value.
I then pass the delta dictionary and current data dictionary to another function, who is responsible for doing the following:
identifying the old value and new value using the delta.key()
building a new dictionary which should contain the a full path to the nested dictionary, which will contain only two values: newValue and oldValue.
What I'm struggling with is that when I do the four loop, it just seems to overwrite the the previous record. The data should get appended. If the path exists, when the update should only be adding to the delta's. Unless the value already exists then I can understand the update.
For example:
Same date -> Different Tags: should append the new tag and it's oldvalue and newvalue.
Same date -> Same tag: Should update the existing tag's
The reason I am trying to do this in this manner is so that I can avoid multiple calls and updates to the collection. Ideally stick to one update.
But my concerns are the following:
Is this the best approach when working with nested dictionaries and MongoDB ?
What issues will this cause when I go to update mongodb using "pymongo". I'm worried it's going to over ride existing records on the update. I want the records to be appended not overwritten.
Is there a different approach that would make more sense?
This is my first attempt 1:
def update_record(_collection, _key, _data, _delta):
today = date.today()
today_formatted = today.strftime("%Y-%m-%d")
_query_criteria = {_key: _data[_key]}
_update_values = {}
_append_delta = {}
x = 0
for delta_key in _delta.keys():
_update_values = {delta_key: _delta[delta_key]}
_append_delta["delta"]["byType"][delta_key][today_formatted] = {"oldValue": _data[delta_key],
"newValue": _delta[delta_key]}
_append_delta["delta"]["byDate"][today_formatted][delta_key] = {"oldValue": _data[delta_key],
"newValue": _delta[delta_key]}
Attempt 2:
def update_record(_collection, _key, _data, _delta):
today = date.today()
today_formatted = today.strftime("%Y-%m-%d")
_query_criteria = {_key: _data[_key]}
_update_values = {}
_append_delta = {}
x = 0
for delta_key in _delta.keys():
_update_values = {delta_key: _delta[delta_key]}
x_dict = {}
y_dict = {}
if x == 0:
_append_delta["delta"]["byType"] = {delta_key: {today_formatted: {}}}
_append_delta["delta"]["byDate"][today_formatted] = {delta_key: {}}
x += 1
_append_delta["delta"]["byType"][delta_key][today_formatted] = {"oldValue": _data[delta_key],
"newValue": _delta[delta_key]}
_append_delta["delta"]["byDate"][today_formatted][delta_key] = {"oldValue": _data[delta_key],
"newValue": _delta[delta_key]}
else:
_append_delta.update(
{"delta":
{"byType": {
delta_key: {today_formatted: {"oldValue": _data[delta_key], "newValue": _delta[delta_key]}}},
"byDate": {
today_formatted: {delta_key: {"oldValue": _data[delta_key], "newValue": _delta[delta_key]}}}
}
}
)
Example of what I want the collection to look like in MongoDB:
[{name: "Apple",
ticker: "appl",
description: "Apple Computers",
currency: "usd",
delta: {
byTag: {
name: {
"2021-06-01": {
oldValue: "appl",
newValue: "Apple"
}
},
description: {
"2021-06-06": {
oldValue: "Apple",
newValue: "Apple Computers"
}
}
},
byDate: {
"2021-06-01": {
name: {
oldValue: "appl",
newValue: "Apple"
}
},
"2021-06-06": {
description: {
oldValue: "Apple",
newValue: "Apple Computers"
}
}
}
}
}]
You have a lot of questions here. You may get a better response if you break them down into bite-size issues.
In terms of dealing with changes to your data, you might want to take a look at dictdiffer. Like a lot of things in python there's usually a good library to achieve what you are looking to do. It won't give you the format you are looking for but will give you a format that the community has determined is best practice for this sort of problem. You get extra great stuff as well, like being able to patch old records with the delta.
Separately, with nested dicts, I think it's easier to create them based on the object structure rather than relying on building from keys. It more verbose but clearer in my opinion. The code below is a sample using classes to give you an idea of this concept:
from pymongo import MongoClient
from datetime import date
from bson.json_util import dumps
db = MongoClient()['mydatabase']
class UpdateRecord:
def __init__(self, name, ticker, description, currency, delta):
self.name = name
self.ticker = ticker
self.description = description
self.currency = currency
self.date = date.today().strftime("%Y-%m-%d")
self.delta = delta
# Need to code to work out the deltas
def by_tags(self):
tags = dict()
for tag in ['name', 'description']:
tags.update({
tag: {
self.date: {
'oldValue': "appl",
'newValue': "Apple"
}
}
})
return tags
def by_date(self):
dates = dict()
for dt in ['2021-06-01', '2021-06-06']:
dates.update({
dt: {
self.date: {
'oldValue': "appl",
'newValue': "Apple"
}
}
})
return dates
def to_dict(self):
return {
'name': self.name,
'ticker': self.ticker,
'description': self.description,
'currency': self.currency,
'delta': {
'byTag': self.by_tags(),
'byDate': self.by_date()
}
}
def update(self, _id):
db.mycollection.update_one({'_id': _id}, {'$push': {'Updates': self.to_dict()}})
delta = {
'oldValue': "appl",
'newValue': "Apple"
}
#
# Test it out
#
dummy_record = {'a': 1}
db.mycollection.insert_one(dummy_record)
record = db.mycollection.find_one()
update_record = UpdateRecord(name='Apple', ticker='appl', description='Apple Computer', currency='usd', delta=delta)
update_record.update(record.get('_id'))
print(dumps(db.mycollection.find_one({}, {'_id': 0}), indent=4))
prints:
{
"a": 1,
"Updates": [
{
"name": "Apple",
"ticker": "appl",
"description": "Apple Computer",
"currency": "usd",
"delta": {
"byTag": {
"name": {
"2021-08-14": {
"oldValue": "appl",
"newValue": "Apple"
}
},
"description": {
"2021-08-14": {
"oldValue": "appl",
"newValue": "Apple"
}
}
},
"byDate": {
"2021-06-01": {
"2021-08-14": {
"oldValue": "appl",
"newValue": "Apple"
}
},
"2021-06-06": {
"2021-08-14": {
"oldValue": "appl",
"newValue": "Apple"
}
}
}
}
}
]
}
I want to create a generic endpoint definition in Fast API Python that reads URL path parameter and then calls a specific method to do a derealisation.
But I always get
422 Unprocessable Entity
So I expect that it works like so:
/answer/aaa -> handle_generic_answer -> read_item_aaa, type body to ModelAAA
/answer/bbb -> handle_generic_answer -> read_item_bbb, type body to ModelBBB
etc.
Here's the generic endpoint code:
#app.post("/answer/{type}")
def handle_generic_answer(type: str, item):
# I also tried
# def handle_generic_answer(type: str, item: Any):
# or
# def handle_generic_answer(type: str, item: Optional):
switcher = {
'aaaa': read_item_aaa,
'bbb': read_item_bbb,
'nothing': unrecognised_answer
}
func = switcher.get(type, unrecognised_answer)
print('answer >> ' + type)
func(item)
then I have separate methods called based on a type value:
def read_item_aaa(item: ModelAAA):
update_aaa(item)
return {"type": "aaa", "result": "success"}
def read_item_bbb(item: ModelBBB):
update_bbb(item)
return {"type": "bbb", "result": "success"}
and a default -
def unrecognised_answer(type):
print("unrecognised_answer")
raise HTTPException(status_code=400, detail="answer type not found")
return {}
models are defined like this:
from pydantic import BaseModel, Field
class ModelAAA(BaseModel):
field1: str
field2: list = []
But whether I call
http://localhost:8000/answer/aaa
or http://localhost:8000/answer/some-other-url
I always get 422:
{
"detail": [
{
"loc": [
"query",
"item"
],
"msg": "field required",
"type": "value_error.missing"
}
]
}
You forgot to annotate body parameter item.
Without this item is treated as query str parameter. For example:
#app.post("/answer/{type}")
def handle_generic_answer(type: str, item: Union[ModelAAA, ModelBBB]):
I want to use the name and description present in tags_metadata inside the method. I don't how to do that is there any way I can use those attributes
from fastapi import FastAPI
tags_metadata = [
{
"name": "select name",
"description": "Operations with users. The **login** logic is also here.",
},
{
"name": "items",
"description": "Manage items. So _fancy_ they have their own docs.",
"externalDocs": {
"description": "Items external docs",
"url": "https://fastapi.tiangolo.com/",
},
},
]
app = FastAPI(openapi_tags=tags_metadata)
#app.get("/users/", tags=["users"])
async def get_users():
return [{"name": "Harry"}, {"name": "Ron"}]
#app.get("/items/", tags=["items"])
async def get_items():
return [{"name": "wand"}, {"name": "flying broom"}]
here I want to select the first name attribute from tags_metadata in the get_users() method
output need to be -- > "select name"
https://fastapi.tiangolo.com/tutorial/metadata/?h=+tags#use-your-tags
Yes there is.
from fastapi import FastAPI
app = FastAPI()
#app.get("/dummy", tags=["dummy2"])
async def dummy():
...
#app.post("/dummy2", tags=["dummy2"])
async def dummy2():
...
Let's assume we have these routes. By checking each route from app.router, we can find the paths.
#app.get("/tags", tags=["tags"])
async def tags():
tags = {}
for route in app.router.__dict__["routes"]:
if hasattr(route, "tags"):
tags[route.__dict__["path"]] = route.__dict__["tags"]
return tags
When we hit the /tags endpoint we will see this.
{
"/dummy":[
"dummy2"
],
"/dummy2":[
"dummy2"
],
"/tags":[
"tags"
]
}