I want to make "partial update" endpoint, but don't want too allow passing null in any field.
Here is the guide from fastapi https://fastapi.tiangolo.com/tutorial/body-updates/#partial-updates-with-patch :
class Item(BaseModel):
name: Optional[str] = None
description: Optional[str] = None
price: Optional[float] = None
tax: float = 10.5
tags: List[str] = []
#app.patch("/items/{item_id}", response_model=Item)
async def update_item(item_id: str, item: Item):
...
update_data = item.dict(exclude_unset=True)
...
With this approach user can pass {"name": null} and corrupt database, because in my case name should always be a string.
So what should I do? The only approach I see so far is playing around with some sentinel objects (using them as "unset" marker instead of None), but this seems hacky and I doubt that pydantic will allow me to do this.
You could use exclude_none in order to exclude values that are equal to None.
Example
item.dict(exclude_none=True)
Source: Pydantic docs
Related
I'm creating an API (FastAPI) that can create database in my catalog. The python function that creates the db takes few arguments. Some are optional (like Description, LocationUri, Parameters) and some are mandatory (CatalogId, etc). I created a Pydantic model that defines these arguments.
class createGlueDatabaseDatabaseInput(BaseModel):
Name: str
Description: Optional[str] = None
LocationUri: Optional[str] = None
Parameters: Optional[dict] = None
class Config:
orm_mode = True
class createGlueDatabase(BaseModel):
CatalogId: str
DB_input: createGlueDatabaseDatabaseInput
class Config:
orm_mode = True
In the above, the catalog id is the only mandatory argument, and the rest are optional. So when the optional parameters are ignored or not provided in the swagger, those values are coming in as "None" to the function. This results in the function failing.
I tried doing the following in my code:
Added response_model_exclude_none=True to my router function that receives the input argument, but this didnt help.
Tried to create everything (other than the catalog id) as optional, still no success.
Can someone help me understand, how to ignore None being sent to my python function? Please let me know if you need any other details. Thanks in advance.
Tried using Pydantic models:
class createGlueDatabaseDatabaseInput(BaseModel):
Name: str
Description: Optional[str] = None
LocationUri: Optional[str] = None
Parameters: Optional[dict] = None
class Config:
orm_mode = True
class createGlueDatabase(BaseModel):
CatalogId: str
DB_input: createGlueDatabaseDatabaseInput
class Config:
orm_mode = True
Tried adding additional args like response_model_exclude_none=True, but didnt work.
Can you provide more code for a better answer?
I think that you need to ignore None in your my python function. This is normal practice. See for example https://fastapi.tiangolo.com/tutorial/body/#use-the-model.
class createGlueDatabaseDatabaseInput(BaseModel):
Name: str
Description: Optional[str] = None
LocationUri: Optional[str] = None
Parameters: Optional[dict] = None
class Config:
orm_mode = True
app = FastAPI()
#app.post("/my_route")
async def create_item(input_data: createGlueDatabaseDatabaseInput):
# here
...
If you pass the input to the POST with the following body
{
"Name": "MyName"
}
then in the place where the comment # here we have
createGlueDatabaseDatabaseInput(Name='MyName', Description=None, LocationUri=None, Parameters=None)
It is serialization of pydantic.
For the Parameters field you can to add the following initial value
class createGlueDatabaseDatabaseInput(BaseModel):
Name: str
Description: Optional[str] = None
LocationUri: Optional[str] = None
Parameters: Optional[dict] = Field(default_factory=dict)
class Config:
orm_mode = True
Then we will have
createGlueDatabaseDatabaseInput(Name='MyName', Description=None, LocationUri=None, Parameters={})
where createGlueDatabaseDatabaseInput.Parameters is empty dict.
I have this model:
class Text(BaseModel):
id: str
text: str = None
class TextsRequest(BaseModel):
data: list[Text]
n_processes: Union[int, None]
So I want to be able to take requests like:
{"data": ["id": "1", "text": "The text 1"], "n_processes": 8}
and
{"data": ["id": "1", "text": "The text 1"]}.
Right now in the second case I get
{'data': [{'id': '1', 'text': 'The text 1'}], 'n_processes': None}
using this code:
app = FastAPI()
#app.post("/make_post/", response_model_exclude_none=True)
async def create_graph(request: TextsRequest):
input_data = jsonable_encoder(request)
So how can I exclude n_processes here?
You can use exclude_none param of Pydantic's model.dict(...):
class Text(BaseModel):
id: str
text: str = None
class TextsRequest(BaseModel):
data: list[Text]
n_processes: Optional[int]
request = TextsRequest(**{"data": [{"id": "1", "text": "The text 1"}]})
print(request.dict(exclude_none=True))
Output:
{'data': [{'id': '1', 'text': 'The text 1'}]}
Also, it's more idiomatic to write Optional[int] instead of Union[int, None].
Pydantic provides the following arguments for exporting models using the model.dict(...) method:
exclude_unset: whether fields which were not explicitly set when
creating the model should be excluded from the returned dictionary;
default False
exclude_none: whether fields which are equal to None should be
excluded from the returned dictionary; default False
Since you are refering to excluding optional unset parameters, you can use the first method (i.e., exclude_unset). This is useful when one would like to exclude a parameter only if it has not been set to either some value or None.
The exclude_none argument, however, ignores that fact that an attribute may have been intentionally set to None, and hence, excludes it from the returned dictionary.
Example:
from pydantic import BaseModel
from typing import List, Union
class Text(BaseModel):
id: str
text: str = None
class TextsRequest(BaseModel):
data: List[Text] # in Python 3.9+ you can use: data: list[Text]
n_processes: Union[int, None] = None
t = TextsRequest(**{'data': [{'id': '1', 'text': 'The text 1'}], 'n_processes': None})
print(t.dict(exclude_none=True))
#> {'data': [{'id': '1', 'text': 'The text 1'}]}
print(t.dict(exclude_unset=True))
#> {'data': [{'id': '1', 'text': 'The text 1'}], 'n_processes': None}
About Optional Parameters
Using Union[int, None] is the same as using Optional[int] (both are equivalent). The most important part, however, to make a parameter optional is the part = None.
As per FastAPI documentation (see admonition Note and Info in the link provided):
Note
FastAPI will know that the value of q is not required because of the
default value = None.
The Union in Union[str, None] will allow your editor to give you
better support and detect errors.
Info
Have in mind that the most important part to make a parameter optional
is the part: = None, as it will use that None as the default value, and that way make the
parameter not required.
The Union[str, None] part allows your editor to provide better
support, but it is not what tells FastAPI that this parameter is
not required.
Hence, regardless of the option you may choose to use, if it is not followed by the = None part, FastAPI won't know that the value of the parameter is optional, and hence, the user will have to provide some value for it. One can also check that through the auto-generated API docs at http://127.0.0.1:8000/docs, where the parameter or request body will appear as a Required field.
For example, any of the below would require the user to pass some body content in their request for the TextsRequest model:
#app.post("/upload")
def upload(t: Union[TextsRequest, None]):
pass
#app.post("/upload")
def upload(t: Optional[TextsRequest]):
pass
If, however, the above TextsRequest definitions were succeeded by = None, for example:
#app.post("/upload")
def upload(t: Union[TextsRequest, None] = None):
pass
#app.post("/upload")
def upload(t: Optional[TextsRequest] = None):
pass
#app.post("/upload")
def upload(t: TextsRequest = None): # this should work as well
pass
the parameter (or body) would be optional, as = None would tell FastAPI that this parameter is not required.
In Python 3.10+
The good news is that in Python 3.10 and above, you don't have to worry about names like Optional and Union, as you can simply use the vertical bar | (also called bitwise or operator, but that meaning is not relevant here) to define an optional parameter (or simply, unions of types). However, the same rule applies to this option as well, i.e., you would still need to add the = None part, if you would like to make the parameter optional (as demonstrated in the example given below).
Example:
#app.post("/upload")
def upload(t: TextsRequest | None = None):
pass
Is there some way to do the following?
from typing import Optional
from fastapi import FastAPI
from pydantic import BaseModel
class Item(BaseModel):
id: str
name: str
description: Optional[str] = None
price: float
tax: Optional[float] = None
app = FastAPI()
#app.post("/items/{item_id}")
async def create_item(item: Item):
return item
I want to have the item_id path parameter value inside the Item model.
Is it possible?
Pydantic in FastAPI is used to define data model. You can do as follows:
from typing import Optional
from fastapi import FastAPI
from pydantic import BaseModel
class Item(BaseModel):
id: Optional[str] = None
name: str
description: Optional[str] = None
price: float
tax: Optional[float] = None
app = FastAPI()
#app.post("/items/{item_id}")
async def create_item(item_id, item: Item):
item.id = item_id
return item
Please, note that id is declared as optional to avoid validation problems with body parameters in the request. Indeed, you are not going to pass the id in the body but in the path.
If necessary, you can also decouple the request- from the response-model (see here). It depends on what you need.
Replace your id: str with id: Optional [str] = None
Replace your create_item (item: Item) with create_item (item_id, item: Item):
Add item.id = item_id under your async. Obviously then you leave the ruo return item
My intention
So, I am developing an API package for one service. I want to make good typehints for every method, which exists in my library
For example, when user types get()., after the dot pycharm will let him know, what response this method will provide.
e.g:
info = get()
info. # and here IDE help with hints.
Pitfalls
But, there are some methods, which provide different responses depending of parameters in methods.
e.g.
# this method responses with object, containing fields:
# count - count of items
# items - list of ids of users
info = get()
# but this method will give additional information. It responses with object, containing fields:
# count - count of items
# items - list of objects with users' information. It has fields:
# id - id of user
# firstname - firstname of user
# lastname - lastname of user
# ... and some others
info = get(fields='firstname')
Objects structure
Now I have such structure (i't simplified)
from typing import List, Union
from pydantic import BaseModel, Field
class UserInfo(BaseModel):
id: int = Field(...)
firstname: str = Field(None)
lastname: str = Field(None)
some_other_fields: str = Field(None)
class GetResponseNoFields(BaseModel):
count: int = Field(...)
items: List[int] = Field(...)
class GetResponseWithFields(BaseModel):
count: int = Field(...)
items: List[UserInfo] = Field(...)
class GetResponseModel(BaseModel):
response: Union[GetResponseNoFields, GetResponseWithFields] = Field(...)
def get(fields=None) -> GetResponseModel:
# some code
pass
The problem
The problem is, when I type get(fields='firsttname').response.items[0]. pycharm shows me typehints only for int. He doesn't think, that items can contain List[UserInfo], he thinks, it only can have List[int]
I have tried
I've tried to use typing.overload decorator, but method 'get' has many parameters, and actually doesn't support default parameter values. Or maybe i didn't do it properly
Here what I have tried with overload (simlified). It didn't work because of 'some_other_param', but I leave it here just in case:
from typing import overload
#overload
def get(fields: None) -> GetResponseNoFields: ...
#overload
def get(fields: str) -> GetResponseWithFields: ...
def get(some_other_param=None, fields=None):
# code here
pass
When I try to call method without parameters, pycharm says, that "Some of the parameters is unfilled"
I want to implement a put or patch request in FastAPI that supports partial update. The official documentation is really confusing and I can't figure out how to do the request. (I don't know that items is in the documentation since my data will be passed with request's body, not a hard-coded dict).
class QuestionSchema(BaseModel):
title: str = Field(..., min_length=3, max_length=50)
answer_true: str = Field(..., min_length=3, max_length=50)
answer_false: List[str] = Field(..., min_length=3, max_length=50)
category_id: int
class QuestionDB(QuestionSchema):
id: int
async def put(id: int, payload: QuestionSchema):
query = (
questions
.update()
.where(id == questions.c.id)
.values(**payload)
.returning(questions.c.id)
)
return await database.execute(query=query)
#router.put("/{id}/", response_model=QuestionDB)
async def update_question(payload: QuestionSchema, id: int = Path(..., gt=0),):
question = await crud.get(id)
if not question:
raise HTTPException(status_code=404, detail="question not found")
## what should be the stored_item_data, as documentation?
stored_item_model = QuestionSchema(**stored_item_data)
update_data = payload.dict(exclude_unset=True)
updated_item = stored_item_model.copy(update=update_data)
response_object = {
"id": question_id,
"title": payload.title,
"answer_true": payload.answer_true,
"answer_false": payload.answer_false,
"category_id": payload.category_id,
}
return response_object
How can I complete my code to get a successful partial update here?
Posting this here for googlers who are looking for an intuitive solution for creating Optional Versions of their pydantic Models without code duplication.
Let's say we have a User model, and we would like to allow for PATCH requests to update the User. But we need to create a schema that tells FastApi what to expect in the content body, and specifically that all the fields are Optional (Since that's the nature of PATCH requests). We can do so without redefining all the fields
from pydantic import BaseModel
from typing import Optional
# Creating our Base User Model
class UserBase(BaseModel):
username: str
email: str
# And a Model that will be used to create an User
class UserCreate(UserBase):
password: str
Code Duplication ❌
class UserOptional(UserCreate):
username: Optional[str]
email: Optional[str]
password: Optional[str]
One Liner ✅
# Now we can make a UserOptional class that will tell FastApi that all the fields are optional.
# Doing it this way cuts down on the duplication of fields
class UserOptional(UserCreate):
__annotations__ = {k: Optional[v] for k, v in UserCreate.__annotations__.items()}
NOTE: Even if one of the fields on the Model is already Optional, it won't make a difference due to the nature of Optional being typing.Union[type passed to Optional, None] in the background.
i.e typing.Union[str, None] == typing.Optional[str]
You can even make it into a function if your going to be using it more than once:
def convert_to_optional(schema):
return {k: Optional[v] for k, v in schema.__annotations__.items()}
class UserOptional(UserCreate):
__annotations__ = convert_to_optional(UserCreate)
I got this answer on the FastAPI's Github issues.
You could make the fields Optional on the base class and create a new QuestionCreate model that extends the QuestionSchema. As an example:
from typing import Optional
class Question(BaseModel):
title: Optional[str] = None # title is optional on the base schema
...
class QuestionCreate(Question):
title: str # Now title is required
The cookiecutter template here provides some good insight too.
I created a library (pydantic-partial) just for that, converting all the fields in the normal DTO model to being optional. See https://medium.com/#david.danier/how-to-handle-patch-requests-with-fastapi-c9a47ac51f04 for a code example and more detailed explanation.
https://github.com/team23/pydantic-partial/
Based on the answer of #cdraper, I made a partial model factory:
from typing import Mapping, Any, List, Type
from pydantic import BaseModel
def model_annotations_with_parents(model: BaseModel) -> Mapping[str, Any]:
parent_models: List[Type] = [
parent_model for parent_model in model.__bases__
if (
issubclass(parent_model, BaseModel)
and hasattr(parent_model, '__annotations__')
)
]
annotations: Mapping[str, Any] = {}
for parent_model in reversed(parent_models):
annotations.update(model_annotations_with_parents(parent_model))
annotations.update(model.__annotations__)
return annotations
def partial_model_factory(model: BaseModel, prefix: str = "Partial", name: str = None) -> BaseModel:
if not name:
name = f"{prefix}{model.__name__}"
return type(
name, (model,),
dict(
__module__=model.__module__,
__annotations__={
k: Optional[v]
for k, v in model_annotations_with_parents(model).items()
}
)
)
def partial_model(cls: BaseModel) -> BaseModel:
return partial_model_factory(cls, name=cls.__name__)
Can be used with the function partial_model_factory:
PartialQuestionSchema = partial_model_factory(QuestionSchema)
Or with decorator partial_model:
#partial_model
class PartialQuestionSchema(QuestionSchema):
pass