Develop Reference

Initialize pydantic model with missing parameters

I have the following two pydantic models for a Users table in my database for a FastAPI application:
from fastapi import Form
from pydantic import BaseModel
class UserCreate(BaseModel):
username: str = Form(...)
password: str = Form(...)
class UserInDb(BaseModel):
id: int
username:str
hashed_password: str
I use the UserCreate class to get form data needed from the client to create a user and then I map it to UserInDb. This is how I initially implemented it:
password = user_create_instance.password
hashed_password = some_hashing_function(password)
user_in_db = UserInDb(**user_create_instance.dict(), hashed_password=hashed_password)
This however throws a value_error.missing error because user_create_instance does not have an id parameter which UserInDb expects to be passed in but the value for that comes from the database.
This issue is fixable by doing something like id: int = None when defining the attribute in the class but it doesn't feel right to do it this way. Is there a better approach?

OpenAPI is missing schemas for some of the Pydantic models in FastAPI app

I am building a FastAPI application, which has a lot of Pydantic models. Even though the application is working just fine, as expected the OpenAPI (Swagger UI) docs do not show the schema for all of these models under the Schemas section.
Here are the contents of pydantic schemas.py
import socket
from datetime import datetime
from enum import Enum
from typing import Any, Dict, List, Optional, Set, Union
from pydantic import BaseModel, Field, validator
from typing_extensions import Literal
ResponseData = Union[List[Any], Dict[str, Any], BaseModel]
# Not visible in Swagger UI
class PageIn(BaseModel):
page_size: int = Field(default=100, gt=0)
num_pages: int = Field(default=1, gt=0, exclude=True)
start_page: int = Field(default=1, gt=0, exclude=True)
# visible under schemas on Swagger UI
class PageOut(PageIn):
total_records: int = 0
total_pages: int = 0
current_page: int = 1
class Config: # pragma: no cover
#staticmethod
def schema_extra(schema, model) -> None:
schema.get("properties").pop("num_pages")
schema.get("properties").pop("start_page")
# Not visible in Swagger UI
class BaseResponse(BaseModel):
host_: str = Field(default_factory=socket.gethostname)
message: Optional[str]
# Not visible in Swagger UI
class APIResponse(BaseResponse):
count: int = 0
location: Optional[str]
page: Optional[PageOut]
data: ResponseData
# Not visible in Swagger UI
class ErrorResponse(BaseResponse):
error: str
# visible under schemas on Swagger UI
class BaseFaultMap(BaseModel):
detection_system: Optional[str] = Field("", example="obhc")
fault_type: Optional[str] = Field("", example="disk")
team: Optional[str] = Field("", example="dctechs")
description: Optional[str] = Field(
"",
example="Hardware raid controller disk failure found. "
"Operation can continue normally,"
"but risk of data loss exist",
)
# Not visible in Swagger UI
class FaultQueryParams(BaseModel):
f_id: Optional[int] = Field(None, description="id for the host", example=12345, title="Fault ID")
hostname: Optional[str]
status: Literal["open", "closed", "all"] = Field("open")
created_by: Optional[str]
environment: Optional[str]
team: Optional[str]
fault_type: Optional[str]
detection_system: Optional[str]
inops_filters: Optional[str] = Field(None)
date_filter: Optional[str] = Field("",)
sort_by: Optional[str] = Field("created",)
sort_order: Literal["asc", "desc"] = Field("desc")
All of these models are actually being used in FastAPI paths to validate the request body. The FaultQueryParams is a custom model, which I use to validate the request query params and is used like below:
query_args: FaultQueryParams = Depends()
The rest of the models are being used in conjunction with Body field. I am not able to figure out why only some of the models are not visible in the Schemas section while others are.
Also another thing I noticed about FaultQueryParams is that the description, examples do not show up against the path endpoint even though they are defined in the model.
Edit 1:
I looked more into and realized that all of the models which are not visible in swagger UI are the ones that are not being used directly in path operations i.e., these models are not being used as response_model or Body types and are sort of helper models which are being used indirectly. So, it seems like FastAPI is not generating the schema for these models.
One exception to the above statement is query_args: FaultQueryParams = Depends() which is being used directly in a path operation to map the Query params for the endpoint against a custom model. This is a problem because swagger is not identifying the meta parameters like title, description, example from the fields of this model & not showing on the UI which is important for the users of this endpoint.
Is there a way to trick FastAPI to generate schema for the custom model FaultQueryParams just like it generates for Body, Query etc ?

FastAPI will generate schemas for models that are used either as a Request Body or Response Model. When declaring query_args: FaultQueryParams = Depends() (using Depends), your endpoint would not expect a request body, but rather query parameters; hence, FaultQueryParams would not be included in the schemas of the OpenAPI docs.
To add additional schemas, you could extend/modify the OpenAPI schema. Example is given below (make sure to add the code for modifying the schema after all routes have been defined, i.e., at the end of your code).
class FaultQueryParams(BaseModel):
f_id: Optional[int] = Field(None, description="id for the host", example=12345, title="Fault ID")
hostname: Optional[str]
status: Literal["open", "closed", "all"] = Field("open")
...
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args
def get_extra_schemas():
return {
"FaultQueryParams": {
"title": "FaultQueryParams",
"type": "object",
"properties": {
"f_id": {
"title": "Fault ID",
"type": "integer",
"description": "id for the host",
"example": 12345
},
"hostname": {
"title": "Hostname",
"type": "string"
},
"status": {
"title": "Status",
"enum": [
"open",
"closed",
"all"
],
"type": "string",
"default": "open"
},
...
}
}
}
from fastapi.openapi.utils import get_openapi
def custom_openapi():
if app.openapi_schema:
return app.openapi_schema
openapi_schema = get_openapi(
title="FastAPI",
version="1.0.0",
description="This is a custom OpenAPI schema",
routes=app.routes,
)
new_schemas = openapi_schema["components"]["schemas"]
new_schemas.update(get_extra_schemas())
openapi_schema["components"]["schemas"] = new_schemas
app.openapi_schema = openapi_schema
return app.openapi_schema
app.openapi = custom_openapi
Some Helpful Notes
Note 1
Instead of manually typing the schema for the extra models that you would like to add to the docs, you can have FastAPI do that for you by adding to your code an endpoint (which you would subsequently remove, after getting the schema) using that model as a request body or response model, for example:
#app.post('/predict')
def predict(query_args: FaultQueryParams):
return query_args
Then, you can get the generated JSON schema at http://127.0.0.1:8000/openapi.json, as described in the documentation. From there, you can either copy and paste the schema of the model to your code and use it directly (as shown in the get_extra_schema() method above) or save it to a file and load the JSON data from the file, as demonstrated below:
import json
...
new_schemas = openapi_schema["components"]["schemas"]
with open('extra_schemas.json') as f:
extra_schemas = json.load(f)
new_schemas.update(extra_schemas)
openapi_schema["components"]["schemas"] = new_schemas
...
Note 2
To declare metadata, such as description, example, etc, for your query parameter, you should define your parameter with Query instead of Field, and since you can't do that with Pydantic models, you could declare a custom dependency class, as decribed here and as shown below:
from fastapi import FastAPI, Query, Depends
from typing import Optional
class FaultQueryParams:
def __init__(
self,
f_id: Optional[int] = Query(None, description="id for the host", example=12345)
):
self.f_id = f_id
app = FastAPI()
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args
The above can be re-written using the #dataclass decorator, as shown below:
from fastapi import FastAPI, Query, Depends
from typing import Optional
from dataclasses import dataclass
#dataclass
class FaultQueryParams:
f_id: Optional[int] = Query(None, description="id for the host", example=12345)
app = FastAPI()
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args

Thank to #Chris for the pointers which ultimately led me to use dataclasses for defining query params in bulk and it just worked fine.
#dataclass
class FaultQueryParams1:
f_id: Optional[int] = Query(None, description="id for the host", example=55555)
hostname: Optional[str] = Query(None, example="test-host1.domain.com")
status: Literal["open", "closed", "all"] = Query(
None, description="fetch open/closed or all records", example="all"
)
created_by: Optional[str] = Query(
None,
description="fetch records created by particular user",
example="user-id",
)

Problem with Python, FastAPI, Pydantic and SQLAlchemy

I'm trying to build a Python FastAPI blog system using SQLAlchemy with SQLite and am having problems using/understanding the response_model parameter of the API decorator. Here's a SQLAlchemy model:
class User(SqlAlchemyBase):
__tablename__ = 'user'
__table_args__ = {"keep_existing": True}
user_uid: int = sa.Column(GUID, primary_key=True, default=GUID_DEFAULT_SQLITE)
first_name: str = sa.Column(sa.String)
last_name: str = sa.Column(sa.String)
email: str = sa.Column(sa.String, index=True, unique=True)
posts: List["Post"] = relationship("Post", backref="user")
created: datetime = sa.Column(sa.DateTime, default=datetime.now(tz=timezone.utc), index=True)
updated: datetime = sa.Column(sa.DateTime, default=datetime.now(
tz=timezone.utc), onupdate=datetime.now(tz=timezone.utc), index=True)
Here's the Pydantic schema's for a User:
from datetime import datetime
from pydantic import BaseModel
class UserBase(BaseModel):
first_name: str
last_name: str
email: str
class UserInDB(UserBase):
user_uid: int
created: datetime
updated: datetime
class Config:
orm_mode = True
class User(UserInDB):
pass
Here's a ULR endpoint to get a single user that has the response_model parameter included:
from typing import List
import fastapi
from starlette import status
from backend.schema.user import User
from backend.services import user_service
router = fastapi.APIRouter()
#router.get("/api/users/{user_uid}", response_model=User, status_code=status.HTTP_200_OK)
async def get_one_user(user_uid: str = None) -> User:
return await user_service.get_user(user_uid)
If I execute the above call in the OpenAPI docs created by FastAPI I get an internal server error in the OpenAPI docs and these errors in the console that's running the FastAPI application:
File "/Users/dougfarrell/projects/blog/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 137, in serialize_response
raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: 6 validation errors for User
response -> first_name
field required (type=value_error.missing)
response -> last_name
field required (type=value_error.missing)
response -> email
field required (type=value_error.missing)
response -> user_uid
field required (type=value_error.missing)
response -> created
field required (type=value_error.missing)
response -> updated
field required (type=value_error.missing)
If I take the response_model parameter out of the #router.get(...) decorator I get back the results of the SQLAlchemy query, but no Pydantic serialization. I don't know what the error stack trace is trying to tell me. Can anyone offer some suggestions, advice or resources that might help me figure out what I'm doing wrong?
I'm using Python version 3.10.1, FastAPI version 0.70.1 and SQLAlchemy version 1.4.29.
Thanks!

Try this:
from typing import List
and change response_model=User to:
response_model=List[User]

How to make pydantic await on a async property (tortoise-orm's reverse ForeignKey)?

(MRE in the bottom of the question)
In tortoise-orm, we have to await on reverse ForeignKey field as such:
comments = await Post.get(id=id).comments
But in fastapi, when returning a Post instance, pydantic is complaining:
pydantic.error_wrappers.ValidationError: 1 validation error for PPost
response -> comments
value is not a valid list (type=type_error.list)
It makes sense as comments property returns coroutine. And I had to use this little hack to get aronud:
post = Post.get(id=id)
return {**post.__dict__, 'comments': await post.comments}
However, the real issue is when I have multiple relations: return a user with his posts with its comments. In that case I had to transform into dict my entiry model in a very ugly way (which doesn't sound good).
Here is the code to reproduce (tried to keep it as simple as possible):
models.py
from tortoise.fields import *
from tortoise.models import Model
from tortoise import Tortoise, run_async
async def init_tortoise():
await Tortoise.init(
db_url='sqlite://db.sqlite3',
modules={'models': ['models']},
)
await Tortoise.generate_schemas()
class User(Model):
name = CharField(80)
class Post(Model):
title = CharField(80)
content = TextField()
owner = ForeignKeyField('models.User', related_name='posts')
class PostComment(Model):
text = CharField(80)
post = ForeignKeyField('models.Post', related_name='comments')
if __name__ == '__main__':
run_async(init_tortoise())
__all__ = [
'User',
'Post',
'PostComment',
'init_tortoise',
]
main.py
import asyncio
from typing import List
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from models import *
app = FastAPI()
asyncio.create_task(init_tortoise())
# pydantic models are prefixed with P
class PPostComment(BaseModel):
text: str
class PPost(BaseModel):
id: int
title: str
content: str
comments: List[PPostComment]
class Config:
orm_mode = True
class PUser(BaseModel):
id: int
name: str
posts: List[PPost]
class Config:
orm_mode = True
#app.get('/posts/{id}', response_model=PPost)
async def index(id: int):
post = await Post.get_or_none(id=id)
return {**post.__dict__, 'comments': await post.comments}
#app.get('/users/{id}', response_model=PUser)
async def index(id: int):
user = await User.get_or_none(id=id)
return {**user.__dict__, 'posts': await user.posts}
/users/1 errors out with:
pydantic.error_wrappers.ValidationError: 1 validation error for PUser
response -> posts -> 0 -> comments
value is not a valid list (type=type_error.list)
Also you may wish to put this into init.py and run:
import asyncio
from models import *
async def main():
await init_tortoise()
u = await User.create(name='drdilyor')
p = await Post.create(title='foo', content='lorem ipsum', owner=u)
c = await PostComment.create(text='spam egg', post=p)
asyncio.run(main())
What I want is to make pydantic automatically await on those async fields (so I can just return Post instance). How is that possible with pydantic?
Changing /posts/{id} to return the post and its owner without comments is actually working when using this way (thanks to #papple23j):
return await Post.get_or_none(id=id).prefetch_related('owner')
But not for reversed foreign keys. Also select_related('comments') didn't help, it is raising AttributeError: can't set attribute.

Sorry, I was sooo dumb.
One solution I though about is to use tortoise.contrib.pydantic package:
PPost = pydantic_model_creator(Post)
# used as
return await PPost.from_tortoise_orm(await Post.get_or_none(id=1))
But as per this question, it is needed to initialize Tortoise before declaring models, otherwise Relation's wont be included. So I was tempted to replace this line:
asyncio.create_task(init_tortoise())
...with:
asyncio.get_event_loop().run_until_complete(init_tortoise())
But it errored out event loop is already running and removing uvloop and installing nest_asyncio helped with that.
The solution I used
As per documentation:
Fetching foreign keys can be done with both async and sync interfaces.
Async fetch:
events = await tournament.events.all()
Sync usage requires that you call fetch_related before the time, and then you can use common functions.
await tournament.fetch_related('events')
After using .fetch_related) (or prefetch_related on a queryset), reverse foreign key would become an iterable, which can be used just as list. But pydantic would still be complaining that is not a valid list, so validators need be used:
class PPost(BaseModel):
comments: List[PPostComment]
#validator('comments', pre=True)
def _iter_to_list(cls, v):
return list(v)
(Note that validator can't be async, as far as I know)
And since I have set orm_mode, I have to be using .from_orm method 😅:
return PPost.from_orm(await Post.get_or_none(id=42))
Remember, a few hours of trial and error can save you several minutes of looking at the README.

You can try using prefetch_related()
For example:
#app.get('/posts/{id}', response_model=PPost)
async def index(id: int):
post = await Post.get_or_none(id=id).prefetch_related('comments')
return {**post.__dict__}

(The following text is translated using DeepL)
There is a way to do this, but it is a bit tricky
First split the pydantic models snippet into schemas.py
from pydantic import BaseModel
from typing import List
# pydantic models are prefixed with P
class PPostComment(BaseModel):
text: str
class Config:
orm_mode = True # add this line
class PPost(BaseModel):
id: int
title: str
content: str
comments: List[PPostComment]
class Config:
orm_mode = True
class PUser(BaseModel):
id: int
name: str
posts: List[PPost]
class Config:
orm_mode = True
Next, rewrite models.py
from tortoise.fields import *
from tortoise.models import Model
from tortoise import Tortoise, run_async
from schemas import *
async def init_tortoise():
await Tortoise.init(
db_url='sqlite://db.sqlite3',
modules={'models': ['models']},
)
await Tortoise.generate_schemas()
class User(Model):
name = CharField(80)
_posts = ReverseRelation["Post"] #1
#property
def posts(self): #3
return [PPost.from_orm(post) for post in self._posts]
class Post(Model):
title = CharField(80)
content = TextField()
owner = ForeignKeyField('models.User', related_name='_posts') #2
_comments = ReverseRelation["PostComment"] #1
#property
def comments(self): #3
return [PPostComment.from_orm(comment) for comment in self._comments]
class PostComment(Model):
text = CharField(80)
post = ForeignKeyField('models.Post', related_name='_comments') #2
if __name__ == '__main__':
run_async(init_tortoise())
__all__ = [
'User',
'Post',
'PostComment',
'init_tortoise',
]
where
#1: Use ReverseRelation to declare the reverse field, here use the prefix of the bottom line to differentiate
#2: Modify the related_name
#3: Write a property function and return the corresponding pydantic model list, here you don't need to use await because the default is to access it with prefetch_related()
Finally, the main.py
import asyncio
from typing import List
from fastapi import FastAPI, HTTPException
from models import *
from schemas import *
from tortoise.query_utils import Prefetch
app = FastAPI()
asyncio.create_task(init_tortoise())
#app.get('/posts/{id}', response_model=PPost)
async def index(id: int):
post = await Post.get_or_none(id=id).prefetch_related('_comments') #1
return PPost.from_orm(post) #2
#app.get('/users/{id}', response_model=PUser)
async def index(id: int):
user = await User.get_or_none(id=id).prefetch_related(
Prefetch('_posts',queryset=Post.all().prefetch_related('_comments')) #3
)
return PUser.from_orm(user) #2
where
#1: Use prefetch_related() to prefetch related data
#2: For a tortoise model with orm_mode = True, you can use from_orm to convert it to a pydantic model.
#3: For multi-layer correlation data structure, you need to write another layer of prefetch_related()

How to generate response decriptions in FastAPI

I want to generate a description of all available responses (along with code 200 example), which are represented in the code, like here.
from typing import Any
import uvicorn
from fastapi import FastAPI, HTTPException
router = FastAPI()
from pydantic import BaseModel
class FileItemBase(BaseModel):
current_project: str = "Test project"
class FileItemInDBBase(FileItemBase):
id: int
folder_path: str
class Config:
orm_mode = True
class FileResponse(FileItemInDBBase):
pass
#router.get("/", response_model=FileResponse)
def example_code() -> Any:
"""
# beautiful description
to demonstrate functionality
"""
demo=True
if demo:
raise HTTPException(418, "That is a teapot.")
if __name__ =="__main__":
uvicorn.run(router)
What I got with this is such a description.
When I try this out - I got an error response (as expected).
What I want - is the description of an error included in the example responses, like here. A Frontend-developer can look at this description and process such cases in the right way without testing the API.
I know how it can be made within OpenAPI specs.
Is there a way to generate this description with FastAPI?

You can add a responses parameter to your path operation.
Then you can pass your model there. It will create a schema for that model.
class FileItemBase(BaseModel):
current_project: str = "Test project"
#app.get("/", response_model=FileItemBase, responses={418: {"model": FileItemBase}})
def example_code():
"""
# beautiful description
to demonstrate functionality
"""
demo = True
if demo:
raise HTTPException(418, "That is a teapot.")