I have the following two pydantic models for a Users table in my database for a FastAPI application:
from fastapi import Form
from pydantic import BaseModel
class UserCreate(BaseModel):
username: str = Form(...)
password: str = Form(...)
class UserInDb(BaseModel):
id: int
username:str
hashed_password: str
I use the UserCreate class to get form data needed from the client to create a user and then I map it to UserInDb. This is how I initially implemented it:
password = user_create_instance.password
hashed_password = some_hashing_function(password)
user_in_db = UserInDb(**user_create_instance.dict(), hashed_password=hashed_password)
This however throws a value_error.missing error because user_create_instance does not have an id parameter which UserInDb expects to be passed in but the value for that comes from the database.
This issue is fixable by doing something like id: int = None when defining the attribute in the class but it doesn't feel right to do it this way. Is there a better approach?
This question already has answers here:
FastAPI - GET request results in typeerror (value is not a valid dict)
(2 answers)
Closed 10 months ago.
I started to build a wepapp with fastapi and Vue. I started at the backend and tried to get some data from a database. As ORM I use SQLAlchemy. What I am tring to achive is to strip down the DB response to some specific columns.
I am fairly new to the world of webapplications, I will also appreciate some good resources. I find it kind of hard to get started in this topic. Despite there being alt of Tutorials, they just cover how to start a basic site and leave the rest to the docs. But I you are not used to the terminology, its quite easy to get lost there.
Anyway, I have this test setup:
My model for the db is
class System(Base):
__tablename__ = 'system'
id = Column(BIGINT(20), primary_key=True)
name = Column(String(200), nullable=False)
type = Column(String(200), nullable=False)
installed_power = Column(Float(asdecimal=True), nullable=False)
date_of_installation = Column(
DateTime, nullable=False, server_default=text("current_timestamp()"))
last_changed = Column(DateTime, nullable=False, server_default=text(
"current_timestamp() ON UPDATE current_timestamp()"))
site_id = Column(ForeignKey('site.id'), nullable=False, index=True)
site = relationship('Site')
and my schema is
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
In my main.py I am doing this
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db)
return query.all()
This dose not work. The error says pydantic.error_wrappers.ValidationError: 1 validation error for System response value is not a valid dict (type=type_error.dict)
If I add all db columns to the schema it works obviously
I also tried something like this, but this did not work either.
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db)
return query.all()
res = []
for row in query:
system = schemas.System()
system.id = row.id
system.name = row.name
system.type = row.type
system.installed_power = row.installed_power
res.append(system)
return res
WITHOUT orm_mode
In your first example, your response_model expects a schema (System), but you send it query.all() which returns a list containing classes from the model you get. (SQLAlchemy does not return a dictionary, which is what pydantic expects by default).
So, at this point, you have to make a choice, either your endpoint must return ONE object, in which case you should not use query.all() but something like query.one_or_none().
Or you want to return a list of objects and your response_model should be:
from typing import List
#app.get("/system", response_model=List[schemas.System])
All that remains is to format your data so that it corresponds to your schema.
for version with only one data:
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).one_or_none()
return schemas.System(id=query.id,name= query.name, type=query.type, installed_power=query.installed_power)
for version with multiple datas:
from typing import List
def make_response_systems(query):
result = []
for data in query:
result.append(schemas.System(id=query.id,name= query.name, type=query.type, installed_power=query.installed_power))
return result
#app.get("/system", response_model=List[schemas.System])
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).all()
return make_response_systems(query)
There are more aesthetic ways to do this. But I think the example above is a good way to understand how it works.
For the rest, you can look at the orm_mode of pydantics models, which you can find in the FastAPI documentation.
To learn, the FastAPI documentation is very complete and easy to access.
WITH orm_mode
Pydantic's orm_mode will tell the Pydantic model to read the data even if it is not a dict, but an ORM model (or any other arbitrary object with attributes).
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
class Config:
orm_mode = True
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).one_or_none()
return query
Documentation fastapi orm_mode : https://fastapi.tiangolo.com/tutorial/sql-databases/?h=orm_mode#use-pydantics-orm_mode
As per pydantic docs
ORM Mode (aka Arbitrary Class Instances)🔗
Pydantic models can be created from arbitrary class instances to support models that map to ORM objects.
To do this:
The Config property orm_mode must be set to True.
The special constructor from_orm must be used to create the model instance.
we need to add orm_mode to the schema config.
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
class Config:
orm_mode = True
Reference: https://pydantic-docs.helpmanual.io/usage/models/#orm-mode-aka-arbitrary-class-instances
I am building a FastAPI application, which has a lot of Pydantic models. Even though the application is working just fine, as expected the OpenAPI (Swagger UI) docs do not show the schema for all of these models under the Schemas section.
Here are the contents of pydantic schemas.py
import socket
from datetime import datetime
from enum import Enum
from typing import Any, Dict, List, Optional, Set, Union
from pydantic import BaseModel, Field, validator
from typing_extensions import Literal
ResponseData = Union[List[Any], Dict[str, Any], BaseModel]
# Not visible in Swagger UI
class PageIn(BaseModel):
page_size: int = Field(default=100, gt=0)
num_pages: int = Field(default=1, gt=0, exclude=True)
start_page: int = Field(default=1, gt=0, exclude=True)
# visible under schemas on Swagger UI
class PageOut(PageIn):
total_records: int = 0
total_pages: int = 0
current_page: int = 1
class Config: # pragma: no cover
#staticmethod
def schema_extra(schema, model) -> None:
schema.get("properties").pop("num_pages")
schema.get("properties").pop("start_page")
# Not visible in Swagger UI
class BaseResponse(BaseModel):
host_: str = Field(default_factory=socket.gethostname)
message: Optional[str]
# Not visible in Swagger UI
class APIResponse(BaseResponse):
count: int = 0
location: Optional[str]
page: Optional[PageOut]
data: ResponseData
# Not visible in Swagger UI
class ErrorResponse(BaseResponse):
error: str
# visible under schemas on Swagger UI
class BaseFaultMap(BaseModel):
detection_system: Optional[str] = Field("", example="obhc")
fault_type: Optional[str] = Field("", example="disk")
team: Optional[str] = Field("", example="dctechs")
description: Optional[str] = Field(
"",
example="Hardware raid controller disk failure found. "
"Operation can continue normally,"
"but risk of data loss exist",
)
# Not visible in Swagger UI
class FaultQueryParams(BaseModel):
f_id: Optional[int] = Field(None, description="id for the host", example=12345, title="Fault ID")
hostname: Optional[str]
status: Literal["open", "closed", "all"] = Field("open")
created_by: Optional[str]
environment: Optional[str]
team: Optional[str]
fault_type: Optional[str]
detection_system: Optional[str]
inops_filters: Optional[str] = Field(None)
date_filter: Optional[str] = Field("",)
sort_by: Optional[str] = Field("created",)
sort_order: Literal["asc", "desc"] = Field("desc")
All of these models are actually being used in FastAPI paths to validate the request body. The FaultQueryParams is a custom model, which I use to validate the request query params and is used like below:
query_args: FaultQueryParams = Depends()
The rest of the models are being used in conjunction with Body field. I am not able to figure out why only some of the models are not visible in the Schemas section while others are.
Also another thing I noticed about FaultQueryParams is that the description, examples do not show up against the path endpoint even though they are defined in the model.
Edit 1:
I looked more into and realized that all of the models which are not visible in swagger UI are the ones that are not being used directly in path operations i.e., these models are not being used as response_model or Body types and are sort of helper models which are being used indirectly. So, it seems like FastAPI is not generating the schema for these models.
One exception to the above statement is query_args: FaultQueryParams = Depends() which is being used directly in a path operation to map the Query params for the endpoint against a custom model. This is a problem because swagger is not identifying the meta parameters like title, description, example from the fields of this model & not showing on the UI which is important for the users of this endpoint.
Is there a way to trick FastAPI to generate schema for the custom model FaultQueryParams just like it generates for Body, Query etc ?
FastAPI will generate schemas for models that are used either as a Request Body or Response Model. When declaring query_args: FaultQueryParams = Depends() (using Depends), your endpoint would not expect a request body, but rather query parameters; hence, FaultQueryParams would not be included in the schemas of the OpenAPI docs.
To add additional schemas, you could extend/modify the OpenAPI schema. Example is given below (make sure to add the code for modifying the schema after all routes have been defined, i.e., at the end of your code).
class FaultQueryParams(BaseModel):
f_id: Optional[int] = Field(None, description="id for the host", example=12345, title="Fault ID")
hostname: Optional[str]
status: Literal["open", "closed", "all"] = Field("open")
...
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args
def get_extra_schemas():
return {
"FaultQueryParams": {
"title": "FaultQueryParams",
"type": "object",
"properties": {
"f_id": {
"title": "Fault ID",
"type": "integer",
"description": "id for the host",
"example": 12345
},
"hostname": {
"title": "Hostname",
"type": "string"
},
"status": {
"title": "Status",
"enum": [
"open",
"closed",
"all"
],
"type": "string",
"default": "open"
},
...
}
}
}
from fastapi.openapi.utils import get_openapi
def custom_openapi():
if app.openapi_schema:
return app.openapi_schema
openapi_schema = get_openapi(
title="FastAPI",
version="1.0.0",
description="This is a custom OpenAPI schema",
routes=app.routes,
)
new_schemas = openapi_schema["components"]["schemas"]
new_schemas.update(get_extra_schemas())
openapi_schema["components"]["schemas"] = new_schemas
app.openapi_schema = openapi_schema
return app.openapi_schema
app.openapi = custom_openapi
Some Helpful Notes
Note 1
Instead of manually typing the schema for the extra models that you would like to add to the docs, you can have FastAPI do that for you by adding to your code an endpoint (which you would subsequently remove, after getting the schema) using that model as a request body or response model, for example:
#app.post('/predict')
def predict(query_args: FaultQueryParams):
return query_args
Then, you can get the generated JSON schema at http://127.0.0.1:8000/openapi.json, as described in the documentation. From there, you can either copy and paste the schema of the model to your code and use it directly (as shown in the get_extra_schema() method above) or save it to a file and load the JSON data from the file, as demonstrated below:
import json
...
new_schemas = openapi_schema["components"]["schemas"]
with open('extra_schemas.json') as f:
extra_schemas = json.load(f)
new_schemas.update(extra_schemas)
openapi_schema["components"]["schemas"] = new_schemas
...
Note 2
To declare metadata, such as description, example, etc, for your query parameter, you should define your parameter with Query instead of Field, and since you can't do that with Pydantic models, you could declare a custom dependency class, as decribed here and as shown below:
from fastapi import FastAPI, Query, Depends
from typing import Optional
class FaultQueryParams:
def __init__(
self,
f_id: Optional[int] = Query(None, description="id for the host", example=12345)
):
self.f_id = f_id
app = FastAPI()
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args
The above can be re-written using the #dataclass decorator, as shown below:
from fastapi import FastAPI, Query, Depends
from typing import Optional
from dataclasses import dataclass
#dataclass
class FaultQueryParams:
f_id: Optional[int] = Query(None, description="id for the host", example=12345)
app = FastAPI()
#app.post('/predict')
def predict(query_args: FaultQueryParams = Depends()):
return query_args
Thank to #Chris for the pointers which ultimately led me to use dataclasses for defining query params in bulk and it just worked fine.
#dataclass
class FaultQueryParams1:
f_id: Optional[int] = Query(None, description="id for the host", example=55555)
hostname: Optional[str] = Query(None, example="test-host1.domain.com")
status: Literal["open", "closed", "all"] = Query(
None, description="fetch open/closed or all records", example="all"
)
created_by: Optional[str] = Query(
None,
description="fetch records created by particular user",
example="user-id",
)
I'm trying to build a Python FastAPI blog system using SQLAlchemy with SQLite and am having problems using/understanding the response_model parameter of the API decorator. Here's a SQLAlchemy model:
class User(SqlAlchemyBase):
__tablename__ = 'user'
__table_args__ = {"keep_existing": True}
user_uid: int = sa.Column(GUID, primary_key=True, default=GUID_DEFAULT_SQLITE)
first_name: str = sa.Column(sa.String)
last_name: str = sa.Column(sa.String)
email: str = sa.Column(sa.String, index=True, unique=True)
posts: List["Post"] = relationship("Post", backref="user")
created: datetime = sa.Column(sa.DateTime, default=datetime.now(tz=timezone.utc), index=True)
updated: datetime = sa.Column(sa.DateTime, default=datetime.now(
tz=timezone.utc), onupdate=datetime.now(tz=timezone.utc), index=True)
Here's the Pydantic schema's for a User:
from datetime import datetime
from pydantic import BaseModel
class UserBase(BaseModel):
first_name: str
last_name: str
email: str
class UserInDB(UserBase):
user_uid: int
created: datetime
updated: datetime
class Config:
orm_mode = True
class User(UserInDB):
pass
Here's a ULR endpoint to get a single user that has the response_model parameter included:
from typing import List
import fastapi
from starlette import status
from backend.schema.user import User
from backend.services import user_service
router = fastapi.APIRouter()
#router.get("/api/users/{user_uid}", response_model=User, status_code=status.HTTP_200_OK)
async def get_one_user(user_uid: str = None) -> User:
return await user_service.get_user(user_uid)
If I execute the above call in the OpenAPI docs created by FastAPI I get an internal server error in the OpenAPI docs and these errors in the console that's running the FastAPI application:
File "/Users/dougfarrell/projects/blog/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 137, in serialize_response
raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: 6 validation errors for User
response -> first_name
field required (type=value_error.missing)
response -> last_name
field required (type=value_error.missing)
response -> email
field required (type=value_error.missing)
response -> user_uid
field required (type=value_error.missing)
response -> created
field required (type=value_error.missing)
response -> updated
field required (type=value_error.missing)
If I take the response_model parameter out of the #router.get(...) decorator I get back the results of the SQLAlchemy query, but no Pydantic serialization. I don't know what the error stack trace is trying to tell me. Can anyone offer some suggestions, advice or resources that might help me figure out what I'm doing wrong?
I'm using Python version 3.10.1, FastAPI version 0.70.1 and SQLAlchemy version 1.4.29.
Thanks!
Try this:
from typing import List
and change response_model=User to:
response_model=List[User]
I want to copy some documents from MongoDB from one DB to another, but only specific properties of this document. Given a source document with the format:
{topic: string, email: {enabled: boolean}, mobile: {enabled: boolean}}
I want to insert into the new DB something with the following format:
{topic: string, email: {enabled: boolean}}
I am doing something like this in order to create the new object that will be inserted:
class Document(object):
topic = ""
email = object()
def make_document(topic, email):
document = Document()
document.topic = topic
document.email = email
return document
settings = source_db.collectionName.find({})
for setting in settings:
new_setting = make_document(setting.topic, setting.email)
db.collectionName.insertOne(new_setting)
My doubts are:
am I creating the new object correctly ?
is the declaration of the email property in the class correct (e.g. email = object())
is there a better (or a more correct) way to do it?
I am really new with Python.
This is the right way of initializing variable(class attribute) inside a class,
class Document(object):
def __init__(self, topic, email):
self.topic = topic
self.email = email
settings = source_db.collectionName.find({})
for setting in settings:
new_setting = Document(setting.topic, setting.email)
db.collectionName.insertOne(new_setting)
Question to you: Since the response you're getting it as dictionary/json, any reason why you trying to make it as class?
{topic: string, email: {enabled: boolean}, mobile: {enabled: boolean}}
Update: Then you can simply achieve this through dictionary parsing. Check this one,
settings = source_db.collectionName.find({})
for setting in settings:
new_data = {"topic": setting["topic"], "email": setting["email"]}
db.collectionName.insertOne(new_data)