This question already has answers here:
FastAPI - GET request results in typeerror (value is not a valid dict)
(2 answers)
Closed 10 months ago.
I started to build a wepapp with fastapi and Vue. I started at the backend and tried to get some data from a database. As ORM I use SQLAlchemy. What I am tring to achive is to strip down the DB response to some specific columns.
I am fairly new to the world of webapplications, I will also appreciate some good resources. I find it kind of hard to get started in this topic. Despite there being alt of Tutorials, they just cover how to start a basic site and leave the rest to the docs. But I you are not used to the terminology, its quite easy to get lost there.
Anyway, I have this test setup:
My model for the db is
class System(Base):
__tablename__ = 'system'
id = Column(BIGINT(20), primary_key=True)
name = Column(String(200), nullable=False)
type = Column(String(200), nullable=False)
installed_power = Column(Float(asdecimal=True), nullable=False)
date_of_installation = Column(
DateTime, nullable=False, server_default=text("current_timestamp()"))
last_changed = Column(DateTime, nullable=False, server_default=text(
"current_timestamp() ON UPDATE current_timestamp()"))
site_id = Column(ForeignKey('site.id'), nullable=False, index=True)
site = relationship('Site')
and my schema is
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
In my main.py I am doing this
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db)
return query.all()
This dose not work. The error says pydantic.error_wrappers.ValidationError: 1 validation error for System response value is not a valid dict (type=type_error.dict)
If I add all db columns to the schema it works obviously
I also tried something like this, but this did not work either.
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db)
return query.all()
res = []
for row in query:
system = schemas.System()
system.id = row.id
system.name = row.name
system.type = row.type
system.installed_power = row.installed_power
res.append(system)
return res
WITHOUT orm_mode
In your first example, your response_model expects a schema (System), but you send it query.all() which returns a list containing classes from the model you get. (SQLAlchemy does not return a dictionary, which is what pydantic expects by default).
So, at this point, you have to make a choice, either your endpoint must return ONE object, in which case you should not use query.all() but something like query.one_or_none().
Or you want to return a list of objects and your response_model should be:
from typing import List
#app.get("/system", response_model=List[schemas.System])
All that remains is to format your data so that it corresponds to your schema.
for version with only one data:
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).one_or_none()
return schemas.System(id=query.id,name= query.name, type=query.type, installed_power=query.installed_power)
for version with multiple datas:
from typing import List
def make_response_systems(query):
result = []
for data in query:
result.append(schemas.System(id=query.id,name= query.name, type=query.type, installed_power=query.installed_power))
return result
#app.get("/system", response_model=List[schemas.System])
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).all()
return make_response_systems(query)
There are more aesthetic ways to do this. But I think the example above is a good way to understand how it works.
For the rest, you can look at the orm_mode of pydantics models, which you can find in the FastAPI documentation.
To learn, the FastAPI documentation is very complete and easy to access.
WITH orm_mode
Pydantic's orm_mode will tell the Pydantic model to read the data even if it is not a dict, but an ORM model (or any other arbitrary object with attributes).
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
class Config:
orm_mode = True
#app.get("/system", response_model=schemas.System)
def get_systems(db: Session = Depends(get_db)):
query = crud.get_system(db).one_or_none()
return query
Documentation fastapi orm_mode : https://fastapi.tiangolo.com/tutorial/sql-databases/?h=orm_mode#use-pydantics-orm_mode
As per pydantic docs
ORM Mode (aka Arbitrary Class Instances)🔗
Pydantic models can be created from arbitrary class instances to support models that map to ORM objects.
To do this:
The Config property orm_mode must be set to True.
The special constructor from_orm must be used to create the model instance.
we need to add orm_mode to the schema config.
class System(BaseModel):
id: int
name: str
type: str
installed_power: int
class Config:
orm_mode = True
Reference: https://pydantic-docs.helpmanual.io/usage/models/#orm-mode-aka-arbitrary-class-instances
Related
I have the following two pydantic models for a Users table in my database for a FastAPI application:
from fastapi import Form
from pydantic import BaseModel
class UserCreate(BaseModel):
username: str = Form(...)
password: str = Form(...)
class UserInDb(BaseModel):
id: int
username:str
hashed_password: str
I use the UserCreate class to get form data needed from the client to create a user and then I map it to UserInDb. This is how I initially implemented it:
password = user_create_instance.password
hashed_password = some_hashing_function(password)
user_in_db = UserInDb(**user_create_instance.dict(), hashed_password=hashed_password)
This however throws a value_error.missing error because user_create_instance does not have an id parameter which UserInDb expects to be passed in but the value for that comes from the database.
This issue is fixable by doing something like id: int = None when defining the attribute in the class but it doesn't feel right to do it this way. Is there a better approach?
Using a ORM, I want to do a POST request letting some fields with a null value, which will be translated in the database for the default value specified there.
The problem is that OpenAPI (Swagger) docs, ignores the default None and still prompts a UUID by default.
from fastapi import FastAPI
from pydantic import BaseModel
from typing import Optional
from uuid import UUID
import uvicorn
class Table(BaseModel):
# ID: Optional[UUID] # the docs show a example UUID, ok
ID: Optional[UUID] = None # the docs still shows a uuid, when it should show a null or valid None value.
app = FastAPI()
#app.post("/table/", response_model=Table)
def create_table(table: Table):
# here we call to sqlalchey orm etc.
return 'nothing important, the important thing is in the docs'
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)
In the OpenAPI schema example (request body) which is at the docs we find:
{
"ID": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
This is not ok, because I specified that the default value is None,so I expected this instead:
{
"ID": null, # null is the equivalent of None here
}
Which will pass a null to the ID and finally will be parsed in the db to the default value (that is a new generated UUID).
When you declare Optional parameters, users shouldn't have to include those parameters in their request (specified with null value) in order to be null. The default value of the parameters will be null, unless the user specifies some other value when sending the request.
Hence, all you have to do is to declare a custom example for the Pydantic model using Config and schema_extra, as described in the documentation and as shown below. The below example will create an empty (i.e., {}) request body in OpenAPI (Swagger UI), which can be successfully submitted (as ID is the only attribute of the model and is optional).
class Table(BaseModel):
ID: Optional[UUID] = None
class Config:
schema_extra = {
"example": {
}
}
#app.post("/table/", response_model=Table)
def create_table(table: Table):
return table
If the Table model included some other required attributes, you could add example values for those, as demonstrated below:
class Table(BaseModel):
ID: Optional[UUID] = None
some_attr: str
class Config:
schema_extra = {
"example": {
"some_attr": "Foo"
}
}
If you would like to keep the auto-generated examples for the rest of the attributes except the one for the ID attribute, you could use the below to remove ID from the model's properties in the generated schema (inspired by Schema customization):
class Table(BaseModel):
ID: Optional[UUID] = None
some_attr: str
some_attr2: float
some_attr3: bool
class Config:
#staticmethod
def schema_extra(schema: Dict[str, Any], model: Type['Table']) -> None:
del schema.get('properties')['ID']
Also, if you would like to add custom example to some of the attributes, you could use Field() (as described here); for example, some_attr: str = Field(example="Foo").
Another possible solution would be to modify the generated OpenAPI schema, as described in Solution 3 of this answer. Though, the above solution is likely more suited to this case.
Note
ID: Optional[UUID] = None is the same as ID: UUID = None. As previously documented in FastAPI website (see this answer):
The Optional in Optional[str] is not used by FastAPI, but will allow
your editor to give you better support and detect errors.
Since then, FastAPI has revised their documentation with the following:
The Union in Union[str, None] will allow your editor to give you
better support and detect errors.
Hence, ID: Union[UUID, None] = None is the same as ID: Optional[UUID] = None and ID: UUID = None. In Python 3.10+, one could also use ID: UUID| None = None (see here).
As per FastAPI documentation (see Info section in the link provided):
Have in mind that the most important part to make a parameter optional
is the part:
= None
or the:
= Query(default=None)
as it will use that None as the default value, and that way make the
parameter not required.
The Union[str, None] part allows your editor to provide better
support, but it is not what tells FastAPI that this parameter is not
required.
Here is my code snippet:
from fastapi_pagination import Page, add_pagination
from fastapi_pagination.ext.sqlalchemy import paginate
#app.get("/clients", response_model=Page[PydanticModel])
def get_items(
db: Session = Depends(get_db) ) -> Any:
items = paginate(
db.query(Model)
.filter(...)
)
...
# do some extra manipulations ..
...
items.items = new_items
return items
When I specify Page[PydenticModel] in the response_model it generates an issue with paginte() because it's not the final response type.
The PydenticModel correspond to new_items and not items (returned from paginate()),
pydantic.error_wrappers.ValidationError: validation errors for
Page[PydanticModel]
Note: I don't want to use Page[Any] in order to keep a good a Swagger docs
I think you need PydenticModel to have orm_mode = True in its config
I am facing the same issue while using pagination in fast API and resolving the issue by adding orm_mode = True in the model class.
Search for orm_mode here for more detail
Sample class with orm_mode = True
class Todo(BaseModel):
id:int
title: str
description: Optional[str]
priority: int
complete: bool
owner_id:int
class Config:
orm_mode = True
I'm handling this request in my code (Python3.9, FastAPI, Pydantic):
https://myapi.com/api?params[A]=1¶ms[B]=2
I tried to make following model:
BaseModel for handling special get request
(for fastapi.Query and pydantic.Field is same)
I also set up aliases for it, but in swagger docs I see next field:
Snap of the swagger docs
There are fields that are specified as extra_data
So, if I specify query params in parameters of my endpoint like this:
#app.get('/')
def my_handler(a: str = Query(None, alias="params[A]")):
return None
Everything works fine. How can I fix it? I want to initialize my pydantic.BaseModel with speacial aliases using this way and avoid usage of query-params in
class MyModel(BaseModel):
a = Field(alias="params[A]")
b = Field(alias="params[B]")
def my_handler(model: MyModel = Depends()):
return model.dict()
I am using flask-restful this is
My class I want to insert
class OrderHistoryResource(Resource):
model = OrderHistoryModel
schema = OrderHistorySchema
order = OrderModel
product = ProductModel
def post(self):
value = req.get_json()
data = cls.schema(many=True).load(value)
data.insert()
In my model
def insert(self):
db.session.add(self)
db.session.commit()
schema
from config.ma import ma
from model.orderhistory import OrderHistoryModel
class OrderHistorySchema(ma.ModelSchema):
class Meta:
model = OrderHistoryModel
include_fk = True
Example Data I want to insert
[
{
"quantity":99,
"flaskSaleStatus":true,
"orderId":"ORDER_64a79028d1704406b6bb83b84ad8c02a_1568776516",
"proId":"PROD_9_1568779885_64a79028d1704406b6bb83b84ad8c02a"
},
{
"quantity":89,
"flaskSaleStatus":true,
"orderId":"ORDER_64a79028d1704406b6bb83b84ad8c02a_1568776516",
"proId":"PROD_9_1568779885_64a79028d1704406b6bb83b84ad8c02a"
}
]
this is what i got after insert method has started
TypeError: insert() takes exactly 2 arguments (0 given)
or there is another way to do this action?
Edited - released marshmallow-sqlalchemy loads directly to instance
You need to loop through the OrderModel instances in your list.
You can then use add_all to add the OrderModel objects to the session, then bulk update - see the docs
Should be something like:
db.session.add_all(data)
db.session.commit()
See this post for brief discussion on why add_all is best when you have complex ORM relationships.
Also - not sure you need to have all your models/schemas as class variables, it's fine to have them imported (or just present in the same file, as long as they're declared before the resource class).
You are calling insert on list cause data is list of model OrderHistoryModel instances.
Also post method doesn't need to be classmethod and you probably had an error there as well.
Since data is list of model instances you can use db.session.add_all method to add them to session in bulk.
def post(self):
value = req.get_json()
data = self.schema(many=True).load(value)
db.session.add_all(data)
db.session.commit()