I am writing a FastAPI application, and have the need to essentially merge some properties from two SQLAlchemy model instances into a single Pydantic model for the response, some properties from Object A and some from Object B, returning a "consolidated" object.
class Message:
id = Column(Integer, primary_key=True)
subject = Column(Text)
message = Column(Text)
class FolderLink:
id = Column(Integer, primary_key=True)
message_id = Column(Integer, ForeignKey('message.id'))
folder_id = Column(Integer, ForeignKey('folder.id'))
is_read = Column(Boolean, nullable=False, default=False)
In application code, I have a Message instance, needing all properties, and the relevant FolderLink instance, from which I need is_read.
My Pydantic schema looks like:
class MessageWithProperties(BaseModel):
id: int # from Message
subject: str # from Message
message: str # from Message
is_read: bool # from FolderLink
And in the view code, the only way I can seem to properly pass objects to the Pydantic model is like so:
objs = []
# [...]
for link in links:
objs.append({**link.message.__dict__, **link.__dict__})
It feels wrong to have to use a dunder method this way. I had experimented with a custom #classmethod constructor on the Pydantic model, but that will not work as I am not directly instantiating them myself, they are instantiated by FastAPI as part of the response handling.
What is the proper way to do this?
What if instead of mashing all the fields togther you create a top level MessageWithProperties class that is built from two inferior Pydantic models, one for Message and one for FolderLink? Something like:
class BaseSchema(pydantic.BaseModel):
class Config:
orm_mode = True
class MessageSchema(BaseSchema):
id: int # from Message
subject: str # from Message
message: str # from Message
class FolderLinkSchema(BaseSchema):
is_read: bool # from FolderLink
class CombinedSchema(BaseSchema):
message: MessageSchema
folderlink: FolderLinkSchema
objs = []
for link in links:
objs.append(CombinedSchema(
message=MessageSchema.from_orm(link.message),
folderlink=FolderLinkSchema.from_orm(link),
))
Now you'll be able to access both the FolderLink and Message
attributes from each object in your objs array.
Related
I've got a Python 3.10.8 FastAPI application using SQLModel and having trouble with many-to-many relationships. I can define them okay using strings to refer to the class on the other side of the many. Here is a simplified sample of one side of the many-to-many schema/model:
schedule.py file
class Schedule(ScheduleBase, table=True):
__tablename__ = "schedules"
id: Optional[int] = Field(default=None, primary_key=True)
plans: List["Plan"] = Relationship(back_populates="schedules", link_model=SchedulePlanLink)
Here's the other side of the many-to-many schema/model:
class Plan(PlanBase, table=True):
__tablename__ = "plans"
id: Optional[int] = Field(default=None, primary_key=True)
schedules: List["Schedule"] = Relationship(back_populates="plans", link_model=SchedulePlanLink)
And here's the association table between them:
class SchedulePlanLink(SchedulePlanLinkBase, table=True):
__tablename__ = "schedule_plan_links"
schedule_id: Optional[int] = Field(
default=None, primary_key=True, foreign_key="schedules.id"
)
plan_id: Optional[int] = Field(
default=None, primary_key=True, foreign_key="plans.id"
)
This works and creates the expected tables and FK's, etc. The problem arises when I try to access the data. I have a SQLModel class that looks like this:
class ScheduleReadWithPlans(ScheduleRead):
plans: Optional[List["PlanRead"]] = []
And I have to have this in order to read the data and return it via a route:
ScheduleReadWithPlans.update_forward_refs()
And I can get the schedule data with the list of plans. But if I try the same thing with plan data trying to get a list of schedules (with the appropriate classes define), I end up getting circular references.
Any ideas about how to define/configure this kind of thing to to work?
If I have a model like this:
class MyModel(DBModel, table=True):
id: Optional[int] = Field( primary_key=True)
Then when saving new records to the database, the ID is automatically assigned, which is great.
However, when I retrieve the model like this I get type errors
model = session.get(MyModel, 1)
id: int = model.id # ID may be None error
Is there a way to auto-assign my IDs but also have the ID type defined when retrieving saved records?
You might find your answer here: https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/#multiple-models-with-inheritance
Basically what you can do is define your models like this:
class MyModelBase(SQLModel):
arg1: str
...
class MyModel(MyModelBase, table=True):
id: Optional[int] = Field( primary_key=True)
class MyModelRead(MyModelBase):
id: int
Like that you are saying MyModelRead must have an attribute id. Note that only MyModelBase is a SQLModel. The other classes are Pydantic objects.
Make sure when defining a field for id, you explicitly state the default value for ID like this
class MyModel(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
Because we don't set the id, it takes the Python's default value of None that we set in Field(default=None)
for more info read docs https://sqlmodel.tiangolo.com/tutorial/automatic-id-none-refresh/
I'm following this tutorial to adapt it to my needs, in this case, to perform a sql module where I need to record the data collected by a webhook from the gitlab issues.
For the database module I'm using SQLAlchemy library and PostgreSQL as database engine.
So, I would like to solve some doubts, I have regarding the use of the Pydantic library, in particular with this example
From what I've read, Pydantic is a library that is used for data validation using classes with attributes.
But I don't quite understand some things...is the integration of Pydantic strictly necessary? The purpose of using Pydantic I understand, but the integration of using Pydantic with SQLAlchemy models I don't understand.
In the tutorial, models.py has the following content:
from sqlalchemy import Boolean, Column, ForeignKey, Integer, String
from sqlalchemy.orm import relationship
from .database import Base
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, unique=True, index=True)
hashed_password = Column(String)
is_active = Column(Boolean, default=True)
items = relationship("Item", back_populates="owner")
class Item(Base):
__tablename__ = "items"
id = Column(Integer, primary_key=True, index=True)
title = Column(String, index=True)
description = Column(String, index=True)
owner_id = Column(Integer, ForeignKey("users.id"))
owner = relationship("User", back_populates="items")
And schemas.py has the following content:
from typing import Optional
from pydantic import BaseModel
class ItemBase(BaseModel):
title: str
description: Optional[str] = None
class ItemCreate(ItemBase):
pass
class Item(ItemBase):
id: int
owner_id: int
class Config:
orm_mode = True
class UserBase(BaseModel):
email: str
class UserCreate(UserBase):
password: str
class User(UserBase):
id: int
is_active: bool
items: list[Item] = []
class Config:
orm_mode = True
I know that the primary means of defining objects in Pydantic is via models and also I know that models are simply classes which inherit from BaseModel.
Why does it create ItemBase, ItemCreate and Item that inherits from ItemBase?
In ItemBase it passes the fields that are strictly necessary in Item table? and defines its type?
The ItemCreate class I have seen that it is used latter in crud.py to create a user, in my case I would have to do the same with the incidents? I mean, I would have to create a clase like this:
class IssueCreate(BaseModel):
pass
There are my examples trying to follow the same workflow:
models.py
import sqlalchemy
from sqlalchemy import Column, Table
from sqlalchemy import Integer, String, Datetime, TIMESTAMP
from .database import Base
class Issues(Base):
__tablename__ = 'issues'
id = Column(Integer, primary_key=True)
gl_assignee_id = Column(Integer, nullable=True)
gl_id_user = Column(Integer, nullable=False)
current_title = Column(String, nullable=False)
previous_title = Column(String, nullable=True)
created_at = Column(TIMESTAMP(timezone=False), nullable=False)
updated_at = Column(TIMESTAMP(timezone=False), nullable=True)
closed_at = Column(TIMESTAMP(timezone=False), nullable=True)
action = Column(String, nullable=False)
And schemas.py
from pydantic import BaseModel
class IssueBase(BaseModel):
updated_at: None
closed_at: None
previous_title: None
class Issue(IssueBase):
id: int
gl_task_id: int
gl_assignee_id: int
gl_id_user: int
current_title: str
action: str
class Config:
orm_mode = True
But I don't know if I'm right doing it in this way, any suggestions are welcome.
The tutorial you mentioned is about FastAPI. Pydantic by itself has nothing to do with SQL, SQLAlchemy or relational databases. It is FastAPI that is showing you a way to use a relational database.
is the integration of pydantic strictly necessary [when using FastAPI]?
Yes. Pydantic is a requirement according to the documentation:
Requirements
Python 3.6+
FastAPI stands on the shoulders of giants:
Starlette for the web parts.
Pydantic for the data parts.
Why does it create ItemBase, ItemCreate and Item that inherits from ItemBase?
Pydantic models are the way FastAPI uses to define the schemas of the data that it receives (requests) and returns (responses). ItemCreate represent the data required to create an item. Item represents the data that is returned when the items are queried. The fields that are common to ItemCreate and Item are placed in ItemBase to avoid duplication.
In ItemBase it passes the fields that are strictly necessary in Item table? and defines its type?
ItemBase has the fields that are common to ItemCreate and Item. It has nothing to do with a table. It is just a way to avoid duplication. Every field of a pydantic model must have a type, there is nothing unusual there.
in my case I would have to do the same with the incidents?
If you have a similar scenario where the schemas of the data that you receive (request) and the data that you return (response) have common fields (same name and type), you could define a model with those fields and have other models inherit from it to avoid duplication.
This could be a (probably simplistic) way of understanding FastAPI and pydantic:
FastAPI transforms requests to pydantic models. Those pydantic models are your input data and are also known as schemas (maybe to avoid confusion with other uses of the word model). You can do whatever you want with those schemas, including using them to create relational database models and persisting them.
Whatever data you want to return as a response needs to be transformed by FastAPI to a pydantic model (schema). It just happens that pydantic supports an orm_mode option that allows it to parse arbitrary objects with attributes instead of dicts. Using that option you can return a relational database model and FastAPI will transform it to the corresponding schema (using pydantic).
FastAPI uses the parsing and validation features of pydantic, but you have to follow a simple rule: the data that you receive must comply with the input schema and the data that you want to return must comply with the output schema. You are in charge of deciding whatever happens in between.
In my flask application I'm trying to inherit from custom flask db.model class to add some simple logic on top of db.model object. Please find details below. I have two questions here:
When I instantiate my child-class Activity by calling Activity.query.first(), neither __init__ method of Activity class nor __init__ method of parent DBActivity class are being called. I would love to understand why they are not being called and how make them being called.
In general, is it a good/viable practice to inherit from flask model classes to implement business logic? If not - would would be a suggested practice?
Appreciate your help a lot!
in app.models I have:
...
class DBActivity(db.Model):
activity_id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('user.id'), nullable=False)
athlete_id = db.Column(db.Integer, db.ForeignKey('user.strava_id'), nullable=False)
json = db.Column(db.JSON)
laps = db.Column(db.JSON)
streams = db.Column(db.JSON)
df_json = db.Column(db.JSON)
intervals = db.Column(db.JSON)
blob = db.Column(db.BLOB)
comment = db.Column(db.Text)
def __init__(self):
print("DBActivity __init__")
I also have a following class inheriting from DBActivity:
class Activity(DBActivity):
def __init__(self):
super().__init__()
print('just returned from super().__init__')
print('running Activity.__init__')
Here is how I instantiate Activity:
with app.app_context():
a = Activity.query.first()
>>>a
<Activity 5567599209>
>>>isinstance(a, Activity)
True
>>>isinstance(a, DBActivity)
True
I'm looking for a way to do the following.
class Foo(db.Model):
__tablename__ = 'foos_foo'
id = db.Column(db.Integer, primary_key=True)
author_id = db.Column(db.Integer, db.ForeignKey('user.id'))
# Not an entity in the table but
# whenever foo.author_name is called,
# it selects the value from User table
author_name = author.name
The reason why I'm looking for a way to do is this:
class Foo(db.Model, Serializable):
I made a Serializable mixin so that foo.serialize would simply return row values in json.
I wish author.name to be part of this serialization. Of course, there are countless other ways to get author's name and insert it inside the serialized output, but for the benefit of clean code, I wish to find a way to include foreign value in the model.
I use the misnomer 'foreign key' because I have no idea what the most appropriate keyword is.
Thank you in advance.
I ended up with using Marshmallow for object serialization.
http://marshmallow.readthedocs.org/en/latest/index.html
These are the implementations.
model.py
class User(db.Model):
name = db.Column()
class UserSerializer(Serializer):
class Meta:
fields = ('id', 'name')
class FooSerializer(Serializer):
author_name = fields.Nested('UserSerializer')
class Meta:
fields = ('id', 'author_name')
view.py
foos = Foo.query.all()
dict = FooSerializer(foos, many=True).data
Objects are not serialize able.
I think you need implement a method that convert to JSON.
json.dumps works well with dicts. So you can also look at.
http://www.marcstober.com/blog/2007/07/07/serializing-arbitrary-python-objects-to-json-using-dict/
You can implement your own method to_json also.