The way PynamoDB is implemented is that it looks to a specific single DynamoDB table:
class UserModel(Model):
class Meta:
# Specific table.
table_name = 'dynamodb-user'
region = 'us-west-1'
The way my infrastructure works is that it has as many dynamodb tables as I have clients, so a single Lambda function has to deal with any amount of separate tables that are identical in structure e.g. represent "UserModel". I can't specify a concrete one.
How would I make this model definition dynamic?
Thanks!
Possible solution:
def create_user_model(table_name: str, region: str):
return type("UserModel", (Model,), {
"key" : UnicodeAttribute(hash_key=True),
"range_key" : UnicodeAttribute(range_key=True),
# Place for other keys
"Meta": type("Meta", (object,), {
"table_name": table_name,
"region": region,
"host": None,
"billing_mode": 'PAY_PER_REQUEST',
})
})
UserModel_dev = create_user_model("user_model_dev", "us-west-1")
UserModel_prod = create_user_model("user_model_prod", "us-west-1")
Update:
A cleaner version:
class UserModel(Model):
key = UnicodeAttribute(hash_key=True)
range_key = UnicodeAttribute(range_key=True)
#staticmethod
def create(table_name: str, region: str):
return type("UserModelDynamic", (UserModel,), {
"Meta": type("Meta", (object,), {
"table_name": table_name,
"region": region,
"host": None,
"billing_mode": 'PAY_PER_REQUEST',
})
})
Open-sourced a solution that is tested and works.
https://github.com/Biomapas/B.DynamoDbCommon/blob/master/b_dynamodb_common/models/model_type_factory.py
Read README.md for more details.
Code:
from typing import TypeVar, Generic, Type
from pynamodb.models import Model
T = TypeVar('T')
class ModelTypeFactory(Generic[T]):
def __init__(self, model_type: Type[T]):
self.__model_type = model_type
# Ensure that given generic belongs to pynamodb.Model class.
if not issubclass(model_type, Model):
raise TypeError('Given model type must inherit from pynamodb.Model class!')
def create(self, custom_table_name: str, custom_region: str) -> Type[T]:
parent_class = self.__model_type
class InnerModel(parent_class):
class Meta:
table_name = custom_table_name
region = custom_region
return InnerModel
Related
I am using Pydantic with FastApi to output ORM data into JSON. I would like to flatten and remap the ORM model to eliminate an unnecessary level in the JSON.
Here's a simplified example to illustrate the problem.
original output: {"id": 1, "billing":
[
{"id": 1, "order_id": 1, "first_name": "foo"},
{"id": 2, "order_id": 1, "first_name": "bar"}
]
}
desired output: {"id": 1, "name": ["foo", "bar"]}
How to map values from nested dict to Pydantic Model? provides a solution that works for dictionaries by using the init function in the Pydantic model class. This example shows how that works with dictionaries:
from pydantic import BaseModel
# The following approach works with a dictionary as the input
order_dict = {"id": 1, "billing": {"first_name": "foo"}}
# desired output: {"id": 1, "name": "foo"}
class Order_Model_For_Dict(BaseModel):
id: int
name: str = None
class Config:
orm_mode = True
def __init__(self, **kwargs):
print(
"kwargs for dictionary:", kwargs
) # kwargs for dictionary: {'id': 1, 'billing': {'first_name': 'foo'}}
kwargs["name"] = kwargs["billing"]["first_name"]
super().__init__(**kwargs)
print(Order_Model_For_Dict.parse_obj(order_dict)) # id=1 name='foo'
(This script is complete, it should run "as is")
However, when working with ORM objects, this approach does not work. It appears that the init function is not called. Here's an example which will not provide the desired output.
from pydantic import BaseModel, root_validator
from typing import List
from sqlalchemy.orm import relationship
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.dialects.postgresql import ARRAY
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from pydantic.utils import GetterDict
class BillingOrm(Base):
__tablename__ = "billing"
id = Column(Integer, primary_key=True, nullable=False)
order_id = Column(ForeignKey("orders.id", ondelete="CASCADE"), nullable=False)
first_name = Column(String(20))
class OrderOrm(Base):
__tablename__ = "orders"
id = Column(Integer, primary_key=True, nullable=False)
billing = relationship("BillingOrm")
class Billing(BaseModel):
id: int
order_id: int
first_name: str
class Config:
orm_mode = True
class Order(BaseModel):
id: int
name: List[str] = None
# billing: List[Billing] # uncomment to verify the relationship is working
class Config:
orm_mode = True
def __init__(self, **kwargs):
# This __init__ function does not run when using from_orm to parse ORM object
print("kwargs for orm:", kwargs)
kwargs["name"] = kwargs["billing"]["first_name"]
super().__init__(**kwargs)
billing_orm_1 = BillingOrm(id=1, order_id=1, first_name="foo")
billing_orm_2 = BillingOrm(id=2, order_id=1, first_name="bar")
order_orm = OrderOrm(id=1)
order_orm.billing.append(billing_orm_1)
order_orm.billing.append(billing_orm_2)
order_model = Order.from_orm(order_orm)
# Output returns 'None' for name instead of ['foo','bar']
print(order_model) # id=1 name=None
(This script is complete, it should run "as is")
The output returns name=None instead of the desired list of names.
In the above example, I am using Order.from_orm to create the Pydantic model. This approach seems to be the same that is used by FastApi when specifying a response model. The desired solution should support use in the FastApi response model as shown in this example:
#router.get("/orders", response_model=List[schemas.Order])
async def list_orders(db: Session = Depends(get_db)):
return get_orders(db)
Update:
Regarding MatsLindh comment to try validators, I replaced the init function with a root validator, however, I'm unable to mutate the return values to include a new attribute. I suspect this issue is because it is a ORM object and not a true dictionary. The following code will extract the names and print them in the desired list. However, I can't see how to include this updated result in the model response:
#root_validator(pre=True)
def flatten(cls, values):
if isinstance(values, GetterDict):
names = [
billing_entry.first_name for billing_entry in values.get("billing")
]
print(names)
# values["name"] = names # error: 'GetterDict' object does not support item assignment
return values
I also found a couple other discussions on this problem that led me to try this approach:
https://github.com/samuelcolvin/pydantic/issues/717
https://gitmemory.com/issue/samuelcolvin/pydantic/821/744047672
What if you override the from_orm class method?
class Order(BaseModel):
id: int
name: List[str] = None
billing: List[Billing]
class Config:
orm_mode = True
#classmethod
def from_orm(cls, obj: Any) -> 'Order':
# `obj` is the orm model instance
if hasattr(obj, 'billing'):
obj.name = obj.billing.first_name
return super().from_orm(obj)
I really missed the handy Django REST Framework serializers while working with the FastAPI + Pydantic stack... So I wrangled with GetterDict to allow defining field getter function in the Pydantic model like this:
class User(FromORM):
fullname: str
class Config(FromORM.Config):
getter_dict = FieldGetter.bind(lambda: User)
#staticmethod
def get_fullname(obj: User) -> str:
return f'{obj.firstname} {obj.lastname}'
where the magic part FieldGetter is implemented as
from typing import Any, Callable, Optional, Type
from types import new_class
from pydantic import BaseModel
from pydantic.utils import GetterDict
class FieldGetter(GetterDict):
model_class_forward_ref: Optional[Callable] = None
model_class: Optional[Type[BaseModel]] = None
def __new__(cls, *args, **kwargs):
inst = super().__new__(cls)
if cls.model_class_forward_ref:
inst.model_class = cls.model_class_forward_ref()
return inst
#classmethod
def bind(cls, model_class_forward_ref: Callable):
sub_class = new_class(f'{cls.__name__}FieldGetter', (cls,))
sub_class.model_class_forward_ref = model_class_forward_ref
return sub_class
def get(self, key: str, default):
if hasattr(self._obj, key):
return super().get(key, default)
getter_fun_name = f'get_{key}'
if not (getter := getattr(self.model_class, getter_fun_name, None)):
raise AttributeError(f'no field getter function found for {key}')
return getter(self._obj)
class FromORM(BaseModel):
class Config:
orm_mode = True
getter_dict = FieldGetter
I'm building a GraphQL application in Python/Graphene using a MongoDB backend (through MongoEngine). Everything has been working well, but noticed that there's not a lot documentation for handling nested lists of embedded documents. I thought one power of GraphQL was the ability to project only the properties you want, but it doesn't appear to be the case fully.
Looking at this collection as an example:
[
{
"name": "John Doe",
"age": 37,
"preferences": [
{
"key": "colour",
"value": "Green"
},
{
"key": "smell",
"value": "onions cooking in butter"
},
...
]
},
...
]
If I want to find a particular object through GraphQL, I would look up through a query like
{
person(name: "John Doe"){edges{node{
name age preferences{edges{node{
key value
}}}
}}}
}
But this could bring back hundreds of nested documents. What I would like to do instead is to identify the requested nested documents as part of the projection request.
{
person(name: "John Doe"){edges{node{
name age preferences(key: "colour"){edges{node{
key value
}}}
}}}
}
My understanding reading the GraphQL spec is these sub-queries are not possible, but wanted to confirm with experts first. And if it is possible, how would I implement it to support these types of requests?
Update Maybe a schema example will provide some more insightful responses.
class PreferenceModel(mongoengine.EmbeddedDocument):
key = mongoengine.fields.StringField()
value = mongoengine.fields.StringField()
class Preference(graphene_mongo.MongoengineObjectType):
class Meta:
interfaces = (graphene.relay.Node, )
model = PreferenceModel
class PersonModel(mongoengine.Document):
meta = {'collection': 'persons'}
name = mongoengine.fields.StringField()
age = mongoengine.fields.IntField()
preferences = mongoengine.fields.EmbeddedDocumentListField(PreferenceModel)
class Person(graphene_mongo.MongoengineObjectType):
class Meta:
interfaces = (graphene.relay.Node, )
model = PersonModel
class Query(graphene.ObjectType):
person = graphene_mongo.MongoengineConnectionField(Person)
schema = graphene.Schema(query=Query, types=[Person])
app = starlette.graphql.GraphQLApp(schema=schema)
Using this above structure, what changes would be necessary to allow for queries/filters on nested objects?
I had a similiar issue but working graphene-django. I solved it using custom resolvers on the DjangoObjectType, like this:
import graphene
from graphene_django import DjangoObjectType
from .models import Question, Choice, SubChoice
class SubChoiceType(DjangoObjectType):
class Meta:
model = SubChoice
fields = "__all__"
class ChoiceType(DjangoObjectType):
sub_choices = graphene.List(SubChoiceType, search_sub_choices=graphene.String())
class Meta:
model = Choice
fields = ("id", "choice_text", "question")
def resolve_sub_choices(self, info, search_sub_choices=None):
if search_sub_choices:
return self.subchoice_set.filter(sub_choice_text__icontains=search_sub_choices)
return self.subchoice_set.all()
class QuestionType(DjangoObjectType):
choices = graphene.List(ChoiceType, search_choices=graphene.String())
class Meta:
model = Question
fields = ("id", "question_text")
def resolve_choices(self, info, search_choices=None):
if search_choices:
return self.choice_set.filter(choice_text__icontains=search_choices)
return self.choice_set.all()
class Query(graphene.ObjectType):
all_questions = graphene.List(QuestionType, search_text=graphene.String())
all_choices = graphene.List(ChoiceType, search_text=graphene.String())
all_sub_choices = graphene.List(SubChoiceType)
def resolve_all_questions(self, info, search_text=None):
qs = Question.objects.all()
if search_text:
qs = qs.filter(question_text__icontains=search_text)
return qs
def resolve_all_choices(self, info, search_text=None):
qs = Choice.objects.all()
if search_text:
qs = qs.filter(choice_text__icontains=search_text)
return qs
def resolve_all_sub_choices(self, info):
qs = SubChoice.objects.all()
return qs
schema = graphene.Schema(query=Query)
you can find the example here: https://github.com/allangz/graphene_subfilters/blob/main/mock_site/polls/schema.py
It may work for you
So I followed this tutorial to kombine fastapi & peewee:
link
And due to this tutorial i got those models (peewee):
class User(peewee.Model):
email = peewee.CharField(unique=True, index=True)
hashed_password = peewee.CharField()
is_active = peewee.BooleanField(default=True)
class Meta:
database = db
class Item(peewee.Model):
title = peewee.CharField(index=True)
description = peewee.CharField(index=True)
owner = peewee.ForeignKeyField(User, backref="items")
class Meta:
database = db
And those basemodels (fastapi):
class PeeweeGetterDict(GetterDict):
def get(self, key: Any, default: Any = None):
res = getattr(self._obj, key, default)
if isinstance(res, peewee.ModelSelect):
return list(res)
return res
class ItemBase(BaseModel):
title: str
description: Optional[str] = None
class ItemCreate(ItemBase):
pass
class Item(ItemBase):
id: int
owner_id: int
class Config:
orm_mode = True
getter_dict = PeeweeGetterDict
class UserBase(BaseModel):
email: str
class UserCreate(UserBase):
password: str
class User(UserBase):
id: int
is_active: bool
items: List[Item] = []
class Config:
orm_mode = True
getter_dict = PeeweeGetterDict
I have this to call the api:
database.db.connect()
database.db.create_tables([User, Item])
database.db.close()
app = FastAPI()
def get_db(db_state=Depends(reset_db_state)):
try:
database.db.connect()
yield
finally:
if not database.db.is_closed():
database.db.close()
#app.get("/users/", response_model=List[schemas.User], dependencies=[Depends(get_db)])
def read_users():
return list(models.User.select()
This is the base. Now begins my problem / question:
If I call this request (GET "/users/"), I get the following JSON as a result (the data is imaginary its only about the structure)
[
{
"email": "123#test.com"
"id": 1
"is_active": 1
"items": [
{
"title": "item1"
"description": "placeholder"
"id": "1"
"owner_id": "1"
}
]
}
]
This is how it is supposed to be BUT i don't want it exactly like this. I want that I only get the user data, without its items.
So... my question:
How can I get the user data without loading the data of the items?
There is a simple way to do that, response_model_exclude is exactly what you are looking for.
#app.get("/users/", response_model=List[schemas.User], response_model_exclude={"items"}, dependencies=[Depends(get_db)])
def read_users():
return list(models.User.select()
Ref: FastAPI Response Model
I am building a GraphQL API using the python packages Flask, SQLAlchemy, Graphene and Graphene-SQLAlchemy. I have followed the SQLAlchemy + Flask Tutorial. I am able to execute queries and mutations to create records. Now I would like to know what is the best way to update an existing record.
Here is my current script schema.py:
from graphene_sqlalchemy import SQLAlchemyObjectType
from database.batch import BatchOwner as BatchOwnerModel
import api_utils # Custom methods to create records in database
import graphene
class BatchOwner(SQLAlchemyObjectType):
"""Batch owners."""
class Meta:
model = BatchOwnerModel
interfaces = (graphene.relay.Node,)
class CreateBatchOwner(graphene.Mutation):
"""Create batch owner."""
class Arguments:
name = graphene.String()
# Class attributes
ok = graphene.Boolean()
batch_owner = graphene.Field(lambda: BatchOwner)
def mutate(self, info, name):
record = {'name': name}
api_utils.create('BatchOwner', record) # Custom methods to create records in database
batch_owner = BatchOwner(name=name)
ok = True
return CreateBatchOwner(batch_owner=batch_owner, ok=ok)
class Query(graphene.ObjectType):
"""Query endpoint for GraphQL API."""
node = graphene.relay.Node.Field()
batch_owner = graphene.relay.Node.Field(BatchOwner)
batch_owners = SQLAlchemyConnectionField(BatchOwner)
class Mutation(graphene.ObjectType):
"""Mutation endpoint for GraphQL API."""
create_batch_owner = CreateBatchOwner.Field()
schema = graphene.Schema(query=Query, mutation=Mutation)
Remarks:
My object BatchOwner has only 2 attributes (Id, name)
To be able to update the BatchOwner name, I assume I need to provide the database Id (not the relay global Id) as an input argument of some update method
But when I query for a BatchOwner from my client, Graphene only returns me the global Id which is base64 encoded (example: QmF0Y2hPd25lcjox, which correspond to BatchOwner:1)
Example of response:
{
"data": {
"batchOwners": {
"edges": [
{
"node": {
"id": "QmF0Y2hPd25lcjox",
"name": "Alexis"
}
}
]
}
}
}
The solution I am thinking of at the moment would be:
Create an update mutation which takes the global Id as an argument
Decode the global Id (how?)
Use the database Id retrieved from the decoded global Id to query on the database and update the corresponding record
Is there a better way to do this?
I have found a solution using the method from_global_id (documented here)
from graphql_relay.node.node import from_global_id
I added the following class to schema.py:
class UpdateBatchOwner(graphene.Mutation):
"""Update batch owner."""
class Arguments:
id = graphene.String()
name = graphene.String()
# Class attributes
ok = graphene.Boolean()
batch_owner = graphene.Field(lambda: BatchOwner)
def mutate(self, info, id, name):
id = from_global_id(id)
record = {'id': id[1], 'name': name}
api_utils.update('BatchOwner', record)
batch_owner = BatchOwner(id=id, name=name)
ok = True
return UpdateBatchOwner(batch_owner=batch_owner, ok=ok)
And I updated the Mutation class:
class Mutation(graphene.ObjectType):
"""Mutation endpoint for GraphQL API."""
create_batch_owner = CreateBatchOwner.Field()
update_batch_owner = UpdateBatchOwner.Field()
I'm wondering if there is a more straight forward way to do this?
I'd like to know whether it is possible to get the "original id" of an object as the result of the query. Whenever I make a request to the server, it returns the node "global identifier", something like U29saWNpdGFjYW9UeXBlOjEzNTkxOA== .
The query is similar to this one:
{
allPatients(active: true) {
edges {
cursor
node {
id
state
name
}
}
}
and the return is:
{
"data": {
"edges": [
{
"cursor": "YXJyYXljb25uZWN0aW9uOjA=",
"node": {
"id": "U29saWNpdGFjYW9UeXBlOjEzNTkxOA==",
"state": "ARI",
"name": "Brad"
}
}
]
}
}
How can I get the "original" id of the object at the database level (e.g. '112') instead of that node unique identifier?
ps.: I am using graphene-python and Relay on the server side.
Overriding default to_global_id method in Node object worked out for me:
class CustomNode(graphene.Node):
class Meta:
name = 'Node'
#staticmethod
def to_global_id(type, id):
return id
class ExampleType(DjangoObjectType):
class Meta:
model = Example
interfaces = (CustomNode,)
First option, remove relay.Node as interface of your objectNode declaration.
Second option, use custom resolve_id fonction to return id original value.
Example
class objectNode(djangoObjectType):
.... Meta ....
id = graphene.Int(source="id")
def resolve_id("commons args ...."):
return self.id
Hope it helps
To expand on the top answer and for those using SQLAlchemy Object Types, this worked for me:
class CustomNode(graphene.Node):
class Meta:
name = 'myNode'
#staticmethod
def to_global_id(type, id):
return id
class ExampleType(SQLAlchemyObjectType):
class Meta:
model = Example
interfaces = (CustomNode, )
If you have other ObjectTypes using relay.Node as the interface, you will need to use a unique name under your CustomNode. Otherwise you will get and assertion error.
With this you can retrive the real id in database:
def get_real_id(node_id: str):
_, product_id_real = relay.Node.from_global_id(global_id=node_id)
return product_id_real