I'm trying to figure out what would be the best way to create classes in a dynamic manner based on the contents of a JSON file. So for example, here's a snippet from the JSON file:
{
"stuff": [{
"name": "burger",
"aka": ["cheeseburger", "hamburger"]
},
{
"name": "fries",
"aka": ["french fries", "potatoes"]
},
{
"name": "meal",
"items": [{
"name": "burger",
"value": "<burger>"
},
{
"name": "fries",
"value": "<fries>"
}
]
}
]
}
And now based on this JSON, I want classes that represent these objects. So for example, something like:
class Burger:
def __init__(self):
self.name = "burger"
self.aka = ["cheeseburger", "hamburger"]
class Meal:
def __init__(self):
self.name = "meal"
self.burger = Burger()
self.fries = Fries()
So basically, based on that JSON, I want to be able to create classes that represent the same attributes and relationships that we see in the JSON. Any ideas about the best way to approach this would be appreciated!
Assuming json variable contains your json data try this:
for d in json:
name = d.pop('name')
t = type(name, (object,), d)
What it does is to call type, which will create new type in python (exactly the same as if you did class name, which correct name set to content of name variable, with base class object and attributes in d. Variable t will contain class object you want.
Related
I've been working with FastAPI for some time, it's a great framework.
However real life scenarios can be surprising, sometimes a non-standard approach is necessary. There's a one case I'd like to ask your help with.
There's a strange external requirement that a model response should be formatted as stated in example:
Desired behavior:
GET /object/1
{status: ‘success’, data: {object: {id:‘1’, category: ‘test’ …}}}
GET /objects
{status: ‘success’, data: {objects: [...]}}}
Current behavior:
GET /object/1 would respond:
{id: 1,field1:"content",... }
GET /objects/ would send a List of Object e.g.,:
{
[
{id: 1,field1:"content",... },
{id: 1,field1:"content",... },
...
]
}
You can substitute 'object' by any class, it's just for description purposes.
How to write a generic response model that will suit those reqs?
I know I can produce response model that would contain status:str and (depending on class) data structure e.g ticket:Ticket or tickets:List[Ticket].
The point is there's a number of classes so I hope there's a more pythonic way to do it.
Thanks for help.
Generic model with static field name
A generic model is a model where one field (or multiple) are annotated with a type variable. Thus the type of that field is unspecified by default and must be specified explicitly during subclassing and/or initialization. But that field is still just an attribute and an attribute must have a name. A fixed name.
To go from your example, say that is your model:
{
"status": "...",
"data": {
"object": {...} # type variable
}
}
Then we could define that model as generic in terms of the type of its object attribute.
This can be done using Pydantic's GenericModel like this:
from typing import Generic, TypeVar
from pydantic import BaseModel
from pydantic.generics import GenericModel
M = TypeVar("M", bound=BaseModel)
class GenericSingleObject(GenericModel, Generic[M]):
object: M
class GenericMultipleObjects(GenericModel, Generic[M]):
objects: list[M]
class BaseGenericResponse(GenericModel):
status: str
class GenericSingleResponse(BaseGenericResponse, Generic[M]):
data: GenericSingleObject[M]
class GenericMultipleResponse(BaseGenericResponse, Generic[M]):
data: GenericMultipleObjects[M]
class Foo(BaseModel):
a: str
b: int
class Bar(BaseModel):
x: float
As you can see, GenericSingleObject reflects the generic type we want for data, whereas GenericSingleResponse is generic in terms of the type parameter M of GenericSingleObject, which is the type of its data attribute.
If we now want to use one of our generic response models, we would need to specify it with a type argument (a concrete model) first, e.g. GenericSingleResponse[Foo].
FastAPI deals with this just fine and can generate the correct OpenAPI documentation. The JSON schema for GenericSingleResponse[Foo] looks like this:
{
"title": "GenericSingleResponse[Foo]",
"type": "object",
"properties": {
"status": {
"title": "Status",
"type": "string"
},
"data": {
"$ref": "#/definitions/GenericSingleObject_Foo_"
}
},
"required": [
"status",
"data"
],
"definitions": {
"Foo": {
"title": "Foo",
"type": "object",
"properties": {
"a": {
"title": "A",
"type": "string"
},
"b": {
"title": "B",
"type": "integer"
}
},
"required": [
"a",
"b"
]
},
"GenericSingleObject_Foo_": {
"title": "GenericSingleObject[Foo]",
"type": "object",
"properties": {
"object": {
"$ref": "#/definitions/Foo"
}
},
"required": [
"object"
]
}
}
}
To demonstrate it with FastAPI:
from fastapi import FastAPI
app = FastAPI()
#app.get("/foo/", response_model=GenericSingleResponse[Foo])
async def get_one_foo() -> dict[str, object]:
return {"status": "foo", "data": {"object": {"a": "spam", "b": 123}}}
Sending a request to that route returns the following:
{
"status": "foo",
"data": {
"object": {
"a": "spam",
"b": 123
}
}
}
Dynamically created model
If you actually want the attribute name to also be different every time, that is obviously no longer possible with static type annotations. In that case we would have to resort to actually creating the model type dynamically via pydantic.create_model.
In that case there is really no point in genericity anymore because type safety is out of the window anyway, at least for the data model. We still have the option to define a GenericResponse model, which we can specify via our dynamically generated models, but this will make every static type checker mad, since we'll be using variables for types. Still, it might make for otherwise concise code.
We just need to define an algorithm for deriving the model parameters:
from typing import Any, Generic, Optional, TypeVar
from pydantic import BaseModel, create_model
from pydantic.generics import GenericModel
M = TypeVar("M", bound=BaseModel)
def create_data_model(
model: type[BaseModel],
plural: bool = False,
custom_plural_name: Optional[str] = None,
**kwargs: Any,
) -> type[BaseModel]:
data_field_name = model.__name__.lower()
if plural:
model_name = f"Multiple{model.__name__}"
if custom_plural_name:
data_field_name = custom_plural_name
else:
data_field_name += "s"
kwargs[data_field_name] = (list[model], ...) # type: ignore[valid-type]
else:
model_name = f"Single{model.__name__}"
kwargs[data_field_name] = (model, ...)
return create_model(model_name, **kwargs)
class GenericResponse(GenericModel, Generic[M]):
status: str
data: M
Using the same Foo and Bar examples as before:
class Foo(BaseModel):
a: str
b: int
class Bar(BaseModel):
x: float
SingleFoo = create_data_model(Foo)
MultipleBar = create_data_model(Bar, plural=True)
This also works as expected with FastAPI including the automatically generated schemas/documentations:
from fastapi import FastAPI
app = FastAPI()
#app.get("/foo/", response_model=GenericResponse[SingleFoo]) # type: ignore[valid-type]
async def get_one_foo() -> dict[str, object]:
return {"status": "foo", "data": {"foo": {"a": "spam", "b": 123}}}
#app.get("/bars/", response_model=GenericResponse[MultipleBar]) # type: ignore[valid-type]
async def get_multiple_bars() -> dict[str, object]:
return {"status": "bars", "data": {"bars": [{"x": 3.14}, {"x": 0}]}}
Output is essentially the same as with the first approach.
You'll have to see, which one works better for you. I find the second option very strange because of the dynamic key/field name. But maybe that is what you need for some reason.
So I have a JSON file
{
"Vehicles": [
{
"Name": "Car",
"ID": 1
},
{
"Name": "Plane",
"ID": 2
}
]
}
and I created the class in python
class vehicleclass:
def __init__(self, vname, vid):
self.name = vname
self.id = vid
what I would like to do is create an instance of the object vehicles for each vehicle in the JSON, I am reading from the file as shown here
with open('vehicle.json') as json_file:
data = json.load(json_file)
I then run this piece of code
for each in data['Vehicles']:
how do I create an instance of vehicleclass using each 'name' iteration in the JSON file
note I realize I can get the value for each 'name' by calling each['Name'] in the for loop
from what I understand I think this should achieve it.
with open("vehicle.json") as json_file: # opens your vehicles.json file
# this will load your file object into json module giving you a dictionary if its a valid json
data = json.load(json_file)
# this list comprehension uses data dictionary to generate your vehicleclass instances
vehicle_instances = [
vehicleclass(vehicle["Name"], vehicle["ID"]) for vehicle in data["Vehicles"]
]
In Marshmallow in order to have a list field you can use:
include_in = fields.List(cls_or_instance=fields.Str(),
default=['sample1', 'sample2'])
This is OK, but I have a new requirement to have a list of dictionaries in a field. A sample payload:
[{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
}]
This payload is part of the bigger schema, so now the question is how should I add and validate such a field?
EDIT-1:
I went a step further and could find out that there is a Dict field type in Marshmallow so until now I have the below code sample:
fields.List(fields.Dict(
keys=fields.String(validate=OneOf(('name', 'age'))),
values=fields.String(required=True)
))
Now new problem arise and I cannot set different data types for fields in the dictionary (name and age). I'd be happy if someone could shed some light on this.
If the items in the list have the same shape, you can use a nested field within fields.List, like so:
class PersonSchema(Schema):
name = fields.Str()
age = fields.Int()
class RootSchema(Schema):
people = fields.List(fields.Nested(PersonSchema))
Another approach for validating list of dictionaries in a field using one schema class.
from marshmallow import Schema, ValidationError
class PeopleSchema(Schema):
name = fields.Str(required=True)
age = fields.Int(required=True)
people = [{
"name": "Ali",
"age": 20
},
{
"name": "Hasan",
"age": 32
},
{
"name": "Ali",
"age": "twenty" # Error: Not an int
}
]
def validate_people():
try:
validated_data = PeopleSchema(many=True).load(people)
except ValidationError as err:
print(err.messages)
validate_people()
Output:
{2: {'age': ['Not a valid integer.']}}
I have the following two classes in my app.models and i'm using the wagtail APIs to get the data as json
class AuthorMeta(Page):
author=models.OneToOneField(User)
city = models.ForeignKey('Cities', related_name='related_author')
class Cities(Page):
name = models.CharField(max_length=30)
So, when I try /api/v1/pages/?type=dashboard.AuthorMeta&fields=title,city, it returns the following data:
{
"meta": {
"total_count": 1
},
"pages": [
{
"id": 11,
"meta": {
"type": "dashboard.AuthorMeta",
"detail_url": "http://localhost:8000/api/v1/pages/11/"
},
"title": "Suneet Choudhary",
"city": {
"id": 10,
"meta": {
"type": "dashboard.Cities",
"detail_url": "http://localhost:8000/api/v1/pages/10/"
}
}
}
]
}
In the city field, it returns the id and meta of the city. How can I get the name of the city in the response here, without making an extra query? :/
I couldn't find any solution in the Documentation. Am I missing something?
Use Django model property to return through the ForeignKey:
class AuthorMeta(Page):
author=models.OneToOneField(User)
city = models.ForeignKey('Cities', related_name='related_author')
city_name = property(get_city_name)
def get_city_name(self):
return self.city.name
Check Term Property to better understand the concept
In case you have the foreign key in a Streamfield, e.g. a PageChooserBlock, you can customize the api response by overwriting the get_api_representation of a block, as described in the example as provided here:
class CustomPageChooserBlock(blocks.PageChooserBlock):
""" Customize the api response. """
def get_api_representation(self, value, context=None):
""" Return the url path instead of the id. """
return value.url_path
I have a schema that using MongoEngine that looks like this
class User(db.Document)
email = db.EmailField(unique=True)
class QueueElement(db.EmbeddedDocument):
accepts = db.ListField(db.ReferenceField('Resource'))
user = db.ReferenceField(User)
class Resource(db.Document):
name = db.StringField(max_length=255, required=True)
current_queue_element = db.EmbeddedDocumentField('QueueElement')
class Queue(db.EmbeddedDocument):
name = db.StringField(max_length=255, required=True)
resources = db.ListField(db.ReferenceField(Resource))
queue_elements = db.ListField(db.EmbeddedDocumentField('QueueElement'))
class Room(db.Document):
name = db.StringField(max_length=255, required=True)
queues = db.ListField(db.EmbeddedDocumentField('Queue'))
and I would like to return a JSON object of a Room object that would include the information about its queues (together with the referenced resources), and the nested queue_elements ( together with their referenced "accepts" references, and user references)
However, when I want to return a Room with its relationships dereferenced:
room = Room.objects(slug=slug).select_related()
if (room):
return ast.literal_eval(room.to_json())
abort(404)
I don't get any dereferencing. I get:
{
"_cls":"Room",
"_id":{
"$oid":"552ab000605cd92f22347d79"
},
"created_at":{
"$date":1428842482049
},
"name":"second",
"queues":[
{
"created_at":{
"$date":1428842781490
},
"name":"myQueue",
"queue_elements":[
{
"accepts":[
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552ab1f8605cd92f22347d7a"
}
],
"created_at":{
"$date":1428849389503
},
"user":{
"$oid":"552ac8c7605cd92f22347d7b"
}
}
],
"resources":[
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552aafb3605cd92f22347d78"
},
{
"$oid":"552ab1f8605cd92f22347d7a"
}
]
}
],
"slug":"secondslug"
}
even though I'm using the select_related() function. I believe this is because MongoEngine may not follow references on embedded documents. Note, I can actually dereference in the python if I do something like this:
room = Room.objects(slug=slug).first().queues[0].queue_elements[0].accepts[0]
return ast.literal_eval(room.to_json())
which yields
{
"_id":{
"$oid":"552aafb3605cd92f22347d78"
},
"created_at":{
"$date":1428842849393
},
"name":"myRes"
}
which is clearly the dereferenced Resource document.
Is there a way I can follow references on embedded documents? Or is this coming up because I'm following a bad pattern, and should be finding a different way to store this information in MongoDB (or indeed, switch to a Relational DB) ? Thanks!