I want to have model that don't need to be created in database as table but use data from other sources like json files and other model objects. So I am using Non-Managed Model. This model is being created dynamically as described here in the docs
Now Let me explain how I want to create the fields in this non-managed model. I have a json file that defines what fields should be in model, Let's call it contact-model.json. fields in this json file are mapped to django fields and dynamic model is created. This part is done.
{
'model_name': 'Contact'
'fields': {
'name': 'CharField',
'email': 'EmailField'
}
}
Now I have model that is storing all data related to above contact-model.json. Code for that model is given below
class GenericAnswer(models.Model):
answer = JSONField()
model = CharField(default='Contact', max_length=30)
And value of that asnwer would be
{
'name': 'Adil Malik',
'email': 'sample#email.com'
}
What I want to do is that when I fetch Contact.objects.all(), It should return objects fetching from GenericAnswer based on model name, in this onctext, it is Contact.
Can I do it. If Yes, Plese Explain How ????
Related
I'm working on a project that uses AI to recognise the speech of an audio file. The output of this AI is a huge JSON object with tons of values. I'll remove some keys, and the final structure will look as follows.
{
text: "<recognised text>",
language: "<detected language>"
segments: [
{startTimestamp: "00:00:00", endTimestamp: "00:00:10", text: "<some text>"},
{startTimestamp: "00:00:10", endTimestamp: "00:00:17", text: "<some text>"},
{startTimestamp: "00:00:17", endTimestamp: "00:00:26", text: "<some text>"},
{ ... },
{ ... }
]
}
Now, I wish to store this new trimmed object in a SQL database because I wish to be able to edit it manually. I'll create a React application to edit segments, delete segments, etc. Additionally, I want to add this feature to the React application, where the information will be saved every 5 seconds using an AJAX call.
Now, I don't understand how I should store this object in the SQL database. Initially, I thought I would store the whole object as a string in a database. Whenever some change is made to the object, I'll send a JSON object from the React application, the backend will sanitize it and then replace the old stringified object in the database with the new sanitised string object. This way updating and deletion will happen with ease but there can be issues in case of searching. But I'm wondering if there are any better approaches to do this.
Could someone guide me on this?
Tech Stack
Frontend - React
Backend - Django 3.2.15
Database - PostgreSQL
Thank you
Now, I don't understand how I should store this object in the SQL database. Initially, I thought I would store the whole object as a string in a database.
If the data has a clear structure, you should not store it as a JSON blob in a relational database. While relational databases have some support for JSON nowadays, it is still not very effective, and normally it means you can not effectively filter, aggregate, and manipulate data, nor can you check referential integrity.
You can work with two models that look like:
from django.db import models
from django.db.models import F, Q
class Subtitle(models.Model):
text = models.CharField(max_length=128)
language = models.CharField(max_length=128)
class Segment(models.Model):
startTimestamp = models.DurationField()
endTimestamp = models.DurationField()
subtitle = models.ForeignKey(
Subtitle, on_delete=models.CASCADE, related_name='segments'
)
text = models.CharField(max_length=512)
class Meta:
ordering = ('subtitle', 'startTimestamp', 'endTimestamp')
constraints = [
models.CheckConstraint(
check=Q(startTimestamp__gt=F('endTimestamp')),
name='start_before_end',
)
]
This will also guarantee that the startTimestamp is before the endTimestamp for example, that these fields store durations (and not "foo" for example).
You can convert from and to JSON with serializers [drf-doc]:
from rest_framework import serializers
class SegmentSerializer(serializers.ModelSerializer):
class Meta:
model = Segment
fields = ['startTimestamp', 'endTimestamp', 'text']
class SubtitleSerializer(serializers.ModelSerializer):
segments = SegmentSerializer(many=True)
class Meta:
model = Subtitle
fields = ['text', 'language', 'segments']
I have a field in my model which is required in the resource body while creating the resource (POST) and is not passed in the request body(will be passed along with URL) while updating the resource (PUT).
I have created my marshmallow model:
class DummySchema():
field_1 = fields.Str(required=True)
field_2 = fields.Id()
If I leave field_1 to required=True, I get the following error on PUT :
{
"message": {
"field_1": [
"Missing data for required field."
]
}
}
I could leave it required in the model, but that would mean it would cause a problem in POST request.
Is there a way I could set the field as optional, but set it required for my POST request so that I can perform my validation?
I think I should've read the documentation thoroughly before :(
I could set a field as partial, so when it'll do validation marshmallow would skip the field
data, errors = DummySchema().load({'field_2': 42}, partial=('field_1',))
REF: https://marshmallow.readthedocs.io/en/2.x-line/quickstart.html#validation
If you want to use it for /POST request then this field can be added in dump_only list.
This can also be used for /PUT request.
class StrategySchema(ma.SQLAlchemyAutoSchema):
class Meta:
model = Strategy
sqla_session = db.session
ordered = True
load_instance = True
dump_only = ( # read-only
"id",
"created_by",
"created_by_id",
"created_at",
"updated_at",
)
dump_only means:
Consider these fields only while dumping a model to json (deserialization)
ignore it while loading a model from json
read-only fields in other words
Overview
I am using Flask-SqlAlchemy and now I am looking into marshmallow to help me serialize and deserialize request data.
I was able to successfully:
Create my models using Flask-SqlAlchemy
Use Flask-Marshmallow to serialize database objects using the same model, by using the Optional Flask-SqlAlchemy Integration
Use marshmallow-jsonapi to quickly generate Json API compliant responses. This required me to declare new Schemas to specify which attributes I want to include (this is duplicate from Flask-SqlAlchemy Models)
Code Samples
Flask-SqlAlchemy Declarative Model
class Space(db.Model):
__tablename__ = 'spaces'
id = sql.Column(sql.Integer, primary_key=True)
name = sql.Column(sql.String)
version = sql.Column(sql.String)
active = sql.Column(sql.Boolean)
flask_marshmallow Schema Declaration (Inherits from SqlAlchemy Model)
ma = flask_marshmallow.Marshmallow(app)
class SpaceSchema(ma.ModelSchema):
class Meta:
model = Space
# API Response
space = Space.query.first()
return SpaceSchema().dump(space).data
# Returns:
{
'id': 123,
'version': '0.1.0',
'name': 'SpaceName',
'active': True
}
marshmallow_json api - requires new Schema Declaration, must include each attribute and type manually
class SpaceJsonSchema(marshmallow_json.Schema):
id = fields.Str(dump_only=True)
name = fields.Str()
version = fields.Str()
active = fields.Bool()
class Meta:
type_ = 'spaces'
self_url = '/spaces/{id}'
self_url_kwargs = {'id': '<id>'}
self_url_many = '/spaces/'
strict = True
# Returns Json API Compliant
{
'data': {
'id': '1',
'type': 'spaces',
'attributes': {
'name': 'Phonebooth',
'active': True,
'version': '0.1.0'
},
'links': {'self': '/spaces/1'}
},
'links': {'self': '/spaces/1'}
}
Issue
As shown in the code, marshmallow-jsonapi allows me to create json api compliant responses, but I end up having to maintain a Declarative Model + Schema Response model.
flask-marshmallow allows me to create Schema responses from the SqlAlchemy models, so I don't have to maintain a separate set of properties for each model.
Question
Is it at all possible to use flask-marshmallow and marshmallow-jsonapi together so 1. Create Marshmallow Schema from a SqlAlchemy model, AND automatically generate json api responses?
I tried creating Schema declaration that inherited from ma.ModelSchema and marshmallow_json.Schema, in both orders, but it does not work (raises exception for missing methods and properties)
marshmallow-jsonapi
marshmallow-jsonapi provides a simple way to produce JSON
API-compliant data in any Python web framework.
flask-marshmallow
Flask-Marshmallow includes useful extras for integrating with
Flask-SQLAlchemy and marshmallow-sqlalchemy.
Not a solution to this exact problem but I ran into similar issues when implementing this library : https://github.com/thomaxxl/safrs (sqlalchemy + flask-restful + jsonapi compliant spec).
I don't remember exactly how I got around it, but if you try it and serialization doesn't work I can help you resolve it if you open an issue in github
I have parent-child relationships in DataStore model: Building entity with reference entity to Office. I perform query on Building model and I would like to limit fields of Office entity in JSON response.
Here is my code:
#Building.query_method(collection_fields=('id', 'name', 'office'), path='buildings', name='list')
def List(self, query):
return query
collection_fields attribute works great only to define parent entity fields (Building), but how to limit fields of child entity?
Here is my response message in JSON:
{ id : 5
name : 'building name'
office: {
name: 'office name',
field1 : 'test',
field1 : 'test',
field1 : 'test'
}
}
I would like to remove some fields from Office object (i.e field1,field2 etc) to reduce JSON response size.
Define limited_message_fields_schema of Office object is not good solution, because it works globally. I would like to format only this single query.
You can create EndpointsAliasProperty in the Building model, where you can transform self.office and use that value in collection_fields:
#EndpointsAliasProperty
def office_ltd(self):
limited = doSomethingWith(self.office)
return limited
#Building.query_method(collection_fields=('id', 'name', 'office_ltd'),
path='buildings', name='list')
def List(self, query):
return query
I have two related models (Events + Locations) with a serialzer shown below:
class Locations
title = models.CharField(max_length=250)
address = model.CharField(max_length=250)
class Events
title = models.CharField(max_length=250)
locations = models.ForeignKey(Locations, related_name='events'
class EventsSerializer(serializers.ModelSerializer):
class Meta:
model = Events
depth = 1
I set the depth to 1 in the serializer so I can get the information from the Locations model instead of a single id. When doing this however, I cant post to events with the location info. I can only perform a post with the title attribute. If I remove the depth option in the serializer, I can perform the post with both the title and location id.
I tried to create a second serializer (EventsSerialzerB) without the depth field with the intention of using the first one as a read-only response, however when I created a second serializer, viewset, and added it to the router, it would automatically override the original viewset.
Is it possible for me to create a serializer that outputs the related model fields, and allows you to post directly to the single model?
// EDIT - Here's what I'm trying to post
$scope.doClick = function (event) {
var test_data = {
title: 'Event Test',
content: 'Some test content here',
location: 2,
date: '2014-12-16T11:00:00Z'
}
// $resource.save() doesn't work?
$http.post('/api/events/', test_data).
success(function(data, status, headers, config) {
console.log('sucess', status);
}).
error(function(data, status, headers, config) {
console.log('error', status);
});
}
So when the serializers are flat, I can post all of these fields. The location field is the id of a location from the related Locations table. When they are nested, I can't include the location field in the test data.
By setting the depth option on the serializer, you are telling it to make any relation nested instead of flat. For the most part, nested serializers should be considered read-only by default, as they are buggy in Django REST Framework 2.4 and there are better ways to handle them in 3.0.
It sounds like you want a nested representation when reading, but a flat representation when writing. While this isn't recommended, as it means GET requests don't match PUT requests, it is possible to do this in a way to makes everyone happy.
In Django REST Framework 3.0, you can try the following to get what you want:
class LocationsSerializer(serializers.ModelSerializer):
class Meta:
model = Locations
fields = ('title', 'address', )
class EventsSerializer(serializers.ModelSerializer):
locations = LocationsSerializer(read_only=True)
class Meta:
model = Events
fields = ('locations', )
class EventViewSet(viewsets.ModelViewSet):
queryet = Event.objects.all()
serializer_class = EventsSerializer
def perform_create(self, serializer):
serializer.save(locations=self.request.data['locations'])
def perform_update(self, serializer):
serializer.save(locations=self.request.data['locations'])
A new LocationsSerializer was created, which will handle the read-only nested representation of the Locations object. By overriding perform_create and perform_update, we can pass in the location id that was passed in with the request body, so the location can still be updated.
Also, you should avoid having model names being plurals. It's confusing when Events.locations is a single location, even though Locations.events is a list of events for the location. Event.location and Location.events reads a bit more clearly, the Django admin will display them reasonably, and your fellow developers will be able to easily understand how the relations are set up.