I am writing a graphene/django ORM query, where I need to aggregate the values of a particular field on all my query result objects and return it with the query. Not quite sure how to do that, as this involves some post-processing. Would appreciate it if someone can offer some guidance.
Here's some sample code. Django model class 'Market' has an integer field 'num_vendors'. The Graphene wrapper is 'MarketNode' that wraps around the 'Market' model class:
Model class:
class Market(models.Model):
num_vendors = models.IntegerField(....)
Graphene class:
class MarketNode(DjangoObjectType):
Meta:
model: Market
I'd like the query to return 'market_count' (there are multiple markets) and 'vendor_count' (sum of all 'vendors' across all markets). So the query would look like:
allMarkets {
market_count
vendor_count
edges {
node {
...
...
num_vendors
...
}
}
}
For the market_count, I am following this example (this works fine):
https://github.com/graphql-python/graphene-django/wiki/Adding-counts-to-DjangoFilterConnectionField
For vendor_count (across all markets), I assume I need to iterate over the results and add all the num_vendors fields, after the query is complete and resolved. How can I achieve this? This must be a fairly common-use case, so I am sure graphene provides some hooks to do this.
You can define MarketConnection with the count fields on it.
Something like:
class MarketConnection(graphene.relay.Connection):
class Meta:
node = Market
market_count = graphene.Int(required=True)
vendor_count = graphene.Int(required=True)
def resolve_market_count(self, info, **kwargs):
return self.iterable.count() if isinstance(self.iterable, QuerySet) else len(self.iterable)
def resolve_vendor_count(self, info, **kwargs):
if isinstance(self.iterable, QuerySet):
return self.iterable.aggregate(Count("vendor"))
return sum([market.num_vendors for market in self.iterable])
and add connection_class to your MarketNode
class MarketNode(DjangoObjectType):
class Meta:
model: Market
connection_class: MarketConnection
Related
I'm using Wagtail, and I want to filter a selection of child pages by a Foreign Key. I've tried the following and I get the error django.core.exceptions.FieldError: Cannot resolve keyword 'use_case' into field when I try children = self.get_children().specific().filter(use_case__slug=slug):
class AiLabResourceMixin(models.Model):
parent_page_types = ['AiLabResourceIndexPage']
use_case = models.ForeignKey(AiLabUseCase, on_delete=models.PROTECT)
content_panels = ArticlePage.content_panels + [
FieldPanel('use_case', widget=forms.Select())
]
class Meta:
abstract = True
class AiLabCaseStudy(AiLabResourceMixin, ArticlePage):
pass
class AiLabBlogPost(AiLabResourceMixin, ArticlePage):
pass
class AiLabExternalLink(AiLabResourceMixin, ArticlePage):
pass
class AiLabResourceIndexPage(RoutablePageMixin, BasePage):
parent_page_types = ['AiLabHomePage']
subpage_types = ['AiLabCaseStudy', 'AiLabBlogPost', 'AiLabExternalLink']
max_count = 1
#route(r'^$')
def all_resources(self, request):
children = self.get_children().specific()
return render(request, 'ai_lab/ai_lab_resource_index_page.html', {
'page': self,
'children': children,
})
#route(r'^([a-z0-9]+(?:-[a-z0-9]+)*)/$')
def filter_by_use_case(self, request, slug):
children = self.get_children().specific().filter(use_case__slug=slug)
return render(request, 'ai_lab/ai_lab_resource_index_page.html', {
'page': self,
'children': children,
})
I've seen this answer, but this assumes I only have one type of page I want to filter. Using something like AiLabCaseStudy.objects.filter(use_case__slug=slug) works, but this only returns AiLabCaseStudys, not AiLabBlogPosts or AiLabExternalLinks.
Any ideas?
At the database level, there is no efficient way to run the filter against all page types at once. Since AiLabResourceMixin is defined as abstract = True, this class has no representation of its own within the database - instead, the use_case field is defined separately for each of AiLabCaseStudy, AiLabBlogPost and AiLabExternalLink. As a result, there's no way for Django or Wagtail to turn .filter(use_case__slug=slug) into a SQL query, since use_case refers to three different places in the database.
A couple of possible ways around this:
If your data model allows, restructure it to use multi-table inheritance - this looks fairly similar to your current definition, except without the abstract = True:
class AiLabResourcePage(ArticlePage):
use_case = models.ForeignKey(AiLabUseCase, on_delete=models.PROTECT)
class AiLabCaseStudy(AiLabResourcePage):
pass
class AiLabBlogPost(AiLabResourcePage):
pass
class AiLabExternalLink(AiLabResourcePage):
pass
AiLabResourcePage will then exist in its own right in the database, and you can query its use_case field with an expression like: AiLabResourcePage.objects.child_of(self).filter(use_case__slug=slug).specific(). There'll be a small performance impact here, since Django has to pull data from one additional table to construct these page objects.
Run a preliminary query on each specific page type to retrieve the matching page IDs, before running the final query with specific():
case_study_ids = list(AiLabCaseStudy.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
blog_post_ids = list(AiLabBlogPost.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
external_link_ids = list(AiLabExternalLink.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
children = Page.objects.filter(id__in=(case_study_ids + blog_post_ids + external_link_ids)).specific()
Try:
children = self.get_children().filter(use_case__slug=slug).specific()
Hello I'm new to flask and I have an application where i am creating different models and schemas for my entities. These two models and schemas are very close to each other except with few differences. I have base classes for my model and schema so i could inherit and re-use the same class. However, i'm having a problem when i need to deserialize them with marshall and return the union result.
I'm using marshmallow,sql-achemy and flaskapi-spec. I am not sure if there's a way to use the marshall_with decorator with multiple schemas since I want to union my results and return the aggregated model.
Here is the endpoint,models and classes I have.
Models;
class BasePublisher(Model):
__abstract__= True
id= Column(db.String(80),primary_key=True,nullable=False)
date = Column(db.DateTime, default=dt.datetime.utcnow, primary_key=True, nullable=False)
views = Column(db.Numeric)
clicks = Column(db.Numeric)
publisher = Column(db.String(80),primary_key=True,nullable=False)
class Facebook(BasePublisher):
__tablename__='facebook_table'
def __init__(self, **kwargs):
db.Model.__init__(self, **kwargs)
class Pinterest(BasePublisher):
__tablename__='pin_table'
def __init__(self, user, **kwargs):
db.Model.__init__(self, user=user, **kwargs)
Schemas
class PublisherSchema(Schema):
date = fields.DateTime(dump_only=True)
type = fields.DateTime(dump_only=True)
views = fields.Number(dump_only=True)
clicks = fields.Number(dump_only=True)
publisher = fields.Str(dump_only=True)
class FacebookSchema(PublisherSchema):
#post_dump
def dump_data(self,data):
data["type"]="Facebok"
class PinterestSchema(PublisherSchema):
#post_dump
def dump_data(self,data):
data["type"]="Pinterest
"
-View
#blueprint.route('/api/sample/publishers/<id>', methods=('GET',))
#use_kwargs({'type': fields.Str(), 'start_date': fields.Str(),'end_date':fields.Str()},location="query")
#marshal_with(facebook_schema)
def get_data(id, type, start_date=None,end_date=None):
facebook_data = Facebook.query.filter_by(id=id)
.filter(Facebook.date.between(start_date,end_date))
.limit(10).all()
Ideally i would like to do this in my view;
pinterest_data = Pinterest.query.filter_by(id=id)
.filter(Pinterest.date.between(start_date,end_date))
.limit(10).all()
facebook_data.query.union(pinterest_data)
Union like this throws an error in flask application and also i have slightly different schemas for each publisher and i don't know how i can return both of them when i de-serialize with marshall
something like this maybe?
#marshal_with(facebook_schema,pinterest_schema)
I've got this filter:
class SchoolFilter(django_filters.FilterSet):
class Meta:
model = School
fields = {
'name': ['icontains'],
'special_id': ['icontains'],
}
Where special_id is a #property of the School Model:
#property
def special_id(self):
type = self.type
unique_id = self.unique_id
code = self.code
if unique_id < 10:
unique_id = f'0{unique_id}'
if int(self.code) < 10:
code = f'0{self.code}'
special_id = f'{code}{type}{id}'
return special_id
I've tried to google some answers, but couldn't find anything. Right now If I use my filter like I do I only receive this error:
'Meta.fields' contains fields that are not defined on this FilterSet: special_id
How could I define the property as a field for this FilterSet? Is it even possible for me to use django-filter with a #property?
Thanks for any answer!
Update:
Figured it out. Not the prettiest solution, but ayyy
class SchoolFilter(django_filters.FilterSet):
special_id = django_filters.CharFilter(field_name="special_id", method="special_id_filter", label="Special School ID")
def special_id_filter(self, queryset, name, value):
schools_pk = []
for obj in queryset:
if obj.special_id == value:
schools_pk.append(obj.pk)
queryset = queryset.filter(pk__in=schools_pk)
return queryset
class Meta:
model = School
fields = {
'name': ['icontains'],
'special_id': ['icontains'],
}
You can't. FilterSet will only filter on actual fields, since FilterSet alters a QuerySet.
QuerySets do a database call based on the filters applied, which means you can only filter on fields actually stored in the database.
You could annotate your QuerySet to add the special_id, but an annotation like this is pretty complex to chain together.
A better way to do this would be to create a custom filter on your FilterSet, but I'm not exactly sure how to do this. If you can explain what special_id is, and exactly why you want to search it through icontains, then I could maybe point you in the right direction.
This is an implementation of a MethodFilter, which I think is similar what you want.
FilterSet operates by filtering queryset (adding where conditions to the underlying sql). Which means, FilterSet can operate only on Columns that are present in the database. Here the special_id is a computed property (It is not a column, it is calculated on the fly using other fields/columns), So it wont work.
The work around is to make special_id a normal field/column, compute the value at runtime and write to database at the time of save.
So I have a serializer that looks like this
class BuildingsSerializer(serializers.ModelSerializer):
masterlisting_set = serializers.PrimaryKeyRelatedField(many=True,
queryset=Masterlistings.objects.all())
and it works great
serializer = BuildingsSerializer(Buildings.objects.get(pk=1))
serializer.data
produces
OrderedDict([
("masterlistings_set", [
"0a06e3d7-87b7-4526-a877-c10f54fa5bc9",
"343643ac-681f-4597-b8f5-ff7e5be65eef",
"449a3ad2-c76c-4cb8-bb86-1be72fafcf64",
])
])
but if I change the queryset in the serializer to
class BuildingsSerializer(serializers.ModelSerializer):
masterlistings_set = serializers.PrimaryKeyRelatedField(many=True, queryset=[])
I still get the same exact result back.
OrderedDict([
("masterlistings_set", [
"0a06e3d7-87b7-4526-a877-c10f54fa5bc9",
"343643ac-681f-4597-b8f5-ff7e5be65eef",
"449a3ad2-c76c-4cb8-bb86-1be72fafcf64",
])
])
Is this supposed to be happening? Am I using querysets incorrectly?
I used [] as an easy example to show that no matter what I put in nothing changes.
Please any insight would be invaluable
It should be noted that masterlistings has a primary key relationship that points to buildings. So a masterlisting belong to a building.
As pointed out by #zymud, queryset argument in PrimaryKeyRelatedField is used for validating field input for creating new entries.
Another solution for filtering out masterlistings_set is to use serializers.SerializerMethodField() as follows:
class BuildingsSerializer(serializers.ModelSerializer):
masterlisting_set = serializers.SerializerMethodField()
def get_masterlisting_set(self, obj):
return MasterListing.objects.filter(building=obj).values_list('pk',flat=True)
queryset in related field limits only acceptable values. So with queryset=[] you will not be able to add new values to masterlisting_set or create new Buildings.
UPDATE. How to use queryset for filtering
This is a little bi tricky - you need to rewrite ManyRelatedField and many_init method in your RelatedField.
# re-define ManyRelatedField `to_representation` method to filter values
# based on queryset
class FilteredManyRelatedField(serializers.ManyRelatedField):
def to_representation(self, iterable):
iterable = self.child_relation.queryset.filter(
pk__in=[value.pk for value in iterable])
return super(FilteredManyRelatedField, self).to_representation(iterable)
# use overridden FilteredManyRelatedField in `many_init`
class FilteredPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
#classmethod
def many_init(cls, *args, **kwargs):
kwargs['child_relation'] = cls(queryset=kwargs.pop('queryset'))
return FilteredManyRelatedField(*args, **kwargs)
I have a problem with serialization of Django inherited models. For example
class Animal(models.Model):
color = models.CharField(max_length=50)
class Dog(Animal):
name = models.CharField(max_length=50)
...
# now I want to serialize Dog model with Animal inherited fields obviously included
print serializers.serialize('xml', Dog.objects.all())
and only Dog model has been serialized.
I can do smth like
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
But it looks ugly and because my models are very big so I have to use SAX parser and with such output it's difficult to parse.
Any idea how to serialize django models with parent class?
**EDIT: ** It use to work ok before this patch has been applied. And the explanation why the patch exist "Model saving was too aggressive about creating new parent class instances during deserialization. Raw save on a model now skips saving of the parent class. " I think there should be an option to be able to serialize "local fields only" by default and second option - "all" - to serialize all inherited fields.
You found your answer in the documentation of the patch.
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
However, if you change Animal to be an abstract base class it will work:
class Animal(models.Model):
color = models.CharField(max_length=50)
class Meta:
abstract = True
class Dog(Animal):
name = models.CharField(max_length=50)
This works as of Django 1.0. See http://docs.djangoproject.com/en/dev/topics/db/models/.
You'll need a custom serializer to support inherited fields, as Django's serializer will only serialize local fields.
I ended up writing my own when dealing with this issue, feel free to copy it: https://github.com/zmathew/django-backbone/blob/master/backbone/serializers.py
In order to use it on its own, you need to do:
serializer = AllFieldsSerializer()
serializer.serialize(queryset, fields=fields)
print serializer.getvalue()
I had the same problem, and i wrote a 'small' queryset serializer which navigates up the inheritance tree and returns all the fields serialized.
It's far from perfect... but works for me :)
a = QuerySetSerializer(MyModel, myqueryset)
a.serialize()
And the snippet:
from __future__ import unicode_literals
import json
import inspect
from django.core import serializers
from django.db.models.base import Model as DjangoBaseModel
class QuerySetSerializer(object):
def __init__(self, model, initial_queryset):
"""
#param model: The model of your queryset
#param initial_queryset: The queryset to serialize
"""
self.model = model
self.initial_queryset = initial_queryset
self.inheritance_tree = self._discover_inheritance_tree()
def serialize(self):
list_of_querysets = self._join_inheritance_tree_objects()
merged_querysets = self._zip_queryset_list(list_of_querysets)
result = []
for related_objects in merged_querysets:
result.append(self._serialize_related_objects(related_objects))
return json.dumps(result)
def _serialize_related_objects(self, related_objects):
"""
In this method, we serialize each instance using the django's serializer function as shown in :
See https://docs.djangoproject.com/en/1.10/topics/serialization/#inherited-models
However, it returns a list with mixed objects... Here we join those related objects into one single dict
"""
serialized_objects = []
for related_object in related_objects:
serialized_object = self._serialize_object(related_object)
fields = serialized_object['fields']
fields['pk'] = serialized_object['pk']
serialized_objects.append(fields)
merged_related_objects = {k: v for d in serialized_objects for k, v in d.items()}
return merged_related_objects
def _serialize_object(self, obj):
data = serializers.serialize('json', [obj, ])
struct = json.loads(data)
return struct[0]
def _discover_inheritance_tree(self):
# We need to find the inheritance tree which excludes abstract classes,
# so we can then join them when serializing the instance
return [x for x in inspect.getmro(self.model) if x is not object and x is not DjangoBaseModel and not x._meta.abstract]
def _join_inheritance_tree_objects(self):
"""
Here we join the required querysets from the non abstract inherited models, which we need so we are able to
serialize them.
Lets say that MyUser inherits from Customer and customer inherits from django's User model
This will return [list(MyUser.objects.filter(...), list(Customer.objects.filter(...), list(User.objects.filter(...)
"""
initial_ids = self._get_initial_ids()
inheritance__querysets = [list(x.objects.filter(id__in=initial_ids).order_by("id")) for x in self.inheritance_tree]
return inheritance__querysets
def _zip_queryset_list(self, list_of_querysets):
"""
At this stage, we have something like:
(
[MyUser1, MyUser2, MyUser3],
[Customer1, Customer2, Customer3],
[User1, User2, User3]
)
And to make it easier to work with, we 'zip' the list of lists so it looks like:
(
[MyUser1, Customer1, User1],
[MyUser2, Customer2, User2],
[MyUser3, Customer3, User3],
)
"""
return zip(*list_of_querysets)
def _get_initial_ids(self):
"""
Returns a list of ids of the initial queryset
"""
return self.initial_queryset.order_by("id").values_list("id", flat=True)
You can define a custom Serializer:
class DogSerializer(serializers.ModelSerializer):
class Meta:
model = Dog
fields = ('color','name')
Use it like:
serializer = DogSerializer(Dog.objects.all(), many=True)
print serializer.data enter code here
Did you look at select_related() ?
as in
serializers.serialize('xml', Dog.objects.select_related().all())