Develop Reference

Aggregating fields in graphene/django queries

I am writing a graphene/django ORM query, where I need to aggregate the values of a particular field on all my query result objects and return it with the query. Not quite sure how to do that, as this involves some post-processing. Would appreciate it if someone can offer some guidance.
Here's some sample code. Django model class 'Market' has an integer field 'num_vendors'. The Graphene wrapper is 'MarketNode' that wraps around the 'Market' model class:
Model class:
class Market(models.Model):
num_vendors = models.IntegerField(....)
Graphene class:
class MarketNode(DjangoObjectType):
Meta:
model: Market
I'd like the query to return 'market_count' (there are multiple markets) and 'vendor_count' (sum of all 'vendors' across all markets). So the query would look like:
allMarkets {
market_count
vendor_count
edges {
node {
...
...
num_vendors
...
}
}
}
For the market_count, I am following this example (this works fine):
https://github.com/graphql-python/graphene-django/wiki/Adding-counts-to-DjangoFilterConnectionField
For vendor_count (across all markets), I assume I need to iterate over the results and add all the num_vendors fields, after the query is complete and resolved. How can I achieve this? This must be a fairly common-use case, so I am sure graphene provides some hooks to do this.

You can define MarketConnection with the count fields on it.
Something like:
class MarketConnection(graphene.relay.Connection):
class Meta:
node = Market
market_count = graphene.Int(required=True)
vendor_count = graphene.Int(required=True)
def resolve_market_count(self, info, **kwargs):
return self.iterable.count() if isinstance(self.iterable, QuerySet) else len(self.iterable)
def resolve_vendor_count(self, info, **kwargs):
if isinstance(self.iterable, QuerySet):
return self.iterable.aggregate(Count("vendor"))
return sum([market.num_vendors for market in self.iterable])
and add connection_class to your MarketNode
class MarketNode(DjangoObjectType):
class Meta:
model: Market
connection_class: MarketConnection

Django-CMS Child plugin to show filtered data from DB table

I have two plugins ProductSelector(parent) and SpecificationSelector(child). I want to set the child up so that when you add it to the parent the Specifications that are shown are the only ones for the product (parent). Right now it pulls in all the specifications from the table. These lines let me filter the data to get what I want.
edit: I found an error that i fixed in the code. I had the PluginBase names the same as the model. This allowed me to use ProductSelector.objects.get(cmsplugin_ptr=instance.parent) in the child to get the parent instance. I still need to figure out how to pass the filtered specification list to the "PluginAdmin Interface"
product = ProductSelector.objects.get(cmsplugin_ptr=instance.parent)
specification = Specifications.objects.filter(product_name__product_name__iexact = product.product_name)
However, I haven't figured out how to send that filtered list to the plugin admin interface.
class ProductSelectorPlugin(CMSPluginBase):
model = ProductSelector
name = "Product Selector"
render_template = "product_selector.html"
allow_children = True
child_classes = ['SpecificationSelectorPlugin']
def render(self, context, instance, placeholder):
context['instance'] = instance
return context
plugin_pool.register_plugin(ProductSelectorPlugin)
class SpecificationSelectorPlugin(CMSPluginBase):
model = SpecificationSelector
render_template = "specification_selector.html"
formfield_overrides = {models.ManyToManyField: {'widget': CheckboxSelectMultiple},}
def render(self, context, instance, placeholder):
product = ProductSelector.objects.get(cmsplugin_ptr=instance.parent)
specification = Specifications.objects.filter(product_name__product_name__iexact = product.product_name)
context['instance'] = instance
return context
plugin_pool.register_plugin(SpecificationSelectorPlugin)
models.py
class ProductSelector(CMSPlugin):
product_name = models.ForeignKey(Product, help_text = "Select the product you want to place")
new_product = models.BooleanField(blank=True)
class SpecificationSelector(CMSPlugin):
specification = models.ManyToManyField(Specifications, blank=True)
def __unicode__(self):
return unicode(self.specification)
Here is an screenshot the Django-cms plugins in the placeholder. Currently it is showing all specs in the table, but I just want it to be the specs for that particular product.
http://imgur.com/3R1LobC
Thank you in advance for the help.

CMSPluginBase inhertis from ModelAdmin which means that you can override the form rendered when adding and editing your plugin.
So you can create a ModelForm subclass like so:
class SpecificationSelectorPluginForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super(SpecificationSelectorPluginForm, self).__init__(*args, **kwargs)
if self.instance.parent_id:
# Assume that the parent is a product instance
parent_plugin = self.instance.parent
product = parent_plugin.get_plugin_instance()[0]
if product:
# It's possible that product is an orphan plugin.
specifications = Specifications.objects.filter(
product_name__product_name__iexact=product.product_name)
self.fields['specification'].queryset = specifications
then change your SpecificationSelectorPlugin to use this form like so:
class SpecificationSelectorPlugin(CMSPluginBase):
form = SpecificationSelectorPluginForm
The above will only work if the specification plugin is a direct child of the product plugin.

many to many relationship between different types using a through table

I've been trying to understand how to get a many to many relationship working correctly. Here's the simplified model of Entity and Event. The relationships I'm trying to model are:
Entities have a many to many with other Entities
Entities have a many to many with Events
From a lookup perspective, here's what works so far:
look up entities that I have a one to many relationship with
look up entities that have a relationship to this entity (reverse/backwards relation)
look up events that I have a one to many relationship with
But I haven't been able to figure out:
look up entities that have a relationship to this event (reverse/backward relation)
class Entity2EntityRel(Model):
from_entity = ForeignKey('Entity', to_field='uuid',
related_name="from_entity")
to_entity = ForeignKey('Entity', to_field='uuid',
related_name="to_entity")
relation = CharField(max_length=64)
def __unicode__(self):
return json.dumps({
"from_entity": str(self.from_entity),
"relation": str(self.relation),
"to_entity": str(self.to_event)
})
class Entity2EventRel(Model):
from_entity = ForeignKey('Entity', to_field='uuid',
related_name="from_entity_ev")
to_event = ForeignKey('Event', to_field='uuid',
related_name="to_event_en")
relation = CharField(max_length=64)
def __unicode__(self):
return json.dumps({
"from_entity": str(self.from_entity),
"relation": str(self.relation),
"to_event": str(self.to_event)
})
class Entity(PolymorphicModel):
uuid = UUIDField(unique=True)
name = CharField(max_length=255, unique=True)
entity_rels = ManyToManyField(
'Entity',
through='Entity2EntityRel',
related_name="related_entity_set",
symmetrical=False)
event_rels = ManyToManyField(
'Event',
through='Entity2EventRel',
related_name="related_event_set",
symmetrical=False)
def add_entity_rel(self, entity, relation_name):
relationship, created = Entity2EntityRel.objects.get_or_create(
from_entity=self,
to_entity=entity,
relation=relation_name)
return relationship
# return entities related to me
def get_entity_rels(self, relation_name):
return self.entity_rels.filter(
to_entity__relation=relation_name,
to_entity__from_entity=self)
# return items that have a relation to me
def get_related_entities(self, relation_name):
return self.related_entity_set.filter(
from_entity__relation=relation_name,
from_entity__to_entity=self)
def add_event_rel(self, event, relation_name):
relationship, created = Entity2EventRel.objects.get_or_create(
from_entity=self,
to_event=event,
relation=relation_name)
return relationship
# return events related to me
def get_event_rels(self, relation_name):
return self.event_rels.filter(
to_event_en__relation=relation_name,
to_event_en__from_entity=self)
Now how do I get the entities that are related to a particular event?

django heterogeneous queryset proxy models

I am trying to figure out how to use proxy classes in Django. I want to receive a queryset where each object belongs to a proxy class of a common super class so that I can run custom sub-classed methods with the same name and my controller logic doesn't need to know or care about which kind of Proxy model it is working with. One thing I don't want to do is to store the information in multiple tables because I want to have unified identifiers for easier reference/management.
I am pretty new to django/python so I would be happy to hear alternative ways to accomplish what I am trying to do.
Here is what I have:
TYPES = (
('aol','AOL'),
('yhoo','Yahoo'),
)
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
type = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
class ConnectAOL(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to AOL
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
class ConnectYahoo(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to Yahoo
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
Now what I want to do is this:
connections = SuperConnect.objects.all()
for connection in connections:
connection.connect()
connection.dostuff
I've looked around and found some hacks but they look questionable and may require me to go to the database for each item in order to retrieve data I probably already have...
Somebody please rescue me :) or I am going to go with this hack:
class MixedQuerySet(QuerySet):
def __getitem__(self, k):
item = super(MixedQuerySet, self).__getitem__(k)
if item.atype == 'aol':
yield(ConnectAOL.objects.get(id=item.id))
elif item.atype == 'yhoo':
yield(ConnectYahoo.objects.get(id=item.id))
else:
raise NotImplementedError
def __iter__(self):
for item in super(MixedQuerySet, self).__iter__():
if item.atype == 'aol':
yield(ConnectAOL.objects.get(id=item.id))
elif item.atype == 'yhoo':
yield(ConnectYahoo.objects.get(id=item.id))
else:
raise NotImplementedError
class MixManager(models.Manager):
def get_query_set(self):
return MixedQuerySet(self.model)
TYPES = (
('aol','AOL'),
('yhoo','Yahoo'),
)
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
atype = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
objects = MixManager()
class ConnectAOL(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to AOL
def deconstruct_constring(self):
return pickle.loads(self.connection_string)
class ConnectYahoo(SuperConnect):
class Meta:
proxy = True
def connect(self):
conn_options = self.deconstruct_constring()
# do special stuff to connect to Yahoo
def deconstruct_constring(self):
return pickle.loads(self.connection_string)

As you mentioned in your question, the problem with your solution is that it generates a SQL query for every object instead of using one SQL in = (id1, id2) query. Proxy models cannot contain additional database fields, so there is no need for extra SQL queries.
Instead, you can convert a SuperConnect object to the appropriate type in SuperConnect.__init__, using the __class__ attribute:
class SuperConnect(models.Model):
name = models.CharField(max_length=90)
type = models.CharField(max_length=45, choices = TYPES)
connection_string = models.TextField(null=True)
def __init__(self, *args, **kwargs):
super(SuperConnect, self).__init__(*args, **kwargs)
if self.type == 'aol':
self.__class__ = ConnectAOL
elif self.type == 'yahoo':
self.__class__ = ConnectYahoo
There is no need for custom managers or querysets, the correct type is set when the SuperConnect object is initialized.

How about putting all the logic in one class. Something like this:
def connect(self):
return getattr(self, "connect_%s" % self.type)()
def connect_aol(self):
pass # AOL stuff
def connect_yahoo(self):
pass # Yahoo! stuff
In the end you have your type field and you should be able to do most (if not all) things that you can do with seperate proxy classes.
If this approach doesn't solve your specific use cases, please clarify.

Django serialization of inherited model

I have a problem with serialization of Django inherited models. For example
class Animal(models.Model):
color = models.CharField(max_length=50)
class Dog(Animal):
name = models.CharField(max_length=50)
...
# now I want to serialize Dog model with Animal inherited fields obviously included
print serializers.serialize('xml', Dog.objects.all())
and only Dog model has been serialized.
I can do smth like
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
But it looks ugly and because my models are very big so I have to use SAX parser and with such output it's difficult to parse.
Any idea how to serialize django models with parent class?
**EDIT: ** It use to work ok before this patch has been applied. And the explanation why the patch exist "Model saving was too aggressive about creating new parent class instances during deserialization. Raw save on a model now skips saving of the parent class. " I think there should be an option to be able to serialize "local fields only" by default and second option - "all" - to serialize all inherited fields.

You found your answer in the documentation of the patch.
all_objects = list(Animal.objects.all()) + list(Dog.objects.all())
print serializers.serialize('xml', all_objects)
However, if you change Animal to be an abstract base class it will work:
class Animal(models.Model):
color = models.CharField(max_length=50)
class Meta:
abstract = True
class Dog(Animal):
name = models.CharField(max_length=50)
This works as of Django 1.0. See http://docs.djangoproject.com/en/dev/topics/db/models/.

You'll need a custom serializer to support inherited fields, as Django's serializer will only serialize local fields.
I ended up writing my own when dealing with this issue, feel free to copy it: https://github.com/zmathew/django-backbone/blob/master/backbone/serializers.py
In order to use it on its own, you need to do:
serializer = AllFieldsSerializer()
serializer.serialize(queryset, fields=fields)
print serializer.getvalue()

I had the same problem, and i wrote a 'small' queryset serializer which navigates up the inheritance tree and returns all the fields serialized.
It's far from perfect... but works for me :)
a = QuerySetSerializer(MyModel, myqueryset)
a.serialize()
And the snippet:
from __future__ import unicode_literals
import json
import inspect
from django.core import serializers
from django.db.models.base import Model as DjangoBaseModel
class QuerySetSerializer(object):
def __init__(self, model, initial_queryset):
"""
#param model: The model of your queryset
#param initial_queryset: The queryset to serialize
"""
self.model = model
self.initial_queryset = initial_queryset
self.inheritance_tree = self._discover_inheritance_tree()
def serialize(self):
list_of_querysets = self._join_inheritance_tree_objects()
merged_querysets = self._zip_queryset_list(list_of_querysets)
result = []
for related_objects in merged_querysets:
result.append(self._serialize_related_objects(related_objects))
return json.dumps(result)
def _serialize_related_objects(self, related_objects):
"""
In this method, we serialize each instance using the django's serializer function as shown in :
See https://docs.djangoproject.com/en/1.10/topics/serialization/#inherited-models
However, it returns a list with mixed objects... Here we join those related objects into one single dict
"""
serialized_objects = []
for related_object in related_objects:
serialized_object = self._serialize_object(related_object)
fields = serialized_object['fields']
fields['pk'] = serialized_object['pk']
serialized_objects.append(fields)
merged_related_objects = {k: v for d in serialized_objects for k, v in d.items()}
return merged_related_objects
def _serialize_object(self, obj):
data = serializers.serialize('json', [obj, ])
struct = json.loads(data)
return struct[0]
def _discover_inheritance_tree(self):
# We need to find the inheritance tree which excludes abstract classes,
# so we can then join them when serializing the instance
return [x for x in inspect.getmro(self.model) if x is not object and x is not DjangoBaseModel and not x._meta.abstract]
def _join_inheritance_tree_objects(self):
"""
Here we join the required querysets from the non abstract inherited models, which we need so we are able to
serialize them.
Lets say that MyUser inherits from Customer and customer inherits from django's User model
This will return [list(MyUser.objects.filter(...), list(Customer.objects.filter(...), list(User.objects.filter(...)
"""
initial_ids = self._get_initial_ids()
inheritance__querysets = [list(x.objects.filter(id__in=initial_ids).order_by("id")) for x in self.inheritance_tree]
return inheritance__querysets
def _zip_queryset_list(self, list_of_querysets):
"""
At this stage, we have something like:
(
[MyUser1, MyUser2, MyUser3],
[Customer1, Customer2, Customer3],
[User1, User2, User3]
)
And to make it easier to work with, we 'zip' the list of lists so it looks like:
(
[MyUser1, Customer1, User1],
[MyUser2, Customer2, User2],
[MyUser3, Customer3, User3],
)
"""
return zip(*list_of_querysets)
def _get_initial_ids(self):
"""
Returns a list of ids of the initial queryset
"""
return self.initial_queryset.order_by("id").values_list("id", flat=True)

You can define a custom Serializer:
class DogSerializer(serializers.ModelSerializer):
class Meta:
model = Dog
fields = ('color','name')
Use it like:
serializer = DogSerializer(Dog.objects.all(), many=True)
print serializer.data enter code here

Did you look at select_related() ?
as in
serializers.serialize('xml', Dog.objects.select_related().all())