I'm having a problem with the datastore trying to replicate a left join to find items from model a that don't have a matching relation in model b:
class Page(db.Model):
url = db.StringProperty(required=True)
class Item(db.Model):
page = db.ReferenceProperty(Page, required=True)
name = db.StringProperty(required=True)
I want to find any pages that don't have any associated items.
You cannot query for items using a "property is null" filter. However, you can add a boolean property to Page that signals if it has items or not:
class Page(db.Model):
url = db.StringProperty(required=True)
has_items = db.BooleanProperty(default=False)
Then override the "put" method of Item to flip the flag. But you might want to encapsulate this logic in the Page model (maybe Page.add_item(self, *args, **kwargs)):
class Item(db.Model):
page = db.ReferenceProperty(Page, required=True)
name = db.StringProperty(required=True)
def put(self):
if not self.page.has_items:
self.page.has_items = True
self.page.put()
return db.put(self)
Hence, the query for pages with no items would be:
pages_with_no_items = Page.all().filter("has_items =", False)
The datastore doesn't support joins, so you can't do this with a single query. You need to do a query for items in A, then for each, do another query to determine if it has any matching items in B.
Did you try it like :
Page.all().filter("item_set = ", None)
Should work.
Related
I'm using Wagtail, and I want to filter a selection of child pages by a Foreign Key. I've tried the following and I get the error django.core.exceptions.FieldError: Cannot resolve keyword 'use_case' into field when I try children = self.get_children().specific().filter(use_case__slug=slug):
class AiLabResourceMixin(models.Model):
parent_page_types = ['AiLabResourceIndexPage']
use_case = models.ForeignKey(AiLabUseCase, on_delete=models.PROTECT)
content_panels = ArticlePage.content_panels + [
FieldPanel('use_case', widget=forms.Select())
]
class Meta:
abstract = True
class AiLabCaseStudy(AiLabResourceMixin, ArticlePage):
pass
class AiLabBlogPost(AiLabResourceMixin, ArticlePage):
pass
class AiLabExternalLink(AiLabResourceMixin, ArticlePage):
pass
class AiLabResourceIndexPage(RoutablePageMixin, BasePage):
parent_page_types = ['AiLabHomePage']
subpage_types = ['AiLabCaseStudy', 'AiLabBlogPost', 'AiLabExternalLink']
max_count = 1
#route(r'^$')
def all_resources(self, request):
children = self.get_children().specific()
return render(request, 'ai_lab/ai_lab_resource_index_page.html', {
'page': self,
'children': children,
})
#route(r'^([a-z0-9]+(?:-[a-z0-9]+)*)/$')
def filter_by_use_case(self, request, slug):
children = self.get_children().specific().filter(use_case__slug=slug)
return render(request, 'ai_lab/ai_lab_resource_index_page.html', {
'page': self,
'children': children,
})
I've seen this answer, but this assumes I only have one type of page I want to filter. Using something like AiLabCaseStudy.objects.filter(use_case__slug=slug) works, but this only returns AiLabCaseStudys, not AiLabBlogPosts or AiLabExternalLinks.
Any ideas?
At the database level, there is no efficient way to run the filter against all page types at once. Since AiLabResourceMixin is defined as abstract = True, this class has no representation of its own within the database - instead, the use_case field is defined separately for each of AiLabCaseStudy, AiLabBlogPost and AiLabExternalLink. As a result, there's no way for Django or Wagtail to turn .filter(use_case__slug=slug) into a SQL query, since use_case refers to three different places in the database.
A couple of possible ways around this:
If your data model allows, restructure it to use multi-table inheritance - this looks fairly similar to your current definition, except without the abstract = True:
class AiLabResourcePage(ArticlePage):
use_case = models.ForeignKey(AiLabUseCase, on_delete=models.PROTECT)
class AiLabCaseStudy(AiLabResourcePage):
pass
class AiLabBlogPost(AiLabResourcePage):
pass
class AiLabExternalLink(AiLabResourcePage):
pass
AiLabResourcePage will then exist in its own right in the database, and you can query its use_case field with an expression like: AiLabResourcePage.objects.child_of(self).filter(use_case__slug=slug).specific(). There'll be a small performance impact here, since Django has to pull data from one additional table to construct these page objects.
Run a preliminary query on each specific page type to retrieve the matching page IDs, before running the final query with specific():
case_study_ids = list(AiLabCaseStudy.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
blog_post_ids = list(AiLabBlogPost.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
external_link_ids = list(AiLabExternalLink.objects.child_of(self).filter(use_case__slug=slug).values_list('id', flat=True))
children = Page.objects.filter(id__in=(case_study_ids + blog_post_ids + external_link_ids)).specific()
Try:
children = self.get_children().filter(use_case__slug=slug).specific()
I'm fairly new to peewee, but have some strong background on SQLAlchemy (and all the vices that come with it). I'm trying to create a custom hybrid expression that correlates to a third (or even N) table. I'll try to demonstrate in an example (non-tested) code:
class BaseModel(Model):
class Meta:
database = database
class Person(BaseModel):
id = PrimaryKeyField(column_name="person_id")
name = CharField(max_length=255, column_name="person_name")
username = CharField(max_length=255, column_name="person_username")
class PersonTree(BaseModel):
id = PrimaryKeyField(column_name="person_tree_id")
name = CharField(max_length=255, column_name="person_tree_name")
code = CharField(max_length=255, column_name="person_tree_code")
person = ForeignKeyField(
column_name="person_id",
model=Person,
field="id",
backref="tree",
)
class Article(BaseModel):
id = PrimaryKeyField(column_name="article_id")
name = CharField(max_length=255, column_name="article_name")
branch = ForeignKeyField(
column_name="person_tree_id",
model=PersonTree,
field="id",
backref="articles",
)
#hybrid_property
def username(self):
"""
This gives me the possibility to grab the direct username of an article
"""
return self.branch.person.username
#username.expression
def username(cls):
"""
What if I wanted to do: Article.query().where(Article.username == "john_doe") ?
"""
pass
With the username hybrid_property on Article, I can get the username of the Person related to an Article using the PersonTree as a correlation, so far so good, but ... What if I wanted to "create a shortcut" to query all Articles created by the "john_doe" Person username, without declaring the JOINs every time I make the query and without relying on .filter(branch__person__username="john_doe")? I know it's possible with SA (to a great extent), but I'm finding this hard to accomplish with peewee.
Just for clarification, here's the SQL I hope to be able to construct:
SELECT
*
FROM
article a
JOIN person_tree pt ON a.person_tree_id = pt.person_tree_id
JOIN person p ON pt.person_id = p.person_id
WHERE
p.username = 'john_doe';
Thanks a lot in advance!
Hybrid properties can be used to allow an attribute to be expressed as a property of a model instance or as a scalar computation in a SQL query.
What you're trying to do, which is add multiple joins and stuff via the property, is not possible using hybrid properties.
What if I wanted to "create a shortcut" to query all Articles created by the "john_doe" Person username
Just add a normal method:
#classmethod
def by_username(cls, username):
return (Article
.select(Article, PersonTree, Person)
.join(PersonTree)
.join(Person)
.where(Person.name == username))
I have the models defined below:
class PrimaryAsset(models.Model):
title = models.Charfield(max_length=200)
class Service(PrimaryAsset):
description = models.Charfield(max_length=200)
class Website(PrimaryAsset):
url = models.Charfield(max_length=200)
class AssetLinks(models.model):
high = models.ForeignKey(PrimaryAsset)
low = models.ForeignKey(PrimaryAsset)
AssetLinks.objects.filter(high=212)[0].low
When I do the filter above, how can I know which instance the objects is (website or service)? Also, is there a way to avoid an N+1 query using prefetch_related in a way that it gets all the child information as well?
You can even use select_related instead of prefetch_related. Something like this should do the trick:
asset = AssetLinks.objects.filter(high=212).select_related(
'high__service', 'high__website',
'low__service', 'low__website',
)[0]
#check for service/website
service = getattr(asset.high, 'service', None)
website = getattr(asset.high, 'website', None)
I can't seem to isolate a single record from this query:
subcust = OwnerCustom.objects.get(carcustom=ncset, owner=sset)
This is the error:
OwnerCustom matching query does not exist
In the actual data, there is only actually one matching record in OwnerCustom for each record in CarCustom. It's supposed to be a kind of many-to-many where there are standard differences listed in CarCustom for each Car, and each owner may maintain their own customizations (overrides) or those default OwnerCustom entries.
Note, there are many different Owner of the same Car. And of course, I'm not actually doing cars, this is a renaming from the original purpose.
Here's the relevant models:
class Car(models.Model):
car_name = models.CharField(max_length=50)
class CarCustom(models.Model):
car = models.ForeignKey(Car, models.PROTECT)
class Owner(models.Model):
car = models.ForeignKey(Car, models.PROTECT)
class OwnerCustom(models.Model):
owner = models.ForeignKey(Owner, models.PROTECT)
carcustom = models.ForeignKey(CarCustom, models.PROTECT)
name = models.CharField(max_length=50)
And the code:
car_queryset = Car.objects.filter(car_name="fancy car")
for nset in car_queryset:
owner_queryset = Owner.objects.filter(car=nset)
for sset in owner_queryset :
carcustom_queryset = CarCustom.objects.filter(car=nset)
for ncset in carcustom_queryset:
subcust = OwnerCustom.objects.get(carcustom=ncset, owner=sset)
I've tried stuff like:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset).first()
Which gives me a NoneType, and then tried:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)[:1].get()
Which gives "matching query does not exist" and this:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)[0]
Gives "list index out of range"
UPDATE: I CAN get a working function by using code like this, but I would think since there is only one (guaranteed by application) matching record possible for OwnerCustom.objects.filter(carcustom=ncset, owner=sset) that I could find a better way to fetch it:
car_queryset = Car.objects.filter(car_name="fancy car")
for nset in car_queryset:
owner_queryset = Owner.objects.filter(car=nset)
for sset in owner_queryset :
carcustom_queryset = CarCustom.objects.filter(car=nset)
for ncset in carcustom_queryset:
subcust_queryset = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)
for subcust in subcust_queryset :
logger.info(subcust.name)
I am creating a sample application storing user detail along with its class information.
Modal classes being used are :
Model class for saving user's class data
class MyData(ndb.Model):
subject = ndb.StringProperty()
teacher = ndb.StringProperty()
strength = ndb.IntegerProperty()
date = ndb.DateTimeProperty()
Model class for user
class MyUser(ndb.Model):
user_name = ndb.StringProperty()
email_id = ndb.StringProperty()
my_data = ndb.StructuredProperty(MyData, repeated = True)
I am able to successfully store data into the datastore and can also make simple query on the MyUser entity using some filters based on email_id and user_name.
But when I try to query MyUser result using filter on a property from the MyUser modal's Structured property that is my_data, its not giving correct result.
I think I am querying incorrectly.
Here is my query function
function to query based upon the repeated structure property
def queryMyUserWithStructuredPropertyFilter():
shail_users_query = MyUser.query(ndb.AND(MyUser.email_id == "napolean#gmail.com", MyUser.my_data.strength > 30))
shail_users_list = shail_users_query.fetch(10)
maindatalist=[]
for each_user in shail_users_list:
logging.info('NEW QUERY :: The user details are : %s %s'% (each_user.user_name, each_user.email_id))
# Class data
myData = each_user.my_data
for each_my_data in myData:
templist = [each_my_data.strength, str(each_my_data.date)]
maindatalist.append(templist)
logging.info('NEW QUERY :: The class data is : %s %s %s %s'% (each_my_data.subject, each_my_data.teacher, str(each_my_data.strength),str(each_my_data.date)))
return maindatalist
I want to fetch that entity with repeated Structured property (my_data) should be a list which has strength > 30.
Please help me in knowing where I am doing wrong.
Thanks.
Queries over StructuredProperties return objects for which at least one of the structured ones satisfies the conditions. If you want to filter those properties, you'll have to do it afterwards.
Something like this should do the trick:
def queryMyUserWithStructuredPropertyFilter():
shail_users_query = MyUser.query(MyUser.email_id == "napolean#gmail.com", MyUser.my_data.strength > 30)
shail_users_list = shail_users_query.fetch(10)
# Here, shail_users_list has at most 10 users with email being
# 'napolean#gmail.com' and at least one element in my_data
# with strength > 30
maindatalist = [
[[data.strength, str(data.date)] for data in user.my_data if data.strength > 30] for user in shail_users_list
]
# Now in maindatalist you have ONLY those my_data with strength > 30
return maindatalist