I'm using google app engine, and am having trouble writing querys to filter ReferenceProperties.
eg.
class Group(db.Model):
name = db.StringProperty(required=True)
creator = db.ReferenceProperty(User)
class GroupMember(db.Model):
group = db.ReferenceProperty(Group)
user = db.ReferenceProperty(User)
And I have tried writing something like this:
members = models.GroupMember.all().filter('group.name =', group_name)
and various other things that don't work. Hopefully someone can give me a prod in the right direction...
If your groups are uniquely named, then your "group.name" is a unique identifier of a Group entity.
That means you can write:
members = models.GroupMember.all().filter(
"group =",model.Group.gql("WHERE name=:1", group_name).get()
)
though you only need to do that if you don't already have the group entity lying around in the stack somewhere.
Google's essay on many-to-many with appengine is here.
If what you want is to get the members of a group, ReferenceProperties have that built-in.
class GroupMember(db.Model):
group = db.ReferenceProperty(Group, collection_name="groupMembers")
user = db.ReferenceProperty(User, collection_name="groupMembers")
Then you can write:
# get the group entity somehow
group = Group.get(group_key)
# do something with the members, such as list the nicknames
nicknames = [x.user.nickname for x in group.groupMembers]
This would require a join, which isn't possible in App Engine. If you want to filter by a property of another model, you need to include that property on the model you're querying against.
This would result in two datastore hits but should work. If you use memcache shouldnt be a problem.
group = models.Group.all().filter("name =", group_name).get()
members = models.GroupMember.all().filter('group =', group)
Using the Models that you defined in your question, lets say you want to list all members of a group called "Space monkeys".
mygroup = Group.gql("WHERE name = :1",'Space monkeys')
for group_member in mygroup.groupmember_set:
print 'group members name is: %s' % (group_member.user.name)
The "groupmember_set" is called a "implicit collection property" and is very useful. You can call it whatever you want by over-riding the default name using the collection_name keyword parameter to ReferenceProperty. For an example see the answer by Thomas L Holaday.
This is all explained in a very good paper by Rafe Kapla: Modeling Entity Relationships.
Related
This seems like a simple question, however wanted something more clear than what I'm doing currently:
Given tables like these (example only):
class People(db.Model):
FirstName = db.StringProperty(multiline=False,required=True)
LastName = db.StringProperty(multiline=False,required=True)
class Animals(db.Model):
AnimalName = db.StringProperty(multiline=False,required=True)
class SpiritAnimal(db.Model):
Person = db.ReferenceProperty(Candidates,required=True)
Animal = db.ReferenceProperty(Candidates,required=True)
There exists a way to fill in 'Person' and 'Animal' using queries to the other two tables like so (example only):
# Query for some person(s)
query = People.all()
query.filter('FirstName', 'Patrick')
query.get()
for person in query:
newSpiritAnimal = SpiritAnimal(
Person = person,
Animal = animal # Assuming pulled previously
)
newSpiritAnimal.put()
Also you can just grab keys, however here is where my question comes into play:
Based off a query such as above, can you just pull the key and use later? Of course you can, but what's the best method to do so?
Let's think about this example:
for person in query:
key_for_later_use = person.key()
Now we can use:
Person = key_for_later_use
One would assume correct? Except this person.key() object doesn't seem to be doing the trick so I looked into it more:
str(person.key())
This provides a key that looks like what you would see in the GAE SDK Console when viewing the 'Datastore Viewer' thus potentially useful, but not having luck with that either.
What's the best way to grab a key off a query, potentially when iterating via for loop?
I've been trying to offload datastore queries by creating a list which I check for something existing, then grab from another list the key:
people_list = [] # Assume populated with 'FirstName'
people_list_keys = [] # Assume populated with person.key()
if 'Patrick' in people_list:
patrick_key = people_list_keys[people.index('Patrick')]
However person.key() doesn't really work, str() around that looks right but doesn't work right.. and by that I mean using that as SpiritAnimal.Person on insert for the ReferenceProperty.
Thoughts?
Oh and I'm seriously not making a SpiritAnimal application, this is all just examples ;)
There might be a disconnect elsewhere, I ran this code:
People(FirstName="Patrick", LastName="Doe").put()
animal = Animals(AnimalName="Tiger").put()
people_list = []
people_list_keys = []
query = People.all()
query.filter('FirstName', 'Patrick')
query.get()
for person in query:
people_list.append(person.FirstName)
people_list_keys.append(person.key())
patrick_key = people_list_keys[people_list.index('Patrick')]
newSpiritAnimal = SpiritAnimal(
Person = patrick_key,
Animal = animal
)
newSpiritAnimal.put()
And the Spirit Animal was 'put' no problem. I don't quite get what your trying to do. Perhaps a little more explanation and I can help a bit more.
I believe this is trival but fairly new to Python.
I am trying to create a model using google app engine.
Basically from a E/R point of view
I have 2 objects with a join table (the join table captures the point in time of the join)
Something like this
Person | Idea | Person_Idea
-------------------------------
person.key idea.key person.key
idea.key
date_of_idea
my Python code would look like
class Person (db.Model):
#some properties here....
class Idea(db.Model):
#some properties here....
class IdeaCreated(db.Model):
person= db.ReferenceProperty(Person)
idea= db.ReferenceProperty(Idea)
created = db.DateTimeProperty(auto_now_add = True)
What I want to be able to do is have a convient way to get all ideas a person has (bypass idea created objects) -sometimes I will need the list of ideas directly.
The only way I can think to do this is to add the follow method on the User class
def allIdeas(self):
ideas = []
for ideacreated in self.ideacreated_set:
ideas.append(ideacreated.idea)
return ideas
Is this the only way to do this? I is there a nicer way that I am missing?
Also assuming I could have a GQL and bypass hydrating the ideaCreated instances (not sure the exact syntax) but putting a GQL query smells wrong to me.
you should use the person as an ancestor/parent for the idea.
idea = Idea(parent=some_person, other_field=field_value).put()
then you can query all ideas where some_person is the ancestor
persons_ideas = Idea.all().ancestor(some_person_key).fetch(1000)
the ancestor key will be included in the Idea entities key and you won't be able to change that the ancestor once the entity is created.
i highly suggest you to use ndb instead of db https://developers.google.com/appengine/docs/python/ndb/
with ndb you could even use StructuredProperty or LocalStructuredProperty
https://developers.google.com/appengine/docs/python/ndb/properties#structured
EDIT:
if you need a many to many relationship look in to ListProperties and store the Persons keys in that property. then you can query for all Ideas with that Key in that property.
class Idea(db.Model):
person = db.StringListProperty()
idea = Idea(person = [str(person.key())], ....).put()
add another person to the idea
idea.person.append(str(another_person.key())).put()
ideas = Idea.filter(person=str(person.key())).fetch(1000)
look into https://developers.google.com/appengine/docs/python/datastore/typesandpropertyclasses#ListProperty
I'm designing datastore models and trying to decide the best approach when thinking about how the query filters are going to work.
Best if I just write out two examples. The first example is if I have a fixed "gender" property with just a string which can be set to either "male" or "female".
class Person(db.Model):
name = db.StringProperty()
gender = db.StringProperty()
p1 = Person(name="Steve", gender="male")
p2 = Person(name="Jane", gender="female")
p1.put()
p2.put()
males = db.GqlQuery("SELECT * FROM Person WHERE gender = :1", "male")
The second example is if the Person entity is an expando model, and I dynamically set a "is_male" or "is_female" dynamic property.
class Person(db.Expando):
name = db.StringProperty()
p1 = Person(name="Steve")
p1.is_male = True
p1.put()
p2 = Person(name="Jane")
p2.is_female = True
p2.put()
males = db.GqlQuery("SELECT * FROM Person WHERE is_male = :1", True)
Now lets say that we gather millions of records and we want to do a query, which one of the two methods above would be faster in production Google App Engine running Python 2.7?
There is absolutely no difference - models look the same in the datastore regardless of whether the property was 'dynamic' or not. The only difference is that a standard property class with no value set will insert a field with value None in the datastore, which takes some extra space, but allows you to query for users with that value not set.
even if its not exactly what you are asking for i think it might give you an idea.
Is there some performance issue between leaving empty ListProperties or using dynamic (expando) properties?
After building a few application on the gae platform I usually use some relationship between different models in the datastore in basically every application. And often I find my self the need to see what record is of the same parent (like matching all entry with same parent)
From the beginning I used the db.ReferenceProperty to get my relations going, like:
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
parentFoo = db.ReferanceProperty(Foo)
fooKey = someFooKeyFromSomePlace
bars = Bar.all()
for bar in bar:
if bar.parentFoo.key() == fooKey:
// do stuff
But lately I've abandoned this approch since the bar.parentFoo.key() makes a sub query to fetch Foo each time. The approach I now use is to store each Foo key as a string on Bar.parentFoo and this way I can string compare this with someFooKeyFromSomePlace and get rid of all the subquery overhead.
Now I've started to look at Entity groups and wondering if this is even a better way to go? I can't really figure out how to use them.
And as for the two approaches above I'm wondering is there any downsides to using them? Could using stored key string comeback and bit me in the * * *. And last but not least is there a faster way to do this?
Tip:
replace...
bar.parentFoo.key() == fooKey
with...
Bar.parentFoo.get_value_for_datastore(bar) == fooKey
To avoid the extra lookup and just fetch the key from the ReferenceProperty
See Property Class
I think you should consider this as well. This will help you fetch all the child entities of a single parent.
bmw = Car(brand="BMW")
bmw.put()
lf = Wheel(parent=bmw,position="left_front")
lf.put()
lb = Wheel(parent=bmw,position="left_back")
lb.put()
bmwWheels = Wheel.all().ancestor(bmw)
For more reference in modeling. you can refer this Appengine Data modeling
I'm not sure what you're trying to do with that example block of code, but I get the feeling it could be accomplished with:
bars = Bar.all().filter("parentFoo " = SomeFoo)
As for entity groups, they are mainly used if you want to alter multiple things in transactions, since appengine restricts that to entities within the same group only; in addition, appengine allows ancestor filters ( http://code.google.com/appengine/docs/python/datastore/queryclass.html#Query_ancestor ), which could be useful depending on what it is you need to do. With the code above, you could very easily also use an ancestor query if you set the parent of Bar to be a Foo.
If your purposes still require a lot of "subquerying" as you put it, there is a neat prefetch pattern that Nick Johnson outlines here: http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine which basically fetches all the properties you need in your entity set as one giant get instead of a bunch of tiny ones, which gets rid of a lot of the overhead. However do note his warnings, especially regarding altering the properties of entities while using this prefetch method.
Not very specific, but that's all the info I can give you until you be more specific about exactly what you're trying to do here.
When you design your modules you also need to consider whether you want to be able to save this within a transaction. However only do this if you need to use transactions.
An alternative approach is to assign the parent like so:
from google.appengine.ext import db
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
def _save_entities( foo_name, bar_name ):
"""Save the model data"""
foo_item = Foo( name = foo_name )
foo_item.put()
bar_item = Bar( parent = foo_item, name = bar_name )
bar_item.put()
def main():
# Run the save in a transaction, if any fail this should all roll back
db.run_in_transaction( _save_transaction, "foo name", "bar name" )
# to query the model data using the ancestor relationship
for item in bar_item.gql("WHERE ANCESTOR IS :ancestor", ancestor = foo_item.key()).fetch(1000):
# do stuff
I'm trying to access an object that is linked to by a db.ReferenceProperty in Google app engine. Here's the model's code:
class InquiryQuestion(db.Model):
inquiry_ref = db.ReferenceProperty(reference_class=GiftInquiry, required=True, collection_name="inquiry_ref")
And I am trying to access it in the following way:
linkedObject = question.inquiry_ref
and then
linkedKey = linkedObject.key
but it's not working. Can anyone please help?
Your naming conventions are a bit confusing. inquiry_ref is both your ReferenceProperty name and your back-reference collection name, so question.inquiry_ref gives you a GiftInquiry Key object, but question.inquiry_ref.inquiry_ref gives you a Query object filtered to InquiryQuestion entities.
Let's say we have the following domain model, with a one-to-many relationship between articles and comments.
class Article(db.Model):
body = db.TextProperty()
class Comment(db.Model):
article = db.ReferenceProperty(Article)
body = db.TextProperty()
comment = Comment.all().get()
# The explicit reference from one comment to one article
# is represented by a Key object
article_key = comment.article
# which gets lazy-loaded to a Model instance by accessing a property
article_body = comment.article.body
# The implicit back-reference from one article to many comments
# is represented by a Query object
article_comments = comment.article.comment_set
# If the article only has one comment, this gives us a round trip
comment = comment.article.comment_set.all().get()
The back reference is just a query. You need to use fetch() or get() to actually retrieve the entity or entities from the datastore:
linkedObject = question.inquiry_ref.get()
should do the trick. Or, you would use fetch() if you were expecting the back ref to refer to more than one entity.
Actually, the way that your class is constructed makes it ambiguous as to what exactly is happening here.
If you have a GiftInquiry entity, it will get an automatic property called inquiry_ref that will be a query (as I described above) that will return all InquiryQuestion entities that have their inquiry_ref property set to that GiftInquiry's Key.
On the other hand, if you have an InquiryQuestion entity, and you want to get the GiftInquiry entity to which its inquiry_ref property is set, you would do this:
linkedObject = db.get(question.inquiry_ref)
as the inquiry_ref is just the Key of the referred-to GiftInquiry, but that is not technically a BackReference.
Check out the explanation of ReferenceProperty and back references from the docs.