This seems like a simple question, however wanted something more clear than what I'm doing currently:
Given tables like these (example only):
class People(db.Model):
FirstName = db.StringProperty(multiline=False,required=True)
LastName = db.StringProperty(multiline=False,required=True)
class Animals(db.Model):
AnimalName = db.StringProperty(multiline=False,required=True)
class SpiritAnimal(db.Model):
Person = db.ReferenceProperty(Candidates,required=True)
Animal = db.ReferenceProperty(Candidates,required=True)
There exists a way to fill in 'Person' and 'Animal' using queries to the other two tables like so (example only):
# Query for some person(s)
query = People.all()
query.filter('FirstName', 'Patrick')
query.get()
for person in query:
newSpiritAnimal = SpiritAnimal(
Person = person,
Animal = animal # Assuming pulled previously
)
newSpiritAnimal.put()
Also you can just grab keys, however here is where my question comes into play:
Based off a query such as above, can you just pull the key and use later? Of course you can, but what's the best method to do so?
Let's think about this example:
for person in query:
key_for_later_use = person.key()
Now we can use:
Person = key_for_later_use
One would assume correct? Except this person.key() object doesn't seem to be doing the trick so I looked into it more:
str(person.key())
This provides a key that looks like what you would see in the GAE SDK Console when viewing the 'Datastore Viewer' thus potentially useful, but not having luck with that either.
What's the best way to grab a key off a query, potentially when iterating via for loop?
I've been trying to offload datastore queries by creating a list which I check for something existing, then grab from another list the key:
people_list = [] # Assume populated with 'FirstName'
people_list_keys = [] # Assume populated with person.key()
if 'Patrick' in people_list:
patrick_key = people_list_keys[people.index('Patrick')]
However person.key() doesn't really work, str() around that looks right but doesn't work right.. and by that I mean using that as SpiritAnimal.Person on insert for the ReferenceProperty.
Thoughts?
Oh and I'm seriously not making a SpiritAnimal application, this is all just examples ;)
There might be a disconnect elsewhere, I ran this code:
People(FirstName="Patrick", LastName="Doe").put()
animal = Animals(AnimalName="Tiger").put()
people_list = []
people_list_keys = []
query = People.all()
query.filter('FirstName', 'Patrick')
query.get()
for person in query:
people_list.append(person.FirstName)
people_list_keys.append(person.key())
patrick_key = people_list_keys[people_list.index('Patrick')]
newSpiritAnimal = SpiritAnimal(
Person = patrick_key,
Animal = animal
)
newSpiritAnimal.put()
And the Spirit Animal was 'put' no problem. I don't quite get what your trying to do. Perhaps a little more explanation and I can help a bit more.
Related
I know there is already a similar question, but I think my case is a bit more complicated because I have a different entry point.
These are my models:
class m_Interaction(models.Model):
fk_ip = models.ForeignKey('m_IP', related_name="interactions")
class m_User(models.Model):
name = models.CharField(max_length=200)
class m_IP(models.Model):
fk_user = models.ForeignKey('m_User', related_name="ips" )
class m_Feature(models.Model):
name = models.CharField(max_length=200)
m2m_interaction = models.ManyToManyField(m_Interaction, related_name='features', through='m_Featurescore')
class m_Featurescore(models.Model):
score = models.FloatField(null=False)
fk_interaction = models.ForeignKey(m_Interaction, related_name='featurescore')
fk_feature = models.ForeignKey(m_Feature, related_name='featurescore')
I start with m_User, follow the reverse relationship over m_IP to the Interactions (m_Interaction). Then I want to get every m_Featurescore.score for each Interaction for a specific instance of m_Feature.
My working query to access at least all interactions in a performant way:
m_User.objects.all().prefetch_related('ips__interactions')
But I can't figure out the correct 'prefetch_related'-statement to access the m_Featurescore.score like this
db_obj_interaction.featurescore.get(fk_feature=db_obj_feature).score
without making a lot of queries.
I already tried almost all combinations of the following:
'ips__interactions__features__featurescore'
Any suggestions?
I found the answer to my own question with the help of noamk in the comments:
I didn't consider that the get()-method in db_obj_interaction.featurescore.get(fk_feature=db_obj_feature).score will issue a new query everytime it's called (it's kinda obvious now).
Therefore I simply restructured my code and now I don't need get() anymore and can use the benefit of the prefetch.
If somebody still needs to filter the Prefetch()-object should be used as suggested by noamk
I'm stuck using Elixir and I currently have a really messy way of searching through a database that i'd like to improve. The documentation provides insight on how to do a basic generative search but I need to step through many different classes and i'd prefer to use Elixir rather than scanning through the list myself.
Here's an example:
Class Student:
hobby = Field(String)
additional_info = OneToOne('AdditionalInformation', inverse='student')
user_profile = OneToOne('UserProfile', inverse='student')
Class AdditionalInformation:
state = Field(String)
city = Field(String)
student = OneToOne('Student', inverse='additional_info')
Class UserProfile:
username = Field(String)
date_signed_up = Field(DateTime)
student = OneToOne('Student', inverse = 'user_profile')
In this example, i'd like to find all students that:
Signed up after 2008
Are from California
Have "video games" as their hobby
I'm thinking there should be a way for me to go:
result = UserProfile.query.filter_by(date_signed_up>2008)
result.query.filter_by(UserProfile.student.hobby='blabla')
result.query....
Currently i'm putting them into a list and looking for a set.
I haven't used Elixir, but I have used SQLAlchemy. I don't think you can do what you want given that current setup. As far as I know, there is no way to filter by relationships directly.
It's unclear whether you're creating new tables or dealing with existing ones, so I'm just going to throw some info at you and hope some of it is helpful.
You can join tables together in SQLAlchemy (assuming there's a foreign key called student_id on UserProfile). This would give you all students who signed up since 2008.
result = Student.query.join(UserProfile).filter(Student.id==UserProfile.student_id).filter(UserProfile.date_signed_up>2008).all()
You can chain .filter() together like I did above, or you can pass multiple args to them. I find this especially useful for dealing with unknown numbers of filters, like you'd get from a search form.
conditions = [UserProfile.date_signed_up>2008]
if something_is_true:
conditions.append(UserProfile.username=="foo")
result = Student.query.join(UserProfile).filter(Student.id==UserProfile.student_id).filter(and_(*conditions)).all()
There's also more complex stuff you can do with hybrid properties, but that doesn't seem appropriate here.
I believe this is trival but fairly new to Python.
I am trying to create a model using google app engine.
Basically from a E/R point of view
I have 2 objects with a join table (the join table captures the point in time of the join)
Something like this
Person | Idea | Person_Idea
-------------------------------
person.key idea.key person.key
idea.key
date_of_idea
my Python code would look like
class Person (db.Model):
#some properties here....
class Idea(db.Model):
#some properties here....
class IdeaCreated(db.Model):
person= db.ReferenceProperty(Person)
idea= db.ReferenceProperty(Idea)
created = db.DateTimeProperty(auto_now_add = True)
What I want to be able to do is have a convient way to get all ideas a person has (bypass idea created objects) -sometimes I will need the list of ideas directly.
The only way I can think to do this is to add the follow method on the User class
def allIdeas(self):
ideas = []
for ideacreated in self.ideacreated_set:
ideas.append(ideacreated.idea)
return ideas
Is this the only way to do this? I is there a nicer way that I am missing?
Also assuming I could have a GQL and bypass hydrating the ideaCreated instances (not sure the exact syntax) but putting a GQL query smells wrong to me.
you should use the person as an ancestor/parent for the idea.
idea = Idea(parent=some_person, other_field=field_value).put()
then you can query all ideas where some_person is the ancestor
persons_ideas = Idea.all().ancestor(some_person_key).fetch(1000)
the ancestor key will be included in the Idea entities key and you won't be able to change that the ancestor once the entity is created.
i highly suggest you to use ndb instead of db https://developers.google.com/appengine/docs/python/ndb/
with ndb you could even use StructuredProperty or LocalStructuredProperty
https://developers.google.com/appengine/docs/python/ndb/properties#structured
EDIT:
if you need a many to many relationship look in to ListProperties and store the Persons keys in that property. then you can query for all Ideas with that Key in that property.
class Idea(db.Model):
person = db.StringListProperty()
idea = Idea(person = [str(person.key())], ....).put()
add another person to the idea
idea.person.append(str(another_person.key())).put()
ideas = Idea.filter(person=str(person.key())).fetch(1000)
look into https://developers.google.com/appengine/docs/python/datastore/typesandpropertyclasses#ListProperty
After building a few application on the gae platform I usually use some relationship between different models in the datastore in basically every application. And often I find my self the need to see what record is of the same parent (like matching all entry with same parent)
From the beginning I used the db.ReferenceProperty to get my relations going, like:
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
parentFoo = db.ReferanceProperty(Foo)
fooKey = someFooKeyFromSomePlace
bars = Bar.all()
for bar in bar:
if bar.parentFoo.key() == fooKey:
// do stuff
But lately I've abandoned this approch since the bar.parentFoo.key() makes a sub query to fetch Foo each time. The approach I now use is to store each Foo key as a string on Bar.parentFoo and this way I can string compare this with someFooKeyFromSomePlace and get rid of all the subquery overhead.
Now I've started to look at Entity groups and wondering if this is even a better way to go? I can't really figure out how to use them.
And as for the two approaches above I'm wondering is there any downsides to using them? Could using stored key string comeback and bit me in the * * *. And last but not least is there a faster way to do this?
Tip:
replace...
bar.parentFoo.key() == fooKey
with...
Bar.parentFoo.get_value_for_datastore(bar) == fooKey
To avoid the extra lookup and just fetch the key from the ReferenceProperty
See Property Class
I think you should consider this as well. This will help you fetch all the child entities of a single parent.
bmw = Car(brand="BMW")
bmw.put()
lf = Wheel(parent=bmw,position="left_front")
lf.put()
lb = Wheel(parent=bmw,position="left_back")
lb.put()
bmwWheels = Wheel.all().ancestor(bmw)
For more reference in modeling. you can refer this Appengine Data modeling
I'm not sure what you're trying to do with that example block of code, but I get the feeling it could be accomplished with:
bars = Bar.all().filter("parentFoo " = SomeFoo)
As for entity groups, they are mainly used if you want to alter multiple things in transactions, since appengine restricts that to entities within the same group only; in addition, appengine allows ancestor filters ( http://code.google.com/appengine/docs/python/datastore/queryclass.html#Query_ancestor ), which could be useful depending on what it is you need to do. With the code above, you could very easily also use an ancestor query if you set the parent of Bar to be a Foo.
If your purposes still require a lot of "subquerying" as you put it, there is a neat prefetch pattern that Nick Johnson outlines here: http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine which basically fetches all the properties you need in your entity set as one giant get instead of a bunch of tiny ones, which gets rid of a lot of the overhead. However do note his warnings, especially regarding altering the properties of entities while using this prefetch method.
Not very specific, but that's all the info I can give you until you be more specific about exactly what you're trying to do here.
When you design your modules you also need to consider whether you want to be able to save this within a transaction. However only do this if you need to use transactions.
An alternative approach is to assign the parent like so:
from google.appengine.ext import db
class Foo(db.Model):
name = db.StringProperty()
class Bar(db.Model):
name = db.StringProperty()
def _save_entities( foo_name, bar_name ):
"""Save the model data"""
foo_item = Foo( name = foo_name )
foo_item.put()
bar_item = Bar( parent = foo_item, name = bar_name )
bar_item.put()
def main():
# Run the save in a transaction, if any fail this should all roll back
db.run_in_transaction( _save_transaction, "foo name", "bar name" )
# to query the model data using the ancestor relationship
for item in bar_item.gql("WHERE ANCESTOR IS :ancestor", ancestor = foo_item.key()).fetch(1000):
# do stuff
I'm using google app engine, and am having trouble writing querys to filter ReferenceProperties.
eg.
class Group(db.Model):
name = db.StringProperty(required=True)
creator = db.ReferenceProperty(User)
class GroupMember(db.Model):
group = db.ReferenceProperty(Group)
user = db.ReferenceProperty(User)
And I have tried writing something like this:
members = models.GroupMember.all().filter('group.name =', group_name)
and various other things that don't work. Hopefully someone can give me a prod in the right direction...
If your groups are uniquely named, then your "group.name" is a unique identifier of a Group entity.
That means you can write:
members = models.GroupMember.all().filter(
"group =",model.Group.gql("WHERE name=:1", group_name).get()
)
though you only need to do that if you don't already have the group entity lying around in the stack somewhere.
Google's essay on many-to-many with appengine is here.
If what you want is to get the members of a group, ReferenceProperties have that built-in.
class GroupMember(db.Model):
group = db.ReferenceProperty(Group, collection_name="groupMembers")
user = db.ReferenceProperty(User, collection_name="groupMembers")
Then you can write:
# get the group entity somehow
group = Group.get(group_key)
# do something with the members, such as list the nicknames
nicknames = [x.user.nickname for x in group.groupMembers]
This would require a join, which isn't possible in App Engine. If you want to filter by a property of another model, you need to include that property on the model you're querying against.
This would result in two datastore hits but should work. If you use memcache shouldnt be a problem.
group = models.Group.all().filter("name =", group_name).get()
members = models.GroupMember.all().filter('group =', group)
Using the Models that you defined in your question, lets say you want to list all members of a group called "Space monkeys".
mygroup = Group.gql("WHERE name = :1",'Space monkeys')
for group_member in mygroup.groupmember_set:
print 'group members name is: %s' % (group_member.user.name)
The "groupmember_set" is called a "implicit collection property" and is very useful. You can call it whatever you want by over-riding the default name using the collection_name keyword parameter to ReferenceProperty. For an example see the answer by Thomas L Holaday.
This is all explained in a very good paper by Rafe Kapla: Modeling Entity Relationships.