Hello I'm new on Datastore and on Python and I have a basic question but it can help me to understand more the Google cloud.
I have 4 entities and let's say I have a parent (match) with children : team, player and event.
class Team(ndb.Model):
d_name = ndb.StringProperty()
d_side = ndb.StringProperty()
class Player(ndb.Model):
d_name = ndb.StringProperty()
date_of_birth = ndb.StringProperty()
d_position = ndb.StringProperty()
d_teamKey = ndb.StringProperty()
class Match(ndb.Model):
d_competition_name = ndb.StringProperty()
d_date = ndb.StringProperty()
d_pool = ndb.StringProperty()
d_season = ndb.StringProperty()
d_team1Key = ndb.StringProperty()
d_team2Key = ndb.StringProperty()
d_winning_teamKey = ndb.StringProperty()
d_match_id = ndb.StringProperty()
d_match_day = ndb.IntegerProperty()
class Event(ndb.Expando):
d_teamKey = ndb.StringProperty()
d_playerKey = ndb.StringProperty()
I know that the query if I want all the matchs day 4 is :
q = ndb.gql("SELECT * FROM Match WHERE d_match_day = 4")
But how can I seach all the players in theses match's children so that I have all the players who have played during the day 4 ?
Thank you !
Add another property to Match: A StructuredProperty, which is a list of Players (and/or Teams):
Players = ndb.StructuredProperty(Player)
Teams = ndb.StructuredProperty(Team)
Then, you can query for 4 and pull the list of Players and/or Teams.
wait.... you add the player as a child to EACH match he plays? that seems unefficient design. A child can only have 1 parent (unless I grossly misunderstood ancestor queries, which is possible, to be fair).
Anyway in one single query I don't think that's doable. I would start getting the keys from match where d_match_day = 4, and from there do a "select * from Players where match_key = " and use the list you just created. (you might need to change match_key to match your actual ancestor key, but the jist of it is there)
Related
I have a project deployed on Google App Engine....I am facing rollback issue. I want to rollback just like Relational db where we do commit before return or end of method. How do we achieve this in ndb???
Please find below code snippet for clarity
class PersonTable(ndb.Model):
personId = ndb.StringProperty()
personName = ndb.StringProperty()
personAddress = ndb.StringProperty()
personOldReference = ndb.StringProperty()
scopeReference = ndb.StringProperty()
class ScopeTable(ndb.Model):
child = ndb.StringProperty()
isPerson = ndb.BooleanProperty()
parent = ndb.StringProperty()
#endpoints.method(ZGScopeRequest, ZGScopeResponse, name='scope', path='scope', http_method='POST')
def scope(self, request):
scope = request.scope
person = request.personName
list = scope.split('.')
length = len(list)
for entry in list:
newId = GenerateRandomId('SCOPE')
child = mobileappmodels.ScopeTable.query(mobileappmodels.ScopeTable.child.IN([entry])).get()
if child:
pass
else:
index = list.index(entry)
parent = list[index-1]
if entry == 'PersonName':
entry = person
fetchPerson = mobileappmodels.CommunityTable.gql("""WHERE personName = :1""",entry).fetch()
update = fetchCommunity[0].key.get()
update.scopeReference = newId
update.put()
save = mobileappmodels.ScopeTable(child=person, isCommunity = True, parent = parent, id=newId)
save.put()
elif entry == 'Global':
save = mobileappmodels.ScopeTable(child=entry, isCommunity = False, parent = '', id=newId)
save.put()
else:
save = mobileappmodels.ScopeTable(child=entry, isCommunity = False, parent = parent, id=newId)
save.put()
return ZGScopeResponse(message="Uploaded")
Suppose my function fails just before return and after 2nd put, how do I rollback???
Help would be really appreciate as I am new to Google App Engine and NDB
Not sure I understand. How could it fail after the last put and before the return?
Perhaps you want to run the code in a transaction:
https://cloud.google.com/appengine/docs/standard/python/ndb/transactions
You can run it in a transaction, and call ndb.Rollback if needed.
I can't seem to isolate a single record from this query:
subcust = OwnerCustom.objects.get(carcustom=ncset, owner=sset)
This is the error:
OwnerCustom matching query does not exist
In the actual data, there is only actually one matching record in OwnerCustom for each record in CarCustom. It's supposed to be a kind of many-to-many where there are standard differences listed in CarCustom for each Car, and each owner may maintain their own customizations (overrides) or those default OwnerCustom entries.
Note, there are many different Owner of the same Car. And of course, I'm not actually doing cars, this is a renaming from the original purpose.
Here's the relevant models:
class Car(models.Model):
car_name = models.CharField(max_length=50)
class CarCustom(models.Model):
car = models.ForeignKey(Car, models.PROTECT)
class Owner(models.Model):
car = models.ForeignKey(Car, models.PROTECT)
class OwnerCustom(models.Model):
owner = models.ForeignKey(Owner, models.PROTECT)
carcustom = models.ForeignKey(CarCustom, models.PROTECT)
name = models.CharField(max_length=50)
And the code:
car_queryset = Car.objects.filter(car_name="fancy car")
for nset in car_queryset:
owner_queryset = Owner.objects.filter(car=nset)
for sset in owner_queryset :
carcustom_queryset = CarCustom.objects.filter(car=nset)
for ncset in carcustom_queryset:
subcust = OwnerCustom.objects.get(carcustom=ncset, owner=sset)
I've tried stuff like:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset).first()
Which gives me a NoneType, and then tried:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)[:1].get()
Which gives "matching query does not exist" and this:
subcust = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)[0]
Gives "list index out of range"
UPDATE: I CAN get a working function by using code like this, but I would think since there is only one (guaranteed by application) matching record possible for OwnerCustom.objects.filter(carcustom=ncset, owner=sset) that I could find a better way to fetch it:
car_queryset = Car.objects.filter(car_name="fancy car")
for nset in car_queryset:
owner_queryset = Owner.objects.filter(car=nset)
for sset in owner_queryset :
carcustom_queryset = CarCustom.objects.filter(car=nset)
for ncset in carcustom_queryset:
subcust_queryset = OwnerCustom.objects.filter(carcustom=ncset, owner=sset)
for subcust in subcust_queryset :
logger.info(subcust.name)
I am creating a sample application storing user detail along with its class information.
Modal classes being used are :
Model class for saving user's class data
class MyData(ndb.Model):
subject = ndb.StringProperty()
teacher = ndb.StringProperty()
strength = ndb.IntegerProperty()
date = ndb.DateTimeProperty()
Model class for user
class MyUser(ndb.Model):
user_name = ndb.StringProperty()
email_id = ndb.StringProperty()
my_data = ndb.StructuredProperty(MyData, repeated = True)
I am able to successfully store data into the datastore and can also make simple query on the MyUser entity using some filters based on email_id and user_name.
But when I try to query MyUser result using filter on a property from the MyUser modal's Structured property that is my_data, its not giving correct result.
I think I am querying incorrectly.
Here is my query function
function to query based upon the repeated structure property
def queryMyUserWithStructuredPropertyFilter():
shail_users_query = MyUser.query(ndb.AND(MyUser.email_id == "napolean#gmail.com", MyUser.my_data.strength > 30))
shail_users_list = shail_users_query.fetch(10)
maindatalist=[]
for each_user in shail_users_list:
logging.info('NEW QUERY :: The user details are : %s %s'% (each_user.user_name, each_user.email_id))
# Class data
myData = each_user.my_data
for each_my_data in myData:
templist = [each_my_data.strength, str(each_my_data.date)]
maindatalist.append(templist)
logging.info('NEW QUERY :: The class data is : %s %s %s %s'% (each_my_data.subject, each_my_data.teacher, str(each_my_data.strength),str(each_my_data.date)))
return maindatalist
I want to fetch that entity with repeated Structured property (my_data) should be a list which has strength > 30.
Please help me in knowing where I am doing wrong.
Thanks.
Queries over StructuredProperties return objects for which at least one of the structured ones satisfies the conditions. If you want to filter those properties, you'll have to do it afterwards.
Something like this should do the trick:
def queryMyUserWithStructuredPropertyFilter():
shail_users_query = MyUser.query(MyUser.email_id == "napolean#gmail.com", MyUser.my_data.strength > 30)
shail_users_list = shail_users_query.fetch(10)
# Here, shail_users_list has at most 10 users with email being
# 'napolean#gmail.com' and at least one element in my_data
# with strength > 30
maindatalist = [
[[data.strength, str(data.date)] for data in user.my_data if data.strength > 30] for user in shail_users_list
]
# Now in maindatalist you have ONLY those my_data with strength > 30
return maindatalist
I am using GAE Python. I have two root entities:
class X(ndb.Model):
subject = ndb.StringProperty()
grade = ndb.StringProperty()
class Y(ndb.Model):
identifier = ndb.StringProperty()
name = ndb.StringProperty()
school = ndb.StringProperty()
year = ndb.StringProperty()
result = ndb.StructuredProperty(X, repeated=True)
Since google stores our data across several data centers, we might not get the most recent data when we do a query as shown below(in case some changes have been "put"):
def post(self):
identifier = self.request.get('identifier')
name = self.request.get('name')
school = self.request.get('school')
year = self.request.get('year')
qry = Y.query(ndb.AND(Y.name==name, Y.school==school, Y.year==year))
record_list = qry.fetch()
My question: How should I modify the above fetch operation to always get the latest data
I have gone through the related google help doc but could not understand how to apply that here
Based on hints from Isaac answer, Would the following be the solution(would "latest_record_data" contain the latest data of the entity):
def post(self):
identifier = self.request.get('identifier')
name = self.request.get('name')
school = self.request.get('school')
year = self.request.get('year')
qry = Y.query(ndb.AND(Y.name==name, Y.school==school, Y.year==year))
record_list = qry.fetch()
record = record_list[0]
latest_record_data = record.key.get()
There's a couple ways on app engine to get strong consistency, most commonly using gets instead of queries and using ancestor queries.
To use a get in your example, you could encode the name into the entity key:
class Y(ndb.Model):
result = ndb.StructuredProperty(X, repeated=True)
def put(name, result):
Y(key=ndb.Key(Y, name), result).put()
def get_records(name):
record_list = ndb.Key(Y, name).get()
return record_list
An ancestor query uses similar concepts to do something more powerful. For example, fetching the latest record with a specific name:
import time
class Y(ndb.Model):
result = ndb.StructuredProperty(X, repeated=True)
#classmethod
def put_result(cls, name, result):
# Don't use integers for last field in key. (one weird trick)
key = ndb.Key('name', name, cls, str(int(time.time())))
cls(key=key, result=result).put()
#classmethod
def get_latest_result(cls, name):
qry = cls.query(ancestor=ndb.Key('name', name)).order(-cls.key)
latest = qry.fetch(1)
if latest:
return latest[0]
The "ancestor" is the first pair of the entity's key. As long as you can put a key with at least the first pair into the query, you'll get strong consistency.
class Author (db.Model)
name = db.StringProperty()
class Book (db.Model)
author = db.ReferenceProperty (collection_name="books", indexed=True)
name = db.StringProperty()
author = Author.get (author_key)
q = Book.all()
q.filter ("author =", author.key())
q.filter ("name =", "BOOK_NAME")
book = q.fetch(1)[0]
book.author.name
Will the last statement (book.author.name) result in a another read on datastore?
Yes
If you are just starting out consider using ndb, which has cacheing of get requests built in.