Develop Reference

How does allocateIds() work in Cloud Datastore Mode?

In the new Datastore Mode documentation, there is mention of allocateIds() method. However, beyond a single paragraph, there isn't an example code that illustrates how this method is used.
I am trying to allocate an ID each time I create a new entity so that I can save the ID as a property of the entity itself.
I assume that in pseudocode, it works like this:
user_id = allocateIds(number_id_ids=1)
user_key = datastore_client.key(kind='User', user_id)
user = datastore.Entity(key=user_key)
user.update({ 'user_id': user_id }) # Allows a get_user_by_id() query
datastore_client.put(user)
How exactly does allocateIds() work in practice?

When you call the allocateIds() function it invokes a new instance of class Key(object) when the consturctor of "Key" is called it takes all of the arguments you provided allocateIds and recombines them through a _combine_args method. That is what produces your key.
(and if you want to see the code yourself)
source: https://googleapis.dev/python/datastore/latest/_modules/google/cloud/datastore/key.html#Key

Yes, allocateIds() should work for the case where you want to get an ID from Datastore mode and use it as both an ID and property value:
from google.cloud import datastore
client = datastore.Client()
# Allocate a single ID in kind User
# Returns list of keys
keys = client.allocate_ids(client.key('User'), 1)
# Get key from list
key = keys[0]
print(key.id)
# Create a User entity using our key
user = datastore.Entity(key)
# Add ID as a field
user.update({
'user_id': key.id
})
# Commit to database
client.put(user)
# Query based on full key
query = client.query(kind='User')
query.key_filter(user.key, '=')
results = list(query.fetch())
print(results)
For most other cases where you just want a single auto-ID, you can skip allocate_ids:
# Create a User entity
# Use an incomplete key so Datastore assigns an ID
user = datastore.Entity(client.key('User'))
# Add some data
user.update({
'foo': 'bar'
})
# Datastore allocates an ID when you call client.put
client.put(user)
# user.key now contains an ID
user_id = user.key.id
print(user_id)
# Query with the ID and key
query = client.query(kind='User')
query.key_filter(user.key, '=')
results = list(query.fetch())
print(results)

use existing field as _id using elasticsearch dsl python DocType

I have class, where I try to set student_id as _id field in elasticsearch. I am referring persistent example from elasticsearch-dsl docs.
from elasticsearch_dsl import DocType, String
ELASTICSEARCH_INDEX = 'student_index'
class StudentDoc(DocType):
'''
Define mapping for Student type
'''
student_id = String(required=True)
name = String(null_value='')
class Meta:
# id = student_id
index = ELASTICSEARCH_INDEX
I tied by setting id in Meta but it not works.
I get solution as override save method and I achieve this
def save(self, **kwargs):
'''
Override to set metadata id
'''
self.meta.id = self.student_id
return super(StudentDoc, self).save(**kwargs)
I am creating this object as
>>> a = StudentDoc(student_id=1, tags=['test'])
>>> a.save()
Is there any direct way to set from Meta without override save method ?

There are a few ways to assign an id:
You can do it like this
a = StudentDoc(meta={'id':1}, student_id=1, tags=['test'])
a.save()
Like this:
a = StudentDoc(student_id=1, tags=['test'])
a.meta.id = 1
a.save()
Also note that before ES 1.5, one was able to specify a field to use as the document _id (in your case, it could have been student_id), but this has been deprecated in 1.5 and from then onwards you must explicitly provide an ID or let ES pick one for you.

Peewee getting column after join

I am unable to read the column of another table which is joined. It throws AttributeError
class Component(Model):
id = IntegerField(primary_key=True)
title = CharField()
class GroupComponentMap(Model):
group = ForeignKeyField(Component, related_name='group_fk')
service = ForeignKeyField(Component, related_name='service_fk')
Now the query is
comp = (Component
.select(Component, GroupComponent.group.alias('group_id'))
.join(GroupComponent, on=(Component.id == GroupComponent.group))
)
for row in comp:
print row.group_id
Now I get an error AttributeError: 'Component' object has no attribute 'group_id'

If you just want to directly patch the group_id attribute onto the selected Component, call .naive(). This instructs peewee that you don't want to reconstruct the graph of joined models -- you just want all attributes patched onto a single Component instance:
for row in comp.naive():
print row.group_id # This will work now.

Peewee - How to Convert a Dict into a Model

Lets say I have
import peewee
class Foo(Model):
name = CharField()
I would like to do the following:
f = {id:1, name:"bar"}
foo = Foo.create_from_dict(f)
Is this native in Peewee? I was unable to spot anything in the source code.
I've wrote this function which works but would rather use the native function if it exists:
#clazz is a string for the name of the Model, i.e. 'Foo'
def model_from_dict(clazz, dictionary):
#convert the string into the actual model class
clazz = reduce(getattr, clazz.split("."), sys.modules[__name__])
model = clazz()
for key in dictionary.keys():
#set the attributes of the model
model.__dict__['_data'][key] = dictionary[key]
return model
I have a web page that displays all the foos and allows the user to edit them. I would like to be able to pass a JSON string to the controller, where I would convert it to a dict and then make Foos out of it, so I can update as necessary.

If you have a dict, you can simply:
class User(Model):
name = CharField()
email = CharField()
d = {'name': 'Charlie', 'email': 'foo#bar.com'}
User.create(**d)

You could use PickledKeyStore which allows you to save any value as a python dict and it works like Python's pickle library.

Python Inspect - Lookup the data type for a property in a GAE db.model Class

class Employee(db.Model):
firstname = db.StringProperty()
lastname = db.StringProperty()
address1 = db.StringProperty()
timezone = db.FloatProperty() #might be -3.5 (can contain fractions)
class TestClassAttributes(webapp.RequestHandler):
"""
Enumerate attributes of a db.Model class
"""
def get(self):
for item in Employee.properties():
self.response.out.write("<br/>" + item)
#for subitem in item.__dict__:
# self.response.out.write("<br/> --" + subitem)
The above will give me a list of the property names for the variable "item".
My idea of item.__dict__ didn't work because item was a str.
How can I then display the data field type for each property, such as db.FloatProperty() for the property called timezone?
GAE = Google App Engine - but I'm sure the same answer would work for any class.
Thanks,
Neal Walters

Iterate using "for name, property in Employee.properties().items()". The property argument is the Property instance, which you can compare using instanceof.

For problems like these, the interactive Python shell is really handy. If you had used it to poke around at your Employee object, you might have discovered the answer to your question through trial and error.
Something like:
>>> from groups.models import Group
>>> Group.properties()
{'avatar': <google.appengine.ext.db.StringProperty object at 0x19f73b0>,
'created_at': <google.appengine.ext.db.DateTimeProperty object at 0x19f7330>,
'description': <google.appengine.ext.db.TextProperty object at 0x19f7210>,
'group_type': <google.appengine.ext.db.StringProperty object at 0x19f73d0>}
From that you know that the properties() method of a db.Model object returns a dict mapping the model's property names to the actual property objects they represent.

I add the same problem, and the first 2 answers did not help me 100%.
I was not able to get the type information, from the meta data of the class or the
instance property, which is bizarre. So I had to use a dictionary.
The method GetType() will return the type of the property as a string.
Here is my answer:
class RFolder(db.Model):
def GetPropertyTypeInstance(self, pname):
for name, property in self.properties().items():
if name==pname:
return property
return None
def GetType(self, pname):
t = self.GetPropertyTypeInstance(pname)
return RFolder.__DB_PROPERTY_INFO[type(t)]
__DB_PROPERTY_INFO = {
db.StringProperty :"String",
db.ByteStringProperty :"ByteString",
db.BooleanProperty :"Boolean",
db.IntegerProperty :"Integer",
db.FloatProperty :"Float",
db.DateTimeProperty :"DateTime",
db.DateProperty :"Date",
db.TimeProperty :"Time",
db.ListProperty :"List",
db.StringListProperty :"StringList",
db.ReferenceProperty :"Reference",
db.SelfReferenceProperty :"SelfReference",
db.UserProperty :"User",
db.BlobProperty :"Blob",
db.TextProperty :"Text",
db.CategoryProperty :"Category",
db.LinkProperty :"Link",
db.EmailProperty :"Email",
db.GeoPtProperty :"GeoPt",
db.IMProperty :"IM",
db.PhoneNumberProperty :"PhoneNumber",
db.PostalAddressProperty :"PostalAddress",
db.RatingProperty :"Rating"
}