Develop Reference

Ndb default order does not preserve insertion order

I used GAE and NDB for a project. I just noticed that if I create several objects, and then I retrieve the list of these objects the order is not preserved (i use the fetch() on the object).
This is a screenshot of the admin page, which shows the same problem:
as you may (if it's too small here is the link) see i've several sessions. Now, i created the sessions that have as name day in order, from 0 to 7.
But as you see the order is not preserved.
I checked and actually the keys are not incremental. Neither the id (id should be incremental, shouldn't it? but anyway in some classes, not this one, I used a hand-made key, so there will be no id).
Is there a way to preserve insertion order?
(or it's just a strange behaviour? or it's my bad?)
PS: if you want to have a look at the code: this is the session model which extends this class i made

Neither keys nor ids are strictly incremental (and incremental by one) in ndb. You can set your own ids and assure they autoincrement properly.
Or you can add to your model(s) a DateTimeProperty:
created = ndb.DateTimeProperty(auto_now_add=True)
And in your view you can use a filter to sort the entities by the date of insertion, for ex:
posts = Post.query().order(-Post.created).fetch()
which will order and fetch your (let's say) Post entities in the descending order of insertion dates.

It's not expected that the order would be preserved unless you perform a query that would retrieve then in a particular order.
What makes you think they should be ordered?

Auto increment property with py2neo (Neo4j)?

I'm using flask with py2neo for my Rest service , I have a user node with the label "User".
how to autoincrement id for the "User" label , in neo4j using py2neo?

You don't, and you probably shouldn't. Neo4j already provides an internal id field that is an auto-incrementing integer. It isn't a property of the node, but is accessible via the id() function, like this:
MATCH (n:Person)
RETURN id(n);
So whenever you create any node, this already happens automatically for free by neo4j, and isn't done by py2neo.
If you need a different type of identifier for your code, I'd recommend something that's plausibly globally unique, like a UUID which is very easy to do in python, rather than an auto-incrementing integer.
The trouble with auto-incrementing numbers as IDs is that since they have a pattern to them (auto-incrementing) people come to rely on the value of the identifier, or come to rely on expectations of how the ID will be assigned. This is almost always a bad idea in databases. The sole purpose of the identifier is to be unique from everything else. It doesn't mean anything, and in some cases isn't even guaranteed not to change. Avoid embedding any reliance on any particular value or assignment scheme into your code.
That's why I like UUIDs, is because their assignment scheme is essentially arbitrary, and they clearly don't mean anything -- so they don't tempt designers to do anything clever with them. :)

Google app engine: better way to make query

Say I have RootEntity, AEntity(child of RootEntity), BEntity(child of AEntity).
class RootEntity(ndb.Model):
rtp = ndb.StringProperty()
class AEntity(ndb.Model):
ap = ndb.IntegerProperty()
class BEntity(ndb.Model):
bp = ndb.StringProperty()
So in different handlers I need to get instances of BEntity with specific ancestor(instance of AEntity).
There is a my query: BEntity.query(ancestor = ndb.Key("RootEntity", 1, "AEntity", AEntity.query(ancestor = ndb.Key("RootEntity", 1)).filter(AEntity.ap == int(some_value)).get().key.integer_id()))
How I can to optimize this query? Make it better, may be less sophisticated?
Upd:
This query is a part of function with #ndb.transactional decorator.

You should not use Entity Groups to represent entity relationships.
Entity groups have a special purpose: to define the scope of transactions. They give you ability to update multiple entities transactionally, as long as they are a part of the same entity group (this limitation has been somewhat relaxed with the new XG transactions). They also allow you to use queries within transactions (not available via XG transactions).
The downside of entity groups is that they have an update limitation of 1 write/second.
In your case my suggestion would be to use separate entities and make references between them. The reference should be a Key of the referenced entity as this is type-safe.
Regarding query simplicity: GAE unfortunately does not support JOINs or reference (multi-entity) queries, so you would still need to combine multiple queries together (as you do now).

There is a give and take with ancestor queries. They are a more verbose and messy to deal with but you get a better structure to your data and consistency in your queries.
To simplify this, if your handler knows the BEntity you want to get, just pass around the key.urlsafe() encoded key, it already has all of your ancestor information encoded.
If this is not possible, try possibly restructuring your data. Since these objects are all of the same ancestor, they belong to the same entity group, thus at most you can insert/update ~1 time per second for objects in that entity group. If you require higher throughput or do not require consistent ancestral queries, then try using ndb.KeyProperty to link entities with a reference to a parent rather than as an ancestor. Then you'd only need to get a single parent to query on rather than the parent and the parent's parent.
You should also try and use IDs whenever possible, so you can avoid having to filter for entities in your datastore by properties and just reference them by ID:
BEntity.query(ancestor = ndb.Key("RootEntity", 1, "AEntity", int(some_value)))
Here, int(some_value) is the integer ID of the AEntity you used when you created that object. Just be sure that you can ensure the IDs you manually create/use will be unique across all instances of that Model that share the same parent.
EDIT:
To clarify, my last example should have been made more clear in that I was suggesting to restructure the data such that int(some_value) be used as the integer ID of the AEntity rather than storing is as a separate property of the Entity - if possible of course. From the example given, a query is performed for the AEntity objects that have a given integer field value of int(some_value) and executed with a get() - implying that you will always expect a single value return for that integer ID making it a good candidate to use as the integer ID for the key of that object eliminating the need for a query.

Is there a function to check whether an ID you want to use for an entity is available?

I think I read something about a function appengine has that can tell whether an ID / key you want to use for an entity is available, or if there was a function to get an available ID to choose. App engine team said also that we should set the ID when the entity is created and not change it. But in practice we can just copy everything to a new entity with the new ID?
Thanks!
Update
I think the function I'm looking for is allocateIDs from the docs:
http://code.google.com/appengine/docs/python/datastore/functions.html

To reserve one or more IDs, use allocate_ids(). To check whether an ID is already taken, just construct a Key for it using Key.from_path(kind, id) and try to db.get() it. Also note that IDs for keys with a parent are taken from separate pools and are only unique among keys with the same parent.

On the page describing transactions, a use case is presented where the entity in question, a SalesAccount is updated, or if the account doesn't exist, it is created instead. The technique is to just try to load the entity with the given key; and if it returns nothing, create it. It's important to do this inside a transaction to avoid the situation where two users are both racing for the same key, and both see that it doesn't exist (and both try to create it).

How do I get the value of a property corresponding to a SQLAlchemy InstrumentedAttribute?

Given a SQLAlchemy mapped class Table and an instance of that class t, how do I get the value of t.colname corresponding to the sqlalchemy.org.attributes.InstrumentedAttribute instance Table.colname?
What if I need to ask the same question with a Column instead of an InstrumentedAttribute?
Given a list of columns in an ORDER BY clause and a row, I would like to find the first n rows that come before or after that row in the given ordering.

To get an objects attribute value corresponding to an InstrumentedAttribute it should be enough to just get the key of the attribute from it's ColumnProperty and fetch it from the object:
t.colname == getattr(t, Table.colname.property.key)
If you have a Column it can get a bit more complicated because the property that corresponds to the Column might have a different key. There currently doesn't seem to be a public API to get from a column to the corresponding property on a mapper. But if you don't need to cover all cases, just fetch the attr using Column.key.
To support descending orderings you'll either need to construct the desc() inside the function or poke a bit at non-public API's. The class of the descending modifier ClauseElement is sqlalchemy.sql.expression._UnaryExpression. To see if it is descending you'll need to check if the .modifier attribute is sqlalchemy.sql.operators.desc_op. If that case you can get at the column inside it via the .element attribute. But as you can see it is a private class, so watch for any changes in that area when upgrading versions.
Checking for descending still doesn't cover all the cases. Fully general support for arbitrary orderings needs to be able to rewrite full SQL expression trees replacing references to a table with corresponding values from an object. Unfortunately this isn't possible with public API's at this moment. The traversal and rewriting part is easy with sqlalchemy.sql.visitors.ReplacingCloningVisitor, the complex part is figuring out which column maps to which attribute given inheritance hierarchies, mappings to joins, aliases and probably some more parts that escape me for now. I'll give a shot at implementing this visitor, maybe I can come up with something robust enough to be worthy of integrating into SQLAlchemy.