I've got two mapped objects, Parent and Child.
class Parent(Base):
__tablename__ = 'parent'
id = ...
name = ...
date_modified = Column(SA_DateTime, default=DateTime.now,
onupdate=DateTime.now, nullable=False)
class Child(Base):
__tablename__ = 'child'
id = ...
name = ...
date_modified = Column(SA_DateTime, default=DateTime.now,
onupdate=DateTime.now, nullable=False)
parent = relationship(Parent, backref='parent')
When a child is updated, I want not only Child.date_modified to be changed, but also Child.parent.date_modified.
I tried to do this like this:
#event.listens_for(Child, 'after_update')
def modified_listener(mapper, connection, target):
if object_session(target).is_modified(target, include_collections=False):
target.parent.date_modified = DateTime.now()
But this doesn't work, because I'm already in a flush and I get something like
SAWarning: Attribute history events accumulated on 1 previously clean instance within inner-flush event handlers have been reset, and will not result in database updates. Consider using set_committed_value() within inner-flush event handlers to avoid this warning.
How can I solve this with SQLAlchemy?
Basic update-parent-when-child-changes using SQLAlchemy events has been covered on this site before here and here, but in your case you're trying to update the parent during the flush, possibly using an update default value from the child, which will be visible after the update, or a new value entirely. Modifying the parent in the event handler is not as straightforward as you might first imagine:
Warning
Mapper-level flush events only allow very limited operations, on attributes local to the row being operated upon only, as well as allowing any SQL to be emitted on the given Connection. Please read fully the notes at Mapper-level Events for guidelines on using these methods; generally, the SessionEvents.before_flush() method should be preferred for general on-flush changes.
As you've noticed, simple
target.parent.date_modified = DateTime.now()
in your event handler warns:
SAWarning: Attribute history events accumulated on 1 previously clean instances within inner-flush event handlers have been reset, and will not result in database updates. Consider using set_committed_value() within inner-flush event handlers to avoid this warning.
set_committed_value() allows setting attributes with no history events, as if the set value was part of the original loaded state.
You've also noticed that receiving a target in an after update event handler does not guarantee that an UPDATE statement was actually emitted:
This method is called for all instances that are marked as “dirty”, even those which have no net changes to their column-based attributes, and for which no UPDATE statement has proceeded.
and
To detect if the column-based attributes on the object have net changes, and therefore resulted in an UPDATE statement, use object_session(instance).is_modified(instance, include_collections=False).
So a solution could be to use the information held in the event target to emit an UPDATE statement on the parent table using the given connection, and then to check if the Parent object is present in the session and set the committed value of it:
from sqlalchemy import event
from sqlalchemy.orm.attributes import set_committed_value
from sqlalchemy.orm.session import object_session
#event.listens_for(Child, 'after_update')
def receive_child_after_update(mapper, connection, target):
session = object_session(target)
if not session.is_modified(target, include_collections=False):
return
new_date_modified = target.date_modified
# Avoid touching the target.parent relationship attribute and
# copy the date_modified value from the child to parent.
# Warning: this will overwrite possible other updates to parent's
# date_modified.
connection.execute(
Parent.__table__.
update().
values(date_modified=new_date_modified).
where(Parent.id == target.parent_id))
parent_key = session.identity_key(Parent, target.parent_id)
try:
the_parent = session.identity_map[parent_key]
except KeyError:
pass
else:
# If the parent object is present in the session, update its
# date_modified attribute **in Python only**, to reflect the
# updated DB state local to this transaction.
set_committed_value(
the_parent, 'date_modified', new_date_modified)
Have you tried:
#event.listens_for(Child, 'after_update')
def modified_listener(mapper, connection, target):
if object_session(target).is_modified(target, include_collections=False):
sa.orm.attributes.set_committed_value(
target.parent,
'date_modified',
DateTime.now())
As per:
http://docs.sqlalchemy.org/en/latest/orm/session_api.html#sqlalchemy.orm.attributes.set_committed_value
Related
I've actually never encountered this error before:
sqlalchemy.exc.InvalidRequestError: stale association proxy, parent object has gone out of scope
After doing some research, it looks like its because the parent object is being garbage collected while the association proxy is working. Fantastic.
However, I'm not sure where it's happening.
Relevant code:
# models.py
class Artist(db.Model):
# ...
tags = association_proxy('_tags', 'tag',
creator=lambda t: ArtistTag(tag=t))
# ...
class Tag(db.Model):
# ...
artist = association_proxy('_artists', 'artist',
creator=lambda a: ArtistTag(artist=a))
# ...
class ArtistTag(db.Model):
# ...
artist_id = db.Column(db.Integer, ForeignKey('artists.id'))
artist = db.relationship('Artist', backref='_tags')
tag_id = db.Column(db.Integer, ForeignKey('tags.id'))
tag = db.relationship('Tag', backref='_artists')
# api/tag.py
from flask.ext.restful import Resource
from ..
class ListArtistTag(Resource):
def get(self, id):
# much safer in actual app
return TagSchema(many=True)
.dump(Artist.query.get(id).tags)
.data
I know it's an old question, but I haven't found a clear solution to a similar problem anywhere on the web, so I've decided to reply here.
The key here is to assign the object that holds the association proxy to a variable before performing any further operations on them. Association proxies aren't regular object properties which would force the GC to hold the reference to the parent object. Actually, the call in form of:
tags = association_proxy('_tags', 'tag', creator=lambda t: ArtistTag(tag=t))
will result in creation of a new AssociationProxy class object, with a weak reference to the target's collection. In low memory conditions, GC may try to collect Artist.query.get(id) result, leaving just the result's tags collection (being a AssociationProxy class object), but it's required that the object having a association proxy to be present, due to SQLAlchemy's implementation (lazy loading mechanism precisely, I believe).
To fix this situation, we need to make sure that the Artist object returned from Artist.query.get(id) call is assigned to a variable, so that the reference count to that object is explicitly of non-zero value. So this:
class ListArtistTag(Resource):
def get(self, id):
# much safer in actual app
return TagSchema(many=True)
.dump(Artist.query.get(id).tags)
.data
becomes this:
class ListArtistTag(Resource):
def get(self, id):
artist = Artist.query.get(id)
return TagSchema(many=True)
.dump(artist.tags)
.data
And it will work as expected. Simple, right?
i have this column in a table:
name = Column(String, default='ts-1')
how do i set the default value to increment automatically so that the names of objects will be 'ts-1', 'ts-2', 'ts-3'... when created.
i tried writing a constructor:
def __init__(self):
self.name = 'ts-' + str(self.id)
but it always returns None.
thanks.
I found a approach to solving this problem using sqlalchemy orm events. turns out to be fairly simple. the idea is to write an event listener on init so that it will be trigger whenever an object is created.
so now i can access the value of id of an instance.
from sqlalchemy import event
#event.listens_for(SomeClass, 'init')
def change_name(target, args, kwargs):
session.add(target)
session.commit()
target.name = 'sheet#' + str(target.id)
Suppose I am using a declarative object like the following:
class MyObject(DeclarativeBase):
...
other_object_id = Column(Integer, ForeignKey('other_object.id'))
other_object = relationship("OtherObject")
...
Suppose I would like to have a convenience method set_other_object_value, which would modify other_object.value, creating an instance of OtherObject and adding it to the session, if necessary:
def set_other_object_value(self, value):
if self.other_object is None:
<create instance of OtherObject and associate with the session>
self.other_object.value = value
The question is: how can I create OtherObject and associate it with the session? A possibly equivalent question: is it possible to access the instance of a Session, to which MyObject was added from within an instance of MyObject?
Use object_session:
from sqlalchemy.orm.session import object_session
# ...
def set_other_object_value(self, value):
if self.other_object is None:
value = OtherObject(...)
self.other_object.value = value
object_session(self).add(value)
But if you have setup you relationships properly, you do not need to add the new value to the session, as sqlalchemy will figure it out and save it to your database on commit anyways.
My question does not really have much to do with sqlalchemy but rather with pure python.
I'd like to control the instantiation of sqlalchemy Model instances. This is a snippet from my code:
class Tag(db.Model):
__tablename__ = 'tags'
query_class = TagQuery
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), unique=True, nullable=False)
def __init__(self, name):
self.name = name
I want to achieve that whenever an entry is instantiated (Tag('django')) that a new instance should be created only if there is not yet another tag with the name django inside the database. Otherwise, instead of initializing a new object, a reference to the already existent row inside the database should be returned by (Tag('django')).
As of now I am ensuring the uniqueness of tags inside the Post Model:
class Post(db.Model):
# ...
# code code code
# ...
def _set_tags(self, taglist):
"""Associate tags with this entry. The taglist is expected to be already
normalized without duplicates."""
# Remove all previous tags
self._tags = []
for tag_name in taglist:
exists = Tag.query.filter(Tag.name==tag_name).first()
# Only add tags to the database that don't exist yet
# TODO: Put this in the init method of Tag (if possible)
if not exists:
self._tags.append(Tag(tag_name))
else:
self._tags.append(exists)
It does its job but still I'd like to know how to ensure the uniqueness of tags inside the Tag class itself so that I could write the _set_tags method like this:
def _set_tags(self, taglist):
# Remove all previous tags
self._tags = []
for tag_name in taglist:
self._tags.append(Tag(tag_name))
While writing this question and testing I learned that I need to use the __new__ method. This is what I've come up with (it even passes the unit tests and I didn't forget to change the _set_tags method):
class Tag(db.Model):
__tablename__ = 'tags'
query_class = TagQuery
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), unique=True, nullable=False)
def __new__(cls, *args, **kwargs):
"""Only add tags to the database that don't exist yet. If tag already
exists return a reference to the tag otherwise a new instance"""
exists = Tag.query.filter(Tag.name==args[0]).first() if args else None
if exists:
return exists
else:
return super(Tag, cls).__new__(cls, *args, **kwargs)
What bothers me are two things:
First: I get a warning:
DeprecationWarning: object.__new__() takes no parameters
Second: When I write it like so I get errors (I also tried to rename the paramater name to n but it did not change anything) :
def __new__(cls, name):
"""Only add tags to the database that don't exist yet. If tag already
exists return a reference to the tag otherwise a new instance"""
exists = Tag.query.filter(Tag.name==name).first()
if exists:
return exists
else:
return super(Tag, cls).__new__(cls, name)
Errors (or similar):
TypeError: __new__() takes exactly 2 arguments (1 given)
I hope you can help me!
I use class method for that.
class Tag(Declarative):
...
#classmethod
def get(cls, tag_name):
tag = cls.query.filter(cls.name == tag_name).first()
if not tag:
tag = cls(tag_name)
return tag
And then
def _set_tags(self, taglist):
self._tags = []
for tag_name in taglist:
self._tags.append(Tag.get(tag_name))
As for __new__, you should not confuse it with __init__. It is expected to be called w/out args, so even if your own constructor asks for some, you should not pass them to super/object unless you know that your super needs them. Typical invocation would be:
def __new__(cls, name=None):
tag = cls.query.filter(cls.name == tag_name).first()
if not tag:
tag = object.__new__(cls)
return tag
However this will not work as expected in your case, since it calls __init__ automatically if __new__ returns instance of cls. You would need to use metaclass or add some checks in __init__.
Don't embed this within the class itself.
Option 1. Create a factory that has the pre-existing pool of objects.
tag_pool = {}
def makeTag( name ):
if name not in tag_pool:
tag_pool[name]= Tag(name)
return tag_pool[name]
Life's much simpler.
tag= makeTag( 'django' )
This will create the item if necessary.
Option 2. Define a "get_or_create" version of the makeTag function. This will query the database. If the item is found, return the object. If no item is found, create it, insert it and return it.
Given the OP's latest error msg:
TypeError: __new__() takes exactly 2 arguments (1 given)
it seems that somewhere the class is getting instantiated without the name parameter, i.e. just Tag(). The traceback for that exception should tell you where that "somewhere" is (but we're not shown it, so that's how far as we can go;-).
That being said, I agree with other answers that a factory function (possibly nicely dressed up as a classmethod -- making factories is one of the best uses of classmethod, after all;-) is the way to go, avoiding the complication that __new__ entails (such as forcing __init__ to find out whether the object's already initialized to avoid re-initializing it!-).
I'm trying to implement a delayed blog post deletion scheme. So instead of an annoying Are you sure?, you get a 2 minute time frame to cancel deletion.
I want to track What will be deleted When with a db.Model class (DeleteQueueItem), as I found no way to delete a task from the queue and suspect I can query what's there.
Creating a DeleteQueueItem entity should automatically set a delete_when property and add a task to the queue. I use the relative path of blog posts as their key_name and want to use that as key_name here, too. This led me to a custom init:
class DeleteQueueItem(db.Model):
"""Model to keep track of items that will be deleted via task queue."""
# URL path to the blog post is handled as key_name
delete_when = db.DateTimeProperty()
def __init__(self, **kwargs):
delay = 120 # Seconds
t = datetime.timedelta(seconds=delay)
deadline = datetime.datetime.now() - t
key_name = kwargs.get('key_name')
db.Model.__init__(self, **kwargs)
self.delete_when = deadline
taskqueue.add(url='/admin/task/delete_page',
countdown=delay,
params={'path': key_name})
This seems to work, until I try to delete the entity:
fetched_item = models.DeleteQueueItem.get_by_key_name(path)
This fails with:
TypeError: __init__() takes exactly 1 non-keyword argument (2 given)
What am I doing wrong?
Generally, you shouldn't try and override the init method of Model classes. While it's possible to get right, the correct constructor behaviour is fairly complex, and may even change between releases, breaking your code (though we try to avoid doing so!). Part of the reason for this is that the constructor has to be used both by your own code, to construct new models, and by the framework, to reconstitute models loaded from the datastore.
A better approach is to use a factory method, which you call instead of the constructor.
Also, you probably want to add the task at the same time as you write the entity, rather than at creation time. If you don't, you end up with a race condition: the task may execute before you've stored the new entity to the datastore!
Here's a suggested refactoring:
class DeleteQueueItem(db.Model):
"""Model to keep track of items that will be deleted via task queue."""
# URL path to the blog post is handled as key_name
delete_when = db.DateTimeProperty()
#classmethod
def new(cls, key_name):
delay = 120 # Seconds
t = datetime.timedelta(seconds=delay)
deadline = datetime.datetime.now() - t
return cls(key_name=key_name, delete_when=deadline)
def put(self, **kwargs):
def _tx():
taskqueue.add(url='/admin/task/delete_page',
countdown=delay,
params={'path': key_name},
transactional=True)
return super(DeleteQueueItem, self).put(**kwargs)
if not self.is_saved():
return db.run_in_transaction(_tx)
else:
return super(DeleteQueueItem, self).put(**kwargs)