I have two tables related via a foreign key, here they are using Declarative Mapping
class Task(DeclarativeBase):
__tablename__ = 'task'
id = Column(Integer, primary_key=True)
state = Column(Integer, default=0)
obs_id = Column(Integer, ForeignKey('obs.id'), nullable=False)
class Obs(DeclarativeBase):
__tablename__ = 'obs'
id = Column(Integer, primary_key=True)
state = Column(Integer, default=0)
So, I would like to update the related task.state when obs.state is changed to value 2. Currently I'm doing it by hand (using a relationship called task)
obs.state = 2
obs.task.state = 2
But I would prefer doing it using a trigger. I have checked that this works in sqlite
CREATE TRIGGER update_task_state UPDATE OF state ON obs
BEGIN
UPDATE task SET state = 2 WHERE (obs_id = old.id) and (new.state = 2);
END;
But I can't find how to express this in sqlalchemy. I have read insert update defaults several times, but can't find the way. I don't know if it's even possible.
You can create trigger in the database with DDL class:
update_task_state = DDL('''\
CREATE TRIGGER update_task_state UPDATE OF state ON obs
BEGIN
UPDATE task SET state = 2 WHERE (obs_id = old.id) and (new.state = 2);
END;''')
event.listen(Obs.__table__, 'after_create', update_task_state)
This is the most reliable way: it will work for bulk updates when ORM is not used and even for updates outside your application. However there disadvantages too:
You have to take care your trigger exists and up to date;
It's not portable so you have to rewrite it if you change database;
SQLAlchemy won't change the new state of already loaded object unless you expire it (e.g. with some event handler).
Below is a less reliable (it will work when changes are made at ORM level only), but much simpler solution:
from sqlalchemy.orm import validates
class Obs(DeclarativeBase):
__tablename__ = 'obs'
id = Column(Integer, primary_key=True)
state = Column(Integer, default=0)
#validates('state')
def update_state(self, key, value):
self.task.state = value
return value
Both my examples work one way, i.e. they update task when obs is changes, but don't touch obs when task is updated. You have to add one more trigger or event handler to support change propagation in both directions.
Related
Imagine I've got the following:
class User:
id = Column(Integer, primary_key=True)
username = Column(String(20), nullable=False)
password_hash = Column(String(HASH_LENGTH), nullable=False)
class LoginAttempts:
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey(User.id))
attempted_at = Column(DateTime, default=datetime.datetime.utcnow)
Now, I want to add a relationship to User called last_attempt that retrieves the most recent login attempt. How might one do this?
This seems like a use case for a relationship to an aliased class, which was added in SQLAlchemy 1.3 – before that you'd use a non primary mapper, or other methods such as a custom primary join. The idea is to create a subquery representing a derived table of latest login attempts per user that is then aliased to LoginAttempts and used as the target of a relationship. The exact query used to derive the latest attempts depends on your DBMS1, but a generic left join "antijoin" will work in most. Start by generating the (sub)query for latest login attempts:
newer_attempts = aliased(LoginAttempts)
# This reads as "find login attempts for which no newer attempt with larger
# attempted_at exists". The same could be achieved using NOT EXISTS as well.
latest_login_attempts_query = select([LoginAttempts]).\
select_from(
outerjoin(LoginAttempts, newer_attempts,
and_(newer_attempts.user_id == LoginAttempts.user_id,
newer_attempts.attempted_at > LoginAttempts.attempted_at))).\
where(newer_attempts.id == None).\
alias()
latest_login_attempts = aliased(LoginAttempts, latest_login_attempts_query)
Then just add the relationship attribute to your User model:
User.last_attempt = relationship(latest_login_attempts, uselist=False,
viewonly=True)
1: For example in Postgresql you could replace the LEFT JOIN subquery with a LATERAL subquery, NOT EXISTS, a query using window functions, or SELECT DISTINCT ON (user_id) ... ORDER BY (user_id, attempted_at DESC).
Although the selected answer is more robust, another way you could accomplish this is to use a lazy=dynamic and order_by:
User.last_attempted = relationship(LoginAttempts, order_by=desc(LoginAttempts.attempted_at), lazy='dynamic')
Be careful though, because this returns a query object (and will require .first() or equivalent), and you will need to use a limit clause:
last_attempted_login = session.query(User).get(my_user_id).last_attempted.limit(1).first()
I have three [MySQL] tables: Person, Role, PersonRole
class Person(Base):
__tablename__ = 'people'
id = Column(INTEGER(unsigned=True), primary_key=True)
full_name = Column(String(120), nullable=False)
email = Column(String(128))
username = Column(String(50))
last_modified = Column(TIMESTAMP, nullable=False,
server_default=text('CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP'))
created = Column(TIMESTAMP, server_default=text('CURRENT_TIMESTAMP'))
roles = relationship('Role',
secondary='person_role',
primaryjoin="and_(Person.id==PersonRole.person_id,"
"Role.active==True,"
"PersonRole.active==True)",
back_populates='people')
def __repr__(self):
"""String representation."""
return '''<Person(id='%s', full_name='%s')>''' % (
str(self.id), str(self.full_name)
)
class Role(Base):
__tablename__ = 'role'
id = Column(INTEGER(unsigned=True), Sequence('role_id_seq'), primary_key=True)
role_name = Column(String(16))
active = Column(Boolean, default=True)
last_modified = Column(TIMESTAMP, nullable=False,
server_default=text('CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP'))
created = Column(TIMESTAMP, server_default=text('CURRENT_TIMESTAMP'))
users = relationship('Person',
secondary='person_role',
primaryjoin="and_(Role.id==PersonRole.role_id,"
"Role.active==True,"
"PersonRole.active==True)",
back_populates='roles')
def __repr__(self):
"""String representation."""
return '''<Role(role_id='%s', role_name='%s')>''' % (
str(self.role_id), str(self.role_name)
)
class PersonRole(Base):
__tablename__ = 'person_role'
ds_id = Column(INTEGER(unsigned=True),
ForeignKey('person.id'), primary_key=True)
role_id = Column(INTEGER(unsigned=True),
ForeignKey('role.id'), primary_key=True)
active = Column(Boolean, default=True)
last_modified = Column(TIMESTAMP, nullable=False,
server_default=text('CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP'))
created = Column(TIMESTAMP,
server_default=text('CURRENT_TIMESTAMP'))
Retrieving a Person like so:
...session setup as s...
person = s.query(Person).get(123)
and to list out their roles:
for role in person.roles:
print(role.id, role.role_name, role.active)
Now, in each table there is an active column. This is to keep track of an [shocker] active status as we dont want to remove data from the table, merely keep it's state. Now to the two issues which have kept me from using SQLAlchemy altogether and has me writing and executing SQL manually -
The active portion in the final loop though displays the active state of role.active and not that of the actual relationship, person_role.active.
Removing a role will remove the relationship row instead of performing the desired action, person_role.active = 0
Even if I deactivate the relationship, adding it again will set off the Duplicate Key error.
Is there a sane, valid way to go about accomplishing this without restructuring my data?
Edit for further clarification:
The two main tables in this case are person and person_role. Person is the main table which holds our users. PersonRole holds the roles a person actually has, in the form of person.id to role.id. The Role table is merely a lookup table for the Role definition (to get the names).
I suppose what I want is a way to intercept how the ORM actually adds/removes the data. Adding should [more-or-less] do an "upsert" and removing should basically run a update query like: update person_role set active = 0 where person_id = %s and role_id = %s.
I have read a lot of the docs but tend to get lost in the terminologies. :S
Not sure I completely understood your problems, but let me try to help you:
1) This is the one I am most confused with: person.roles maps to a collection of Role entities. Ain't that what you expect?
2, 3) You have set up a relationship between Person and Role, using PersonRole as secondary. Deactivating means setting PersonRole.status to inactive, not removing it. Trying to add it again will show a Duplicate Key error indeed!
I think you want to run a query to load a PersonRole entity by ds_id and role_id, update its status to active and persist changes. I understand that sometimes this may be tedious to perform, so maybe a immediate solution that would not require you to move data around would be to map Person and Role to PersonRole. So, in spite of having Person.roles you would have Person.personRoles, so you have access to PersonRole entities and may set its status.
Of course this is a very immediate solution. SQLAlchemy is extremely featured so maybe you can intercept the Person.roles removal and customize its behaviour to set PersonRole to inactive. You may want to read more about cascading on the docs as well.
Hope I was able to clarify things a little :)
Here's a SQLAlchemy class:
class MyGroup(Base):
__tablename__ = 'my_group'
group_id = Column(Integer, Sequence('my_seq'), primary_key=True)
group_name = Column(String(200), nullable=False, index=True)
date_created = Column(DateTime, default=func.now())
date_updated = Column(DateTime, default=func.now(), onupdate=func.now())
Anytime I add a group_name or (for example) update the group_name, the date_updated field will get updated. That's great.
But sometimes there are cases where I want to mark a group as "updated" even if the group record itself did not change (for example, if data in a different but related table is updated).
I could do it manually:
group = session.query(MyGroup).filter(MyGroup.group_name=='Some Group').one()
group.date_updated = datetime.datetime.utcnow()
session.commit()
but I'd really rather let the model do it in its own way, rather than recreate a Python process to manually update the date. (For example, to avoid mistakes like where maybe the model uses now() while the manual function mistakenly uses utcnow())
Is there a way with SQLAlchemy to "touch" a record (kind of like UNIX touch) that wouldn't alter any of the record's other values but would trigger the onupdate= function?
Just to add to this answer, the documentation states the following:
The Column.default and Column.onupdate keyword arguments also accept Python functions. These functions are invoked at the time of insert or update if no other value for that column is supplied, and the value returned is used for the column’s value.
Key part being: are invoked at the time of insert or update if no other value for that column is supplied.
Key part of the key part: if no other value for that column is supplied
So with a simple update statement with empty values, does the trick:
from sqlalchemy import update
stmt = update(ModelName).where(ModelName.column.in_(column_values)).values()
db.engine.execute(update_product_mapping_info)
I am using the sqlalchemy.sql.expression.update for this, documentation here.
Here's the Model column definition I have:
from datetime import datetime
last_updated = Column(db.DateTime, onupdate=datetime.utcnow)
To show a complete example, building on #Chayemor's answer I did the following:
import sqlalchemy.sql.functions as func
from sqlalchemy import Column, Integer, String, update
from . import database as db
Base = declarative_base()
class Object(Base):
__tablename__ = "objects"
id = Column(Integer, primary_key=True, nullable=False)
last_update = Column(
DateTime,
server_default=func.now(),
onupdate=func.current_timestamp()
)
def touch(self):
stmt = update(Game).where(Game.id == self.id)
db.engine.execute(stmt)
From here, running obj.touch() updates its last_update field in the database without changing any other data.
Another way to do this is to call orm.attributes.flag_modified on an instance and attribute. SQLAlchemy will mark the attribute as modified even though it is unchanged and generate an update.
with Session() as s, s.begin():
mg = s.execute(sa.select(MyGroup)).scalar_one()
orm.attributes.flag_modified(mg, 'group_name')
Note that the "dummy" update will be included in the generated SQL's SET clause
UPDATE tbl
SET group_name=%(group_name)s,
date_updated=now()
WHERE tbl.group_id = %(tbl_group_id)s
in contrast with that generated by Chayemor's answer:
UPDATE tbl
SET date_updated=now()
WHERE tbl.group_name = %(group_name_1)s
This may be significant (consider triggers for example).
I haven't looked at the source but from the docs it seems that this is only triggered by issuing a SQL UPDATE command:
onupdate – A scalar, Python callable, or ClauseElement representing a default value to be applied to the column within UPDATE statements, which wil be invoked upon update if this column is not present in the SET clause of the update. This is a shortcut to using ColumnDefault as a positional argument with for_update=True.
If your concern is ensuring that your "touch" uses the same function as the onupdate function you could define a method on your model to perform the touch and have the onupdate parameter point this method.
I think something like this would work:
class MyGroup(Base):
__tablename__ = 'my_group'
group_id = Column(Integer, Sequence('my_seq'), primary_key=True)
group_name = Column(String(200), nullable=False, index=True)
date_created = Column(DateTime, default=func.now())
date_updated = Column(
DateTime,
default=self.get_todays_date,
onupdate=self.get_todays_date)
def get_todays_date(self):
return datetime.datetime.utcnow()
def update_date_updated(self):
self.date_updated = self.get_todays_date()
You could then update your record like this:
group.update_date_updated()
session.commit()
I have a Flask python application that has a set of related tables that are chained together through foreign keys. I would like to be able to return an aggregate list of records from one table that are related to a distant table. However, I am struggling to understand how sqlalchemy does this through object relationships.
For example, there are three objects I'd like to join (challenge and badge) with two tables (talent_challenge and badge) to be able to query for all badges related to a specific challenge. In SQL, this would look something like:
SELECT b.id, b.name
FROM badge b
INNER JOIN talent_challenge tc ON tc.talent_id = b.talent_id
WHERE tc.challenge_id = 21
The 'talent' and 'challenge' tables are not needed in this case, since I only need the talent and challenge IDs (in 'talent_challenge') for the relationship. All of the interesting detail is in the badge table.
I am able to use sqlalchemy to access the related talent from a challenge using:
talents = db.relationship('TalentModel', secondary='talent_challenge')
And I can then reference talent.badges for each of those talents to get the relevant badges related to my initial challenge. However, there can be redundancy, and this list of badges isn't contained in a single object.
A stripped-down version of the three models are:
class TalentModel(db.Model):
__tablename__ = 'talent'
# Identity
id = db.Column(db.Integer, primary_key=True)
# Relationships
challenges = db.relationship('ChallengeModel', secondary='talent_challenge',)
# badges (derived as backref from BadgeModel)
class ChallengeModel(db.Model):
__tablename__ = 'challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
member_id = db.Column(db.Integer, db.ForeignKey('member.id'))
# Relationships
talents = db.relationship('TalentModel', secondary='talent_challenge', order_by='desc(TalentModel.created_at)')
class BadgeModel(db.Model):
__tablename__ = 'badge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
# Parents
talent = db.relationship('TalentModel', foreign_keys=[talent_id], backref="badges")
I also have a model for the associative table, 'talent_challenge':
class TalentChallengeModel(db.Model):
__tablename__ = 'talent_challenge'
# Identity
id = db.Column(db.Integer, primary_key=True)
talent_id = db.Column(db.Integer, db.ForeignKey('talent.id'))
challenge_id = db.Column(db.Integer, db.ForeignKey('challenge.id'))
# Parents
talent = db.relationship('TalentModel', uselist=False, foreign_keys=[talent_id])
challenge = db.relationship('ChallengeModel', uselist=False, foreign_keys=[challenge_id])
I would like to better understand sqlalchemy (or specifically, flask-sqlalchemy) to allow me to construct this list of badges from the challenge object. Is db.session.query of BadgeModel my only option?
UPDATED 1/23/2015:
My blocker on my project was solved by using the following:
#property
def badges(self):
from app.models.sift import BadgeModel
from app.models.relationships.talent import TalentChallengeModel
the_badges = BadgeModel.query\
.join(TalentChallengeModel, TalentChallengeModel.talent_id==BadgeModel.talent_id)\
.filter(TalentChallengeModel.challenge_id==self.id)\
.all()
return the_badges
Wrapping the query in a function got around the issues I was having with the name BadgeModel not being defined and not being able to be imported in the model otherwise. The #property decorator allows me to just reference this as challenge.badges later in the view.
However, I am still interested in understanding how to do this as a relationship. Some searching elsewhere led me to believe this would work:
badges = db.relationship('BadgeModel',
secondary="join(BadgeModel, TalentChallengeModel, BadgeModel.talent_id == TalentChallengeModel.talent_id)",
secondaryjoin="remote([id]) == foreign(TalentChallengeModel.challenge_id)",
primaryjoin="BadgeModel.talent_id == foreign(TalentChallengeModel.talent_id)",
viewonly=True,
)
Because of other unresolved issues in my application environment, I can't fully test this (e.g., adding this code breaks Flask-User in my site) but would like to know if this is correct syntax and if there is any disadvantage to this over the query-in-function solution.
Let's say I have the following structure (using Flask-SqlAlchemy):
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False, index=True)
# The following line throws an error at runtime.
variant = db.Column(db.Integer, nullable=False, index=True,
default=select(func.count(User.id)).where(User.name == self.name))
def __init__(self, name):
super(User, self).__init__()
self.name = name
#property
def clause(self):
return '/'.join([str(self.variant), self.name])
Problem is, "User is not defined." I would like to model a system with Users who may choose the same name but add a field to differentiate between users in a systemic way without using (thereby exposing) the "id" field.
Anyone know how to make a self-referential query to use to populate a default value?
The issue of the default not referring to User here is solved by just assigning "default" to the Column once User is available. However, that's not going to solve the problem here because "self" means nothing either, there is no User method being called so you can't just refer to "self". The challenge with this statement is that you want it to be rendered as an inline sub-SELECT but it still needs to know the in-memory value of ".name". So you have to assign that sub-SELECT per-object in some way.
The usual way people approach ORM-level INSERT defaults like this is usually by using a before_insert handler.
Another way that's worth pointing out is by creating a SQL level INSERT trigger. This is overall the most "traditional" approach, as here you need to have access to the row being inserted; triggers define a means of getting at the row values that are being inserted.
As far as using a default at the column level, you'd need to use a callable function as the default which can look at the current value of the row being inserted, but at the moment that means that your SELECT statement will not be rendered inline with the INSERT statement, you'd need to pre-execute the SELECT which is not really what we want here.
Anyway, the basic task of rendering a SQL expression into the INSERT while also having that SQL expression refer to some local per-object state is achieved by assigning that expression to the attribute, the ORM picks up on this at flush time. Below we do this in the constructor, but this can also occur inside of before_insert() as well:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, index=True)
variant = Column(Integer, nullable=False, index=True)
def __init__(self, name):
self.name = name
self.variant = select([func.count(User.id)]).where(User.name == self.name)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
s.add(User(name='n1'))
s.commit()
s.add(User(name='n1'))
s.commit()
print s.query(User.variant).all()