I am trying to update the order of an existing object when using the SQLAlchemy Ordering List. Here's the example model used in the SQLAlchemy docs:
class Slide(Base):
__tablename__ = 'slide'
id = Column(Integer, primary_key=True)
name = Column(String)
bullets = relationship("Bullet", order_by="Bullet.position",
collection_class=ordering_list('position'))
class Bullet(Base):
__tablename__ = 'bullet'
id = Column(Integer, primary_key=True)
slide_id = Column(Integer, ForeignKey('slide.id'))
position = Column(Integer)
text = Column(String)
I understand how to call append() with new objects and that works fine.
But how do I update the order of an existing object?
I have tried this and get some hard to understand results.
existing_item = session.query(Bullet).filter(Bullet.id == bullet_id).one()
slide.bullets.insert(new_order, existing_item)
slide.bullets.reorder()
session.commit()
There doesn't seem to be any documentation that explains how to update an existing object. Am I missing something?
I think I figured this out. I needed to call remove() on the object in question before re-inserting it.
Not sure if there's a better way, but this works.
existing_item = session.query(Bullet).filter(Bullet.id == bullet_id).one()
slide.bullets.remove(existing_item)
slide.bullets.insert(new_order, existing_item)
slide.bullets.reorder()
session.commit()
Also worth noting: I am using count_from=1 so that requires that I subtract 1 from new_order so that the end-user isn't confused by an order that starts at 0.
Related
Consider a many-to-many relationship of Things where the order of the children (or parents) matters. So the association object ("Link") has an additional property "position". When I fill the structure, I obviously can't use the "association-jumping" append() method because I need to explicitly access the Link object to set the "position" attribute. Sad but understandable.
What I'm struggling with now is to retrieve the children of a Thing in the order given by the Link's position property. Is that in any way possible? If not, the association_proxy is not of any use to me in either populating nor retrieving the data which makes me wonder what it is good for at all. I've tried to wrap my head around the use cases in the documentation but they don't seem to apply for my situation.
Here's a self-contained example:
from sqlalchemy import create_engine, CHAR, Column, Integer,\
String, ForeignKey
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Thing(Base):
__tablename__ = 'thing'
id = Column(Integer, primary_key=True)
name = Column(String(255))
up = association_proxy('coll_up', 'up')
down = association_proxy('coll_down', 'down')
class Link(Base):
__tablename__ = 'link'
id_up = Column(Integer, ForeignKey('thing.id'),
primary_key=True)
id_down = Column(Integer, ForeignKey('thing.id'),
primary_key=True)
position = Column(Integer)
up = relationship('Thing', primaryjoin=id_up == Thing.id,
backref='coll_down')
down = relationship('Thing', primaryjoin=id_down == Thing.id,
backref='coll_up')
if __name__ == '__main__':
# I know that for convenient append()ing stuff I'd need an __init__()
# method, but see comments below
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
if __name__ == '__main__':
db = Session()
root = Thing(name='Root thing')
db.add(root)
# inserting "reversed" to have something to sort
for i in reversed(range(5)):
t = Thing(name='Thing %d' % i)
# If Link had an __init__ method I could use root.append(t), but...
db.add(t)
# ...in order to set position, I need to explicitly use the association
# object anyway
l = Link(up=root, down=t, position=i)
db.add(l)
db.commit()
# OK, so far the association_proxy construct hasn't been of any use. Let's
# see about retrieving the data...
root = db.query(Thing).filter_by(name='Root thing').one()
for thing in root.down:
print(thing.name)
# This, as expected. prints the elements in the order in which they were
# inserted (which is "reversed"). How can I sort them by the "position"
# property of the "Link" element, or is the associaton object useless in
# this scenario?
I'd suggest investigating the order_by parameter of relationships. You can can order the ORM's organization of related object by the child's properties. (Ordering will not work without session.commit()).
Assuming you want to order by Link.position:
class Link(Base):
__tablename__ = 'link'
id_up = Column(Integer, ForeignKey('thing.id'), primary_key=True)
id_down = Column(Integer, ForeignKey('thing.id'), primary_key=True)
position = Column(Integer)
# Note the syntax for order_by -- the ORM model is referred to as a string
up = relationship(
'Thing', primaryjoin=id_up == Thing.id,
order_by='Link.position',
backref='coll_down'
)
down = relationship(
'Thing',
primaryjoin=id_down == Thing.id,
order_by='Link.position',
backref='coll_up'
)
If you need more rigid ordering, meaning, ordering before the child is committed to the session, you can define an orderinglist on the relationship which "intercept" and reorganize relationships in the ORM. (Ordering will work without session.commit()).
from sqlalchemy.ext.orderinglist import ordering_list
class Link(Base):
...
up = relationship(
'Thing',
primaryjoin=id_up == Thing.id,
order_by='Link.position',
collection_class=ordering_list('position')
backref='coll_down'
)
...
As a next step, if you're looking to organize your position based on ids (perhaps position = id_up - id_down), I'd suggest looking into sqlalchemy's event API. An elegant approach would be trigger recalculation of relevant attributes based on a relevant update. (Ex: id_up is increased by 1, increase position by 1).
I have three models (note that this is done in Flask-SQLAlchemy, but if you can only write an answer for vanilla SQLAlchemy, that is fine with me.) Irrelevant fields are removed for clarity.
class KPI(db.Model):
__tablename__ = 'kpis'
id = db.Column(db.Integer, primary_key=True)
identifier = db.Column(db.String(length=50))
class Report(db.Model):
__tablename__ = 'reports'
id = db.Column(db.Integer, primary_key=True)
class ReportKPI(db.Model):
report_id = db.Column(db.Integer, db.ForeignKey('reports.id'), primary_key=True)
kpi_id = db.Column(db.Integer, db.ForeignKey('kpis.id'), primary_key=True)
report = db.relationship('Report', backref=db.backref('values'))
kpi = db.relationship('KPI')
My goal is to find all Report objects that don’t measure a specific KPI (ie. there is no ReportKPI object whose KPI relationship has identifier set to a specific value).
One of my tries look like
Report.query \
.join(ReportKPI) \
.join(KPI) \
.filter(KPI.identifier != 'reflection')
but this gives back more Report objects that actually exist (I guess I get one for every ReportKPI that has a KPI with anything but “reflection”.)
Is the thing I want to achieve actually possible with SQLAlchemy? If so, what is the magic word (pleas doesn’t seem to work…)
An EXISTS subquery expression is a good fit for your goal. A shorthand way to write such a query would be:
Report.query.\
filter(db.not_(Report.values.any(
ReportKPI.kpi.has(identifier='reflection'))))
but this produces 2 nested EXISTS expressions, though a join in an EXISTS would do as well:
Report.query.\
filter(db.not_(
ReportKPI.query.
filter_by(report_id=Report.id).
join(KPI).
filter_by(identifier='reflection').
exists()))
Finally, a LEFT JOIN with an IS NULL is an option as well:
Report.query.\
outerjoin(db.join(ReportKPI, KPI),
db.and_(ReportKPI.report_id == Report.id,
KPI.identifier == 'reflection')).\
filter(KPI.id.is_(None))
I'd like to add a custom attribute "searchable" to the columns in an sqlalchemy model. The purpose is to extract data for just these columns (using ModelFoo.__ table__.columns) and put the data into solr. Is there a way I can mark certain columns using a custom attribute?
class ModelFoo(AppBase):
__tablename__ = 'foo'
id = Column("id", Integer, primary_key=True, autoincrement=True)
os = Column(String, nullable=False, searchable=True)
platform = Column(String, searchable=True)
by default, I get the following error when I try the above:
sqlalchemy.exc.ArgumentError: Unknown arguments passed to Column: ['searchable']
I am looking for a generic way to add only "searchable" columns to solr, something along these lines:
for table in Base.metadata.tables.values():
keys = [str(key) for key in table.columns if key.searchable]
solr.add(session.query(*keys).all())
in the above code, I am looking for some short solution or alternative to get "key.searchable" to work. Hope this clarifies the question.
I solved this using a separate attribute in model:
class ModelFoo(Base):
__tablename__ = 'foo'
id = Column("id", Integer, primary_key=True, autoincrement=True)
os = Column(String, nullable=False)
platform = Column(String)
search_cols = ["os", "value"]
for k, v in list(Base._decl_class_registry.items()):
if (hasattr(v, "search_cols")):
cols = [getattr(v, val) for val in v.search_cols]
query = sess.query(*cols)
solr.add(query.all())
I'm a little unclear on your question but it looks like your are trying to do something like this from the link you provided:
si.add(Book.objects.all())
Where Book.objects.all() is a list of records from an ORM mapped table. Note that the docs say a list of objects is also acceptable. I think to solution here is to use the SQLAlchemy query method to build records with only the fields you want. Using your example with would look like this:
si.add(session.query(ModelFoo.os, ModelFoo.platform).all())
Does adding sqlalchemy.orm.relationship to a model changes internal DB schema? Do I need to add a DB migration script if the only thing that has changed in the ORM mapping is the orm.relationship being added?
For example original mapping:
class Service(Base):
__tablename__ = 'service'
id = sql.Column(sql.String(64), primary_key=True)
type = sql.Column(sql.String(255))
extra = sql.Column(sql.Text())
class Endpoint(Base):
__tablename__ = 'endpoint'
id = sql.Column(sql.String(64), primary_key=True)
legacy_endpoint_id = sql.Column(sql.String(64))
interface = sql.Column(sql.String(8), primary_key=True)
region = sql.Column('region', sql.String(255))
service_id = sql.Column(sql.String(64),
sql.ForeignKey('service.id'),
nullable=False)
url = sql.Column(sql.Text())
extra = sql.Column(sql.Text())
Modified mapping:
## -3,6 +3,7 ##
id = sql.Column(sql.String(64), primary_key=True)
type = sql.Column(sql.String(255))
extra = sql.Column(sql.Text())
+ endpoints = sql.orm.relationship("Endpoint")
class Endpoint(Base):
## -16,4 +17,5 ##
nullable=False)
url = sql.Column(sql.Text())
extra = sql.Column(sql.Text())
+ service = sql.orm.relationship("Service")
With the change above, do I need to add DB migration script?
I am assuming it is a one to many relationship (service to endpoints)
No, you don't need to add anything to the migration script unless it's a many to many relationship. In that case an association table may be required, but then it'd (association table) already be part of your model.
http://docs.sqlalchemy.org/en/latest/orm/relationships.html#many-to-many
Also specifying the relationship only in one of the models is enough, for the other direction you can use backref option:
class Service(Base):
__tablename__ = 'service'
id = sql.Column(sql.String(64), primary_key=True)
type = sql.Column(sql.String(255))
extra = sql.Column(sql.Text())
endpoints = sql.orm.relationship("Endpoint", backref="service")
This is a two way relationship now.
Now .service on endpoint object will give you the corresponding service object. And .endpoints on service object will give you an array of endpoint objects. In case it's a one to one then you may specify uselist=False in relationship, then you'd get back a scalar instead of a list
So I'm quite new to SQLAlchemy.
I have a model Showing which has about 10,000 rows in the table. Here is the class:
class Showing(Base):
__tablename__ = "showings"
id = Column(Integer, primary_key=True)
time = Column(DateTime)
link = Column(String)
film_id = Column(Integer, ForeignKey('films.id'))
cinema_id = Column(Integer, ForeignKey('cinemas.id'))
def __eq__(self, other):
if self.time == other.time and self.cinema == other.cinema and self.film == other.film:
return True
else:
return False
Could anyone give me some guidance on the fastest way to insert a new showing if it doesn't exist already. I think it is slightly more complicated because a showing is only unique if the time, cinmea, and film are unique on a showing.
I currently have this code:
def AddShowings(self, showing_times, cinema, film):
all_showings = self.session.query(Showing).options(joinedload(Showing.cinema), joinedload(Showing.film)).all()
for showing_time in showing_times:
tmp_showing = Showing(time=showing_time[0], film=film, cinema=cinema, link=showing_time[1])
if tmp_showing not in all_showings:
self.session.add(tmp_showing)
self.session.commit()
all_showings.append(tmp_showing)
which works, but seems to be very slow. Any help is much appreciated.
If any such object is unique based on a combination of columns, you need to mark these as a composite primary key. Add the primary_key=True keyword parameter to each of these columns, dropping your id column altogether:
class Showing(Base):
__tablename__ = "showings"
time = Column(DateTime, primary_key=True)
link = Column(String)
film_id = Column(Integer, ForeignKey('films.id'), primary_key=True)
cinema_id = Column(Integer, ForeignKey('cinemas.id'), primary_key=True)
That way your database can handle these rows more efficiently (no need for an incrementing column), and SQLAlchemy now automatically knows if two instances of Showing are the same thing.
I believe you can then just merge your new Showing back into the session:
def AddShowings(self, showing_times, cinema, film):
for showing_time in showing_times:
self.session.merge(
Showing(time=showing_time[0], link=showing_time[1],
film=film, cinema=cinema)
)