I have a many-to-many relationship, with instances of OsmAdminUnit (polygon geometries) grouped into OsmAdminAgg instances.
The model definitions are essentially:
class OsmAdminUnit(db.Model):
__tablename__ = 'osm_admin'
id = db.Column(db.Integer, primary_key=True)
geometry = db.Column(Geometry(
geometry_type='GEOMETRY',
srid=3857), nullable=False)
agg_units = db.relationship('OsmAdminAgg',
secondary=aggregations,
backref=db.backref('osm_admin', lazy='dynamic'))
class OsmAdminAgg(db.Model):
__tablename__ = 'admin_agg'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), unique=True, nullable=False)
Now what I am struggling to do is selecting OsmAdminUnits that belong to a certain OsmAdminAgg AND getting the polgyons merged by applying ST_Union from GeoAlchemy.
Selecting all admin units that belong to admin agg with id=1 works:
units = OsmAdminUnit.query.filter(OsmAdminUnit.agg_units.any(id=1)).all()
But I don't get how I can apply ST_Union on that result.
My approach so far was:
union = db.session.query(
OsmAdminUnit.geometry.ST_Union().ST_AsGeoJSON().label('agg_union')
).filter(OsmAdminUnit.agg_units.any(id=1)).subquery()
So how do I get the union of these geometries, and get it as GeoJSON?
Btw, I am building this on top of Flask, using SQLAlchemy, Flask-SQLAlchemy, Geoalchemy2.
Try this:
from sqlalchemy.sql.functions import func
union = db.session.query(func.ST_AsGeoJSON(func.ST_Union(
OsmAdminUnit.geometry)).label('agg_union')
).filter(OsmAdminUnit.agg_units.any(id=1)).subquery()
You can see a basic template for this in the GeoAlchemy 2 docs. Essentially, you need to pass func to the query, rather than the model, to select the union itself.
In your case, something like:
import sqlalchemy
union = db.session.query(
sqlalchemy.func.ST_AsGeoJSON(
sqlalchemy.func.ST_Union(OsmAdminUnit.geometry)
).label('agg_union')
).filter(
OsmAdminUnit.agg_units.any(id=1)
).all()
This grabs the union of geometry values for OsmAdminUnit records matching the filter, and returns it as stringified GeoJSON.
The accepted answer didn't work for me, I think the import may be different in the version of sqlalchemy I'm using.
Related
I have got a not very common join and filter problem.
Here are my models;
class Order(Base):
id = Column(Integer, primary_key=True)
order_id = Column(String(19), nullable=False)
... (other fields)
class Discard(Base):
id = Column(Integer, primary_key=True)
order_id = Column(String(19), nullable=False)
I want to query all and full instances of Order but just exclude those that have a match in Discard.order_id based on Order.order_id field. As you can see there is no relationship between order_id fields.
I've tried outer left join, notin_ but ended up with no success.
With this answer I've achieved desired results.
Here is my code;
orders = (
session.query(Order)
.outerjoin(Discard, Order.order_id == Discard.order_id)
.filter(Discard.order_id == None) # noqa: E711
.all()
)
I was paying too much attention to flake8 wrong syntax message at Discard.order_id == None and was using Discard.order_id is None. It appeared out they were rendered differently by sqlalchemy.
I have two tables, Products and Orders, inside my Flask-SqlAlchemy setup, and they are linked so an order can have several products:
class Products(db.Model):
id = db.Column(db.Integer, primary_key=True)
....
class Orders(db.Model):
guid = db.Column(db.String(36), default=generate_uuid, primary_key=True)
products = db.relationship(
"Products", secondary=order_products_table, backref="orders")
....
linked via:
order_products_table = db.Table("order_products_table",
db.Column('orders_guid', db.String(36), db.ForeignKey('orders.guid')),
db.Column('products_id', db.Integer, db.ForeignKey('products.id'))
# db.Column('license', dbString(36))
)
For my purposes, each product in an order will receive a unique license string, which logically should be added to the order_products_table rows of each product in an order.
How do I declare this third license column on the join table order_products_table so it gets populated it as I insert an Order?
I've since found the documentation for the Association Object from the SQLAlchemy docs, which allows for exactly this expansion to the join table.
Updated setup:
# Instead of a table, provide a model for the JOIN table with additional fields
# and explicit keys and back_populates:
class OrderProducts(db.Model):
__tablename__ = 'order_products_table'
orders_guid = db.Column(db.String(36), db.ForeignKey(
'orders.guid'), primary_key=True)
products_id = db.Column(db.Integer, db.ForeignKey(
'products.id'), primary_key=True)
order = db.relationship("Orders", back_populates="products")
products = db.relationship("Products", back_populates="order")
licenses = db.Column(db.String(36), nullable=False)
class Products(db.Model):
id = db.Column(db.Integer, primary_key=True)
order = db.relationship(OrderProducts, back_populates="order")
....
class Orders(db.Model):
guid = db.Column(db.String(36), default=generate_uuid, primary_key=True)
products = db.relationship(OrderProducts, back_populates="products")
....
What is really tricky (but also shown on the documentation page), is how you insert the data. In my case it goes something like this:
o = Orders(...) # insert other data
for id in products:
# Create OrderProducts join rows with the extra data, e.g. licenses
join = OrderProducts(licenses="Foo")
# To the JOIN add the products
join.products = Products.query.get(id)
# Add the populated JOIN as the Order products
o.products.append(join)
# Finally commit to database
db.session.add(o)
db.session.commit()
I was at first trying to populate the Order.products (or o.products in the example code) directly, which will give you an error about using a Products class when it expects a OrderProducts class.
I also struggled with the whole field naming and referencing of the back_populates. Again, the example above and on the docs show this. Note the pluralization is entirely to do with how you want your fields named.
I have three models (note that this is done in Flask-SQLAlchemy, but if you can only write an answer for vanilla SQLAlchemy, that is fine with me.) Irrelevant fields are removed for clarity.
class KPI(db.Model):
__tablename__ = 'kpis'
id = db.Column(db.Integer, primary_key=True)
identifier = db.Column(db.String(length=50))
class Report(db.Model):
__tablename__ = 'reports'
id = db.Column(db.Integer, primary_key=True)
class ReportKPI(db.Model):
report_id = db.Column(db.Integer, db.ForeignKey('reports.id'), primary_key=True)
kpi_id = db.Column(db.Integer, db.ForeignKey('kpis.id'), primary_key=True)
report = db.relationship('Report', backref=db.backref('values'))
kpi = db.relationship('KPI')
My goal is to find all Report objects that don’t measure a specific KPI (ie. there is no ReportKPI object whose KPI relationship has identifier set to a specific value).
One of my tries look like
Report.query \
.join(ReportKPI) \
.join(KPI) \
.filter(KPI.identifier != 'reflection')
but this gives back more Report objects that actually exist (I guess I get one for every ReportKPI that has a KPI with anything but “reflection”.)
Is the thing I want to achieve actually possible with SQLAlchemy? If so, what is the magic word (pleas doesn’t seem to work…)
An EXISTS subquery expression is a good fit for your goal. A shorthand way to write such a query would be:
Report.query.\
filter(db.not_(Report.values.any(
ReportKPI.kpi.has(identifier='reflection'))))
but this produces 2 nested EXISTS expressions, though a join in an EXISTS would do as well:
Report.query.\
filter(db.not_(
ReportKPI.query.
filter_by(report_id=Report.id).
join(KPI).
filter_by(identifier='reflection').
exists()))
Finally, a LEFT JOIN with an IS NULL is an option as well:
Report.query.\
outerjoin(db.join(ReportKPI, KPI),
db.and_(ReportKPI.report_id == Report.id,
KPI.identifier == 'reflection')).\
filter(KPI.id.is_(None))
I am creating a website using Flask and SQLAlchemy. This website keeps track of classes that a student has taken. I would like to find a way to search my database using SQLAlchemy to find all unique classes that have been entered. Here is code from my models.py for Class:
class Class(db.Model):
__tablename__ = 'classes'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(100))
body = db.Column(db.Text)
created = db.Column(db.DateTime, default=datetime.datetime.now)
user_email = db.Column(db.String(100), db.ForeignKey(User.email))
user = db.relationship(User)
In other words, I would like to get all unique values from the title column and pass that to my views.py.
Using the model query structure you could do this
Class.query.with_entities(Class.title).distinct()
query = session.query(Class.title.distinct().label("title"))
titles = [row.title for row in query.all()]
titles = [r.title for r in session.query(Class.title).distinct()]
As #van has pointed out, what you are looking for is:
session.query(your_table.column1.distinct()).all(); #SELECT DISTINCT(column1) FROM your_table
but I will add that in most cases, you are also looking to add another filter on the results. In which case you can do
session.query(your_table.column1.distinct()).filter_by(column2 = 'some_column2_value').all();
which translates to sql
SELECT DISTINCT(column1) FROM your_table WHERE column2 = 'some_column2_value';
I'd like to add a custom attribute "searchable" to the columns in an sqlalchemy model. The purpose is to extract data for just these columns (using ModelFoo.__ table__.columns) and put the data into solr. Is there a way I can mark certain columns using a custom attribute?
class ModelFoo(AppBase):
__tablename__ = 'foo'
id = Column("id", Integer, primary_key=True, autoincrement=True)
os = Column(String, nullable=False, searchable=True)
platform = Column(String, searchable=True)
by default, I get the following error when I try the above:
sqlalchemy.exc.ArgumentError: Unknown arguments passed to Column: ['searchable']
I am looking for a generic way to add only "searchable" columns to solr, something along these lines:
for table in Base.metadata.tables.values():
keys = [str(key) for key in table.columns if key.searchable]
solr.add(session.query(*keys).all())
in the above code, I am looking for some short solution or alternative to get "key.searchable" to work. Hope this clarifies the question.
I solved this using a separate attribute in model:
class ModelFoo(Base):
__tablename__ = 'foo'
id = Column("id", Integer, primary_key=True, autoincrement=True)
os = Column(String, nullable=False)
platform = Column(String)
search_cols = ["os", "value"]
for k, v in list(Base._decl_class_registry.items()):
if (hasattr(v, "search_cols")):
cols = [getattr(v, val) for val in v.search_cols]
query = sess.query(*cols)
solr.add(query.all())
I'm a little unclear on your question but it looks like your are trying to do something like this from the link you provided:
si.add(Book.objects.all())
Where Book.objects.all() is a list of records from an ORM mapped table. Note that the docs say a list of objects is also acceptable. I think to solution here is to use the SQLAlchemy query method to build records with only the fields you want. Using your example with would look like this:
si.add(session.query(ModelFoo.os, ModelFoo.platform).all())