I'm using SQLAlchemy and I what I liked with Django ORM was the Manager I could implement to override the initial query of an object.
Is something like this exist in SQLAlchemy? I'd like to always exclude items that have "visible = False", when I do something like :
session.query(BlogPost).all()
Is it possible?
Thanks!
EDIT: the original version almost worked. The following version actually works.
It sounds like what you're trying to do is arrange for the query entity to be something other than SELECT table.* FROM table. In sqlalchemy, you can map any "selectable" to a class; There are some caveats, though; if the selectable is not a table, inserting data can be tricky. Something like this approaches a workable solution. You probably do want to have a regular table mapped to permit inserts, so the first part is a totally normal table, class and mapper.
blog_post_table = Table("blog_posts", metadata,
Column('id', Integer, primary_key=True),
Column('visible', Boolean, default=True),
...
)
class BlogPost(object):
pass
blog_post_mapper = mapper(BlogPost, blog_post_table)
Or, if you were using the declarative extension, it'll all be one
class BlogPost(Base):
__tablename__ = 'blog_posts'
id = Column(Integer, primary_key=True)
visible = Column(Boolean, default=True)
Now, we need a select expression to represent the visible posts.
visible_blog_posts_expr = sqlalchemy.sql.select(
[BlogPost.id,
BlogPost.visible]) \
.where(BlogPost.visible == True) \
.alias()
Or, since naming all of the columns of the desired query is tedious (not to mention in violation of DRY), you can use the same construct as session.query(BlogPost) and extract the 'statement'. You don't actually want it bound to a session, though, so call the class directly.
visible_blog_posts_expr = \
sqlalchemy.orm.Query(BlogPost) \
.filter(BlogPost.visible == True) \
.statement \
.alias()
And we map that too.
visible_blog_posts = mapper(BlogPost, visible_blog_posts_expr, non_primary=True)
You can then use the visible_blog_posts mapper instead of BlogPosts with Session.query, and you will still get BlogPost, which can be updated and saved as normal.
posts = session.query(visible_blog_posts).all()
assert all(post.visible for post in posts)
For this particular example, there's not much difference between explicit mapper use and declarative extension, you still must call mapper for the non-primary mappings. At best, it allows you to type SomeClass.colname instead of some_table.c.colname (or SomeClass.__table__.colname, or BlogPost.metadata.tables[BlogPost.__tablename__] or ... and so on).
The mistakes I made in the original example, which are now corrected. I was missing some missing []'s in the call to sqlalchemy.sql.select, which expects the columns to be in a sequence. when using a select statement to mapper, sqlalchemy insists that the statement be aliased, so that it can be named (SELECT .... ) AS some_subselect_alias_5
You can do e.g.
session.query(BlogPost).filter_by(visible=True)
which should give you just the posts you need.
Related
I haven't been able to find an answer to this, but I'm sure it must be somewhere.
My question is similar to this question: sqlalchemy: how to join several tables by one query?
But I need a query result, not a tuple. I don't have access to the models, so I can't change it, and I can't modify the functions to use a tuple.
I have two tables, UserInformation and MemberInformation, both with a foreign key and relationship to Principal, but not to each other.
How can I get all the records and columns from both tables in one query?
I've tried:
query = DBSession.query(MemberInformation).join(UserInformation, MemberInformation.pId == UserInformation.pId)
but it only returns the columns of MemberInformation
and:
query = DBSession.query(MemberInformation, UserInformation).join(UserInformation, MemberInformation.pId == UserInformation.pId)
but that returns a tuple.
What am I missing here?
Old question, but worth answering because i see it's got a lot of view activity.
You need to create a relationship and then tell SQLAlchemy how to load the related data. Not sure what your tables / relationship looks like, but it might look something like this:
# Create relationship
MemberInformation.user = relationship(
"UserInformation",
foreign_keys=[MemberInformation.pId],
lazy="joined",
)
# Execute query
query = DBSession.query(MemberInformation) \
.options(joinedload(MemberInformation.user)) \
.all()
# All objects are in memory. Evaluating the following will NOT result in additional
# database interaction
for member in query:
print(f'Member: {member} User: {member.user}')
# member is a MemberInformation object, member.user is a UserInformation object
Ideally, the relationship would be defined in your models. If can, however, be defined at run time list the example above.
Only way I found to do this is to use statement instead of query:
stmt = select([table1, table2.col.label('table2_col')]).select_from(join(table1, table2, table1.t1_id == table2.t2_id))
obj = session.execute(stmt).fetchall()
Suppose I have a one-to-many relationship like this:
class Book(Base):
__tablename__ = "books"
id = Column(Integer)
...
library_id = Column(Integer, ForeignKey("libraries.id"))
class Library(Base):
__tablename__ = "books"
id = Column(Integer)
...
books = relationship(Book, backref="library")
Now, if I have an ID of a book, is there a way to retrieve it from the Library.books relationship, "get me a book with id=10 in this particular library"? Something like:
try:
the_book = some_library.books.by_primary_key(10)
except SomeException:
print "The book with id 10 is not found in this particular library"
Workarounds I can think of (but which I'd rather avoid using):
book = session.query(Book).get(10)
if book and book.library_id != library_id:
raise SomeException("The book with id 10 is not found in this particular library")
or
book = session.query(Book).filter(Book.id==10).filter(Book.library_id=library.id).one()
Reason: imagine there are several different relationships (scifi_books, books_on_loan etc.) which specify different primaryjoin conditions - manually querying would require writing individual queries for all of them, while SQLAlchemy already knows how to retrieve items for that relationship. Also, I'd prefer to load the books all at once (by accessing library.books) than issuing individual queries.
Another option, which works but is inefficient and inelegant is:
for b in library.books:
if b.id == book_id:
return b
What I'm currently using is:
library_books = {b.id:b for b in library.books}
for data in list_of_dicts_containing_book_id:
if data['id'] in library_books:
library_books[data['id']].do_something(data)
else:
print "Book %s is not in the library" % data['id']
I just hope there's a nicer built-in way of quickly retrieving items from a relationship by their id
UPD: I've asked the question in the sqlalchemy mail list.
SQLAlchemy's query object has with_parent method which does exactly that:
with_parent(instance, property=None)
Add filtering criterion that relates the given instance to a child object or collection, using its attribute state as well as an established relationship() configuration.
so in my example the code would look like
q = session.query(Book)
q = q.with_parent(my_library, "scifi_books")
q = q.filter(Book.id==10).one()
This will issue a separate query though, even if the my_library.scifi_books relation is already loaded. There seems to be no "built-in" way to retrieve an item from an already-loaded relation by its PK, so the easiest is to just convert the relation to a dict and use that to look up items:
book_lookup = {b.id: b for b in my_library.scifi_books}
book = books_lookup[10]
See SQLAlchemy docs on querying with joins. So you want something like this (be aware that this is untested):
query(Book, Library). \
filter(Book.id==10). \
filter(Book.library.id==needed_library_id).all()
If Book -> Library reference would be scalar, you could use has():
query.filter(Library.books.has(id=10))
To make batch queries for multiple books at once, you can use in_() operator:
query(Library).join('books', Book).filter(Book.id.in_([1, 2, 10])).all()
I'm not sure what this is called since it is new to me, but here is what I want to do:
I have two tables in my database: TableA and TableB. TableA has pk a_id and another field called a_code. TableB has pk b_id and another field called b_code.
I have these tables mapped in my sqlalchemy code and they work fine. I want to create a third object called TableC that doesn't actually exist in my database, but that contains combinations of a_code and b_code, something like this:
class TableC:
a_code = String
b_code = String
Then I'd like to query TableC like:
TableC.query.filter(and_(
TableC.a_code == x,
TableC.b_code == y)).all()
Question 1) Does this type of thing have a name? 2) How do I do the mapping (using declarative would be nice)?
I don't really have a complete understanding of the query you are trying to express, weather it's a union or a join or some third thing, but that aside, it certainly is possible to map an arbitrary selectable (anything you can pass to a database that returns rows).
I'll start with the assumption that you want some kind of union of TableA and TableB, which would be all of the rows in A, and also all of the rows in B. This is easy enough to change to a different concept if you reveal more information about the shape of the data you are expressing.
We'll start by setting up the real tables, and classes to map them, in the declarative style.
from sqlalchemy import *
import sqlalchemy.ext.declarative
Base = sqlalchemy.ext.declarative.declarative_base()
class TableA(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
a_code = Column(String)
class TableB(Base):
__tablename__ = 'b'
id = Column(Integer, primary_key=True)
b_code = Column(String)
Since we've used declarative, we don't actually have table instances to work from, which is neccesary for the next part. There are many ways to access the tables, but the way I prefer is to use sqlalchemy mapping introspection methods, since that will work no matter how the class was mapped.
from sqlalchemy.orm.attributes import manager_of_class
a_table = manager_of_class(TableA).mapper.mapped_table
b_table = manager_of_class(TableB).mapper.mapped_table
Next, we need an actual sql expression that represents the data we are interested in.
This is a union, which results in columns that look the same as the columns defined in the first class, id and a_code. We could rename it, but that's not a very important part of the example.
ab_view_sel = sqlalchemy.alias(a_table.select().union(b_table.select()))
Finally, we map a class to this. It is possible to use declarative for this, but it's actually more code to do it that way instead of classic mapping style, not less. Notice that the class inherits from object, not base
class ViewAB(object):
pass
sqlalchemy.orm.mapper(ViewAB, ab_view_sel)
And that's pretty much it. Of course there are some limitations with this; the most obvious being there's no (trivial) way to save instances of ViewAB back to the database.
There isn't really a concept of 'virtual tables', but it is possible to send a single query that 'joins' the data from multiple tables. This is probably as close as you can get to what you want.
For example, one way to do this in sqlalchemy/elixir would be (and this isn't far off from what you've shown, we're just not querying a 'virtual' table):
result = session.query(TableA, TableB).filter(TableA.a_code==x).filter(TableB.b_code==y).all()
This is similar to an SQL inner join, with some qualifying conditions in the filter statements. This isn't going to give you an sqlalchemy table object, but will give you a list of objects from each real table.
It looks like SQLAlchemy allows you to map an arbitrary query to a class. e.g. From SQLAlchemy: one classes – two tables:
usersaddresses = sql.join(t_users, t_addresses,
t_users.c.id == t_addresses.c.user_id)
class UserAddress(object):
def __repr__(self):
return "<FullUser(%s,%s,%s)" % (self.id, self.name, self.address)
mapper(UserAddress, usersaddresses, properties={
'id': [t_users.c.id, t_addresses.c.user_id],
})
f = session.query(UserAddress).filter_by(name='Hagar').one()
I'm using SA 0.6.6, Declarative style, against Postgres 8.3, to map Python objects to a database. I have a table that is self referencing and I'm trying to make a relationship property for it's children. No matter what I try, I end up with a NoReferencedTableError.
My code looks exactly like the sample code from the SA website for how to do this very thing.
Here's the class.
class FilterFolder(Base):
__tablename__ = 'FilterFolder'
id = Column(Integer,primary_key=True)
name = Column(String)
isShared = Column(Boolean,default=False)
isGlobal = Column(Boolean,default=False)
parentFolderId = Column(Integer,ForeignKey('FilterFolder.id'))
childFolders = relationship("FilterFolder",
backref=backref('parentFolder', remote_side=id)
)
Here's the error I get:
NoReferencedTableError: Foreign key assocated with column 'FilterFolder.parentFolderId' could not find table 'FilterFolder' with which to generate a foreign key to target column 'id'
Any ideas what I'm doing wrong here?
This was a foolish mistake on my part. I typically specify my FK's by specifying the Entity type, not the string. I am using different schemas, so when defining the FK entity as a string I also need the schema.
Broken:
parentFolderId = Column(Integer,ForeignKey('FilterFolder.id'))
Fixed:
parentFolderId = Column(Integer,ForeignKey('SchemaName.FilterFolder.id'))
I checked your code with SQLAlchemy 0.6.6 and sqlite. I was able to create the tables, add a parent and child combination, and retrieve them again using a session.query.
As far as I can tell, the exception you mentioned (NoReferencedTableError) is thrown in schema.py (in the SQLAlchemy source) exclusively, and is not database specific.
Some questions: Do you see the same bug if you use an sqlite URL instead of the Postgres one? How are you creating your schema? Mine looks something like this:
engine = create_engine(db_url)
FilterFolder.metadata.create_all(self.dbengine)
I'm using SQLAlchemy declarative base to define my model. I defined a property name that is computed from one the columns (title):
class Entry(Base):
__tablename__ = "blog_entry"
id = Column(Integer, primary_key=True)
title = Column(Unicode(255))
...
#property
def name(self):
return re.sub(r'[^a-zA-Z0-9 ]','',self.title).replace(' ','-').lower()
When trying to perform a query using name, SQLAlchemy throws an error:
Session.query(Entry).filter(Entry.name == my_name).first()
>>> ArgumentError: filter() argument must be of type sqlalchemy.sql.ClauseElement or string
After investigating for a while, I found that maybe comparable_using() could help, but I couldn't find any example that shows a comparator that references another column of the table.
Is this even possible or is there a better approach?
From SqlAlchemy 0.7 you can achieve this using hybrid_property
see the docs here: http://www.sqlalchemy.org/docs/orm/extensions/hybrid.html
Can you imagine what SQL should be issued for your query? The database knows nothing about name, it has neither a way to calculate it, nor to use any index to speed up the search.
My best bet is a full scan, fetching title for every record, calculating name then filtering by it. You can rawly do it by [x for x in Session.query(Entry).all() if x.name==my_name][0]. With a bit more of sophistication, you'll only fetch id and title in the filtering pass, and then fetch the full record(s) by id.
Note that a full scan is usually not nice from performance POV, unless your table is quite small.