what is the difference in SQLAlchemy between contains_eager and joinedload.
I read the manual about contains_eager, and manual about joinedload too. They are both can be used for loading one-to-many related rows, or many-to-one.
they are generating the same sql:
query = session.query(User).\
outerjoin(adalias, User.addresses).\
options(contains_eager(User.addresses, alias=adalias)).all()
...
SELECT
users.user_id AS users_user_id,
users.user_name AS users_user_name,
adalias.address_id AS adalias_address_id,
adalias.user_id AS adalias_user_id,
adalias.email_address AS adalias_email_address,
(...other columns...)
FROM users
LEFT OUTER JOIN email_addresses AS email_addresses_1
ON users.user_id = email_addresses_1.user_id
>>> jack = session.query(User).\
... options(joinedload(User.addresses)).\
... filter_by(name='jack').all()
SELECT
addresses_1.id AS addresses_1_id,
addresses_1.email_address AS addresses_1_email_address,
addresses_1.user_id AS addresses_1_user_id,
users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname,
users.password AS users_password
FROM users
LEFT OUTER JOIN addresses AS addresses_1
ON users.id = addresses_1.user_id
WHERE users.name = ?
['jack']
Can anybody show more particular code examples?
Don't forget many to many and one to one. The difference is that with contains_eager() you instruct SQLA about an existing join or joins that should be used to populate a relationship. This way you can also populate using a filtered subset.
joinedload() on the other hand goes to great lengths to make the joins required for populating entirely transparent and they should not affect the outcome of the original query, as is explained in The Zen of Joined Eager Loading. In other words you cannot use the joined relations for filtering etc.
Related
If I have an operator table and a user table, and I have more than one relationship to the user table within operator where pm is a relationship() that joins on operator.pm_id == user.id and sales is a relationship() that joins on operator.sales_id == user.id, how can I reference a username column from within the user table in a select statement, similar to the following:
stmt = select([operator, operator.pm.username.label('pm'), operator.sales.username.label('sales')])
This obviously doesn't work, but how can I accomplish something similar?
The core idea is this:
stmt = select([operator.name, user.name])
stmt = stmt.select_from(operator.join(user, operator.pm_id==user.id))
You can generate joins of columns from multiple tables by using the select_from statement in this way.
The last part where you want to add
operator.sales(different username from user table)
I still don't fully understand. But maybe you can use the scheme above to solve this too.
EDIT:
To make this explicit: You can chain this operation indefinitely, e.g.
stmt = stmt.select_from(table1.join(table2, table1.key==table2.key).join(table3, table2.key==table3.key))
etc..
I have a User that can have multiple Accounts. There can be multiple Units on each Account.
I build a filter dictionary and get relevant units:
units = Unit.objects.filter(**unit_filter)
However, I would also like to get distinct users. I can easily get their ids:
user_dicts = units.values('account__user').distinct()
or to be more exact:
user_ids = [rec.get('account__user') for rec in
units.values('account__user').distinct()]
So then I can filter Users using User.objects.filter(id__in=user_ids). (I can also use the generator expression instead of list comprehension, but that is not the point.)
I am not sure but evaluating id in seems to me not very efficient. Is there a better way how to get unique users from filtered units?
Edit:
I add SQL queries (did not test them, may be wrong) to make obvious what I am trying to do in Django ORM. Actually I am trying to use JOIN instead of WHERE clause.
I hope for this:
WITH selected_units AS
(SELECT id, account_id FROM units)
SELECT DISTINCT u.id FROM user u
JOIN account a on a.user_id=u.id
JOIN unit ut ON ut.account=a.id
JOIN selected_units s ON s.id=ut.id;
But with id_in I get this:
WITH selected_units AS
(SELECT id, account_id FROM units)
SELECT DISTINCT u.id FROM user u
JOIN account a on a.user_id=u.id
JOIN unit ut ON ut.account_id=a.id
WHERE ut.id IN (SELECT id FROM selected_units);
I think you can do it using subqueries, something like this:
User.objects.filter(account__unit__in=units)
Or less pretty:
User.objects.filter(id__in=units.values_list('account__user__id', flat=True))
I am working with a database that does not have relationships created between tables, and changing schema is not an option for me.
I'm trying to describe in orm how to join two tables without describing Foregin keys. To make make things worst I need a custom ON clause in my SQL
Here is my ORM(more or less):
class Table1(Base):
__tablename__ = "table1"
id1 = Column(String)
id2 = Column(String)
class Table2(Base):
__tablename__ = "table2"
id1 = Column(String)
id2 = Column(String)
Goal
What I'm trying to create is relationship that joins tables like this:
.....
FROM Table1
JOIN Table2 ON (Table1.id1 = Table2.id1 OR Table1.id2 = Table2.id2)
My Attempt
I tried adding following Table1 but documentation does not explain how is this wrong in terms I can understand:
table2 = relationship("Table2",
primaryjoin=or_(foreign(id1) == remote(Table2.id1),
foreign(id2) == remote(Table2.id2)))
But when tested this I got wrong SQL query back(I expected to see in SQL the join I described above):
str(query(Table1,Table2))
SELECT "table1".id1, "table1".id2, "table2".id1, "table2".id2
FROM "table1","table2"
Note
I don't really undersatnd what remote and foregin do but I tried to infer from documentation where do they belong, without then I would get error on import saying:
ArgumentError: Could not locate any relevant foreign key columns for primary join condition 'my full primaryjoin code' on relationship Table1.other_table. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation.
I don't think that I can use ForeignKey or ForeignKeyContraint because none of my colums are constraned to other table's values.
The expression
str(query(Table1,Table2))
produces a cross join between the 2 tables, as you've observed. This is the expected behaviour. If you want to use inner joins etc., you'll have to be explicit about it:
str(query(Table1, Table2).join(Table1.table2))
This joins along the relationship attribute table2. The attribute indicates how this join should happen.
Documentation on foreign() and remote() is a bit scattered to my own taste as well, but it is established in "Adjacency List Relationships" and "Non-relational Comparisons / Materialized Path" that when foreign and remote annotations are on different sides of the expression (in the ON clause), the relationship is considered to be many-to-one. When they are on the same side or remote is omitted it is considered one-to-many. So your relationship is considered to be many-to-one.
They are just an alternative to foreign_keys and remote_side parameters.
In a legacy database we have, there is a pretty special datastructure where we have two many-to-many relations joining the same two tables companies to paymentschedules.
There is a many-to-many relation using an association table called companies_paymentschedules, and a second many-to-many relation using an association table called companies_comp_paymentschedules.
Both relations serve different purposes.
companies_paymentschedules stores paymentschedules for which the company has a discount, companies_comp_paymentschedules stores paymentschedules that are linked to the company.
(I know that this could be simplified by replacing these tables with a single lookup table, but that is not an option in this legacy database.)
The problem is that I need to join both types of companies (discounted and linked) in the same query. SQLAlchemy joins both tables without problems, but it also joins the companies table, and calls them both "companies", which leads to a SQL syntax error (using MSSQL BTW).
This is the query:
q = Paymentschedules.query
# join companies if a company is specified
if company is not None:
q = q.join(Paymentschedules.companies)
q = q.join(Paymentschedules.companies_with_reduction)
The many-to-many relations are both defined in our companies model, and look like this:
paymentschedules_with_reduction = relationship("Paymentschedules", secondary=companies_paymentschedules, backref="companies_with_reduction")
paymentschedules = relationship("Paymentschedules", secondary=companies_comp_paymentschedules, backref="companies")
The problem is that the JOINS trigger SQLAlchemy to create a SQL statement that looks like this:
FROM paymentschedules
JOIN companies_comp_paymentschedules AS companies_comp_paymentschedules_1 ON paymentschedules.pmsd_id = companies_comp_paymentschedules_1.pmsd_id
JOIN companies ON companies.comp_id = companies_comp_paymentschedules_1.comp_id
JOIN companies_paymentschedules AS companies_paymentschedules_1 ON paymentschedules.pmsd_id = companies_paymentschedules_1.pmsd_id
JOIN companies ON companies.comp_id = companies_paymentschedules_1.comp_id
The two lookup tables have different names, but the related companies table is called "companies" in both cases, causing a SQL error:
[SQL Server Native Client 11.0][SQL Server]The objects "companies" and "companies" in the FROM clause have the same exposed names. Use correlation names to distinguish them. (1013) (SQLExecDirectW); ...]
I have been looking for a way to alias a join, or perhaps alias one of the relations from my lookup-tables to the companies table, but I was unable to do so.
Is there a way to alias a joined many-to-many table?
update:
Based on the suggestion by #IljaEverilä I found this: http://docs.sqlalchemy.org/en/latest/orm/query.html?highlight=onclause (see "Joins to a Target with an ON Clause") as a method to alias a joined table, but the example only shows how to alias a one-to-many type join. In my case I need to alias the other side of my lookup table, so I can't apply the example code to my situation.
I am using PeeWee with MySQL. I have two tables that need a full join to keep records from both left and right sides. MySQL doesn't support this directly, but I have used "Method 2" in this helpful artice - http://www.xaprb.com/blog/2006/05/26/how-to-write-full-outer-join-in-mysql/ to create a Full Join SQL statement that seems to work for my data.
It requires a "UNION ALL" of "LEFT OUTER JOIN" and "RIGHT OUTER JOIN", using an excusion of duplicate data in the 2nd result set.
I'm matching up backup-tape barcodes in the two tables.
SQL
SELECT * FROM mediarecall AS mr
LEFT OUTER JOIN media AS m ON mr.alternateCode = m.tapeLabel
UNION ALL
SELECT * FROM mediarecall AS mr
RIGHT OUTER JOIN media AS m ON mr.alternateCode = m.tapeLabel
WHERE mr.alternateCode IS NULL
However, when I come to bring this into my python script using PeeWee, I discovered that there doesn't seem to be a JOIN.RIGHT_OUTER to allow me to re-create this SQL. I have used plently of JOIN.LEFT_OUTER in the past, but this is the first time I have needed a Full Join.
I can make PeeWee work with a RawQuery(), of course, but I'd love to keep my code looking more elegant if I can.
Has anyone managed to re-create a Full Join with MySQL and PeeWee without resorting to RawQuery?
I had envisaged something like the following (which I know is invalid):-
left_media = (MediaRecall
.select()
.join(Media,JOIN.LEFT_OUTER,
on=(MediaRecall.alternateCode == Media.tapeLabel)
)
)
right_media = (MediaRecall
.select()
.join(Media,JOIN.RIGHT_OUTER,
on=(MediaRecall.alternateCode == Media.tapeLabel)
)
)
.where(MediaRecall.alternateCode >> None) # Exclude duplicates
all_media = ( left_media | right_media) # UNION of the 2 results, which I
# can then use .where(), etc on
You can add support for right outer:
from peewee import JOIN
JOIN['RIGHT_OUTER'] = 'RIGHT OUTER'
Then you can use JOIN.RIGHT_OUTER.