From: StartTable.objects.annotate(name=F('object_type_2__destination_table__name'))
Django writes a query containing this automatically:
LEFT OUTER JOIN "object" T4 ON ("start_table"."object_type_2_id" = T4."id")
LEFT OUTER JOIN "destination_table" ON (T4."id" = "destination_table"."object_id")
Is there a way to have Django make this more efficient by writing this instead?:
JOIN destination_table ON destination_table.object_id = start_table.object_type_2_id
Some context to keep in mind; the start_table has several foreign key fields that all refer to the same object table, but for different reasons, which is why I've given object_type_2_id as the column name.
Related
I am wanting to map a class object to a table that is a join between two tables, and all the columns from one table and only one column from the joined table being selected (mapped).
join_table = join(table1, table2, tabl1.c.description==table2.c.description)
model_table_join= select([table1, table2.c.description]).select_from(join_table).alias()
Am I doing this right?
If all you want to do is pull in one extra column from a JOIN, I'd not muck about with an arbitrary select mapping. As the documentation points out:
The practice of mapping to arbitrary SELECT statements, especially complex ones as above, is almost never needed; it necessarily tends to produce complex queries which are often less efficient than that which would be produced by direct query construction. The practice is to some degree based on the very early history of SQLAlchemy where the mapper() construct was meant to represent the primary querying interface; in modern usage, the Query object can be used to construct virtually any SELECT statement, including complex composites, and should be favored over the “map-to-selectable” approach.
You'd just either select that extra column in your application:
session.query(Table1Model, Table2Model.description).join(Table2Model)
or you can register a relationship on the Table1Model and an association property that always pulls in the extra column:
class Table1Model(Base):
# ...
_table2 = relationship('Table2Model', lazy='join')
description = association_proxy('_table2', 'description')
The association property manages the Table2Model.description column of the joined row as you interact with it on Table1Model instances.
That said, if you must stick with a join() query as the base, then you could just exclude the extra, duplicated columns from the join, with a exclude_properties mapper argument:
join_table = join(table1, table2, table1.c.description == table2.c.description)
class JoinedTableModel(Base):
__table__ = join_table
__mapper_args__ = {
'exclude_properties' : [table1.c.description]
}
The new model then uses all the columns from the join to create attributes with the same names, except for those listed in `exclude_properties.
Or you can keep using duplicated column names in the model simply by giving them a new name:
join_table = join(table1, table2, table1.c.description == table2.c.description)
class JoinedTableModel(Base):
__table__ = join_table
table1_description = table1.c.description
You can rename any column from the join this way, at which point they will no longer conflict with other columns with the same base name from the other table.
I am working with a database that does not have relationships created between tables, and changing schema is not an option for me.
I'm trying to describe in orm how to join two tables without describing Foregin keys. To make make things worst I need a custom ON clause in my SQL
Here is my ORM(more or less):
class Table1(Base):
__tablename__ = "table1"
id1 = Column(String)
id2 = Column(String)
class Table2(Base):
__tablename__ = "table2"
id1 = Column(String)
id2 = Column(String)
Goal
What I'm trying to create is relationship that joins tables like this:
.....
FROM Table1
JOIN Table2 ON (Table1.id1 = Table2.id1 OR Table1.id2 = Table2.id2)
My Attempt
I tried adding following Table1 but documentation does not explain how is this wrong in terms I can understand:
table2 = relationship("Table2",
primaryjoin=or_(foreign(id1) == remote(Table2.id1),
foreign(id2) == remote(Table2.id2)))
But when tested this I got wrong SQL query back(I expected to see in SQL the join I described above):
str(query(Table1,Table2))
SELECT "table1".id1, "table1".id2, "table2".id1, "table2".id2
FROM "table1","table2"
Note
I don't really undersatnd what remote and foregin do but I tried to infer from documentation where do they belong, without then I would get error on import saying:
ArgumentError: Could not locate any relevant foreign key columns for primary join condition 'my full primaryjoin code' on relationship Table1.other_table. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation.
I don't think that I can use ForeignKey or ForeignKeyContraint because none of my colums are constraned to other table's values.
The expression
str(query(Table1,Table2))
produces a cross join between the 2 tables, as you've observed. This is the expected behaviour. If you want to use inner joins etc., you'll have to be explicit about it:
str(query(Table1, Table2).join(Table1.table2))
This joins along the relationship attribute table2. The attribute indicates how this join should happen.
Documentation on foreign() and remote() is a bit scattered to my own taste as well, but it is established in "Adjacency List Relationships" and "Non-relational Comparisons / Materialized Path" that when foreign and remote annotations are on different sides of the expression (in the ON clause), the relationship is considered to be many-to-one. When they are on the same side or remote is omitted it is considered one-to-many. So your relationship is considered to be many-to-one.
They are just an alternative to foreign_keys and remote_side parameters.
In a legacy database we have, there is a pretty special datastructure where we have two many-to-many relations joining the same two tables companies to paymentschedules.
There is a many-to-many relation using an association table called companies_paymentschedules, and a second many-to-many relation using an association table called companies_comp_paymentschedules.
Both relations serve different purposes.
companies_paymentschedules stores paymentschedules for which the company has a discount, companies_comp_paymentschedules stores paymentschedules that are linked to the company.
(I know that this could be simplified by replacing these tables with a single lookup table, but that is not an option in this legacy database.)
The problem is that I need to join both types of companies (discounted and linked) in the same query. SQLAlchemy joins both tables without problems, but it also joins the companies table, and calls them both "companies", which leads to a SQL syntax error (using MSSQL BTW).
This is the query:
q = Paymentschedules.query
# join companies if a company is specified
if company is not None:
q = q.join(Paymentschedules.companies)
q = q.join(Paymentschedules.companies_with_reduction)
The many-to-many relations are both defined in our companies model, and look like this:
paymentschedules_with_reduction = relationship("Paymentschedules", secondary=companies_paymentschedules, backref="companies_with_reduction")
paymentschedules = relationship("Paymentschedules", secondary=companies_comp_paymentschedules, backref="companies")
The problem is that the JOINS trigger SQLAlchemy to create a SQL statement that looks like this:
FROM paymentschedules
JOIN companies_comp_paymentschedules AS companies_comp_paymentschedules_1 ON paymentschedules.pmsd_id = companies_comp_paymentschedules_1.pmsd_id
JOIN companies ON companies.comp_id = companies_comp_paymentschedules_1.comp_id
JOIN companies_paymentschedules AS companies_paymentschedules_1 ON paymentschedules.pmsd_id = companies_paymentschedules_1.pmsd_id
JOIN companies ON companies.comp_id = companies_paymentschedules_1.comp_id
The two lookup tables have different names, but the related companies table is called "companies" in both cases, causing a SQL error:
[SQL Server Native Client 11.0][SQL Server]The objects "companies" and "companies" in the FROM clause have the same exposed names. Use correlation names to distinguish them. (1013) (SQLExecDirectW); ...]
I have been looking for a way to alias a join, or perhaps alias one of the relations from my lookup-tables to the companies table, but I was unable to do so.
Is there a way to alias a joined many-to-many table?
update:
Based on the suggestion by #IljaEverilä I found this: http://docs.sqlalchemy.org/en/latest/orm/query.html?highlight=onclause (see "Joins to a Target with an ON Clause") as a method to alias a joined table, but the example only shows how to alias a one-to-many type join. In my case I need to alias the other side of my lookup table, so I can't apply the example code to my situation.
I am trying to query all objects in a table without backreferences from another model.
class A(models.Model):
pass
class B(models.Model):
reference = models.ForeignKey(A)
In order to get all A objects with no references from any B objects, I do
A.objects.filter(b__isnull=True)
The Django documentation on isnull does not mention backreferences at all.
Can I get into trouble with this or is it just poorly documented?
I tried this with Django 1.10.3, using your same code example.
Let's take a look at the raw SQL statement that Django creates:
>>> print(A.objects.filter(b__isnull=True).query)
SELECT "backrefs_a"."id" FROM "backrefs_a" LEFT OUTER JOIN "backrefs_b" ON ("backrefs_a"."id" = "backrefs_b"."reference_id") WHERE "backrefs_b"."id" IS NULL
Technically, this isn't the exact same SQL that Django will send to the database, but it's close enough. For more discussion, see https://stackoverflow.com/a/1074224/5044893
If we pretty it up a bit, you can see that the query is actually quite safe:
SELECT "backrefs_a"."id"
FROM "backrefs_a"
LEFT OUTER JOIN "backrefs_b" ON ("backrefs_a"."id" = "backrefs_b"."reference_id")
WHERE "backrefs_b"."id" IS NULL
The LEFT in the LEFT OUTER JOIN ensures that you will get records from A, even when there are no matching records in B. And, because Django won't save a B with a NULL id, you can rest assured that it will work as you expect.
Situation: I have two tables, say 'master' and 'detail', where 'master' has two columns that refer to 'detail': 'foo_id', 'bar_id'. That is, I need to join detail twice with different names. I want to do:
SELECT master.id, foo.name, bar.name, other stuff ...
FROM master
JOIN detail AS foo ON foo.id = master.foo_id
JOIN detail AS bar ON bar.id = master.bar_id
how do I do that using SQLAlchemy?
Note that I am not using ORM. Also I am refering to database objects from metadata (strings), therefore I do: table.c["foo_id"] instead of table.c.foo_id (if this information is going to be relevant to the statement construction).
Fighting for quite a long time with the problem, i've solved it couple of minutes after posting my question here. The solution was to store and then reuse all aliased tables. Before I was using alias in join and then same alias for table to fetch column reference in SELECT, which resulted in redundant and broken 'FROM detail AS foo, detail AS bar, master JOIN detail ... JOIN detail ..."
Working solution:
create an empty dictionary for aliased tables: { "table name": Table object }
for each master-detail join register new table with an alias in the dictionary:
detail_table = Table(name,...).alias(alias) tables[alias] = detail_table
join next aliased table:
expression = join(expression, detail_table)
When collecting fields for SELECT, do not get another Table(name, ...) but get from your aliased table list:
column = tables[table_name].c[column_name]