I'm using SQLAlchemy ORM for a Flask project where I want to join across an eagerly loaded model but this leads to two joins to the same intermediary model. If you run the code below you'll see in the generated SQL that there are two joins between the Author model and the Book model. If the lazy=joined bit is removed the sql generated is perfect.
I don't know if I'm doing something wrong or this is by design. How do I get SQLAlchemy to emit the right SQL while maintaining the joinedload in this case?
Note: I have tried this with MySQL and SQLite and it happens with both those dbs.
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
name = Column(String)
pseudo = Column(String)
books = relationship("Book", lazy='joined')
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>" % (
self.name, self.fullname, self.password)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
author_id = Column(Integer, ForeignKey('authors.id'))
name = Column(String)
user = relationship("Author", back_populates="books")
pages = relationship("Page")
class Page(Base):
__tablename__ = 'pages'
id = Column(Integer, primary_key=True)
book_id = Column(Integer, ForeignKey('books.id'))
text = Column(String)
book = relationship("Book", back_populates="pages")
Base.metadata.create_all(engine)
session = Session()
print(str(session.query(Author).outerjoin(Author.books, Page)))
It is by design – read The Zen of Joined Eager Loading:
It is critical to understand the distinction that while Query.join() is used to alter the results of a query, joinedload() goes through great lengths to not alter the results of the query, and instead hide the effects of the rendered join to only allow for related objects to be present.
There are multiple somewhat similar questions in sqlalchemy, though couldn't find one that'd fit the bill exactly.
If you manually add a join, and want to use it to eager load a relationship as well, you need contains_eager():
session.query(Author).\
outerjoin(Author.books, Book.pages).\
options(contains_eager(Author.books).contains_eager(Book.pages))
Note that the relationship definitions Author.books and Book.pages would seem to be missing the back_populates= argument.
Related
I'm misunderstanding something about SqlAlchemy ForeignKey constraints. My understanding is that the insertion of B below should raise a ForeignKeyConstraint exception because there's no A with "my_a" as its name. Isn't that what a ForeignKey constraint does? Require the existence of the value in the table column mapped by the constraint when the constrained table is updated?
from sqlalchemy import Column, create_engine, ForeignKey, Integer, VARCHAR
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class A(Base):
__tablename__ = 'table_A'
id = Column(Integer, primary_key=True)
name = Column(VARCHAR(32))
class B(Base):
__tablename__ = 'table_B'
id = Column(Integer, primary_key=True)
a_name = Column(VARCHAR(32), ForeignKey('table_A.name'), nullable=False)
engine = create_engine('sqlite:////tmp/AB.db.foo')
Base.metadata.create_all(engine)
Session = sessionmaker()
Session.configure(bind=engine)
b = B(a_name="my_a")
session = Session()
session.add(b)
session.commit()
session.close()
SQLite – even modern versions – does not enforce foreign keys by default.
Assuming the [SQLite] library is compiled with foreign key constraints enabled, it must still be enabled by the application at runtime, using the PRAGMA foreign_keys command.
SQLite documentation
SQLAlchemy documentation
I made a simple table using sqlalchemy.
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Table, Column, ForeignKey, MetaData, Sequence
from sqlalchemy.orm import relationship
from sqlalchemy import INTEGER, REAL, VARCHAR, TIMESTAMP, BOOLEAN, Integer
import psycopg2 as pg
import psycopg2.extras
db_string = "postgresql://someDBInfoString"
db = create_engine(db_string)
base = declarative_base()
Session = sessionmaker(db)
session = Session()
class Women(base):
__tablename__ = "WOMEN"
id = Column(Integer, primary_key=True)
name = Column(VARCHAR)
man = relationship("Men", uselist=False, back_populates="woman")
I tried several ways as below to make the id column have Serial type or an equivalent of it that can autoincrement.
1) Setting it as PK
2) Adding autoincrement=True keyword arg
id = Column(Integer, autoincrement=True, primary_key=True)
3) Passing Sequence
id = Column(Integer, Sequence('seq_tramos_id', start=1, increment=1), primary_key=True)
But in every method, id column is set as a plain Integer type on DBMS and inserting a data through DBMS (Postgresql) requires manually including id along with the other value. Inserting a data with ORM using session as in session.add(somestring) is no problem as id is automatically set just as I intended. So this is a problem of how ORM-created table works on DBMS.
What else can I try?
edit : added import and detailed description
Basing this question off this similar post, but about SORT ORDER
I understand that you can change the lazy to dynamic in the relatonship, and then that will allow you to query against the relationship before loading, but is there a way to LIMIT the return results directly from a selectin or on of the other loading techniques?
Use case is, Im trying to pass the record into Marshmallow and limit the number of nested records returned. dynamic at that point doesnt work, as Marshmallow includes it as an all() and selectin appears to just included it unqueryable at time of load, and again Marshmallow get the entire record set.
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Example(Base):
__tablename__ = 'examples'
id = Column(Integer, primary_key=True)
related_items = relationship('RelatedItem', back_populates='example', order_by='RelatedItem.id')
class RelatedItem(Base):
__tablename__ = 'related_items'
id = Column(Integer, primary_key=True)
example_id = Column(Integer, ForeignKey('examples.id'), nullable=False)
example = relationship('Example', back_populates='related_items')
I am trying to create an application using SQLAlchemy. It worked fine as long as I only had one file with one Class. Now I want to have multiple classes/tables in different files. I stumbled upon this question, and tried to do it like it was suggested there: I now have three files
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
blind.py
from sqlalchemy import Column, String
from .base import Base
class Blind(Base):
__tablename__ = 'blinds'
blind = Column(String)
data_processor_uuid = Column(String, primary_key=True)
data_source_uuid = Column(String)
timestamp = Column(String, primary_key=True)
and data.py
from sqlalchemy import Column, Integer, String, Float
from .base import Base
class Datum(Base):
__tablename__ = 'data'
data_source_uuid = Column(Integer, primary_key=True)
sensor_type = Column(String)
timestamp = Column(String, primary_key=True)
value = Column(Float)
I now want to initialize the database in db_setup.py using
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .base import Base
engine = create_engine('sqlite:///test.db', echo=True)
Base.metadata.bind = engine
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
def get_db_session():
return session
This works, however, it does not create the tables in the database as expected. When I try to insert something into the table, I get an error saying "table does not exist". Can someone tell me what I am doing wrong here?
The problem was that I wasn't importing the class definitions for Blinds and Datum anywhere, so they weren't evaluated! Before I split them up into different files, I had imported them to get to Base. Thanks to #IljaEverilä for this answer!
How does one generate the SQL/Migrate Code/Whatever with SQLAlchemy when using the Declarative Base?
from sqlalchemy import *
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('mysql://root:password#localhost/mydb_dev', echo=True)
metadata = MetaData(bind=engine)
class User(Base):
__table__ = Table("users", metadata, autoload=True)
id = Column(Integer, primary_key=True)
display_name = Column(String)
email = Column(String)
password = Column(String)
def __repr__(self):
return "<User(id='{}', display_name='{}', email='{}')>".format(self.id, self.display_name, self.email)
class Site(Base):
__table__ = Table("sites", metadata, autoload=True)
id = Column(Integer, primary_key=True)
name = Column(String)
urls = relationship("URL")
def __repr__(self):
return "<Site(id='{}', name='{}')>".format(self.id, self.name)
I have this so far, I'd like to see what SQLAlchemy would generate as a schema.
Or, does SQLAlchemy do this at all? Is this a case where I create and manage the database and it's schema separately, and I just update my entities to reflect it?
Do understand that I am used to Doctrine2, and im very new to SQLAlchemy
Thanks!
You can create the db models by calling:
metadata.create_all()
http://docs.sqlalchemy.org/en/latest/core/metadata.html#sqlalchemy.schema.MetaData.create_all
Note that this only works for the creation of previously non-existent models, and doesn't handle updates or downgrades. Check out alembic for even finer control:
http://alembic.zzzcomputing.com/en/latest/