I am importing data from csv files into a table created using the SQLAlchemy declarative api.
I receive updates to this data which I want to stage in a temporary table with the same structure for preprocessing.
E.g:
from sqlalchemy import Column,Integer
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyModel(Base):
__tablename__ = "mymodel"
test_column = Column(Integer,primary_key=True)
I can use MyModel.__table__.create() to create this table.
Can I use a similar construct to create another table with the same model and a different name?
What would be the recommended way to achieve this?
Just extend your existing table and change its name
class StagingMyModel(MyModel):
__tablename__ = "staging_mymodel"
This worked for me:
from sqlalchemy import Column,Integer
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyBase(Base):
__abstract__ = True
test_column = Column(Integer,primary_key=True)
class MyModel(MyBase):
__tablename__ = "mymodel"
class MyStagingModel(MyBase):
__tablename__ = "mymodel_staging"
Reference: https://docs.sqlalchemy.org/en/14/orm/declarative_tables.html#using-deferredreflection
See also https://sparrigan.github.io/sql/sqla/2016/01/03/dynamic-tables.html for an approach that probably jibes with what you were originally thinking of with ModelName.__table__.create().
Related
I'm using SQLAlchemy ORM for a Flask project where I want to join across an eagerly loaded model but this leads to two joins to the same intermediary model. If you run the code below you'll see in the generated SQL that there are two joins between the Author model and the Book model. If the lazy=joined bit is removed the sql generated is perfect.
I don't know if I'm doing something wrong or this is by design. How do I get SQLAlchemy to emit the right SQL while maintaining the joinedload in this case?
Note: I have tried this with MySQL and SQLite and it happens with both those dbs.
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
name = Column(String)
pseudo = Column(String)
books = relationship("Book", lazy='joined')
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>" % (
self.name, self.fullname, self.password)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
author_id = Column(Integer, ForeignKey('authors.id'))
name = Column(String)
user = relationship("Author", back_populates="books")
pages = relationship("Page")
class Page(Base):
__tablename__ = 'pages'
id = Column(Integer, primary_key=True)
book_id = Column(Integer, ForeignKey('books.id'))
text = Column(String)
book = relationship("Book", back_populates="pages")
Base.metadata.create_all(engine)
session = Session()
print(str(session.query(Author).outerjoin(Author.books, Page)))
It is by design – read The Zen of Joined Eager Loading:
It is critical to understand the distinction that while Query.join() is used to alter the results of a query, joinedload() goes through great lengths to not alter the results of the query, and instead hide the effects of the rendered join to only allow for related objects to be present.
There are multiple somewhat similar questions in sqlalchemy, though couldn't find one that'd fit the bill exactly.
If you manually add a join, and want to use it to eager load a relationship as well, you need contains_eager():
session.query(Author).\
outerjoin(Author.books, Book.pages).\
options(contains_eager(Author.books).contains_eager(Book.pages))
Note that the relationship definitions Author.books and Book.pages would seem to be missing the back_populates= argument.
To create the User table I have to use drop_all and then create_all methods. But these two functions re-initiate an entire database. I wonder if there is a way to create the User table without erasing (or dropping) any existing tables in a database?
import sqlalchemy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True)
name = sqlalchemy.Column(sqlalchemy.String)
def __init__(self, code=None, *args, **kwargs):
self.name = name
url = 'postgresql+psycopg2://user:pass#01.02.03.04/my_db'
engine = sqlalchemy.create_engine(url)
session = sqlalchemy.orm.scoped_session(sqlalchemy.orm.sessionmaker())
session.configure(bind=engine, autoflush=False, expire_on_commit=False)
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
You can create/drop individual tables:
User.__table__.drop(engine)
User.__table__.create(engine)
from app import db
from models import User
User.__table__.create(db.engine)
User.__table__.drop(db.engine)
Another way to accomplish the task:
Base.metadata.tables['users'].create(engine)
Base.metadata.tables['users'].drop(engine)
I have some standard SQLAlchemy models that I reuse across projects. Something like this:
from sqlalchemy import Column, Integer, String, Unicode
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
slug = Column(String(250), nullable=False, unique=True)
title = Column(Unicode(250), nullable=False)
def __call__(self):
return self.title
I'd like to put this in a shared library and import it into each new project instead of cutting and pasting it, but I can't, because the declarative_base instance is defined separately in the project. If there's more than one, they won't share sessions. How do I work around this?
Here's another question that suggests using mixin classes. Could that work? Will SQLAlchemy accurately import foreign keys from mixin classes?
When you call
Base = declarative_base()
SA create new metadata for this Base.
To reuse your models you must bind metadata of main models to reusable models, but before any import of your reusable models by:
Base.metadata = my_main_app.db.metadata
MixIn classes useful for repeating column declarations, and extending class methods.
For connecting reusable apps based on MixIns you must define concrete class in code manualy for each model.
Will SQLAlchemy accurately import
foreign keys from mixin classes?
MixIn class with foreign key and constraint
from sqlalchemy.schema import UniqueConstraint
from sqlalchemy.ext.declarative import declared_attr
class MessageMixIn(object):
ttime = Column(DateTime)
#declared_attr
def sometable_id(cls):
return Column(Integer, ForeignKey('sometable.id'))
#declared_attr
def __table_args__(cls):
return (UniqueConstraint('sometable_id', 'ttime'), {})
I want to dynamically get the SQLAlchemy mapper class that instances of InstrumentedList use.
I have a 1-M ParentClass-ChildClass (let's call the column myRelation), and the parentInstance.myRelation are InstrumentedList instances. I could hack it and grab the class of the first instance in InstrumentedList, but this doesn't work if there are no objects in the InstrumentedList.
The reason: I need to append a Python dictionary containing the properties of mapper class X to the InstrumentedList, but I don't know the mapper class at runtime. Since I can't append a dict, I need to get the mapper class.
Thanks.
the relationship is associated with the instrumented list via property. so start with this plain vanilla mapping:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class A(Base):
__tablename__ = "a"
id = Column(Integer, primary_key=True)
bs = relationship("B")
class B(Base):
__tablename__ = "b"
id = Column(Integer, primary_key=True)
a_id = Column(Integer, ForeignKey('a.id'))
in any recent version of SQLAlchemy you can see it like this:
print A.bs.property.mapper.class_
in 0.8 there's a little more API available and you can do this:
from sqlalchemy import inspect
print inspect(A.bs).mapper.class_
docs for inspect-> http://docs.sqlalchemy.org/en/rel_0_8/core/inspection.html
docs for "mapper->class_" -> http://docs.sqlalchemy.org/en/rel_0_8/orm/mapper_config.html#sqlalchemy.orm.mapper.Mapper.class_
I have some standard SQLAlchemy models that I reuse across projects. Something like this:
from sqlalchemy import Column, Integer, String, Unicode
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
slug = Column(String(250), nullable=False, unique=True)
title = Column(Unicode(250), nullable=False)
def __call__(self):
return self.title
I'd like to put this in a shared library and import it into each new project instead of cutting and pasting it, but I can't, because the declarative_base instance is defined separately in the project. If there's more than one, they won't share sessions. How do I work around this?
Here's another question that suggests using mixin classes. Could that work? Will SQLAlchemy accurately import foreign keys from mixin classes?
When you call
Base = declarative_base()
SA create new metadata for this Base.
To reuse your models you must bind metadata of main models to reusable models, but before any import of your reusable models by:
Base.metadata = my_main_app.db.metadata
MixIn classes useful for repeating column declarations, and extending class methods.
For connecting reusable apps based on MixIns you must define concrete class in code manualy for each model.
Will SQLAlchemy accurately import
foreign keys from mixin classes?
MixIn class with foreign key and constraint
from sqlalchemy.schema import UniqueConstraint
from sqlalchemy.ext.declarative import declared_attr
class MessageMixIn(object):
ttime = Column(DateTime)
#declared_attr
def sometable_id(cls):
return Column(Integer, ForeignKey('sometable.id'))
#declared_attr
def __table_args__(cls):
return (UniqueConstraint('sometable_id', 'ttime'), {})