I have three related classes: Parent, Child, SubChild. The relationship is one-many in both cases. I've set it up so child.parent references correctly and of course so that sub_child.child.parent works as well.
The thing is that I never actually need to know sub_child.child, but I do need to know the sub_child's ultimate parent. I'd like to set up a relationship such that sub_child.parent would return a reference to the ultimate Parent object.
Is this possible, or is it just a bad idea? I've read the documentation, but don't see much that looks promising.
I'm using python2 and sqlalchemy orm on mysql as a backend.
Have a look at http://docs.sqlalchemy.org/en/latest/orm/nonstandard_mappings.html
With this method you should be able to create a mapping over the three tables you mentioned and assign the columns of the participating tables as attributes to the mapping class.
metadata = MetaData()
parent = Table('parent', metadata,
Column('id', Integer, primary_key=True),
Column('child', Integer, ForeignKey('child.id')),
)
child = Table('child', metadata,
Column('id', Integer, primary_key=True),
Column('subchild', Integer, ForeignKey('subchild.id')),
)
subchild = Table('subchild', metadata,
Column('id', Integer, primary_key=True),
Column('some_column', String),
)
joined = join(parent, child, subchild)
Base = declarative_base()
class Parent(Base):
__table__ = joined
id = column_property(parent.c.id, child.c.id, subchild.c.id)
subchild_attr = subchild.c.some_column
Related
I have an issue, where I use the following:
class Docs(Base):
__tablename__ = "docs"
__table__ = Table(__tablename__, Base.metadata,
Column("dID", Integer, primary_key=True),
Column("d_type", Integer, primary_key=True),
Column("d_category", Integer, primary_key=True),
autoload_with=get_global_db_engine())
__table_args__ = (UniqueConstraint("dID", "d_type", "d_category"),)
# Class Globals
COLUMNS = __table__.columns.keys()
Problem is, when I loop throughCOLUMNS - it doesn't list all the columns of __table__, it holds only the columns I pre-defined inside the Table ( 3 cols ).
How do I get COLUMNS to return all of them ?
You'll have to set extend_existing to extend your defined table with reflected columns.
From the documenation:
Table.extend_existing will also work in conjunction with Table.autoload to run a new reflection operation against the database, even if a Table of the same name is already present in the target MetaData; newly reflected Column objects and other options will be added into the state of the Table, potentially overwriting existing columns and options of the same name.
So,
__table__ = Table(__tablename__, Base.metadata,
Column("dID", Integer, primary_key=True),
Column("d_type", Integer, primary_key=True),
Column("d_category", Integer, primary_key=True),
extend_existing=True,
autoload_with=get_global_db_engine())
should do the trick.
The situation is a little bit simplified. I have two migration files for sqlalchemy-migrate:
In First I create table volume_usage_cache, then autoload it, create copy of its columns and print it:
from sqlalchemy import Column, DateTime
from sqlalchemy import Boolean, BigInteger, MetaData, Integer, String, Table
def upgrade(migrate_engine):
meta = MetaData()
meta.bind = migrate_engine
# Create new table
volume_usage_cache = Table('volume_usage_cache', meta,
Column('deleted', Boolean(create_constraint=True, name=None)),
Column('id', Integer(), primary_key=True, nullable=False),
Column('curr_write_bytes', BigInteger(), default=0),
mysql_engine='InnoDB',
mysql_charset='utf8'
)
volume_usage_cache.create()
volume_usage_cache = Table('volume_usage_cache', meta, autoload=True)
columns = []
[columns.append(column.copy()) for column in volume_usage_cache.columns]
print columns
And I get in log what I expected:
[Column('deleted', Boolean(), table=None), Column('id', Integer(), table=None,
primary_key=True, nullable=False), Column('curr_write_bytes', BigInteger(),
table=None, default=ColumnDefault(0))]
But if I make a copy of columns in Second migration file (that is runed after First):
from sqlalchemy import MetaData, String, Integer, Boolean, Table, Column, Index
def upgrade(migrate_engine):
meta = MetaData()
meta.bind = migrate_engine
table = Table("volume_usage_cache", meta, autoload=True)
columns = []
for column in table.columns:
columns.append(column.copy())
print columns
I get a different result:
[Column('deleted', INTEGER(), table=None, default=ColumnDefault(0)),
Column(u'id', INTEGER(), table=None, primary_key=True, nullable=False),
Column(u'curr_write_bytes', NullType(), table=None)]
Why curr_write_bytes column has NullType?
The are two problems:
First:
In First file we are using old metadata that already contains all columns with need types
So if we create new MetaData instance, SqlAlchemy will load info about table from database and will get the same result as in Second file.
Second:
There is no support in sqlAlchemy for BigInteger column type (in sqlite). And Sqlite doesn't support types of column at all. So we can create table with column BigInteger (and it will work), but after autoload type of such column will be automatically converted to NullType.
I'm trying to create a simple representation for an entity deduplication schema using mysql, and using sqlalchemy for programmatic access.
I'm trying to achieve a specific effect which I think is kind of a self-referential query but i'm not sure:
Essentially I have an 'entities' table (with unique entity_id) and an associated Entity object,
and then an entity_groups table which (for simplicity) has a 'group_id' and 'entity_id' columns, so that I 'register' an entity with a group by creating a row for that relation.
this table too is associated with an ORM object - EntityGroup.
Question is, how do i get the EntityGroup object reference all entities in the group?
I expect I need to write something like:
mapper(EntityGroup, entity_groups_table,
properties={
'entities': relationship(
Entity,
.... ?
)
},
and i'm alittle fuzzy on the details. Basically I need all the rows in entity_groups that have the same group_id as the row represented by the object. And then I need to materialize
all the Entity objects associated those rows' entity_id column. This sounds like something achievable by a more verbose Query() operation in sqlalchemy, but i'm not sure how to combine that with the relationship() construct (if at all - perhaps go manual? )
Any help will be useful, I hope I was clear and to the point
You really should not do it using a Query, as if you configure the relationships properly you will get this automatically. Assuming that you use entity_group table solely to store the relationship and nothing else, you should just configure many-to-many relationship as documented. Fully working example should help:
from sqlalchemy import create_engine, Column, Integer, String, MetaData, ForeignKey, Table
from sqlalchemy.orm import relationship, mapper, scoped_session, sessionmaker, backref
from sqlalchemy.ext.associationproxy import association_proxy
# Configure test DB
engine = create_engine(u'sqlite:///:memory:', echo=False)
session = scoped_session(sessionmaker(bind=engine, autoflush=False))
metadata = MetaData()
# tables
entities_table = Table('entities', metadata,
Column('entity_id', Integer, primary_key=True),
)
groups_table = Table('groups', metadata,
Column('group_id', Integer, primary_key=True),
)
entity_groups_table = Table('entity_groups', metadata,
Column('entity_id', Integer, ForeignKey('entities.entity_id'), primary_key=True),
Column('group_id', Integer, ForeignKey('groups.group_id'), primary_key=True),
)
# object model
class Group(object):
def __repr__(self): return "<Group: %d>" % (self.group_id,)
class Entity(object):
def __repr__(self): return "<Entity: %d>" % (self.entity_id,)
# mappers
mapper(Group, groups_table)
mapper(Entity, entities_table,
properties={'groups': relationship(Group, secondary=entity_groups_table, backref='entities')},
)
# create db schema
metadata.create_all(engine)
# == TESTS
# create entities
e1 = Entity()
e2 = Entity()
g1 = Group()
g2 = Group()
g3 = Group()
g1.entities.append(e1)
g2.entities.append(e2)
g3.entities.append(e1)
g3.entities.append(e2)
session.add(e1)
session.add(e2)
session.commit()
# query...
session.expunge_all()
# check Peter
for g in session.query(Group).all():
print "group: ", g, " has ", g.entities
should produce something like:
group: <Group: 1> has [<Entity: 1>]
group: <Group: 2> has [<Entity: 1>, <Entity: 2>]
group: <Group: 3> has [<Entity: 2>]
Sometimes it is useful to map a class against a join instead of a single table when using SQLAlchemy's declarative extension. When column names collide, usually in a one-to-many because all primary keys are named id by default, you can use .alias() to prefix every column with its table name. That is inconvenient if you've already written code that assumes your mapped class has non-prefixed names.
For example:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, Column, Integer, ForeignKeyConstraint
Base = declarative_base()
t1 = Table('t1',
Base.metadata,
Column('id', Integer, primary_key=True))
t2 = Table('t2',
Base.metadata,
Column('id', Integer, primary_key=True),
Column('fkey', Integer),
ForeignKeyConstraint(['fkey'], [t1.c.id]))
class ST(Base):
__table__ = t1.join(t2)
class ST2(Base):
__table__ = t1.join(t2).alias()
ST has id, fkey properties with each name mapping to the first table in the join that uses the overridden name, so the mapped class does not expose t2's primary key. ST2 has t1_id, t2_id and t2_fkey properties.
Is there a convenient way to alias only some of the columns from each table in the join so the mapped class exposes the more convenient non-prefixed property names for most mapped columns?
You can create alias for each column separately with its label() method. So it's possible something similar to the following (not tested):
from sqlalchemy import select
def alias_dups(join):
dups = set(col.key for col in join.left.columns) & \
set(col.key for col in join.right.columns)
columns = []
for col in join.columns:
if col.key in dups:
col = col.label('%s_%s' % (col.table.name, col.key))
columns.append(col)
return select(columns, from_obj=[join]).alias()
class ST2(Base):
__table__ = alias_dups(t1.join(t2))
Hi have have the following tables
nfiletable = Table(
'NFILE', base.metadata,
Column('fileid', Integer, primary_key=True),
Column('path', String(300)),
Column('filename', String(50)),
Column('filesize', Integer),
schema='NATIVEFILES')#,autoload=True,autoload_with=engine)
sheetnames_table=Table(
'SHEETNAMES', base.metadata, schema='NATIVEFILES',
autoload=True, autoload_with=engine)
nfile_sheet_table=Table(
'NFILE_SHEETNAME',base.metadata,
Column('fileid', Integer, ForeignKey(nfiletable.c.fileid)),
Column('sheetid', Integer, ForeignKey(sheetnames_table.c.sheet_id)),
schema='NATIVEFILES')
and mappers:
nfile_mapper=mapper(Nfile,nfiletable)
mapper(Sheet, sheetnames_table, properties={
'files': relation(
Nfile, secondary=nfile_sheet_table,
primaryjoin=(sheetnames_table.c.sheet_id==nfile_sheet_table.c.sheetid),
secondaryjoin=(nfile_sheet_table.c.fileid==nfiletable.c.fileid),
foreign_keys=[nfile_sheet_table.c.sheetid,nfile_sheet_table.c.fileid],
backref='sheets')
})
when i do the following
upl = Session.query(Nfile).filter_by(fileid=k).one()
sheetdb=[]
for sheet in sheetstoadd:
s = sheetcache[sheetname]
sheetdb.append(s)
upl.sheets = sheetdb
Session.save(upl)
Session.flush()
the line upl.sheets = sheetdb takes forever.
It seems that all files for each sheet in sheetdb are loaded from the db.
How can I prevent this?
if NFile.sheets references a huge collection, put "lazy='dynamic'" on the backref:
mapper(Sheet, sheetnames_table, properties={
'files': relation(
Nfile, secondary=nfile_sheet_table,
backref=backref('sheets', lazy='dynamic'))
})
All the primaryjoin/secondaryjoin/foreign_keys stuff is also not needed since your nfile_sheet_table has ForeignKey constructs on it.