Develop Reference

SQLalchemy ORM add to session with dynamic column

I have 2 tables (not the true setup of my database).
tableclass.py:
class TableMeaningEN(sqla_base):
__tablename__ = 'MeaningEN'
id = sqla.Column(sqla.Integer, primary_key=True)
meaning = sqla.Column(sqla.String, primary_key=True)
class TableReadingON(sqla_base):
__tablename__ = 'ReadingON'
id = sqla.Column(sqla.Integer, primary_key=True)
reading = sqla.Column(sqla.String, primary_key=True)
Different column names
As you can see, both have a column id, but TableMeaningEN has meaning and TableReadingON has reading.
Normally (assuming you already have a session) you would add something like this:
session.add(TableMeaningEN(id=1, meaning='test'))
However I want to add dynamically entries to tables, so I have:
import tableclass as tc
for t_name in ['MeaningEN', 'ReadingON']:
session.add(getattr(tc, 'Table{}'.format(t_name))(id=1, ??='test'))
Question
How do I solve the problem that ?? is in one table meaning and in the other reading?
Tried:
columns = sqla.inspect(getattr(tc, 'Table{}'.format(overwrite))).columns.keys()
session.add(getattr(tc, 'Table{}'.format(t_name))(id=1, columns[1]='test'))
I tried this, but however that's not allowed.

If tables are really different by just one attribute, you could create a constructor for both which takes positional arguments as well, such as:
class TableMeaningEN(sqla_base):
def __init__(self, id, meaning):
self.id, self.meaning = id, meaning
# similar __init__ for the other class/table
# then use the following:
session.add(getattr(tc, 'Table{}'.format(t_name))(1, 'test'))
An alternative would be to dynamically create the keyword arguments, assuming there is naming convention:
import tableclass as tc
for t_name in ['MeaningEN', 'ReadingON']:
cls = getattr(tc, 'Table{}'.format(t_name))
fld_name = t_name.lower()[:-2]
kw = {'id': 1, fld_name: 'test'}
session.add(cls(**kw))

sqlalchemy generic foreign key (like in django ORM)

Does sqlalchemy have something like django's GenericForeignKey? And is it right to use generic foreign fields.
My problem is: I have several models (for example, Post, Project, Vacancy, nothing special there) and I want to add comments to each of them. And I want to use only one Comment model. Does it worth to? Or should I use PostComment, ProjectComment etc.? Pros/cons of both ways?
Thanks!

The simplest pattern which I use most often is that you actually have separate Comment tables for each relationship. This may seem frightening at first, but it doesn't incur any additional code versus using any other approach - the tables are created automatically, and the models are referred to using the pattern Post.Comment, Project.Comment, etc. The definition of Comment is maintained in one place. This approach from a referential point of view is the most simple and efficient, as well as the most DBA friendly as different kinds of Comments are kept in their own tables which can be sized individually.
Another pattern to use is a single Comment table, but distinct association tables. This pattern offers the use case that you might want a Comment linked to more than one kind of object at a time (like a Post and a Project at the same time). This pattern is still reasonably efficient.
Thirdly, there's the polymorphic association table. This pattern uses a fixed number of tables to represent the collections and the related class without sacrificing referential integrity. This pattern tries to come the closest to the Django-style "generic foreign key" while still maintaining referential integrity, though it's not as simple as the previous two approaches.
Imitating the pattern used by ROR/Django, where there are no real foreign keys used and rows are matched using application logic, is also possible.
The first three patterns are illustrated in modern form in the SQLAlchemy distribution under examples/generic_associations/.
The ROR/Django pattern, since it gets asked about so often, I will also add to the SQLAlchemy examples, even though I don't like it much. The approach I'm using is not exactly the same as what Django does as they seem to make use of a "contenttypes" table to keep track of types, that seems kind of superfluous to me, but the general idea of an integer column that points to any number of tables based on a discriminator column is present. Here it is:
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy import create_engine, Integer, Column, \
String, and_
from sqlalchemy.orm import Session, relationship, foreign, remote, backref
from sqlalchemy import event
class Base(object):
"""Base class which provides automated table name
and surrogate primary key column.
"""
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id = Column(Integer, primary_key=True)
Base = declarative_base(cls=Base)
class Address(Base):
"""The Address class.
This represents all address records in a
single table.
"""
street = Column(String)
city = Column(String)
zip = Column(String)
discriminator = Column(String)
"""Refers to the type of parent."""
parent_id = Column(Integer)
"""Refers to the primary key of the parent.
This could refer to any table.
"""
#property
def parent(self):
"""Provides in-Python access to the "parent" by choosing
the appropriate relationship.
"""
return getattr(self, "parent_%s" % self.discriminator)
def __repr__(self):
return "%s(street=%r, city=%r, zip=%r)" % \
(self.__class__.__name__, self.street,
self.city, self.zip)
class HasAddresses(object):
"""HasAddresses mixin, creates a relationship to
the address_association table for each parent.
"""
#event.listens_for(HasAddresses, "mapper_configured", propagate=True)
def setup_listener(mapper, class_):
name = class_.__name__
discriminator = name.lower()
class_.addresses = relationship(Address,
primaryjoin=and_(
class_.id == foreign(remote(Address.parent_id)),
Address.discriminator == discriminator
),
backref=backref(
"parent_%s" % discriminator,
primaryjoin=remote(class_.id) == foreign(Address.parent_id)
)
)
#event.listens_for(class_.addresses, "append")
def append_address(target, value, initiator):
value.discriminator = discriminator
class Customer(HasAddresses, Base):
name = Column(String)
class Supplier(HasAddresses, Base):
company_name = Column(String)
engine = create_engine('sqlite://', echo=True)
Base.metadata.create_all(engine)
session = Session(engine)
session.add_all([
Customer(
name='customer 1',
addresses=[
Address(
street='123 anywhere street',
city="New York",
zip="10110"),
Address(
street='40 main street',
city="San Francisco",
zip="95732")
]
),
Supplier(
company_name="Ace Hammers",
addresses=[
Address(
street='2569 west elm',
city="Detroit",
zip="56785")
]
),
])
session.commit()
for customer in session.query(Customer):
for address in customer.addresses:
print(address)
print(address.parent)

I know this is probably a terrible way to do this, but it was a quick fix for me.
class GenericRelation(object):
def __init__(self, object_id, object_type):
self.object_id = object_id
self.object_type = object_type
def __composite_values__(self):
return (self.object_id, self.object_type)
class Permission(AbstractBase):
#__abstract__ = True
_object = None
_generic = composite(
GenericRelation,
sql.Column('object_id', data_types.UUID, nullable=False),
sql.Column('object_type', sql.String, nullable=False),
)
permission_type = sql.Column(sql.Integer)
#property
def object(self):
session = object_session(self)
if self._object or not session:
return self._object
else:
object_class = eval(self.object_type)
self._object = session.query(object_class).filter(object_class.id == self.object_id).first()
return self._object
#object.setter
def object(self, value):
self._object = value
self.object_type = value.__class__.__name__
self.object_id = value.id

SQLAlchemy Declarative: Adding a static text attribute to a column

I am using: SQLAlchemy 0.7.9 and Python 2.7.3 with Bottle 0.11.4. I am an amateur at python.
I have a class (with many columns) derived from declarative base like this:
class Base(object):
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id = Column(Integer, primary_key = True)
def to_dict(self):
serialized = dict((column_name, getattr(self, column_name))
for column_name in self.__table__.c.keys())
return serialized
Base = declarative_base(cls=Base)
class Case(Base):
version = Column(Integer)
title = Column(String(32))
plausible_dd = Column(Text)
frame = Column(Text)
primary_task = Column(Text)
secondary_task = Column(Text)
eval_objectives = Column(Text)
...
I am currently using this 'route' in Bottle to dump out a row/class in json like this:
#app.route('/<name>/:record')
def default(name, record, db):
myClass = getattr(sys.modules[__name__], name)
parms = db.query(myClass).filter(myClass.id == record)
result = json.dumps(([parm.to_dict() for parm in parms]))
return result
My first question is: How can I have each column have some static text that I can use as a proper name such that I can iterate over the columns and get their values AND proper names? For example:
class Case(Base):
version = Column(Integer)
version.pn = "Version Number"
My second question is: Does the following do what I am looking for? I have seen examples of this, but I don't understand the explanation.
Example from sqlalchemy.org:
id = Column("some_table_id", Integer)
My interpretation of the example:
version = Column("Version Number", Integer)
Obviously I don't want a table column to be created. I just want the column to have an "attribute" in the generic sense. Thank you in advance.

info dictionary could be used for that. In your model class define it like this:
class Case(Base):
version = Column(Integer, info={'description': 'Version Number'})
Then it can accessed as the table column property:
desc = Case.__table__.c.version.info.get('description', '<no description>')
Update
Here's one way to iterate through all the columns in the table and get their names, values and descriptions. This example uses dict comprehension, which is available since Python 2.7.
class Case(Base):
# Column definitions go here...
def as_dict(self):
return {c.name: (getattr(self, c.name), c.info.get('description'))
for c in self.__table__.c}

KeyError when adding objects to SQLAlchemy association object

I have two tables, tablet and correspondent:
class Correspondent(db.Model, GlyphMixin):
# PK column and tablename etc. come from the mixin
name = db.Column(db.String(100), nullable=False, unique=True)
# association proxy
tablets = association_proxy('correspondent_tablets', 'tablet')
def __init__(self, name, tablets=None):
self.name = name
if tablets:
self.tablets = tablets
class Tablet(db.Model, GlyphMixin):
# PK column and tablename etc. come from the mixin
area = db.Column(db.String(100), nullable=False, unique=True)
# association proxy
correspondents = association_proxy('tablet_correspondents', 'correspondent')
def __init__(self, area, correspondents=None):
self.area = area
if correspondents:
self.correspondents = correspondents
class Tablet_Correspondent(db.Model):
__tablename__ = "tablet_correspondent"
tablet_id = db.Column("tablet_id",
db.Integer(), db.ForeignKey("tablet.id"), primary_key=True)
correspondent_id = db.Column("correspondent_id",
db.Integer(), db.ForeignKey("correspondent.id"), primary_key=True)
# relations
tablet = db.relationship(
"Tablet",
backref="tablet_correspondents",
cascade="all, delete-orphan",
single_parent=True)
correspondent = db.relationship(
"Correspondent",
backref="correspondent_tablets",
cascade="all, delete-orphan",
single_parent=True)
def __init__(self, tablet=None, correspondent=None):
self.tablet = tablet
self.correspondent = correspondent
I can add records to tablet and correspondent, and doing e.g. Tablet.query.first().correspondents simply returns an empty list, as you would expect. If I manually insert a row into my tablet_correspondent table using existing tablet and correspondent IDs, the list is populated, again as you would expect.
However, if I try to do
cor = Correspondent.query.first()
tab = Tablet.query.first()
tab.correspondents.append(cor)
I get:
KeyError: 'tablet_correspondents'
I'm pretty sure I'm leaving out something fairly elementary here.

The problem with your code is in the .__init__ method. If you are to debug-watch/print() the parameters, you will notice that the parameter tablet is actually an instance of Correspondent:
class Tablet_Correspondent(db.Model):
def __init__(self, tablet=None, correspondent=None):
print "in __init__: ", tablet, correspondent
self.tablet = tablet
self.correspondent = correspondent
The reason for this is the way SA creates new values. From documentation Creation of New Values:
When a list append() event (or set add(), dictionary __setitem__(), or
scalar assignment event) is intercepted by the association proxy, it
instantiates a new instance of the “intermediary” object using its
constructor, passing as a single argument the given value.
In your case when you call tab.correspondents.append(cor), the Tablet_Correspondent.__init__ is called with single argument cor.
Solution? If you will only be adding Correspondents to the Tablet, then just switch the parameters in the __init__. In fact, remove the second parameter completely.
If, however, you will also be using cor.tablets.append(tab), then you need to explicitely use the creator argument to the association_proxy as explained in the documentation linked to above:
class Tablet(db.Model, GlyphMixin):
# ...
correspondents = association_proxy('tablet_correspondents', 'correspondent', creator=lambda cor: Tablet_Correspondent(correspondent=cor))
class Correspondent(db.Model, GlyphMixin):
# ...
tablets = association_proxy('correspondent_tablets', 'tablet', creator=lambda tab: Tablet_Correspondent(tablet=tab))

Like van said, the problem stay in the __init__ method of the Association Object.
In fact, if Tablet or Correspondent classes don't define an __init__ method or don't pass any parameter, the solution doesn't work (no argument expected).
I found an alternative solution. It's easy to detect which class has to be proxied, so it can be assigned to the right field (and still work on adding more associations):
class Tablet_Correspondent(db.Model):
# ...
def __init__(self, proxied=None):
if type(proxied) is Tablet:
self.tablet = proxied
elif type(proxied) is Correspondent:
self.correspondent = proxied

Mixin being added to every child class table when using joined table inheritance

I have the following setup:
class Attribute(object):
a = Column(Integer)
class Thing(Base, Attribute):
b = Column(Integer)
class Subthing(Thing):
c = COlumn(Integer)
However, both Thing and Subthing will have the Attribute mixin, meaning they'll both have the columns specified in Attribute:
Thing: a | b
Subthing: a | c
I only want these mixin columns present in Thing and not Subthing:
Thing: a | b
Subthing: c
Is this possible or will I have to resort to making columns and methods manually instead of using a mixin for every Thing?

Subthing gets a from the parent by definition of inheritance. If you don't want that behavior, then Subthing cannot inherit from that object. A possible solution is to introduce yet another base class.
class Attribute(object):
#declared_attr # the actual syntax on SQLAlchemy for doing mixins
def a(cls):
return Column(Integer)
class BaseThing(Base):
pass
class Thing(BaseThing, Attribute):
b = Column(Integer)
class Subthing(BaseThing):
c = Column(Integer)
Then Subthing only has column c, and Thing has a, and b.

This is a behavioral inconsistency that's developed as the result from the behavior of #declared_attr slowly deviating from a behavioral contract that was only tested against the "column copy" use case. The original use case of the "column on mixin" was to apply it to all inheriting classes equally, however #declared_attr, which was developed a bit later, didn't adopt this behavior, hence inconsistent.
Changing "column copy" to only take effect for the non-subclass is a backwards-incompatible behavioral change, so is in the upcoming 0.8 only. This is ticket #2565 (http://www.sqlalchemy.org/trac/ticket/2565) and is resolved in r9baa197f2c67.
test:
from sqlalchemy import *
from sqlalchemy.ext.declarative import declarative_base, declared_attr
Base = declarative_base()
class Mixin(object):
a = Column(Integer)
#declared_attr
def b(cls):
return Column(Integer)
class A(Mixin, Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
class B(A):
__tablename__ = 'b'
id = Column(Integer, ForeignKey('a.id'), primary_key=True)
assert 'a' in A.__table__.c
assert 'b' in A.__table__.c
assert 'a' not in B.__table__.c
assert 'b' not in B.__table__.c

I just ran into the same issue. It turns out if you define columns in your mixin class using #declared_attr, SQLAlchemy behaves properly. Columns declared directly in the mixin leak into sub-classes when using joined-table inheritance.