Does sqlalchemy have something like django's GenericForeignKey? And is it right to use generic foreign fields.
My problem is: I have several models (for example, Post, Project, Vacancy, nothing special there) and I want to add comments to each of them. And I want to use only one Comment model. Does it worth to? Or should I use PostComment, ProjectComment etc.? Pros/cons of both ways?
Thanks!
The simplest pattern which I use most often is that you actually have separate Comment tables for each relationship. This may seem frightening at first, but it doesn't incur any additional code versus using any other approach - the tables are created automatically, and the models are referred to using the pattern Post.Comment, Project.Comment, etc. The definition of Comment is maintained in one place. This approach from a referential point of view is the most simple and efficient, as well as the most DBA friendly as different kinds of Comments are kept in their own tables which can be sized individually.
Another pattern to use is a single Comment table, but distinct association tables. This pattern offers the use case that you might want a Comment linked to more than one kind of object at a time (like a Post and a Project at the same time). This pattern is still reasonably efficient.
Thirdly, there's the polymorphic association table. This pattern uses a fixed number of tables to represent the collections and the related class without sacrificing referential integrity. This pattern tries to come the closest to the Django-style "generic foreign key" while still maintaining referential integrity, though it's not as simple as the previous two approaches.
Imitating the pattern used by ROR/Django, where there are no real foreign keys used and rows are matched using application logic, is also possible.
The first three patterns are illustrated in modern form in the SQLAlchemy distribution under examples/generic_associations/.
The ROR/Django pattern, since it gets asked about so often, I will also add to the SQLAlchemy examples, even though I don't like it much. The approach I'm using is not exactly the same as what Django does as they seem to make use of a "contenttypes" table to keep track of types, that seems kind of superfluous to me, but the general idea of an integer column that points to any number of tables based on a discriminator column is present. Here it is:
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy import create_engine, Integer, Column, \
String, and_
from sqlalchemy.orm import Session, relationship, foreign, remote, backref
from sqlalchemy import event
class Base(object):
"""Base class which provides automated table name
and surrogate primary key column.
"""
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id = Column(Integer, primary_key=True)
Base = declarative_base(cls=Base)
class Address(Base):
"""The Address class.
This represents all address records in a
single table.
"""
street = Column(String)
city = Column(String)
zip = Column(String)
discriminator = Column(String)
"""Refers to the type of parent."""
parent_id = Column(Integer)
"""Refers to the primary key of the parent.
This could refer to any table.
"""
#property
def parent(self):
"""Provides in-Python access to the "parent" by choosing
the appropriate relationship.
"""
return getattr(self, "parent_%s" % self.discriminator)
def __repr__(self):
return "%s(street=%r, city=%r, zip=%r)" % \
(self.__class__.__name__, self.street,
self.city, self.zip)
class HasAddresses(object):
"""HasAddresses mixin, creates a relationship to
the address_association table for each parent.
"""
#event.listens_for(HasAddresses, "mapper_configured", propagate=True)
def setup_listener(mapper, class_):
name = class_.__name__
discriminator = name.lower()
class_.addresses = relationship(Address,
primaryjoin=and_(
class_.id == foreign(remote(Address.parent_id)),
Address.discriminator == discriminator
),
backref=backref(
"parent_%s" % discriminator,
primaryjoin=remote(class_.id) == foreign(Address.parent_id)
)
)
#event.listens_for(class_.addresses, "append")
def append_address(target, value, initiator):
value.discriminator = discriminator
class Customer(HasAddresses, Base):
name = Column(String)
class Supplier(HasAddresses, Base):
company_name = Column(String)
engine = create_engine('sqlite://', echo=True)
Base.metadata.create_all(engine)
session = Session(engine)
session.add_all([
Customer(
name='customer 1',
addresses=[
Address(
street='123 anywhere street',
city="New York",
zip="10110"),
Address(
street='40 main street',
city="San Francisco",
zip="95732")
]
),
Supplier(
company_name="Ace Hammers",
addresses=[
Address(
street='2569 west elm',
city="Detroit",
zip="56785")
]
),
])
session.commit()
for customer in session.query(Customer):
for address in customer.addresses:
print(address)
print(address.parent)
I know this is probably a terrible way to do this, but it was a quick fix for me.
class GenericRelation(object):
def __init__(self, object_id, object_type):
self.object_id = object_id
self.object_type = object_type
def __composite_values__(self):
return (self.object_id, self.object_type)
class Permission(AbstractBase):
#__abstract__ = True
_object = None
_generic = composite(
GenericRelation,
sql.Column('object_id', data_types.UUID, nullable=False),
sql.Column('object_type', sql.String, nullable=False),
)
permission_type = sql.Column(sql.Integer)
#property
def object(self):
session = object_session(self)
if self._object or not session:
return self._object
else:
object_class = eval(self.object_type)
self._object = session.query(object_class).filter(object_class.id == self.object_id).first()
return self._object
#object.setter
def object(self, value):
self._object = value
self.object_type = value.__class__.__name__
self.object_id = value.id
Related
I'm making a simple lookup application for Japanese characters (Kanji), where the user can search the database using any of the information available.
My database structure
Kanji:
id
character (A kanji like 頑)
heisig6 (a number indicating the order of showing Kanji)
kanjiorigin (a number indicating the order of showing Kanji)
MeaningEN (1 kanji_id can have multiple entries with different meanings):
kanji_id (FOREIGN KEY(kanji_id) REFERENCES "Kanji" (id)
meaning
User handling
The user can choose to search by 'id', 'character', 'heisig6', 'kanjiorigin' or 'meaning' and it should then return all information in all those fields. (All fields return only 1 result, except meanings, which can return multiple results)
Code, EDIT 4+5: my code with thanks to #ApolloFortyNine and #sqlalchemy on IRC, EDIT 6: join --> outerjoin (otherwise won't find information that has no Origins)
import sqlalchemy as sqla
import sqlalchemy.orm as sqlo
from tableclass import TableKanji, TableMeaningEN, TableMisc, TableOriginKanji # See tableclass.py
# Searches database with argument search method
class SearchDatabase():
def __init__(self):
#self.db_name = "sqlite:///Kanji_story.db"
self.engine = sqla.create_engine("sqlite:///Kanji.db", echo=True)
# Bind the engine to the metadata of the Base class so that the
# declaratives can be accessed through a DBSession instance
tc.sqla_base.metadata.bind = self.engine
# For making sessions to connect to db
self.db_session = sqlo.sessionmaker(bind=self.engine)
def retrieve(self, s_input, s_method):
# s_input: search input
# s_method: search method
print("\nRetrieving results with input: {} and method: {}".format(s_input, s_method))
data = [] # Data to return
# User searches on non-empty string
if s_input:
session = self.db_session()
# Find id in other table than Kanji
if s_method == 'meaning':
s_table = TableMeaningEN # 'MeaningEN'
elif s_method == 'okanji':
s_table = TableOriginKanji # 'OriginKanji'
else:
s_table = TableKanji # 'Kanji'
result = session.query(TableKanji).outerjoin(TableMeaningEN).outerjoin(
(TableOriginKanji, TableKanji.origin_kanji)
).filter(getattr(s_table, s_method) == s_input).all()
print("result: {}".format(result))
for r in result:
print("r: {}".format(r))
meanings = [m.meaning for m in r.meaning_en]
print(meanings)
# TODO transform into origin kanji's
origins = [str(o.okanji_id) for o in r.okanji_id]
print(origins)
data.append({'character': r.character, 'meanings': meanings,
'indexes': [r.id, r.heisig6, r.kanjiorigin], 'origins': origins})
session.close()
if not data:
data = [{'character': 'X', 'meanings': ['invalid', 'search', 'result']}]
return(data)
Question EDIT 4+5
Is this an efficient query?: result = session.query(TableKanji).join(TableMeaningEN).filter(getattr(s_table, s_method) == s_input).all() (The .join statement is necessary, because otherwise e.g. session.query(TableKanji).filter(TableMeaningEN.meaning == 'love').all() returns all the meanings in my database for some reason? So is this either the right query or is my relationship() in my tableclass.py not properly defined?
fixed (see lambda: in tableclass.py) kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="OriginKanji") <-- what is wrong about this? It gives the error:
File "/path/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 1805, in get_property
"Mapper '%s' has no property '%s'" % (self, key))
sqlalchemy.exc.InvalidRequestError: Mapper 'Mapper|TableKanji|Kanji' has no property 'OriginKanji'
Edit 2: tableclass.py (EDIT 3+4+5: updated)
import sqlalchemy as sqla
from sqlalchemy.orm import relationship
import sqlalchemy.ext.declarative as sqld
sqla_base = sqld.declarative_base()
class TableKanji(sqla_base):
__tablename__ = 'Kanji'
id = sqla.Column(sqla.Integer, primary_key=True)
character = sqla.Column(sqla.String, nullable=False)
radical = sqla.Column(sqla.Integer) # Can be defined as Boolean
heisig6 = sqla.Column(sqla.Integer, unique=True, nullable=True)
kanjiorigin = sqla.Column(sqla.Integer, unique=True, nullable=True)
cjk = sqla.Column(sqla.String, unique=True, nullable=True)
meaning_en = relationship("TableMeaningEN", back_populates="kanji") # backref="Kanji")
okanji_id = relationship("TableOriginKanji", foreign_keys=lambda: TableOriginKanji.kanji_id, back_populates="kanji")
class TableMeaningEN(sqla_base):
__tablename__ = 'MeaningEN'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
meaning = sqla.Column(sqla.String, primary_key=True)
kanji = relationship("TableKanji", back_populates="meaning_en")
class TableOriginKanji(sqla_base):
__tablename__ = 'OriginKanji'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
okanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
order = sqla.Column(sqla.Integer)
#okanji = relationship("TableKanji", foreign_keys=[kanji_id], backref="okanji")
kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="okanji_id")
We would really have to be able to see your database schema to give real critique, but assuming no foreign keys, what you said is basically the best you can do.
SQLAlchemy really begins to shine when you have complicated relations going on however. For example, if you properly had foreign keys set, you could do something like the following.
# Assuming kanji is a tc.tableMeaningEN.kanji_id object
kanji_meaning = kanji.meanings
And that would return the meanings for the kanji as an array, without any further queries.
You can go quite deep with relationships, so I'm linking the documentation here. http://docs.sqlalchemy.org/en/latest/orm/relationships.html
EDIT: Actually, you don't need to manually join at all, SQLAlchemy will do it for you.
The case is wrong on your classes, but I'm not sure if SQLAlchemy is case sensitive there or not. If it works, then just move on.
If you query the a table (self.session.query(User).filter(User.username == self.name).first()) you should have an object of the table type (User here).
So in your case, querying the TableKanji table alone will return an object of that type.
kanji_obj = session.query(TableKanji).filter(TableKanji.id == id).first()
# This will return an array of all meaning_ens that match the foreign key
meaning_arr = kanji_obj.meaning_en
# This will return a single meeting, just to show each member of the arr is of type TableMeaningEn
meaning_arr[0].meaning
I have a project made use of some of these features, hope it helps:
https://github.com/ApolloFortyNine/SongSense
Database declaration (with relationships): https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/database.py
Automatic joins: https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/getfriend.py#L134
I really like my database structure, but as for the rest it's pretty awful. Hope it still helps though.
I have the following relationship set up in a model:
role_profiles = Table('roleprofile', Base.metadata,
Column('role_id', Integer, ForeignKey('role.id')),
Column('profile_id', Integer, ForeignKey('profile.id'))
)
class profile(Base):
__tablename__ = 'profile'
# Columns...
roles = relationship('role', secondary=role_profiles, backref='profiles')
class role(Base):
__tablename__ = 'role'
# Columns...
So as I now understand that it works is that the roles property on the profile object will contain a list of role classes (which it does).
What I want to do is to serialize for each property of the model class generically. It works fine for the top class profile and I determine that there is a list of roles that I should recurse into:
# I need a statement here to check if the field.value is a backref
#if field.value is backref:
# continue
if isinstance(field.value, list):
# Get the json for the list
value = serialize.serialize_to_json(field.value)
else:
# Get the json for the value
value = cls._serialize(field.value)
The problem is that the backref of the relationship adds a pointer back to the profile. The same profile is then serialized and it recurse the roles over and over again until stack overflow.
Is there a way to determine that the property is a backref added by the relationship?
Update
Maybe I should add that it works fine in this case if I remove the backref since I don't need it but I would like to keep it in.
Update
As a temporary fix I added a class property to my base class:
class BaseModelMixin(object):
"""Base mixin for models using stamped data"""
__backref__ = None
and add it like this:
class role(Base):
__tablename__ = 'role'
__backref__ = ('profiles', )
# Columns...
and use it like this in my recursion:
if self.__backref__ and property_name in self.__backref__:
continue
If there is a better way please let me know because this doesn't look optimal.
Not sure if this is the best practice, but this code works for me. It returns True if the attribute is a reference, False if a regular column type.
def is_relation(orm_object, attr_name):
return hasattr(getattr(orm_object.__class__, attr_name).property, 'mapper')
You can create a __relationships__ in your class BaseModelMixin as a #property, which has a list of all relationships name which are not as a backref name in a model.
class BaseModelMixin(object):
"""Base mixin for models using stamped data"""
#property
def __relationships__(self):
"""
Return a list of relationships name which are not as a backref
name in model
"""
back_ref_relationships = list()
items = self.__mapper__.relationships.items()
for (key, value) in items:
if isinstance(value.backref, tuple):
back_ref_relationships.append(key)
return back_ref_relationships
As you have two class profile and role, so
>>> p = profile()
>>> p.__relationships__
# ['roles']
>>> r = role()
>>> r.__relationships__
# []
have a look at inspect
e.g.
from sqlalchemy import inspect
mapper = inspect(MyModelClass)
# dir(mapper)
# mapper.relationships.keys()
I have two tables, tablet and correspondent:
class Correspondent(db.Model, GlyphMixin):
# PK column and tablename etc. come from the mixin
name = db.Column(db.String(100), nullable=False, unique=True)
# association proxy
tablets = association_proxy('correspondent_tablets', 'tablet')
def __init__(self, name, tablets=None):
self.name = name
if tablets:
self.tablets = tablets
class Tablet(db.Model, GlyphMixin):
# PK column and tablename etc. come from the mixin
area = db.Column(db.String(100), nullable=False, unique=True)
# association proxy
correspondents = association_proxy('tablet_correspondents', 'correspondent')
def __init__(self, area, correspondents=None):
self.area = area
if correspondents:
self.correspondents = correspondents
class Tablet_Correspondent(db.Model):
__tablename__ = "tablet_correspondent"
tablet_id = db.Column("tablet_id",
db.Integer(), db.ForeignKey("tablet.id"), primary_key=True)
correspondent_id = db.Column("correspondent_id",
db.Integer(), db.ForeignKey("correspondent.id"), primary_key=True)
# relations
tablet = db.relationship(
"Tablet",
backref="tablet_correspondents",
cascade="all, delete-orphan",
single_parent=True)
correspondent = db.relationship(
"Correspondent",
backref="correspondent_tablets",
cascade="all, delete-orphan",
single_parent=True)
def __init__(self, tablet=None, correspondent=None):
self.tablet = tablet
self.correspondent = correspondent
I can add records to tablet and correspondent, and doing e.g. Tablet.query.first().correspondents simply returns an empty list, as you would expect. If I manually insert a row into my tablet_correspondent table using existing tablet and correspondent IDs, the list is populated, again as you would expect.
However, if I try to do
cor = Correspondent.query.first()
tab = Tablet.query.first()
tab.correspondents.append(cor)
I get:
KeyError: 'tablet_correspondents'
I'm pretty sure I'm leaving out something fairly elementary here.
The problem with your code is in the .__init__ method. If you are to debug-watch/print() the parameters, you will notice that the parameter tablet is actually an instance of Correspondent:
class Tablet_Correspondent(db.Model):
def __init__(self, tablet=None, correspondent=None):
print "in __init__: ", tablet, correspondent
self.tablet = tablet
self.correspondent = correspondent
The reason for this is the way SA creates new values. From documentation Creation of New Values:
When a list append() event (or set add(), dictionary __setitem__(), or
scalar assignment event) is intercepted by the association proxy, it
instantiates a new instance of the “intermediary” object using its
constructor, passing as a single argument the given value.
In your case when you call tab.correspondents.append(cor), the Tablet_Correspondent.__init__ is called with single argument cor.
Solution? If you will only be adding Correspondents to the Tablet, then just switch the parameters in the __init__. In fact, remove the second parameter completely.
If, however, you will also be using cor.tablets.append(tab), then you need to explicitely use the creator argument to the association_proxy as explained in the documentation linked to above:
class Tablet(db.Model, GlyphMixin):
# ...
correspondents = association_proxy('tablet_correspondents', 'correspondent', creator=lambda cor: Tablet_Correspondent(correspondent=cor))
class Correspondent(db.Model, GlyphMixin):
# ...
tablets = association_proxy('correspondent_tablets', 'tablet', creator=lambda tab: Tablet_Correspondent(tablet=tab))
Like van said, the problem stay in the __init__ method of the Association Object.
In fact, if Tablet or Correspondent classes don't define an __init__ method or don't pass any parameter, the solution doesn't work (no argument expected).
I found an alternative solution. It's easy to detect which class has to be proxied, so it can be assigned to the right field (and still work on adding more associations):
class Tablet_Correspondent(db.Model):
# ...
def __init__(self, proxied=None):
if type(proxied) is Tablet:
self.tablet = proxied
elif type(proxied) is Correspondent:
self.correspondent = proxied
So say I have some classes X, Y and Z using SQLAlchemy declarative syntax to define some simple columns and relationships
Requirements:
At the class level, (X|Y|Z).primary_keys returns a collection of
the respective class' primary keys' (InstrumentedAttribute
objects) I also want (X|Y|Z).relations to reference the class'
relations in the same way
At the instance level, I would like the same attributes to reference
those attributes' instantiated values, whether they've been
populated using my own constructors, individual attributes
setters, or whatever SQLAlchemy does when it retrieves rows from
the db.
So far I have the following.
import collections
import sqlalchemy
import sqlalchemy.ext.declarative
from sqlalchemy import MetaData, Column, Table, ForeignKey, Integer, String, Date, Text
from sqlalchemy.orm import relationship, backref
class IndexedMeta(sqlalchemy.ext.declarative.DeclarativeMeta):
"""Metaclass to initialize some class-level collections on models"""
def __new__(cls, name, bases, defaultdict):
cls.pk_columns = set()
cls.relations = collections.namedtuple('RelationshipItem', 'one many')( set(), set())
return super().__new__(cls, name, bases, defaultdict)
Base = sqlalchemy.ext.declarative.declarative_base(metaclass=IndexedMeta)
def build_class_lens(cls, key, inst):
"""Populates the 'indexes' of primary key and relationship attributes with the attributes' names. Additionally, separates "x to many" relationships from "x to one" relationships and associates "x to one" relathionships with the local-side foreign key column"""
if isinstance(inst.property, sqlalchemy.orm.properties.ColumnProperty):
if inst.property.columns[0].primary_key:
cls.pk_columns.add(inst.key)
elif isinstance(inst.property, sqlalchemy.orm.properties.RelationshipProperty):
if inst.property.direction.name == ('MANYTOONE' or 'ONETOONE'):
local_column = cls.__mapper__.get_property_by_column(inst.property.local_side[0]).key
cls.relations.one.add( (local_column, inst.key) )
else:
cls.relations.many.add(inst.key)
sqlalchemy.event.listen(Base, 'attribute_instrument', build_class_lens)
class Meeting(Base):
__tablename__ = 'meetings'
def __init__(self, memo):
self.memo = memo
id = Column(Integer, primary_key=True)
date = Column(Date)
memo = Column('note', String(60), nullable=True)
category_name = Column('category', String(60), ForeignKey('categories.name'))
category = relationship("Category", backref=backref('meetings'))
topics = relationship("Topic",
secondary=meetings_topics,
backref="meetings")
...
...
Ok, so that gets me by on the class level, though I feel like I am doing silly things with metaclasses, and I get some strange intermittent errors where the 'sqlalchemy' module allegedly isn't recognized in build_class_lens and evals to Nonetype.
I am not quite sure how I should proceed at the instance level.
I've looked into the events interface. I see the ORM event init, but it seems to run prior to the __init__ function defined on my models, meaning the instance attributes haven't yet been populated at that time, so I can't build my 'lens' on them.
I also wonder if the Attribute event set might be of help. That is my next try, though i still wonder if it is the most appropriate way.
All in all I really wonder if I am missing some really elegant way to approach this problem.
I think the metaclass thing with declarative goes by the old XML saying, "if you have a problem, and use XML, now you have two problems". The metaclass in Python is useful pretty much as a hook to detect the construction of new classes, and that's about it. We now have enough events that there shouldn't be any need to use a metaclass beyond what declarative already does.
In this case I'd go a little further and say that the approach of trying to actively build up these collections is not really worth it - it's much easier to generate them lazily, as below:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
import collections
from sqlalchemy.orm.properties import RelationshipProperty
class memoized_classproperty(object):
"""A decorator that evaluates once at the class level,
assigns the new value to the class.
"""
def __init__(self, fget, doc=None):
self.fget = fget
self.__doc__ = doc or fget.__doc__
self.__name__ = fget.__name__
def __get__(desc, self, cls):
result = desc.fget(cls)
setattr(cls, desc.__name__, result)
return result
class Lens(object):
#memoized_classproperty
def pk_columns(cls):
return class_mapper(cls).primary_key
#memoized_classproperty
def relations(cls):
props = collections.namedtuple('RelationshipItem', 'one many')(set(), set())
# 0.8 will have "inspect(cls).relationships" here
mapper = class_mapper(cls)
for item in mapper.iterate_properties:
if isinstance(item, RelationshipProperty):
if item.direction.name == ('MANYTOONE' or 'ONETOONE'):
local_column = mapper.get_property_by_column(item.local_side[0]).key
props.one.add((local_column, item.key))
else:
props.many.add(item.key)
return props
Base= declarative_base(cls=Lens)
meetings_topics = Table("meetings_topics", Base.metadata,
Column('topic_id', Integer, ForeignKey('topic.id')),
Column('meetings_id', Integer, ForeignKey('meetings.id')),
)
class Meeting(Base):
__tablename__ = 'meetings'
def __init__(self, memo):
self.memo = memo
id = Column(Integer, primary_key=True)
date = Column(Date)
memo = Column('note', String(60), nullable=True)
category_name = Column('category', String(60), ForeignKey('categories.name'))
category = relationship("Category", backref=backref('meetings'))
topics = relationship("Topic",
secondary=meetings_topics,
backref="meetings")
class Category(Base):
__tablename__ = 'categories'
name = Column(String(50), primary_key=True)
class Topic(Base):
__tablename__ = 'topic'
id = Column(Integer, primary_key=True)
print Meeting.pk_columns
print Meeting.relations.one
# assignment is OK, since prop is memoized
Meeting.relations.one.add("FOO")
print Meeting.relations.one
I have a sqlalchemy model, where all most all tables/objects have a notes field. So to try follow the DRY principle, I moved the field to a mixin class.
class NotesMixin(object):
notes = sa.Column(sa.String(4000) , nullable=False, default='')
class Service(Base, NotesMixin):
__tablename__ = "service"
service_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
class Datacenter(Base, NotesMixin):
__tablename__ = "datacenter"
datacenter_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
class Network(Base, NotesMixin, StatusMixin):
__tablename__ = "network"
network_id = sa.Column(sa.Integer, primary_key=True)
etc...
Now the notes column is the first column in the model/db. I know it does not affect the functionality of my app, but it irritates me a bit to see notes before id, etc. Any way to move it to the end?
Found a cleaner solution:
Use the sqlalchemy.ext.declarative.declared_attr decorator in sqlalchemy 0.6.5 (sqlalchemy.util.classproperty in sqlalchemy <= 0.6.4)
class NotesMixin(object):
#declared_attr
def notes(cls):
return sa.Column(sa.String(4000) , nullable=False, default='')
According to the docs, this is "for columns that have foreign keys, as well as for the variety of mapper-level constructs that require destination-explicit context". While this is strictly speaking not the case here, it does so by calling the method (and creating the column) when the subclass is constructed, thus avoiding the need to make a copy. Which means the mixin column will come at the end. Probably a better solution than hacking _creation_order...
The easy answer: just create the database tables yourself, instead of having sqlalchemy do it with metadata.create_all().
If you don't find that acceptable, I'm afraid this would require a (small) change in sqlalchemy.ext.declarative itself, or you'd have to create your own metaclass and pass it to declarative_base() with the metaclass keyword argument. That class will then get used instead of the default DeclarativeMeta.
Explanation: sqlalchemy uses the creation order of the column properties, which it stores in the "private" attribute ._creation_order (generated when Column() is called). The declarative extension does mixin columns by creating a copy of the column object from your mixin class, and adding that to the class. The ._creation_order of this copy is set to the same value as the original property of the mixin class. As the mixin class is of course created first, it's column properties will have a lower creation order than the subclass.
So, to make your request possible, a new creation order should be assigned when the copy is made, rather than taking the original. You could try and make your own metaclass based on this explanation, and use that. But you might also try and ask the sqlalchemy developers. Maybe they are willing to accept this as a bug/feature request? At least, it seems like a minor (one line) change, that would not have a any effect other than the change you ask for (which arguably is better too).
One can also change the order of columns upon CREATE TABLE compilation (here exemplified for the postgresql dialect):
from sqlalchemy.schema import CreateTable
from sqlalchemy.ext.compiler import compiles
#compiles(CreateTable, 'postgresql')
def _compile_create_table(element, compiler, **kwargs):
element.columns = element.columns[::-1] # reverse order of columns
return compiler.visit_create_table(element)
This then works with metadata.create_all().
I know it has been a while, but I found a very simple solution for this:
class PriorityColumn(Column):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._creation_order = 1
This is a drop-in replacement for Column, if you are working with Mixins and you want your Derived class' attributes to be first.
class A:
a = Column(Integer)
b = Column(String)
class B(A, Base):
c = PriorityColumn(Integer)
d = PriorityColumn(Float)
# Your table will look like this:
# B(c, d, a, b)
I found that I could set the column order (to the last position) on the Mixin using:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
# get highest column order of all Column objects of this class.
last_position = max([value._creation_order
for key, value in vars(cls).items()
if isinstance(value, Column)])
col._creation_order = last_position + 0.5
return col
class Service(Base, NotesMixin):
__tablename__ = "service"
service_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
To set the column order based on the location of another column (similar to
alter table `some_table` modify `some_colum` `some_type` after
`some_other_column;
see https://stackoverflow.com/a/3822219/488331)
You can use:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
col._creation_order = cls.some_other_column._creation_order + 0.5
return col
NOTE: If you use + 1 you end up 2 columns back. I don't really understand why you can even use a decimal.
To set the column order based off of the location of the first column (make this always the 4th column) you could do:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
# get lowest column order of all Column objects of this class.
start_position = min([value._creation_order
for key, value in vars(cls).items()
if isinstance(value, Column)])
col._creation_order = start_position + 3.5
return col