I have 2 tables (not the true setup of my database).
tableclass.py:
class TableMeaningEN(sqla_base):
__tablename__ = 'MeaningEN'
id = sqla.Column(sqla.Integer, primary_key=True)
meaning = sqla.Column(sqla.String, primary_key=True)
class TableReadingON(sqla_base):
__tablename__ = 'ReadingON'
id = sqla.Column(sqla.Integer, primary_key=True)
reading = sqla.Column(sqla.String, primary_key=True)
Different column names
As you can see, both have a column id, but TableMeaningEN has meaning and TableReadingON has reading.
Normally (assuming you already have a session) you would add something like this:
session.add(TableMeaningEN(id=1, meaning='test'))
However I want to add dynamically entries to tables, so I have:
import tableclass as tc
for t_name in ['MeaningEN', 'ReadingON']:
session.add(getattr(tc, 'Table{}'.format(t_name))(id=1, ??='test'))
Question
How do I solve the problem that ?? is in one table meaning and in the other reading?
Tried:
columns = sqla.inspect(getattr(tc, 'Table{}'.format(overwrite))).columns.keys()
session.add(getattr(tc, 'Table{}'.format(t_name))(id=1, columns[1]='test'))
I tried this, but however that's not allowed.
If tables are really different by just one attribute, you could create a constructor for both which takes positional arguments as well, such as:
class TableMeaningEN(sqla_base):
def __init__(self, id, meaning):
self.id, self.meaning = id, meaning
# similar __init__ for the other class/table
# then use the following:
session.add(getattr(tc, 'Table{}'.format(t_name))(1, 'test'))
An alternative would be to dynamically create the keyword arguments, assuming there is naming convention:
import tableclass as tc
for t_name in ['MeaningEN', 'ReadingON']:
cls = getattr(tc, 'Table{}'.format(t_name))
fld_name = t_name.lower()[:-2]
kw = {'id': 1, fld_name: 'test'}
session.add(cls(**kw))
Here's a little table for holding IP address ranges in the form of a start address, end address, and number of IPs within the range
class IpRange(Base):
__tablename__ = 'ip_range'
ip_range_id = Column(Integer, Sequence('ip_range_id_seq'), primary_key=True)
start_ip = Column(String(15))
end_ip = Column(String(15))
num_ips = Column(Integer)
What I'd love to do is be able to create an object using a variety of styles, and have the class figure out how to populate its own fields.
foo = IpRange(ip='192.168.0.1')
foo = IpRange(ip='192.168.0.0/24')
foo = IpRange(ip='192.168.0.0-192.168.0.255')
It wouldn't be too hard to write a function that could parse various IP address/range notations:
def parseIp(desired_format):
... stuff to parse any valid IP address/network format ...
if desired_format == 'start':
return start_ip
if desired_format == 'end':
return end_ip
if desired_format == 'num_ips':
return num_ips
And I was hoping I could then use the default method for my Columns to get the data that each column needed:
class IpRange(Base):
__tablename__ = 'ip_range'
ip_range_id = Column(Integer, Sequence('ip_range_id_seq'), primary_key=True)
start_ip = Column(String(15), default=parseIp('start'))
end_ip = Column(String(15), default=parseIp('end'))
num_ips = Column(Integer, default=parseIp('num_ips')
However, that is not valid SQLAlchemy syntax. The documentation speaks of context-sensitive default columns, but the syntax does not allow parameters to be passed to the function. So even though I could call the function with default=parseIp, I wouldn't be able to tell it what kind of return value I'm looking for.
Is there a way to do this within Column specification for SQLAlchemy?
Or as an alternative idea, should I turn parseIp into a helper script that just generates the new IpRange object itself and returns it back to the caller? Something like:
def parseIp(ipstring):
... parse data ...
return IpRange(start_ip=parsed_start_ip, end_ip=parsed_end_ip, num_ips=parsed_num_ips)
>>> ipobj = parseIp('192.168.0.0/24')
Just override the constructor. Example:
class IpRange(Base):
def __init__(self, ip=None, **kwargs):
if ip is not None:
self.start_ip, self.end_ip = my_super_awesome_cidr_parser(ip)
else:
super(IpRange, self).__init__(**kwargs)
This question already has answers here:
How to serialize SqlAlchemy result to JSON?
(37 answers)
Closed 4 years ago.
I'm trying to jsonify a SQLAlchemy result set in Flask/Python.
The Flask mailing list suggested the following method http://librelist.com/browser//flask/2011/2/16/jsonify-sqlalchemy-pagination-collection-result/#04a0754b63387f87e59dda564bde426e :
return jsonify(json_list = qryresult)
However I'm getting the following error back:
TypeError: <flaskext.sqlalchemy.BaseQuery object at 0x102c2df90>
is not JSON serializable
What am I overlooking here?
I have found this question: How to serialize SqlAlchemy result to JSON? which seems very similar however I didn't know whether Flask had some magic to make it easier as the mailing list post suggested.
Edit: for clarification, this is what my model looks like
class Rating(db.Model):
__tablename__ = 'rating'
id = db.Column(db.Integer, primary_key=True)
fullurl = db.Column(db.String())
url = db.Column(db.String())
comments = db.Column(db.Text)
overall = db.Column(db.Integer)
shipping = db.Column(db.Integer)
cost = db.Column(db.Integer)
honesty = db.Column(db.Integer)
communication = db.Column(db.Integer)
name = db.Column(db.String())
ipaddr = db.Column(db.String())
date = db.Column(db.String())
def __init__(self, fullurl, url, comments, overall, shipping, cost, honesty, communication, name, ipaddr, date):
self.fullurl = fullurl
self.url = url
self.comments = comments
self.overall = overall
self.shipping = shipping
self.cost = cost
self.honesty = honesty
self.communication = communication
self.name = name
self.ipaddr = ipaddr
self.date = date
It seems that you actually haven't executed your query. Try following:
return jsonify(json_list = qryresult.all())
[Edit]: Problem with jsonify is, that usually the objects cannot be jsonified automatically. Even Python's datetime fails ;)
What I have done in the past, is adding an extra property (like serialize) to classes that need to be serialized.
def dump_datetime(value):
"""Deserialize datetime object into string form for JSON processing."""
if value is None:
return None
return [value.strftime("%Y-%m-%d"), value.strftime("%H:%M:%S")]
class Foo(db.Model):
# ... SQLAlchemy defs here..
def __init__(self, ...):
# self.foo = ...
pass
#property
def serialize(self):
"""Return object data in easily serializable format"""
return {
'id' : self.id,
'modified_at': dump_datetime(self.modified_at),
# This is an example how to deal with Many2Many relations
'many2many' : self.serialize_many2many
}
#property
def serialize_many2many(self):
"""
Return object's relations in easily serializable format.
NB! Calls many2many's serialize property.
"""
return [ item.serialize for item in self.many2many]
And now for views I can just do:
return jsonify(json_list=[i.serialize for i in qryresult.all()])
[Edit 2019]:
In case you have more complex objects or circular references, use a library like marshmallow).
Here's what's usually sufficient for me:
I create a serialization mixin which I use with my models. The serialization function basically fetches whatever attributes the SQLAlchemy inspector exposes and puts it in a dict.
from sqlalchemy.inspection import inspect
class Serializer(object):
def serialize(self):
return {c: getattr(self, c) for c in inspect(self).attrs.keys()}
#staticmethod
def serialize_list(l):
return [m.serialize() for m in l]
All that's needed now is to extend the SQLAlchemy model with the Serializer mixin class.
If there are fields you do not wish to expose, or that need special formatting, simply override the serialize() function in the model subclass.
class User(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String)
password = db.Column(db.String)
# ...
def serialize(self):
d = Serializer.serialize(self)
del d['password']
return d
In your controllers, all you have to do is to call the serialize() function (or serialize_list(l) if the query results in a list) on the results:
def get_user(id):
user = User.query.get(id)
return json.dumps(user.serialize())
def get_users():
users = User.query.all()
return json.dumps(User.serialize_list(users))
I had the same need, to serialize into json. Take a look at this question. It shows how to discover columns programmatically. So, from that I created the code below. It works for me, and I'll be using it in my web app. Happy coding!
def to_json(inst, cls):
"""
Jsonify the sql alchemy query result.
"""
convert = dict()
# add your coversions for things like datetime's
# and what-not that aren't serializable.
d = dict()
for c in cls.__table__.columns:
v = getattr(inst, c.name)
if c.type in convert.keys() and v is not None:
try:
d[c.name] = convert[c.type](v)
except:
d[c.name] = "Error: Failed to covert using ", str(convert[c.type])
elif v is None:
d[c.name] = str()
else:
d[c.name] = v
return json.dumps(d)
class Person(base):
__tablename__ = 'person'
id = Column(Integer, Sequence('person_id_seq'), primary_key=True)
first_name = Column(Text)
last_name = Column(Text)
email = Column(Text)
#property
def json(self):
return to_json(self, self.__class__)
Here's my approach:
https://github.com/n0nSmoker/SQLAlchemy-serializer
pip install SQLAlchemy-serializer
You can easily add mixin to your model and then just call
.to_dict() method on its instance.
You also can write your own mixin on base of SerializerMixin.
For a flat query (no joins) you can do this
#app.route('/results/')
def results():
data = Table.query.all()
result = [d.__dict__ for d in data]
return jsonify(result=result)
and if you only want to return certain columns from the database you can do this
#app.route('/results/')
def results():
cols = ['id', 'url', 'shipping']
data = Table.query.all()
result = [{col: getattr(d, col) for col in cols} for d in data]
return jsonify(result=result)
Ok, I've been working on this for a few hours, and I've developed what I believe to be the most pythonic solution yet. The following code snippets are python3 but shouldn't be too horribly painful to backport if you need.
The first thing we're gonna do is start with a mixin that makes your db models act kinda like dicts:
from sqlalchemy.inspection import inspect
class ModelMixin:
"""Provide dict-like interface to db.Model subclasses."""
def __getitem__(self, key):
"""Expose object attributes like dict values."""
return getattr(self, key)
def keys(self):
"""Identify what db columns we have."""
return inspect(self).attrs.keys()
Now we're going to define our model, inheriting the mixin:
class MyModel(db.Model, ModelMixin):
id = db.Column(db.Integer, primary_key=True)
foo = db.Column(...)
bar = db.Column(...)
# etc ...
That's all it takes to be able to pass an instance of MyModel() to dict() and get a real live dict instance out of it, which gets us quite a long way towards making jsonify() understand it. Next, we need to extend JSONEncoder to get us the rest of the way:
from flask.json import JSONEncoder
from contextlib import suppress
class MyJSONEncoder(JSONEncoder):
def default(self, obj):
# Optional: convert datetime objects to ISO format
with suppress(AttributeError):
return obj.isoformat()
return dict(obj)
app.json_encoder = MyJSONEncoder
Bonus points: if your model contains computed fields (that is, you want your JSON output to contain fields that aren't actually stored in the database), that's easy too. Just define your computed fields as #propertys, and extend the keys() method like so:
class MyModel(db.Model, ModelMixin):
id = db.Column(db.Integer, primary_key=True)
foo = db.Column(...)
bar = db.Column(...)
#property
def computed_field(self):
return 'this value did not come from the db'
def keys(self):
return super().keys() + ['computed_field']
Now it's trivial to jsonify:
#app.route('/whatever', methods=['GET'])
def whatever():
return jsonify(dict(results=MyModel.query.all()))
If you are using flask-restful you can use marshal:
from flask.ext.restful import Resource, fields, marshal
topic_fields = {
'title': fields.String,
'content': fields.String,
'uri': fields.Url('topic'),
'creator': fields.String,
'created': fields.DateTime(dt_format='rfc822')
}
class TopicListApi(Resource):
def get(self):
return {'topics': [marshal(topic, topic_fields) for topic in DbTopic.query.all()]}
You need to explicitly list what you are returning and what type it is, which I prefer anyway for an api. Serialization is easily taken care of (no need for jsonify), dates are also not a problem. Note that the content for the uri field is automatically generated based on the topic endpoint and the id.
Here's my answer if you're using the declarative base (with help from some of the answers already posted):
# in your models definition where you define and extend declarative_base()
from sqlalchemy.ext.declarative import declarative_base
...
Base = declarative_base()
Base.query = db_session.query_property()
...
# define a new class (call "Model" or whatever) with an as_dict() method defined
class Model():
def as_dict(self):
return { c.name: getattr(self, c.name) for c in self.__table__.columns }
# and extend both the Base and Model class in your model definition, e.g.
class Rating(Base, Model):
____tablename__ = 'rating'
id = db.Column(db.Integer, primary_key=True)
fullurl = db.Column(db.String())
url = db.Column(db.String())
comments = db.Column(db.Text)
...
# then after you query and have a resultset (rs) of ratings
rs = Rating.query.all()
# you can jsonify it with
s = json.dumps([r.as_dict() for r in rs], default=alchemyencoder)
print (s)
# or if you have a single row
r = Rating.query.first()
# you can jsonify it with
s = json.dumps(r.as_dict(), default=alchemyencoder)
# you will need this alchemyencoder where your are calling json.dumps to handle datetime and decimal format
# credit to Joonas # http://codeandlife.com/2014/12/07/sqlalchemy-results-to-json-the-easy-way/
def alchemyencoder(obj):
"""JSON encoder function for SQLAlchemy special classes."""
if isinstance(obj, datetime.date):
return obj.isoformat()
elif isinstance(obj, decimal.Decimal):
return float(obj)
Flask-Restful 0.3.6 the Request Parsing recommend marshmallow
marshmallow is an ORM/ODM/framework-agnostic library for converting
complex datatypes, such as objects, to and from native Python
datatypes.
A simple marshmallow example is showing below.
from marshmallow import Schema, fields
class UserSchema(Schema):
name = fields.Str()
email = fields.Email()
created_at = fields.DateTime()
from marshmallow import pprint
user = User(name="Monty", email="monty#python.org")
schema = UserSchema()
result = schema.dump(user)
pprint(result)
# {"name": "Monty",
# "email": "monty#python.org",
# "created_at": "2014-08-17T14:54:16.049594+00:00"}
The core features contain
Declaring Schemas
Serializing Objects (“Dumping”)
Deserializing Objects (“Loading”)
Handling Collections of Objects
Validation
Specifying Attribute Names
Specifying Serialization/Deserialization Keys
Refactoring: Implicit Field Creation
Ordering Output
“Read-only” and “Write-only” Fields
Specify Default Serialization/Deserialization Values
Nesting Schemas
Custom Fields
Here is a way to add an as_dict() method on every class, as well as any other method you want to have on every single class.
Not sure if this is the desired way or not, but it works...
class Base(object):
def as_dict(self):
return dict((c.name,
getattr(self, c.name))
for c in self.__table__.columns)
Base = declarative_base(cls=Base)
I've been looking at this problem for the better part of a day, and here's what I've come up with (credit to https://stackoverflow.com/a/5249214/196358 for pointing me in this direction).
(Note: I'm using flask-sqlalchemy, so my model declaration format is a bit different from straight sqlalchemy).
In my models.py file:
import json
class Serializer(object):
__public__ = None
"Must be implemented by implementors"
def to_serializable_dict(self):
dict = {}
for public_key in self.__public__:
value = getattr(self, public_key)
if value:
dict[public_key] = value
return dict
class SWEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Serializer):
return obj.to_serializable_dict()
if isinstance(obj, (datetime)):
return obj.isoformat()
return json.JSONEncoder.default(self, obj)
def SWJsonify(*args, **kwargs):
return current_app.response_class(json.dumps(dict(*args, **kwargs), cls=SWEncoder, indent=None if request.is_xhr else 2), mimetype='application/json')
# stolen from https://github.com/mitsuhiko/flask/blob/master/flask/helpers.py
and all my model objects look like this:
class User(db.Model, Serializer):
__public__ = ['id','username']
... field definitions ...
In my views I call SWJsonify wherever I would have called Jsonify, like so:
#app.route('/posts')
def posts():
posts = Post.query.limit(PER_PAGE).all()
return SWJsonify({'posts':posts })
Seems to work pretty well. Even on relationships. I haven't gotten far with it, so YMMV, but so far it feels pretty "right" to me.
Suggestions welcome.
I was looking for something like the rails approach used in ActiveRecord to_json and implemented something similar using this Mixin after being unsatisfied with other suggestions. It handles nested models, and including or excluding attributes of the top level or nested models.
class Serializer(object):
def serialize(self, include={}, exclude=[], only=[]):
serialized = {}
for key in inspect(self).attrs.keys():
to_be_serialized = True
value = getattr(self, key)
if key in exclude or (only and key not in only):
to_be_serialized = False
elif isinstance(value, BaseQuery):
to_be_serialized = False
if key in include:
to_be_serialized = True
nested_params = include.get(key, {})
value = [i.serialize(**nested_params) for i in value]
if to_be_serialized:
serialized[key] = value
return serialized
Then, to get the BaseQuery serializable I extended BaseQuery
class SerializableBaseQuery(BaseQuery):
def serialize(self, include={}, exclude=[], only=[]):
return [m.serialize(include, exclude, only) for m in self]
For the following models
class ContactInfo(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
full_name = db.Column(db.String())
source = db.Column(db.String())
source_id = db.Column(db.String())
email_addresses = db.relationship('EmailAddress', backref='contact_info', lazy='dynamic')
phone_numbers = db.relationship('PhoneNumber', backref='contact_info', lazy='dynamic')
class EmailAddress(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
email_address = db.Column(db.String())
type = db.Column(db.String())
contact_info_id = db.Column(db.Integer, db.ForeignKey('contact_info.id'))
class PhoneNumber(db.Model, Serializer):
id = db.Column(db.Integer, primary_key=True)
phone_number = db.Column(db.String())
type = db.Column(db.String())
contact_info_id = db.Column(db.Integer, db.ForeignKey('contact_info.id'))
phone_numbers = db.relationship('Invite', backref='phone_number', lazy='dynamic')
You could do something like
#app.route("/contact/search", methods=['GET'])
def contact_search():
contact_name = request.args.get("name")
matching_contacts = ContactInfo.query.filter(ContactInfo.full_name.like("%{}%".format(contact_name)))
serialized_contact_info = matching_contacts.serialize(
include={
"phone_numbers" : {
"exclude" : ["contact_info", "contact_info_id"]
},
"email_addresses" : {
"exclude" : ["contact_info", "contact_info_id"]
}
}
)
return jsonify(serialized_contact_info)
I was working with a sql query defaultdict of lists of RowProxy objects named jobDict
It took me a while to figure out what Type the objects were.
This was a really simple quick way to resolve to some clean jsonEncoding just by typecasting the row to a list and by initially defining the dict with a value of list.
jobDict = defaultdict(list)
def set_default(obj):
# trickyness needed here via import to know type
if isinstance(obj, RowProxy):
return list(obj)
raise TypeError
jsonEncoded = json.dumps(jobDict, default=set_default)
I just want to add my method to do this.
just define a custome json encoder to serilize your db models.
class ParentEncoder(json.JSONEncoder):
def default(self, obj):
# convert object to a dict
d = {}
if isinstance(obj, Parent):
return {"id": obj.id, "name": obj.name, 'children': list(obj.child)}
if isinstance(obj, Child):
return {"id": obj.id, "name": obj.name}
d.update(obj.__dict__)
return d
then in your view function
parents = Parent.query.all()
dat = json.dumps({"data": parents}, cls=ParentEncoder)
resp = Response(response=dat, status=200, mimetype="application/json")
return (resp)
it works well though the parent have relationships
It's been a lot of times and there are lots of valid answers, but the following code block seems to work:
my_object = SqlAlchemyModel()
my_serializable_obj = my_object.__dict__
del my_serializable_obj["_sa_instance_state"]
print(jsonify(my_serializable_object))
I'm aware that this is not a perfect solution, nor as elegant as the others, however for those who want o quick fix, they might try this.
I frequently make models with Text columns that hold Markdown formatted richtext. My models look like this:
class Document(Base):
id = Column(Integer, primary_key=True)
title = Column(Unicode(250))
description = Column(Text)
description_html = Column(Text)
My edit forms (a) read from and write to description and then (b) write the Markdown formatted version to description_html. My (Jinja2) view templates (c) load the HTML version with {{ doc.description_html|safe }}.
I'd like to cut down these three recurring operations into one Column definition, like this:
class Document(Base):
id = Column(Integer, primary_key=True)
title = Column(Unicode(250))
description = Column(MarkdownText)
Where MarkdownText is a new column type that:
Makes two columns in the database table (description and description_html),
Upon writes to the column, also writes a Markdown formatted version to the html column, and
Provides a __html__() method that returns the contents of the html column. This will allow it to be used from a Jinja2 template as {{ doc.description }} without the safe filter.
Question: Is #1 possible? Can I define a column that makes two columns?
Here we go - now with composite columns:
from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, Text
from sqlalchemy.orm import composite, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('sqlite:///')
session = sessionmaker(bind=engine)()
Base = declarative_base()
class MarkdownText(object):
def __init__(self, text):
self._text = text
self._html = "<html>%s</html>" % text
#classmethod
def _from_db(cls, text, html):
mt = MarkdownText(text)
mt._html = html
return mt
def __composite_values__(self):
return (self._text, self._html)
def __str__(self):
return self._text
#property
def __html__(self):
return self._html
class Foo(Base):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)
a = composite(MarkdownText._from_db,
Column('_text', Text),
Column('_html', Text))
def __init__(self, a):
self.a = MarkdownText(a)
def __repr__(self):
return '(%s)' % (self.a)
Base.metadata.create_all(engine)
session.add_all([Foo('test'), Foo('nips')])
session.commit()
x = session.query(Foo).all()
print x
print x[0].a.__html__
print x[0].a
And this gives us nicely:
[(test), (nips)]
<html>test</html>
test
Instead of answering to your bullet points, I better ask you this: "Do you really want to store both plain text and html text in database?". Here's how I would do it:
def text2html(text):
# TODO: Implement me!
pass
class Document(Base):
id = Column(Integer, primary_key=True)
title = Column(Unicode(250))
description = Column(Text)
#property
def description_html(self):
return text2html(self.description)
And in view the html description can be accessed just as document.description_html...
I have a sqlalchemy model, where all most all tables/objects have a notes field. So to try follow the DRY principle, I moved the field to a mixin class.
class NotesMixin(object):
notes = sa.Column(sa.String(4000) , nullable=False, default='')
class Service(Base, NotesMixin):
__tablename__ = "service"
service_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
class Datacenter(Base, NotesMixin):
__tablename__ = "datacenter"
datacenter_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
class Network(Base, NotesMixin, StatusMixin):
__tablename__ = "network"
network_id = sa.Column(sa.Integer, primary_key=True)
etc...
Now the notes column is the first column in the model/db. I know it does not affect the functionality of my app, but it irritates me a bit to see notes before id, etc. Any way to move it to the end?
Found a cleaner solution:
Use the sqlalchemy.ext.declarative.declared_attr decorator in sqlalchemy 0.6.5 (sqlalchemy.util.classproperty in sqlalchemy <= 0.6.4)
class NotesMixin(object):
#declared_attr
def notes(cls):
return sa.Column(sa.String(4000) , nullable=False, default='')
According to the docs, this is "for columns that have foreign keys, as well as for the variety of mapper-level constructs that require destination-explicit context". While this is strictly speaking not the case here, it does so by calling the method (and creating the column) when the subclass is constructed, thus avoiding the need to make a copy. Which means the mixin column will come at the end. Probably a better solution than hacking _creation_order...
The easy answer: just create the database tables yourself, instead of having sqlalchemy do it with metadata.create_all().
If you don't find that acceptable, I'm afraid this would require a (small) change in sqlalchemy.ext.declarative itself, or you'd have to create your own metaclass and pass it to declarative_base() with the metaclass keyword argument. That class will then get used instead of the default DeclarativeMeta.
Explanation: sqlalchemy uses the creation order of the column properties, which it stores in the "private" attribute ._creation_order (generated when Column() is called). The declarative extension does mixin columns by creating a copy of the column object from your mixin class, and adding that to the class. The ._creation_order of this copy is set to the same value as the original property of the mixin class. As the mixin class is of course created first, it's column properties will have a lower creation order than the subclass.
So, to make your request possible, a new creation order should be assigned when the copy is made, rather than taking the original. You could try and make your own metaclass based on this explanation, and use that. But you might also try and ask the sqlalchemy developers. Maybe they are willing to accept this as a bug/feature request? At least, it seems like a minor (one line) change, that would not have a any effect other than the change you ask for (which arguably is better too).
One can also change the order of columns upon CREATE TABLE compilation (here exemplified for the postgresql dialect):
from sqlalchemy.schema import CreateTable
from sqlalchemy.ext.compiler import compiles
#compiles(CreateTable, 'postgresql')
def _compile_create_table(element, compiler, **kwargs):
element.columns = element.columns[::-1] # reverse order of columns
return compiler.visit_create_table(element)
This then works with metadata.create_all().
I know it has been a while, but I found a very simple solution for this:
class PriorityColumn(Column):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._creation_order = 1
This is a drop-in replacement for Column, if you are working with Mixins and you want your Derived class' attributes to be first.
class A:
a = Column(Integer)
b = Column(String)
class B(A, Base):
c = PriorityColumn(Integer)
d = PriorityColumn(Float)
# Your table will look like this:
# B(c, d, a, b)
I found that I could set the column order (to the last position) on the Mixin using:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
# get highest column order of all Column objects of this class.
last_position = max([value._creation_order
for key, value in vars(cls).items()
if isinstance(value, Column)])
col._creation_order = last_position + 0.5
return col
class Service(Base, NotesMixin):
__tablename__ = "service"
service_id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(255), nullable=False, index=True, unique=True)
To set the column order based on the location of another column (similar to
alter table `some_table` modify `some_colum` `some_type` after
`some_other_column;
see https://stackoverflow.com/a/3822219/488331)
You can use:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
col._creation_order = cls.some_other_column._creation_order + 0.5
return col
NOTE: If you use + 1 you end up 2 columns back. I don't really understand why you can even use a decimal.
To set the column order based off of the location of the first column (make this always the 4th column) you could do:
#declared_attr
def notes(cls):
col = sa.Column(sa.String(4000) , nullable=False, default='')
# get lowest column order of all Column objects of this class.
start_position = min([value._creation_order
for key, value in vars(cls).items()
if isinstance(value, Column)])
col._creation_order = start_position + 3.5
return col