According to the documentation, using order_columns you can specify which columns allow sorting, which adds blue arrows in the header to select sorting in ascending or descending order.
However I also want to order by a relationship called "softwareproduct" to another table but when I add that to order_columns, it crashes (as it is not a real column but a relationship). The documentation also lists order_rel_fields, which I tried as well but that doesn't add a sorting function to the "softwareproduct" "column"/relationship:
Add_columns, edit_columns, show_columns and list_columns work perfectly fine, only order doesn't, even though "softwareproduct" isn't technically a real column but a relationship.
How can I let the users sort on such relationships?
models.py
[...]
class Softwareproduct(Model):
suffix = Column(String(200), primary_key=True)
label = Column(String(200), nullable=False)
[...]
def __repr__(self):
return self.label
class Citation(Model):
suffix = Column(String(200), primary_key=True)
swp_suffix = Column(String(200), ForeignKey("softwareproduct.suffix"),nullable=False)
softwareproduct = relationship("Softwareproduct")
label = Column(String(200), nullable=False)
def __repr__(self):
return self.label
views.py
class CitationView(ModelView):
datamodel = SQLAInterface(Citation)
label_columns = {'label':'Citation', 'suffix': 'ID'}
add_columns = ['softwareproduct', "label", "suffix", "classified"]
edit_columns = ['softwareproduct', "label", "suffix","classified"]
show_columns = ['softwareproduct', "label", "suffix","classified"]
list_columns = ['softwareproduct', "label", "suffix","classified"]
order_columns= ["label","suffix"]
order_rel_fields = {'softwareproduct': ('label', 'asc')}
related_views = [ClassifiedView]
Change
order_columns= ["label","suffix"]
To
base_order = ("label", "asc")
The main issue is with me trying to use the append function, and getting an SQL integrity error when I call session.commit(), even when the object is not being re-created. Here's my case:
I have a one to many relationship in a Flask SQLAlchemy database.
It's a Tag-Category relationship, tags belong to only one category, and by definition a category has many tags. For the sake of explanation, tags are being used on X objects.
In my Flask models file, I did this:
class Category(db.Model):
__tablename__ = 'category'
id = db.Column(db.Integer, primary_key = True)
category = db.Column(db.Text, unique = True)
tags = db.relationship('Tag', backref = 'category', lazy = 'dynamic')
class Tag(db.Model):
__tablename__ = 'tags'
id = db.Column(db.Integer, primary_key = True)
tag = db.Column(db.Text, unique = True)
category_id = db.Column(db.Integer, db.ForeignKey('category.id'))
This should define a Category class to which I can "append" tags using the session.append(Tag) function.
Categories and tags are unique, I wanna take input from the user, and add the category/tag as the user specifies. In some cases, the user might want to just add a tag to an existing category.
The way I'm taking input is by using a form, from which I receive the category and tag, and I create the objects and add them to the database if they don't exist already, and if they do, I just want to append the tags to the X's or to the categories.
Here's my code:
category = Category(category = request.form['categories']) #Category related to its respective list of tags
tag_list = request.form.getlist('tags') # tags are received as a list
for tag_entity in tag_list:
tag = Tag(tag = tag_entity)
X.tags.append(tag)
# Checking if the category exists or not in the database
list1 = []
if len(db.session.query(Category.tags).filter(Category.category == request.form['categories']).all()) != 0:
list1 = db.session.query(Category.tags).filter(Category.category == request.form['categories']).all()
list1.append(tag)
for entry in list1:
print entry.tag
list1 = [entry.tag for entry in list1]
# Here I should check if the tag already exists as well:
db.session.add(tag)
# Only add category if it doesn't already exist
if len(db.session.query(Category.id).filter(Category.category == request.form['categories']).all()) == 0:
print "I'm here"
db.session.add(Category(category = request.form['categories'], tags = list1))
else:
print "woops im here"
newCat = Category(category=request.form['categories'])
newCat.tags.extend(list1)
db.session.merge(newCat)
db.session.add(X)
db.session.commit()
I know my code is really messy and it looks like it doesn't do what it's supposed to, that's a result of me just failing over and over again.
My main issue was with me trying to call category.tags.append(Tag), then getting an integrity error when session.commit() was being called even when the category was not even re-created.
I hope what I wrote makes sense, any help would be appreciated.
I've created models for my database:
class Album(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
year = db.Column(db.String(4))
tracklist = db.relationship('Track', secondary=tracklist,
backref=db.backref('albums',
lazy='dynamic'), lazy='dynamic')
class Track(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
class Artist(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(128))
releases = db.relationship('Track', secondary=releases,
backref=db.backref('artists',
lazy='dynamic'), lazy='dynamic')
They are many-to-many related Album <--> Track <--> Artist
Next, I have this form:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
My idea is to give the user freedom in filling desired combination of forms (but at least one is required), so I've got this function, which recieves SearchForm().data (an immutable dict 'field_name': 'data'):
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
if form.search_by_album.data:
query = query.filter(Album.title == form.search_by_album.data)
if form.search_by_artist.data:
query = query.filter(Artist.name == form.search_by_artist.data)
if form.search_track.data:
query = query.filter(Track.title == form.search_track.data)
if form.year.data:
query = query.filter(Album.year == form.year.data)
result = query.all()
return result
My question is if there is a more abstract way of adding filters in the function above? If one day I decide to add more columns to my tables (or even create new tables), I will have to add more monstrous ifs to constrcut_query(), which will eventually grow enormous. Or such an abstractions is not a pythonic way because "Explicit is better than implicit"?
PS
I know about forms from models, but I don't think that they are my case
One way would be associating the filter-attribute with the fields at some place, e.g. as a class attribute on the form itself:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
# map form fields to database fields/attributes
field_to_attr = {search_by_album: Album.title,
search_by_artist: Artist.name,
search_track: Track.title,
year: Album.year}
When building the query, you could then build the where clause in a pretty comfortable way:
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
for field in form:
if field.data:
query = query.filter(form.field_to_attr[field] == field.data)
# or:
# for field, attr in form.field_to_attr.items():
# if field.data:
# query = query.filter(attr == field.data)
result = query.all()
return result
Adding new fields and attributes to filter on would then only translate to the creating the field and its mapping to an attribute.
I have model, which is look like this
class Question(db.Model):
__tablename__ = 'questions'
id = db.Column(db.Integer, primary_key=True)
question_title = db.Column(db.String(64))
question_text = db.Column(db.String(256), nullable=False)
votest = db.Column(db.Integer, default=0)
answers = db.relationship('Answer', backref='question')
def __repr__(self):
return '<QTitle: %s>\n<QText: %s>' % (self.question_title,
self.question_text)
The thing is to check if column question_title is empty, and assign it to first 64 symbols of column question_text + '...'.
Which is the best way to implement this?
Maybe the best way is not to implement this in model, but make this check in view?
The best way is to do this in your view after you read the data from a form (if this is the case).
You could create a helper function which takes as arguments question_title and question_text and verifies if the question_title is empty and if it is then assign to it the first 64 characters from question_text. And you call this function in your view after you get the data from the form (or from wherever you get the data). And after that you put the data in the database.
Check this out:
# Query all Question objects with empty question_title
empties = session.query(Question).filter_by(question_title=None).all()
# Iterate over them
for empty_q in empties:
# Apply the logic
empty_q.question_title = empty_q.question_text[:64] + '...'
session.add(empty_q)
session.commit()
print "Updated Empty Questions!"
I didn't test it, but it should work!
To provide an activity log in my SQLAlchemy-based app, I have a model like this:
class ActivityLog(Base):
__tablename__ = 'activitylog'
id = Column(Integer, primary_key=True)
activity_by_id = Column(Integer, ForeignKey('users.id'), nullable=False)
activity_by = relation(User, primaryjoin=activity_by_id == User.id)
activity_at = Column(DateTime, default=datetime.utcnow, nullable=False)
activity_type = Column(SmallInteger, nullable=False)
target_table = Column(Unicode(20), nullable=False)
target_id = Column(Integer, nullable=False)
target_title = Column(Unicode(255), nullable=False)
The log contains entries for multiple tables, so I can't use ForeignKey relations. Log entries are made like this:
doc = Document(name=u'mydoc', title=u'My Test Document',
created_by=user, edited_by=user)
session.add(doc)
session.flush() # See note below
log = ActivityLog(activity_by=user, activity_type=ACTIVITY_ADD,
target_table=Document.__table__.name, target_id=doc.id,
target_title=doc.title)
session.add(log)
This leaves me with three problems:
I have to flush the session before my doc object gets an id. If I had used a ForeignKey column and a relation mapper, I could have simply called ActivityLog(target=doc) and let SQLAlchemy do the work. Is there any way to work around needing to flush by hand?
The target_table parameter is too verbose. I suppose I could solve this with a target property setter in ActivityLog that automatically retrieves the table name and id from a given instance.
Biggest of all, I'm not sure how to retrieve a model instance from the database. Given an ActivityLog instance log, calling self.session.query(log.target_table).get(log.target_id) does not work, as query() expects a model as parameter.
One workaround appears to be to use polymorphism and derive all my models from a base model which ActivityLog recognises. Something like this:
class Entity(Base):
__tablename__ = 'entities'
id = Column(Integer, primary_key=True)
title = Column(Unicode(255), nullable=False)
edited_at = Column(DateTime, onupdate=datetime.utcnow, nullable=False)
entity_type = Column(Unicode(20), nullable=False)
__mapper_args__ = {'polymorphic_on': entity_type}
class Document(Entity):
__tablename__ = 'documents'
__mapper_args__ = {'polymorphic_identity': 'document'}
body = Column(UnicodeText, nullable=False)
class ActivityLog(Base):
__tablename__ = 'activitylog'
id = Column(Integer, primary_key=True)
...
target_id = Column(Integer, ForeignKey('entities.id'), nullable=False)
target = relation(Entity)
If I do this, ActivityLog(...).target will give me a Document instance when it refers to a Document, but I'm not sure it's worth the overhead of having two tables for everything. Should I go ahead and do it this way?
One way to solve this is polymorphic associations. It should solve all 3 of your issues and also make database foreign key constraints work. See the polymorphic association example in SQLAlchemy source. Mike Bayer has an old blogpost that discusses this in greater detail.
Definitely go through the blogpost and examples Ants linked to. I did not find the explanation confusion, but rather assuming some more experience on the topic.
Few things I can suggest are:
ForeignKeys: in general I agree they are a good thing go have, but I am not sure it is conceptually important in your case: you seem to be using this ActivityLog as an orthogonal cross-cutting concern (AOP); but version with foreign keys would effectively make your business objects aware of the ActivityLog. Another problem with having FK for audit purposes using schema setup you have is that if you allow object deletion, FK requirement will delete all the ActivityLog entries for this object.
Automatic logging: you are doing all this logging manually whenever you create/modify(/delete) the object. With SA you could implement a SessionExtension with before_commit which would do the job for you automatically.
In this way you completely can avoid writing parts like below:
log = ActivityLog(activity_by=user, activity_type=ACTIVITY_ADD,
target_table=Document.__table__.name, target_id=doc.id,
target_title=doc.title)
session.add(log)
EDIT-1: complete sample code added
The code is based on the first non-FK version from http://techspot.zzzeek.org/?p=13.
The choice not to use FK is based on the fact that for audit purposes when the
main object is deleted, it should not cascade to delete all the audit log entries.
Also this keeps auditable objects unaware of the fact they are being audited.
Implementation uses a SA one-to-many relationship. It is possible that some
objects are modified many times, which will result in many audit log entries.
By default SA will load the relationship objects when adding a new entry to the
list. Assuming that during "normal" usage we would like only to add new audit
log entry, we use lazy='noload' flag so that the relation from the main object
will never be loaded. It is loaded when navigated from the other side though,
and also can be loaded from the main object using custom query, which is shown
in the example as well using activitylog_readonly readonly property.
Code (runnable with some tests):
from datetime import datetime
from sqlalchemy import create_engine, Column, Integer, SmallInteger, String, DateTime, ForeignKey, Table, UnicodeText, Unicode, and_
from sqlalchemy.orm import relationship, dynamic_loader, scoped_session, sessionmaker, class_mapper, backref
from sqlalchemy.orm.session import Session
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.interfaces import SessionExtension
import logging
logging.basicConfig(level=logging.INFO)
_logger = logging.getLogger()
ACTIVITY_ADD = 1
ACTIVITY_MOD = 2
ACTIVITY_DEL = 3
class ActivityLogSessionExtension(SessionExtension):
_logger = logging.getLogger('ActivityLogSessionExtension')
def before_commit(self, session):
self._logger.debug("before_commit: %s", session)
for d in session.new:
self._logger.info("before_commit >> add: %s", d)
if hasattr(d, 'create_activitylog'):
log = d.create_activitylog(ACTIVITY_ADD)
for d in session.dirty:
self._logger.info("before_commit >> mod: %s", d)
if hasattr(d, 'create_activitylog'):
log = d.create_activitylog(ACTIVITY_MOD)
for d in session.deleted:
self._logger.info("before_commit >> del: %s", d)
if hasattr(d, 'create_activitylog'):
log = d.create_activitylog(ACTIVITY_DEL)
# Configure test data SA
engine = create_engine('sqlite:///:memory:', echo=False)
session = scoped_session(sessionmaker(bind=engine, autoflush=False, extension=ActivityLogSessionExtension()))
Base = declarative_base()
Base.query = session.query_property()
class _BaseMixin(object):
""" Just a helper mixin class to set properties on object creation.
Also provides a convenient default __repr__() function, but be aware that
also relationships are printed, which might result in loading relations.
"""
def __init__(self, **kwargs):
for k,v in kwargs.items():
setattr(self, k, v)
def __repr__(self):
return "<%s(%s)>" % (self.__class__.__name__,
', '.join('%s=%r' % (k, self.__dict__[k])
for k in sorted(self.__dict__) if '_sa_' != k[:4] and '_backref_' != k[:9])
)
class User(Base, _BaseMixin):
__tablename__ = u'users'
id = Column(Integer, primary_key=True)
name = Column(String)
class Document(Base, _BaseMixin):
__tablename__ = u'documents'
id = Column(Integer, primary_key=True)
title = Column(Unicode(255), nullable=False)
body = Column(UnicodeText, nullable=False)
class Folder(Base, _BaseMixin):
__tablename__ = u'folders'
id = Column(Integer, primary_key=True)
title = Column(Unicode(255), nullable=False)
comment = Column(UnicodeText, nullable=False)
class ActivityLog(Base, _BaseMixin):
__tablename__ = u'activitylog'
id = Column(Integer, primary_key=True)
activity_by_id = Column(Integer, ForeignKey('users.id'), nullable=False)
activity_by = relationship(User) # #note: no need to specify the primaryjoin
activity_at = Column(DateTime, default=datetime.utcnow, nullable=False)
activity_type = Column(SmallInteger, nullable=False)
target_table = Column(Unicode(20), nullable=False)
target_id = Column(Integer, nullable=False)
target_title = Column(Unicode(255), nullable=False)
# backref relation for auditable
target = property(lambda self: getattr(self, '_backref_%s' % self.target_table))
def _get_user():
""" This method returns the User object for the current user.
#todo: proper implementation required
#hack: currently returns the 'user2'
"""
return session.query(User).filter_by(name='user2').one()
# auditable support function
# based on first non-FK version from http://techspot.zzzeek.org/?p=13
def auditable(cls, name):
def create_activitylog(self, activity_type):
log = ActivityLog(activity_by=_get_user(),
activity_type=activity_type,
target_table=table.name,
target_title=self.title,
)
getattr(self, name).append(log)
return log
mapper = class_mapper(cls)
table = mapper.local_table
cls.create_activitylog = create_activitylog
def _get_activitylog(self):
return Session.object_session(self).query(ActivityLog).with_parent(self).all()
setattr(cls, '%s_readonly' %(name,), property(_get_activitylog))
# no constraints, therefore define constraints in an ad-hoc fashion.
primaryjoin = and_(
list(table.primary_key)[0] == ActivityLog.__table__.c.target_id,
ActivityLog.__table__.c.target_table == table.name
)
foreign_keys = [ActivityLog.__table__.c.target_id]
mapper.add_property(name,
# #note: because we use the relationship, by default all previous
# ActivityLog items will be loaded for an object when new one is
# added. To avoid this, use either dynamic_loader (http://www.sqlalchemy.org/docs/reference/orm/mapping.html#sqlalchemy.orm.dynamic_loader)
# or lazy='noload'. This is the trade-off decision to be made.
# Additional benefit of using lazy='noload' is that one can also
# record DEL operations in the same way as ADD, MOD
relationship(
ActivityLog,
lazy='noload', # important for relationship
primaryjoin=primaryjoin,
foreign_keys=foreign_keys,
backref=backref('_backref_%s' % table.name,
primaryjoin=list(table.primary_key)[0] == ActivityLog.__table__.c.target_id,
foreign_keys=foreign_keys)
)
)
# this will define which classes support the ActivityLog interface
auditable(Document, 'activitylogs')
auditable(Folder, 'activitylogs')
# create db schema
Base.metadata.create_all(engine)
## >>>>> TESTS >>>>>>
# create some basic data first
u1 = User(name='user1')
u2 = User(name='user2')
session.add(u1)
session.add(u2)
session.commit()
session.expunge_all()
# --check--
assert not(_get_user() is None)
##############################
## ADD
##############################
_logger.info('-' * 80)
d1 = Document(title=u'Document-1', body=u'Doc1 some body skipped the body')
# when not using SessionExtension for any reason, this can be called manually
#d1.create_activitylog(ACTIVITY_ADD)
session.add(d1)
session.commit()
f1 = Folder(title=u'Folder-1', comment=u'This folder is empty')
# when not using SessionExtension for any reason, this can be called manually
#f1.create_activitylog(ACTIVITY_ADD)
session.add(f1)
session.commit()
# --check--
session.expunge_all()
logs = session.query(ActivityLog).all()
_logger.debug(logs)
assert len(logs) == 2
assert logs[0].activity_type == ACTIVITY_ADD
assert logs[0].target.title == u'Document-1'
assert logs[0].target.title == logs[0].target_title
assert logs[1].activity_type == ACTIVITY_ADD
assert logs[1].target.title == u'Folder-1'
assert logs[1].target.title == logs[1].target_title
##############################
## MOD(ify)
##############################
_logger.info('-' * 80)
session.expunge_all()
d1 = session.query(Document).filter_by(id=1).one()
assert d1.title == u'Document-1'
assert d1.body == u'Doc1 some body skipped the body'
assert d1.activitylogs == []
d1.title = u'Modified: Document-1'
d1.body = u'Modified: body'
# when not using SessionExtension (or it does not work, this can be called manually)
#d1.create_activitylog(ACTIVITY_MOD)
session.commit()
_logger.debug(d1.activitylogs_readonly)
# --check--
session.expunge_all()
logs = session.query(ActivityLog).all()
assert len(logs)==3
assert logs[2].activity_type == ACTIVITY_MOD
assert logs[2].target.title == u'Modified: Document-1'
assert logs[2].target.title == logs[2].target_title
##############################
## DEL(ete)
##############################
_logger.info('-' * 80)
session.expunge_all()
d1 = session.query(Document).filter_by(id=1).one()
# when not using SessionExtension for any reason, this can be called manually,
#d1.create_activitylog(ACTIVITY_DEL)
session.delete(d1)
session.commit()
session.expunge_all()
# --check--
session.expunge_all()
logs = session.query(ActivityLog).all()
assert len(logs)==4
assert logs[0].target is None
assert logs[2].target is None
assert logs[3].activity_type == ACTIVITY_DEL
assert logs[3].target is None
##############################
## print all activity logs
##############################
_logger.info('=' * 80)
logs = session.query(ActivityLog).all()
for log in logs:
_ = log.target
_logger.info("%s -> %s", log, log.target)
##############################
## navigate from main object
##############################
_logger.info('=' * 80)
session.expunge_all()
f1 = session.query(Folder).filter_by(id=1).one()
_logger.info(f1.activitylogs_readonly)