Develop Reference

Use sqlalchemy ORM to access and update a nested field in a json column

I have a table with one column containing json data. The schema of the table is like this
class User(Base):
__tablename__ = "user_account"
id = Column(Integer, primary_key=True)
data = Column(JSON, default={})
The data in the data column has a form like this:
{
'field_a': 1234,
'field_b': 5678
}
To access the data in the fields, i use the specific json functions from sqlite/SQL server and use hybrid props to have easier access. So the table looks like this.
class User(Base):
__tablename__ = "user_account"
id = Column(Integer, primary_key=True)
data = Column(JSON, default={})
#hybrid_property
def field_a(self):
return self.data.get('field_a')
#field_a.setter
def field_a(self, value):
self.data['field_a'] = value
#field_a.expression
def field_a(cls):
return func.json_extract(cls.data, '$.field_a')
#field_a.update_expression
def field_a(cls, value):
return [
(cls.data, func.json_set(cls.data, '$.field_a', value))
]
Now, i can make queries to access the data and to update the data using core and orm functions like the following:
# core query
session.execute(sa.select(User.id, User.field_a))
# core update
session.execute(sa.update(User).where(User.id == 1).values({'field_a':8888}))
# orm query
session.query(User.field_a).all()
# orm update
session.query(User).filter(User.id == 1).update({'field_a': 6666})
However what i would like to do.
user = session.query(User).filter(User.id==1).one()
# the update statement shall only modify field_a in data and not update the
# whole json data in the column
user.field_a = 5555
session.commit()
With the design like above this will issue an update statement of the complete data in the data column, while i would like it to issue only the partial update via func.json... functions. This poses a problem for me since i could have another process which issued an UPDATE of field_b in the time between querying and updating.
Is a structure like the one i want even possible? Actually, i am not even interested in the complete data column but just in a couple of nested field within that column.

Since i did not get any responses here, i went to sqlalchemy github discussions and posted the same question there (link). I got the following answer from #zzzeek:
the ORM won't do that directly and there's no hook in the unit of work that can make that happen for an ordinary change event. you need to emit the update() statement directly instead.

Generic query method based of an object from its model class

In my model class, I want to create a generic method say get_list(obj) which accept an argument of its object which contains values of their corresponding attribute, and returns all appropriate records that match with corresponding column.
Suppose that I have a users class in my model, and to use the get_list(obj) method. I just need to pass an object of users with its values. Obviously, this will save a lot of time instead of creating repetitive filter_by().
class Users(db.Model):
__tablename__ = 'users'
id = Column(Integer, primary_key=True, nullable=False, autoincrement=True)
username = Column(String(200), nullable=False)
password = Column(String(200))
email = Column(String(200), nullable=False, unique=True)
def as_query(self):
query = []
for c in self.__table__.columns:
if getattr(self, c.name) is not None:
query.append(c.name+'='+str(getattr(self, c.name)))
return ' and '.join(query)
#classmethod
def get_list(cls, statement):
return cls.query.filter_by(statement).all()
To use the method, we can expect something more like this
user = Users(username='admin')
results = Users.get_list(user.as_query()) # result as a list
I'm aware that we can achieve the same thing with just write the query into filter_by instead of creating a meaningless object. However, in one of my APIs the object will be created automatically-meaning it will be automatic on the fly.
However, this solution is only a hack to just flatten the attribute and its value into filter_by() statement and obviously not working?
Do you have a better solution for this?

Not sure, If this is what you were looking for, but I believe with Python's dict unpacking, it is already possible to unpack all the object attributes to use as filter in the filter_by function.
results = session.query(models.Users).filter_by(**user.dict()).all()
You can also use the or_ method to match any of the attribute values like below:
session.query(Users).filter(or_(**user.dict()))

flask sqlalchemy UniqueConstraint with foreignkey attribute

I have an app I am building with Flask that contains models for Projects and Plates, where Plates have Project as a foreignkey.
Each project has a year, given as an integer (so 17 for 2017); and each plate has a number and a name, constructed from the plate.project.year and plate.number. For example, Plate 106 from a project done this year would have the name '17-0106'. I would like this name to be unique.
Here are my models:
class Project(Model):
__tablename__ = 'projects'
id = Column(Integer, primary_key=True)
name = Column(String(64),unique=True)
year = Column(Integer,default=datetime.now().year-2000)
class Plate(Model):
__tablename__ = 'plates'
id = Column(Integer, primary_key=True)
number = Column(Integer)
project_id = Column(Integer, ForeignKey('projects.id'))
project = relationship('Project',backref=backref('plates',cascade='all, delete-orphan'))
#property
def name(self):
return str(self.project.year) + '-' + str(self.number).zfill(4)
My first idea was to make the number unique amongst the plates that have the same project.year attribute, so I have tried variations on
__table_args__ = (UniqueConstraint('project.year', 'number', name='_year_number_uc'),), but this needs to access the other table.
Is there a way to do this in the database? Or, failing that, an __init__ method that checks for uniqueness of either the number/project.year combination, or the name property?

There are multiple solutions to your problem. For example, you can de-normalize project.year-number combination and store it as a separate Plate field. Then you can put a unique key on it. The question is how you're going to maintain that value. The two obvious options are triggers (assuming your DB supports triggers and you're ok to use them) or sqla Events, see http://docs.sqlalchemy.org/en/latest/orm/events.html#
Both solutions won't emit an extra SELECT query. Which I believe is important for you.
your question is somewhat similar to Can SQLAlchemy events be used to update a denormalized data cache?

Computing field values on save?

class Quote(db.Model):
id = db.Column(db.Integer, primary_key=True)
content = db.Column(db.Text)
votes = db.Column(db.Integer)
author_id = db.Column(db.Integer, db.ForeignKey('author.id'))
date_added = db.Column(db.DateTime,default=datetime.datetime.now())
last_letter = db.Column(db.String(1))
I have a Model that looks like the above. I want last_letter to be the last letter of whatever the content is. Where should I place this logic so that it will occur every time a model is saved? I'm reading about Hybrid Properties and stuff and I'm not sure which way is the correct one to go.

1.the Naive way: you can use sqlalchemy column default value to set something like:
last_letter = db.Column(db.char, default=content[len(content)-1:])
didn't check if that would actually work, guess not.
2.you can also do something like adding this init to the class:
def __init__(self,id,content,votes,auther_id,date_added):
self.id = id
self.content = content
#yadda yadda etc
self.last_letter = content[len(content)-1:] #or something similiar
or you could use "listen" to the "before insert" event and add this dynamically as explained here.
you can use sql computed column with an sql trigger (in the db) without sqlalchemy.
you can probably use a sqlalchemy mapper sql expression as a hybrid property, also I didn't try that myself, look simple enough and probably is the most elegant way to do this.

last_letter could be decorated with #property and defined
#property
def last_letter(self):
return self.content[-1]
Disclaimer: I just learned how to use decorators and am using them everywhere

SQLAlchemy: query custom property based on table field

I'm using SQLAlchemy declarative base to define my model. I defined a property name that is computed from one the columns (title):
class Entry(Base):
__tablename__ = "blog_entry"
id = Column(Integer, primary_key=True)
title = Column(Unicode(255))
...
#property
def name(self):
return re.sub(r'[^a-zA-Z0-9 ]','',self.title).replace(' ','-').lower()
When trying to perform a query using name, SQLAlchemy throws an error:
Session.query(Entry).filter(Entry.name == my_name).first()
>>> ArgumentError: filter() argument must be of type sqlalchemy.sql.ClauseElement or string
After investigating for a while, I found that maybe comparable_using() could help, but I couldn't find any example that shows a comparator that references another column of the table.
Is this even possible or is there a better approach?

From SqlAlchemy 0.7 you can achieve this using hybrid_property
see the docs here: http://www.sqlalchemy.org/docs/orm/extensions/hybrid.html

Can you imagine what SQL should be issued for your query? The database knows nothing about name, it has neither a way to calculate it, nor to use any index to speed up the search.
My best bet is a full scan, fetching title for every record, calculating name then filtering by it. You can rawly do it by [x for x in Session.query(Entry).all() if x.name==my_name][0]. With a bit more of sophistication, you'll only fetch id and title in the filtering pass, and then fetch the full record(s) by id.
Note that a full scan is usually not nice from performance POV, unless your table is quite small.