I'm working on a video game auction website for buying/selling in-game items.
I want to be able to query the Auctions table and sort them by the "hottest" auctions. This is based on the number of bids/hour placed on an auction.
Here's the auction model:
class Auctions(db.Model):
id = db.Column(db.Integer, primary_key=True, index=True)
posted = db.Column(db.DateTime())
end = db.Column(db.DateTime())
...
bids = db.relationship('Bids', backref='auctions', lazy='dynamic', order_by='desc(Bids.amount)', cascade="all, delete-orphan")
Here's the Bids model:
class Bids(db.Model):
id = db.Column(db.Integer, primary_key=True, index=True)
bidder_id = db.Column(db.Integer, db.ForeignKey('user.id'), index=True)
auction_id = db.Column(db.Integer, db.ForeignKey('auctions.id'), index=True)
amount = db.Column(db.Integer)
posted = db.Column(db.DateTime())
I'm able to sort them by the amount of bids like this:
hot_stmt = db.session.query(models.Bids.auction_id, func.count('*').label('bid_count')).group_by(models.Bids.auction_id).subquery()
hot = db.session.query(models.Auctions, hot_stmt.c.bid_count).outerjoin(hot_stmt, (models.Auctions.id == hot_stmt.c.auction_id)).order_by(hot_stmt.c.bid_count.desc()).limit(5)
I can calculate and list bids/hour with this:
for auc, count in hot:
time_delta = datetime.utcnow() - auc.posted
auc_hours = time_delta.seconds / 60 / 60
print(auc.id, count / auc_hours)
How could I sort the query by bids/hour so that the query returns the top 5 hottest auctions?
One useful approach is to create a dictionary with auctions as keys and bids/hr as values:
d = {}
for auc, count in hot:
time_delta = datetime.utcnow() - auc.posted
auc_hours = time_delta.seconds / 60 / 60
d[auc] = count / auc_hours
Make a list of the auctions:
aucs = [auc for auc, count in hot]
Sort the list aucs based on the values (use the reverse keyword to put the highest values at the beginning of the list, since the sort function goes lowest-to-highest by default):
aucs.sort(key=d.get, reverse=True)
Related
Checked the sqlAlchemy docs but cannot see example of multiple columns query with filter and using FUNC.
How to compose a query based on my model to return result like this:
SELECT
COUNT(amount)a_cnt,
SUM(amount)a_sum,
AVG(amount)a_avg
FROM public.transaction
WHERE acc_id = 1
AND "traDate" >= '2019-11-20'
AND "traDate" <= '2019-12-01'
******************
a_cnt || a_sum || a_avg
------------------------
3 || 12 || 4
Please see below my model, and query functions, one with Class other with session, still unsure which one I should be using in this case. Both result in printing the query syntax.
Model:
class Transaction(db.Model):
id = db.Column(db.Integer, primary_key=True)
traDate = db.Column(db.Date, nullable=False)
amount = db.Column(db.Float, nullable=False)
desc = db.Column(db.String, nullable=False)
card = db.Column(db.String(1), nullable=False)
tag_id = db.Column(db.Integer, db.ForeignKey('tag.id'), nullable=True)
acc_id = db.Column(db.Integer, db.ForeignKey('account.id'), nullable=False)
uplDate = db.Column(db.DateTime, nullable=False, default=datetime.now)
### this?
def sum_filtered(account_id, date_from, date_to):
return db.session.query(db.func.count(Transaction.amount).label('a_cnt'), db.func.sum(Transaction.amount).label('a_sum'), db.func.avg(Transaction.amount).label('a_avg')).filter_by(acc_id = account_id).filter(Transaction.traDate >= date_from, Transaction.traDate <= date_to)
### OR this?
def sum_filtered(account_id, date_from, date_to):
return Transaction.query.with_entities(func.sum(Transaction.amount).label('a_sum')).filter_by(acc_id = account_id).filter(Transaction.traDate >= date_from, Transaction.traDate <= date_to)
app:
#app.route(...)
templateData = {
...
'total_amnt' : model.Transaction.sum_filtered(accountid, f_from, f_to),
...
}
return render_template('/list.html', **templateData)
html:
...
<span class="input-group-text">Total £{{ total_amnt }}</span><!-- shows the query syntax-->
<span class="input-group-text">Total £{{ total_amnt.a_sum }}</span><!-- shows nothing-->
...
What am I missing?
Found this Docs. If no better answer provided, I will accept this.
def sum_filtered(account_id, date_from, date_to):
result = db.session.execute('SELECT COUNT(amount)a_cnt, AVG(amount)a_avg, SUM(amount)a_sum FROM transaction WHERE acc_id = :p1 AND "traDate" BETWEEN :p2 AND :p3',{'p1' : account_id, 'p2' : date_from, 'p3' : date_to})
return result.fetchone()
App:
'sum_avg_cnt' : model.Transaction.sum_filtered(accountid, f_from, f_to),
Then html:
{{ sum_avg_cnt.a_cnt }}
I've created models for my database:
class Album(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
year = db.Column(db.String(4))
tracklist = db.relationship('Track', secondary=tracklist,
backref=db.backref('albums',
lazy='dynamic'), lazy='dynamic')
class Track(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
class Artist(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(128))
releases = db.relationship('Track', secondary=releases,
backref=db.backref('artists',
lazy='dynamic'), lazy='dynamic')
They are many-to-many related Album <--> Track <--> Artist
Next, I have this form:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
My idea is to give the user freedom in filling desired combination of forms (but at least one is required), so I've got this function, which recieves SearchForm().data (an immutable dict 'field_name': 'data'):
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
if form.search_by_album.data:
query = query.filter(Album.title == form.search_by_album.data)
if form.search_by_artist.data:
query = query.filter(Artist.name == form.search_by_artist.data)
if form.search_track.data:
query = query.filter(Track.title == form.search_track.data)
if form.year.data:
query = query.filter(Album.year == form.year.data)
result = query.all()
return result
My question is if there is a more abstract way of adding filters in the function above? If one day I decide to add more columns to my tables (or even create new tables), I will have to add more monstrous ifs to constrcut_query(), which will eventually grow enormous. Or such an abstractions is not a pythonic way because "Explicit is better than implicit"?
PS
I know about forms from models, but I don't think that they are my case
One way would be associating the filter-attribute with the fields at some place, e.g. as a class attribute on the form itself:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
# map form fields to database fields/attributes
field_to_attr = {search_by_album: Album.title,
search_by_artist: Artist.name,
search_track: Track.title,
year: Album.year}
When building the query, you could then build the where clause in a pretty comfortable way:
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
for field in form:
if field.data:
query = query.filter(form.field_to_attr[field] == field.data)
# or:
# for field, attr in form.field_to_attr.items():
# if field.data:
# query = query.filter(attr == field.data)
result = query.all()
return result
Adding new fields and attributes to filter on would then only translate to the creating the field and its mapping to an attribute.
I have a table
class Term(CommonFunctions, Base):
__tablename__ = 'terms'
id = Column(Integer, primary_key=True,autoincrement=True)
term_begin = Column(Date, nullable=False)
term_end = Column(Date)
term_served = Column(Integer) # term_number # calculatable?
office_type_id = Column(Integer, ForeignKey(OfficeType.id))
office_type = relationship('OfficeType', backref='terms')
state_id = Column(Integer, ForeignKey(State.id))
state = relationship('State', backref='terms')
district_id = Column(Integer, ForeignKey(District.id))
district = relationship('District', backref='terms')
office_class = Column(Integer)
# ... other fieldds
I am trying to run a report, to find the ID pairs, of rows that have the same set of data for (state_id,district_id,office_type_id, office_class)
for a specific office_type_id within a specific date range.
The query I have right now - (institution = office_type_id)
date = request.args.get('date')
institution = request.args.get('institution')
term_alias = aliased(schema.Term)
composition = Session.query(schema.Term.id, term_alias.id).\
filter(schema.Term.id != term_alias.id).\
filter(schema.Term.office_class == term_alias.office_class).\
filter(schema.Term.state_id == term_alias.state_id).\
filter(schema.Term.office_type_id == term_alias.office_type_id).\
filter(schema.Term.office_type_id == institution).\
filter(schema.Term.office_class != 0).\
filter(and_(schema.Term.term_begin <= date, or_(schema.Term.term_end >= date,
schema.Term.term_end == None))).\
all()
This works - in a sense. I get back valid results, but it reproduces the result twice, once for each version of the pair.
For Example :
[(127,196), (196,127)]
My question is, How can I update the query, to include only pairs, that are not already represented by a logically equivalent pair.
I would like the above set to be either [(127, 196)] or [(196,127)] not both.
Thanks for reading
A common way is to impose a particular (arbitrary) ordering:
Session.query(...).filter(..., schema.Term.id < term_alias.id)
If you can get back a "reflexive" pair (pair of identical IDs), you need apply a distinct as well.
Session.query(...).filter(..., schema.Term.id < term_alias.id).distinct()
Hello I have a problem with join at flask-sqlalchemy. I am a beginner at database and flask.
These are my classes:
class Shopping(db.Model):
__tablename__ = 'shoppings'
id = db.Column(db.Integer, primary_key=True)
product_name = db.Column(db.String(30), index=True, unique=False)
price=db.Column(db.Float(10), index=True)
date=db.Column(db.Date())
s_type_id = db.Column(db.Integer, db.ForeignKey('shopping_types.id'))
def __repr__(self):
return 'Alisveris yeri :{0} Tutar :{1} Tarih: {2}'.format(self.product_name,self.price,self.date)
def __list__(self):
return [self.product_name,self.price,self.date]
class Shopping_Type(db.Model):
__tablename__='shopping_types'
id=db.Column(db.Integer,primary_key=True)
type_name=db.Column(db.String(30), index=True, unique=True)
types = db.relationship('Shopping', backref = 'shopping_types', lazy = 'dynamic')
def __repr__(self):
return '{0}'.format(self.type_name)
when I try on python terminal and run:
select shoppings.product_name ,shoppings.price, shoppings.date, shopping_types.type_name from shoppings join shopping_types ON shoppings.s_type_id=shopping_types.id
query
I get what I want but when I run flask-sqlalchemy command:
rslt=db.session.query(spng).join(st)
spng:Shopping(class)
st:Shopping_Type(class)
I get only Shopping data.
I want to get Shopping + Shopping_Type data.
Thank you.
rslt = db.session.query(spng, st).join(st)
The result would be an enumerable of tuples of (Shopping, Shopping_Type)
rslt = db.session.query(spng, st).filter(spng.s_type_id == st.id).all()
for x, y in rslt:
print("Product ID: {} Product Name: {} Sopping Type: {}".format(x.id, x.product_name, y.tye_name))
type(rslt) is a tuple, contains elements as the number of tables joining.
I have an association object in SQLAlchemy that has some extra information (actually a single field) for 2 other objects.
The first object is a Photo model, the second object is a PhotoSet and the association object is called PhotoInSet which holds the position attribute which tells us in what position is the Photo in the current PhotoSet.
class Photo(Base):
__tablename__ = 'photos'
id = Column(Integer, primary_key=True)
filename = Column(String(128), index=True)
title = Column(String(256))
description = Column(Text)
pub_date = Column(SADateTime)
class PhotoInSet(Base):
__tablename__ = 'set_order'
photo_id = Column(Integer, ForeignKey('photos.id'), primary_key=True)
photoset_id = Column(Integer, ForeignKey('photo_set.id'), primary_key=True)
position = Column(Integer)
photo = relationship('Photo', backref='sets')
def __repr__(self):
return '<PhotoInSet %r>' % self.position
class PhotoSet(Base):
__tablename__ = 'photo_set'
id = Column(Integer, primary_key=True)
name = Column(String(256))
description = Column(Text)
timestamp = Column(SADateTime)
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship('User', backref=backref('sets', lazy='dynamic'))
photo_id = Column(Integer, ForeignKey('photos.id'))
photos = relationship('PhotoInSet', backref=backref('set', lazy='select'))
I have no problems creating a new PhotoSet saving the position and creating the relationship, which is (roughly) done like this:
# Create the Set
new_set = PhotoSet(name, user)
# Add the photos with positions applied in the order they came
new_set.photos.extend(
[
PhotoInSet(position=pos, photo=photo)
for pos, photo in
enumerate(photo_selection)
]
)
But I am having a lot of trouble attempting to figure out how to update the position when the order changes.
If I had, say, 3 Photo objects with ids: 1, 2, and 3, and positions 1, 2, and 3 respectively, would look like this after creation:
>>> _set = PhotoSet.get(1)
>>> _set.photos
[<PhotoInSet 1>, <PhotoInSet 2>, <PhotoInSet 3>]
If the order changes, (lets invert the order for this example), is there anyway SQLAlchemy can help me update the position value? So far I am not happy with any of the approaches I can come up with.
What would be the most concise way to do this?
Take a look at the Ordering List extension:
orderinglist is a helper for mutable ordered relationships. It will
intercept list operations performed on a relationship()-managed
collection and automatically synchronize changes in list position onto
a target scalar attribute.
I believe you could change your schema to look like:
from sqlalchemy.ext.orderinglist import ordering_list
# Photo and PhotoInSet stay the same...
class PhotoSet(Base):
__tablename__ = 'photo_set'
id = Column(Integer, primary_key=True)
name = Column(String(256))
description = Column(Text)
photo_id = Column(Integer, ForeignKey('photos.id'))
photos = relationship('PhotoInSet',
order_by="PhotoInSet.position",
collection_class=ordering_list('position'),
backref=backref('set', lazy='select'))
# Sample usage...
session = Session()
# Create two photos, add them to the set...
p_set = PhotoSet(name=u'TestSet')
p = Photo(title=u'Test')
p2 = Photo(title='uTest2')
p_set.photos.append(PhotoInSet(photo=p))
p_set.photos.append(PhotoInSet(photo=p2))
session.add(p_set)
session.commit()
print 'Original list of titles...'
print [x.photo.title for x in p_set.photos]
print ''
# Change the order...
p_set.photos.reverse()
# Any time you change the order of the list in a way that the existing
# items are in a different place, you need to call "reorder". It will not
# automatically try change the position value for you unless you are appending
# an object with a null position value.
p_set.photos.reorder()
session.commit()
p_set = session.query(PhotoSet).first()
print 'List after reordering...'
print [x.photo.title for x in p_set.photos]
The results of this script...
Original list of titles...
[u'Test', u'uTest2']
List after reordering...
[u'uTest2', u'Test']
In your comment, you said...
So this would mean that if I assign a new list to _set.photos I get the positioning for free?
I doubt this is the case.