Develop Reference

How would I do these multiple joins as a Django queryset?

I have this query that joins multiple tables together:
select
p.player_id
, d.player_data_1
, l.year
, l.league
, s.stat_1
, l.stat_1_league_average
from
stats s
inner join players p on p.player_id = s.player_id
left join player_data d on d.other_player_id = p.other_player_id
left join league_averages as l on l.year = s.year and l.league = s.year
where
p.player_id = 123
My models look like this:
class Stats(models.Model):
player_id = models.ForeignKey(Player)
stat_1 = models.IntegerField()
year = models.IntegerField()
league = models.IntegerField()
class Player(models.Model):
player_id = models.IntegerField(primary_key=True)
other_player_id = models.ForeignKey(PlayerData)
class PlayerData(models.Model):
other_player_id = models.IntegerField(primary_key=True)
player_data_1 = models.TextField()
class LeagueAverages(models.Model):
year = models.IntegerField()
league = models.IntegerField()
stat_1_league_average = models.DecimalField()
I can do something like this:
Stats.objects.filter(player_id=123).select_related('player')
to do the first join. For the second join, I tried:
Stats.objects.filter(player_id=123).select_related('player').select_related('player_data')
but I got this error:
django.core.exceptions.FieldError: Invalid field name(s) given in select_related: 'player_data'. Choices are: player
How would I do the third join considering that year and league aren't foreign keys in any of the tables? Thanks!

select_related(*fields) Returns a QuerySet that will “follow” foreign-key relationships, [...]
According to the django documentation select_related follows foreign-key relationships. player_data is neighter a foreign key, nor even an field of Stats. If you'd want to INNER join PlayerData and Player you could follow its foreign-keys. In your case use the
double-underscore to get to PlayerData:
Stats.objects.all()
.select_related('player_id')
.select_related('player_id__other_player_id')
As for joining LeagueAverages: There is not a way to join models without an appropriate foreign key, but to use raw sql. Have a look at a related question: Django JOIN query without foreign key. By using .raw(), your LEFT join (which by the way is also not that easy without using raw: Django Custom Left Outer Join) could also be taken care of.
Quick notes about your models:
Each model by default has an automatically incrementing primary key that can be accessed via .id or .pk. So there is no need to add for example player_id
A models.ForeignKey field references an object not it's id. Therefore it's more intuitive to rename for example player_id to player. If you name your field player django allows you automatically to access it's id via player_id

Sqlalchemy complex NOT IN another table query

First of all, i would like to apologize as my SQL knowledge level is still very low. Basically the problem is the following: I have two distinct tables, no direct relationship between them, but they share two columns: storm_id and userid.
Basically, i would like to query all posts from storm_id, that are not from a banned user and some extra filters.
Here are the models:
Post
class Post(db.Model):
id = db.Column(db.Integer, primary_key = True)
...
userid = db.Column(db.String(100))
...
storm_id = db.Column(db.Integer, db.ForeignKey('storm.id'))
Banneduser
class Banneduser(db.Model):
id = db.Column(db.Integer, primary_key=True)
sn = db.Column(db.String(60))
userid = db.Column(db.String(100))
name = db.Column(db.String(60))
storm_id = db.Column(db.Integer, db.ForeignKey('storm.id'))
Both Post and Banneduser are another table (Storm) children. And here is the query i am trying to output. As you can see, i am trying to filter:
verified posts
by descending order
with a limit (i put it apart from the query as the elif has other filters)
# we query banned users id
bannedusers = db.session.query(Banneduser.userid)
# we do the query except the limit, as in the if..elif there are more filtering queries
joined = db.session.query(Post, Banneduser)\
.filter(Post.storm_id==stormid)\
.filter(Post.verified==True)\
# here comes the trouble
.filter(~Post.userid.in_(bannedusers))\
.order_by(Post.timenow.desc())\
try:
if contentsettings.filterby == 'all':
posts = joined.limit(contentsettings.maxposts)
print((posts.all()))
# i am not sure if this is pythonic
posts = [item[0] for item in posts]
return render_template("stream.html", storm=storm, wall=posts)
elif ... other queries
I got two problems, one basic and one underlying problem:
1/ .filter(~Post.userid.in_(bannedusers))\ gives one output EACH TIME post.userid is not in bannedusers, so i get N repeated posts. I try to filter this with distinct, but it does not work
2/ Underlying problem: i am not sure if my approach is the correct one (the ddbb model structure/relationship plus the queries)

Use SQL EXISTS. Your query should be like this:
db.session.query(Post)\
.filter(Post.storm_id==stormid)\
.filter(Post.verified==True)\
.filter(~ exists().where(Banneduser.storm_id==Post.storm_id))\
.order_by(Post.timenow.desc())

Column with default checking same table

Using SQLAlchemy, I'd like to autogenerate an identifier for a model:
class Foo(Model):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)
date = Column(Date, default=datetime.now)
reference = Column(Unicode, default=generate_reference)
Basically, I want generate_reference returning a field like:
FOO201410-001
where 2014 is current's year, 10 current's month and 001 the next id for current month, calculated using a query such as:
SELECT COUNT(*)
FROM foo
WHERE strftime('%m', datetime(date, 'unixepoch')) == strftime('%m', date('now'))
AND strftime('%y', datetime(date, 'unixepoch')) == strftime('%y', date('now'))
I've taken the syntax form from what I've read from SQLite3, though it's just an example. Basically, I want to know how many other records have the same month/year, and assign the last part with that count.
I've tried doing defaults with select expressions, but as my table is not yet created, it looks like I cannot create a select expression from it yet.
Thanks!

Here's a way to do it with default instead of event:
def foo_reference_default(context):
now = datetime.now()
month, year = now.month, now.year
ref = context.connection.execute(db.select([Foo.__table__]).where(
db.and_(db.extract('month', Foo.date)==month,
db.extract('year', Foo.date)==year)
).count()).scalar() + 1
return u'FOO{year}{month}-{ref:03}'.format(year=year, month=month, ref=ref)
class Foo(Model):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)
date = Column(Date, default=datetime.now)
reference = Column(Unicode, default=foo_reference_default)
This is effectively the same as your answer: A prior SELECT is done in order to populate the INSERT. (Note that I added +1 so it would start at 001 instead of 000.)
You could, of course, use a lambda to embed the function into default, but I don't recommend it -- you only want to call now() once. Calling it multiple times introduces a slim but real chance of getting inconsistent data on month and year edges.

I finally found a solution that works fine, thanks to other answers:
class Foo(Model):
__tablename__ = 'foo'
id = Column(Integer, primary_key=True)
date = Column(Date, default=datetime.now)
reference = Column(Unicode)
#db.event.listens_for(Foo, 'before_insert')
def receive_before_insert(mapper, connection, foo):
ref = foo.query.filter(db.and_(db.extract('month', Foo.date)==datetime.now().month,
db.extract('year', Foo.date)==datetime.now().year)
).count()
foo.reference = 'FOO{year}{month}-{ref:03}'.format(year=datetime.now().year,
month=datetime.now().month,
ref=ref)
though, I'm leaving the question open, in case someone suggests something that could
be directly embedded within the default key.

Updating dynamically determined fields with peewee

I have a peewee model like the following:
class Parrot(Model):
is_alive = BooleanField()
bought = DateField()
color = CharField()
name = CharField()
id = IntegerField()
I get this data from the user and look for the corresponding id in the (MySQL) database. What I want to do now is to update those attributes which are not set/empty at the moment. For example, if the new data has the following attributes:
is_alive = True
bought = '1965-03-14'
color = None
name = 'norwegian'
id = 17
and the data from the database has:
is_alive = False
bought = None
color = 'blue'
name = ''
id = 17
I would like to update the bought date and the name (which are not set or empty), but without changing the is_alive status. In this case, I could get the new and old data in separate class instances, manually create a list of attributes and compare them one for one, updating where necessary, and finally saving the result to the database. However, I feel there might be a better way for handling this, which could also be used for any class with any attributes. Is there?

MySQL Solution:
UPDATE my_table SET
bought = ( case when bought is NULL OR bought = '' ) then ? end )
, name = ( case when name is NULL OR name = '' ) then ? end )
-- include other field values if any, here
WHERE
id = ?
Use your scripting language to set the parameter values.
In case of the parameters matching the old values, then update will not be performed, by default.

SQLAlchemy Return All Distinct Column Values

I am creating a website using Flask and SQLAlchemy. This website keeps track of classes that a student has taken. I would like to find a way to search my database using SQLAlchemy to find all unique classes that have been entered. Here is code from my models.py for Class:
class Class(db.Model):
__tablename__ = 'classes'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(100))
body = db.Column(db.Text)
created = db.Column(db.DateTime, default=datetime.datetime.now)
user_email = db.Column(db.String(100), db.ForeignKey(User.email))
user = db.relationship(User)
In other words, I would like to get all unique values from the title column and pass that to my views.py.

Using the model query structure you could do this
Class.query.with_entities(Class.title).distinct()

query = session.query(Class.title.distinct().label("title"))
titles = [row.title for row in query.all()]

titles = [r.title for r in session.query(Class.title).distinct()]

As #van has pointed out, what you are looking for is:
session.query(your_table.column1.distinct()).all(); #SELECT DISTINCT(column1) FROM your_table
but I will add that in most cases, you are also looking to add another filter on the results. In which case you can do
session.query(your_table.column1.distinct()).filter_by(column2 = 'some_column2_value').all();
which translates to sql
SELECT DISTINCT(column1) FROM your_table WHERE column2 = 'some_column2_value';