The main issue is with me trying to use the append function, and getting an SQL integrity error when I call session.commit(), even when the object is not being re-created. Here's my case:
I have a one to many relationship in a Flask SQLAlchemy database.
It's a Tag-Category relationship, tags belong to only one category, and by definition a category has many tags. For the sake of explanation, tags are being used on X objects.
In my Flask models file, I did this:
class Category(db.Model):
__tablename__ = 'category'
id = db.Column(db.Integer, primary_key = True)
category = db.Column(db.Text, unique = True)
tags = db.relationship('Tag', backref = 'category', lazy = 'dynamic')
class Tag(db.Model):
__tablename__ = 'tags'
id = db.Column(db.Integer, primary_key = True)
tag = db.Column(db.Text, unique = True)
category_id = db.Column(db.Integer, db.ForeignKey('category.id'))
This should define a Category class to which I can "append" tags using the session.append(Tag) function.
Categories and tags are unique, I wanna take input from the user, and add the category/tag as the user specifies. In some cases, the user might want to just add a tag to an existing category.
The way I'm taking input is by using a form, from which I receive the category and tag, and I create the objects and add them to the database if they don't exist already, and if they do, I just want to append the tags to the X's or to the categories.
Here's my code:
category = Category(category = request.form['categories']) #Category related to its respective list of tags
tag_list = request.form.getlist('tags') # tags are received as a list
for tag_entity in tag_list:
tag = Tag(tag = tag_entity)
X.tags.append(tag)
# Checking if the category exists or not in the database
list1 = []
if len(db.session.query(Category.tags).filter(Category.category == request.form['categories']).all()) != 0:
list1 = db.session.query(Category.tags).filter(Category.category == request.form['categories']).all()
list1.append(tag)
for entry in list1:
print entry.tag
list1 = [entry.tag for entry in list1]
# Here I should check if the tag already exists as well:
db.session.add(tag)
# Only add category if it doesn't already exist
if len(db.session.query(Category.id).filter(Category.category == request.form['categories']).all()) == 0:
print "I'm here"
db.session.add(Category(category = request.form['categories'], tags = list1))
else:
print "woops im here"
newCat = Category(category=request.form['categories'])
newCat.tags.extend(list1)
db.session.merge(newCat)
db.session.add(X)
db.session.commit()
I know my code is really messy and it looks like it doesn't do what it's supposed to, that's a result of me just failing over and over again.
My main issue was with me trying to call category.tags.append(Tag), then getting an integrity error when session.commit() was being called even when the category was not even re-created.
I hope what I wrote makes sense, any help would be appreciated.
Related
I'm trying to make have a route in my fastAPI that gives back a list of all parents.portfolios and all the children or stocks that are associated with each of them PLUS the extra data that is in the association table (for that relationship).
The response is suppose to look somewhat like this
[ { "parrent1_attr1": bla,
"parrent1_attr2": bla,
"children": [ {
"child1_attr1": bla,
"child1_attr2": bla},
{"child2_attr1": bla,
"child2_attr2": bla}]
},
etc...]
Right now the route that produces this looks like this:
#router.get("/")
def get_all_portfolios(db: Session = Depends(get_db), current_user: int = Depends(oauth2.get_current_user)):
results = db.query(models.Portfolio).options(joinedload(models.Portfolio.stocks)).all()
return results
But this gives me the wrong result.
This results in this.
[ { "parrent1_attr1": bla,
"parrent1_attr2": bla,
"children": [ {
"association_table_attr1": bla
"association_table_attr2": bla},]
So I get data from the association table back instead of from the children.
The models I have are here.
class Portfolio(Base):
__tablename__ = "portfolios"
id = Column(Integer, primary_key=True, nullable=False)
...
stocks = relationship("PortfolioStock", back_populates="portfolio")
class Stock(Base):
__tablename__ = "stocks"
id = Column(Integer, primary_key=True, nullable=False)
...
portfolios = relationship("PortfolioStock", back_populates="stock")
class PortfolioStock(Base):
__tablename__ = "portfolio_stocks"
id = Column(Integer, primary_key=True)
stock_id = Column(Integer, ForeignKey("stocks.id", ondelete="CASCADE"))
portfolio_id = Column(Integer, ForeignKey("portfolios.id", ondelete="CASCADE"))
count = Column(Integer, nullable=True)
buy_in = Column(Float, nullable=True)
stock = relationship("Stock", back_populates="portfolios")
portfolio = relationship("Portfolio", back_populates="stocks")
Let me know if you need more information. I appreciate your help.
I find it to be easier to give the association some name of its own because it is confusing but in this case Portfolio.stocks is actually a list of the association objects and NOT actual stocks. You have to get those off the association object. In my example below I go and get stock with assoc.stock.id. That should not trigger another query because we used joinedload to pre-load it. If the stock had a name we'd reference it with assoc.stock.name.
with Session(engine) as session:
q = session.query(Portfolio).options(joinedload(Portfolio.stocks).joinedload(PortfolioStock.stock))
for portfolio in q.all():
print (f"Listing associated stocks for portfolio {portfolio.id}")
for assoc in portfolio.stocks:
print (f" Buy in {assoc.buy_in}, count {assoc.count} and stock id {assoc.stock.id}")
The query looks something like this:
SELECT portfolios.id AS portfolios_id, stocks_1.id AS stocks_1_id, portfolio_stocks_1.id AS portfolio_stocks_1_id, portfolio_stocks_1.stock_id AS portfolio_stocks_1_stock_id, portfolio_stocks_1.portfolio_id AS portfolio_stocks_1_portfolio_id, portfolio_stocks_1.count AS portfolio_stocks_1_count, portfolio_stocks_1.buy_in AS portfolio_stocks_1_buy_in
FROM portfolios LEFT OUTER JOIN portfolio_stocks AS portfolio_stocks_1 ON portfolios.id = portfolio_stocks_1.portfolio_id LEFT OUTER JOIN stocks AS stocks_1 ON stocks_1.id = portfolio_stocks_1.stock_id
For anybody that is looking for an answer, here is how I fixed it.
I used the query from Ian that he mentioned above, (thank you a ton for that).
And then I just manually declared the structure I wanted to have.
The whole code looks like this
results = (
db.query(models.Portfolio)
.options(joinedload(models.Portfolio.stocks).joinedload(models.PortfolioStock.stock))
.all()
)
result_list = []
for portfolio in results:
result_dict = portfolio.__dict__
stock_list = []
for sto in result_dict["stocks"]:
sto_dict = sto.__dict__
temp_sto = {}
temp_sto = sto_dict["stock"]
setattr(temp_sto, "buy_in", sto_dict["buy_in"])
setattr(temp_sto, "count", sto_dict["count"])
stock_list.append(temp_sto)
result_dict["stocks"] = stock_list
result_list.append(result_dict)
return result_list
What I'm doing here is firstly declare an empty list where our final results will be stored and which will be returned.
Then we iterate over the query (because the query gives us a list back).
So we have each "SQL alchemy Model" now as portfolio in there.
Then we can convert this into a dictionary and assign it a new variable with result_dict = portfolio.__dict__ The __dict__ method converts the model into a Python dictionary that you can work with easily.
Since that result_dict contains a list of PortfolioStock models which is the association table model in. These are stored in the stocks key we have to iterate over them to get those values as well. And here we just repeat the process.
We convert the model into a dictionary with __dict__ then make a new empty dictionary temp_sto={} and set it equal to the stock key which is the key that is linking the child to our association table. So now we have the child or stock we want to access. We can simply set our new empty dicitonary equal to that so we inherit all the information contained within.
And then we just have to add all other information from the association table that we might want which can be accessed via the dictionary we defined at the beginning of the for loop sto_dict.
Once we have this we append it to an empty list we have defined outside of this for loop but inside the portfolio loop.
Set the result_dict["stocks"] key (so basically the key where you want all children to be contained in) equal to that list we just appended all the dictionaries to and then append that dictionary to our result_list.
And last thing to do is just to return that list, and we're done.
I have provided an agnostic approach hopefully down below
query = db.query(Parent).options(joinedload(Parent.relationship).joinedload(AssociationTable.child).all()
result_list = []
for parent in query:
parent_dict = parent.__dict__
child_list = []
for association in parent_dict["relationship_name"]:
association_dict = association.__dict__
temp_child_dict = {}
temp_child_dict = association_dict["child"]
setattr(temp_child_dict, "name_you_want", association_dict["key_of_value_you_want"])
# repeat as many times as you need
child_list.append(temp_child_dict)
parent_dict["children"] = child_list
result_list.append(parent_dict)
return result_list
I hope this helps you if you are in a similar situation.
I have two models like this:
class Question(models.Model):
ques_id = models.IntegerField()
test_id = models.ForeignKey('exam.Test')
ques = models.TextField()
class UserAnswer(models.Model):
user = models.ForeignKey('exam.User')
test_id = models.ForeignKey('exam.Test')
ques_id=models.ForeignKey('exam.Question')
user_ans = models.TextField()
I need to execute this query to get the correct 'ques' field values.
SELECT A.ques_id, B.ques, A.user_ans FROM useranswer A
inner join question B on B.ques_id= A.ques_id and B.test_id =A.test_id
WHERE A.user_id=1 and B.test_id='101'
So far what I have done:
UserAnswer.objects.filter(test_id=test_id, user_id=user_id).values('ques_id', 'ques_id__ques','user_ans')
But it doesn't returning the right 'ques' field values because it doesn't considering the B.test_id =A.test_id section. How to retrieve it???
First of all, your field names are misleading. Do not
suffix foreign key fields with _id! Accessing them as attributes returns model instances and django provides the _id suffixed attributes to access the actual keys implicitly:
class Question(models.Model):
test = models.ForeignKey('exam.Test')
ques = models.TextField()
# is this supposed to be the primary key? Don't do that
# ques_id = models.IntegerField()
class UserAnswer(models.Model):
user = models.ForeignKey('exam.User')
ques = models.ForeignKey('exam.Question')
user_ans = models.TextField()
# This is redundant as ques belongs to a test already
# test = models.ForeignKey('exam.Test')
I assume you want to get all the answers for one user and one test:
UserAnswer.objects.filter(
ques__test_id=test_id,
user_id=user_id
).values('ques_id', 'ques__ques', 'user_ans')
I've created models for my database:
class Album(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
year = db.Column(db.String(4))
tracklist = db.relationship('Track', secondary=tracklist,
backref=db.backref('albums',
lazy='dynamic'), lazy='dynamic')
class Track(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(128))
class Artist(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(128))
releases = db.relationship('Track', secondary=releases,
backref=db.backref('artists',
lazy='dynamic'), lazy='dynamic')
They are many-to-many related Album <--> Track <--> Artist
Next, I have this form:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
My idea is to give the user freedom in filling desired combination of forms (but at least one is required), so I've got this function, which recieves SearchForm().data (an immutable dict 'field_name': 'data'):
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
if form.search_by_album.data:
query = query.filter(Album.title == form.search_by_album.data)
if form.search_by_artist.data:
query = query.filter(Artist.name == form.search_by_artist.data)
if form.search_track.data:
query = query.filter(Track.title == form.search_track.data)
if form.year.data:
query = query.filter(Album.year == form.year.data)
result = query.all()
return result
My question is if there is a more abstract way of adding filters in the function above? If one day I decide to add more columns to my tables (or even create new tables), I will have to add more monstrous ifs to constrcut_query(), which will eventually grow enormous. Or such an abstractions is not a pythonic way because "Explicit is better than implicit"?
PS
I know about forms from models, but I don't think that they are my case
One way would be associating the filter-attribute with the fields at some place, e.g. as a class attribute on the form itself:
class SearchForm(FlaskForm):
search_by_album = StringField('Album', validators=[Optional()])
search_by_artist = StringField('Artist', validators=[Optional()])
search_track = StringField('Track', validators=[Optional()])
year = StringField('Year', validators=[Optional(), Length(max=4)])
# map form fields to database fields/attributes
field_to_attr = {search_by_album: Album.title,
search_by_artist: Artist.name,
search_track: Track.title,
year: Album.year}
When building the query, you could then build the where clause in a pretty comfortable way:
def construct_query(form):
query = db.session.query(*[field.label.text for field in form if field.data and field.name != 'csrf_token'])
for field in form:
if field.data:
query = query.filter(form.field_to_attr[field] == field.data)
# or:
# for field, attr in form.field_to_attr.items():
# if field.data:
# query = query.filter(attr == field.data)
result = query.all()
return result
Adding new fields and attributes to filter on would then only translate to the creating the field and its mapping to an attribute.
I'm making a simple lookup application for Japanese characters (Kanji), where the user can search the database using any of the information available.
My database structure
Kanji:
id
character (A kanji like 頑)
heisig6 (a number indicating the order of showing Kanji)
kanjiorigin (a number indicating the order of showing Kanji)
MeaningEN (1 kanji_id can have multiple entries with different meanings):
kanji_id (FOREIGN KEY(kanji_id) REFERENCES "Kanji" (id)
meaning
User handling
The user can choose to search by 'id', 'character', 'heisig6', 'kanjiorigin' or 'meaning' and it should then return all information in all those fields. (All fields return only 1 result, except meanings, which can return multiple results)
Code, EDIT 4+5: my code with thanks to #ApolloFortyNine and #sqlalchemy on IRC, EDIT 6: join --> outerjoin (otherwise won't find information that has no Origins)
import sqlalchemy as sqla
import sqlalchemy.orm as sqlo
from tableclass import TableKanji, TableMeaningEN, TableMisc, TableOriginKanji # See tableclass.py
# Searches database with argument search method
class SearchDatabase():
def __init__(self):
#self.db_name = "sqlite:///Kanji_story.db"
self.engine = sqla.create_engine("sqlite:///Kanji.db", echo=True)
# Bind the engine to the metadata of the Base class so that the
# declaratives can be accessed through a DBSession instance
tc.sqla_base.metadata.bind = self.engine
# For making sessions to connect to db
self.db_session = sqlo.sessionmaker(bind=self.engine)
def retrieve(self, s_input, s_method):
# s_input: search input
# s_method: search method
print("\nRetrieving results with input: {} and method: {}".format(s_input, s_method))
data = [] # Data to return
# User searches on non-empty string
if s_input:
session = self.db_session()
# Find id in other table than Kanji
if s_method == 'meaning':
s_table = TableMeaningEN # 'MeaningEN'
elif s_method == 'okanji':
s_table = TableOriginKanji # 'OriginKanji'
else:
s_table = TableKanji # 'Kanji'
result = session.query(TableKanji).outerjoin(TableMeaningEN).outerjoin(
(TableOriginKanji, TableKanji.origin_kanji)
).filter(getattr(s_table, s_method) == s_input).all()
print("result: {}".format(result))
for r in result:
print("r: {}".format(r))
meanings = [m.meaning for m in r.meaning_en]
print(meanings)
# TODO transform into origin kanji's
origins = [str(o.okanji_id) for o in r.okanji_id]
print(origins)
data.append({'character': r.character, 'meanings': meanings,
'indexes': [r.id, r.heisig6, r.kanjiorigin], 'origins': origins})
session.close()
if not data:
data = [{'character': 'X', 'meanings': ['invalid', 'search', 'result']}]
return(data)
Question EDIT 4+5
Is this an efficient query?: result = session.query(TableKanji).join(TableMeaningEN).filter(getattr(s_table, s_method) == s_input).all() (The .join statement is necessary, because otherwise e.g. session.query(TableKanji).filter(TableMeaningEN.meaning == 'love').all() returns all the meanings in my database for some reason? So is this either the right query or is my relationship() in my tableclass.py not properly defined?
fixed (see lambda: in tableclass.py) kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="OriginKanji") <-- what is wrong about this? It gives the error:
File "/path/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 1805, in get_property
"Mapper '%s' has no property '%s'" % (self, key))
sqlalchemy.exc.InvalidRequestError: Mapper 'Mapper|TableKanji|Kanji' has no property 'OriginKanji'
Edit 2: tableclass.py (EDIT 3+4+5: updated)
import sqlalchemy as sqla
from sqlalchemy.orm import relationship
import sqlalchemy.ext.declarative as sqld
sqla_base = sqld.declarative_base()
class TableKanji(sqla_base):
__tablename__ = 'Kanji'
id = sqla.Column(sqla.Integer, primary_key=True)
character = sqla.Column(sqla.String, nullable=False)
radical = sqla.Column(sqla.Integer) # Can be defined as Boolean
heisig6 = sqla.Column(sqla.Integer, unique=True, nullable=True)
kanjiorigin = sqla.Column(sqla.Integer, unique=True, nullable=True)
cjk = sqla.Column(sqla.String, unique=True, nullable=True)
meaning_en = relationship("TableMeaningEN", back_populates="kanji") # backref="Kanji")
okanji_id = relationship("TableOriginKanji", foreign_keys=lambda: TableOriginKanji.kanji_id, back_populates="kanji")
class TableMeaningEN(sqla_base):
__tablename__ = 'MeaningEN'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
meaning = sqla.Column(sqla.String, primary_key=True)
kanji = relationship("TableKanji", back_populates="meaning_en")
class TableOriginKanji(sqla_base):
__tablename__ = 'OriginKanji'
kanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
okanji_id = sqla.Column(sqla.Integer, sqla.ForeignKey('Kanji.id'), primary_key=True)
order = sqla.Column(sqla.Integer)
#okanji = relationship("TableKanji", foreign_keys=[kanji_id], backref="okanji")
kanji = relationship("TableKanji", foreign_keys=[kanji_id], back_populates="okanji_id")
We would really have to be able to see your database schema to give real critique, but assuming no foreign keys, what you said is basically the best you can do.
SQLAlchemy really begins to shine when you have complicated relations going on however. For example, if you properly had foreign keys set, you could do something like the following.
# Assuming kanji is a tc.tableMeaningEN.kanji_id object
kanji_meaning = kanji.meanings
And that would return the meanings for the kanji as an array, without any further queries.
You can go quite deep with relationships, so I'm linking the documentation here. http://docs.sqlalchemy.org/en/latest/orm/relationships.html
EDIT: Actually, you don't need to manually join at all, SQLAlchemy will do it for you.
The case is wrong on your classes, but I'm not sure if SQLAlchemy is case sensitive there or not. If it works, then just move on.
If you query the a table (self.session.query(User).filter(User.username == self.name).first()) you should have an object of the table type (User here).
So in your case, querying the TableKanji table alone will return an object of that type.
kanji_obj = session.query(TableKanji).filter(TableKanji.id == id).first()
# This will return an array of all meaning_ens that match the foreign key
meaning_arr = kanji_obj.meaning_en
# This will return a single meeting, just to show each member of the arr is of type TableMeaningEn
meaning_arr[0].meaning
I have a project made use of some of these features, hope it helps:
https://github.com/ApolloFortyNine/SongSense
Database declaration (with relationships): https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/database.py
Automatic joins: https://github.com/ApolloFortyNine/SongSense/blob/master/songsense/getfriend.py#L134
I really like my database structure, but as for the rest it's pretty awful. Hope it still helps though.
I have model, which is look like this
class Question(db.Model):
__tablename__ = 'questions'
id = db.Column(db.Integer, primary_key=True)
question_title = db.Column(db.String(64))
question_text = db.Column(db.String(256), nullable=False)
votest = db.Column(db.Integer, default=0)
answers = db.relationship('Answer', backref='question')
def __repr__(self):
return '<QTitle: %s>\n<QText: %s>' % (self.question_title,
self.question_text)
The thing is to check if column question_title is empty, and assign it to first 64 symbols of column question_text + '...'.
Which is the best way to implement this?
Maybe the best way is not to implement this in model, but make this check in view?
The best way is to do this in your view after you read the data from a form (if this is the case).
You could create a helper function which takes as arguments question_title and question_text and verifies if the question_title is empty and if it is then assign to it the first 64 characters from question_text. And you call this function in your view after you get the data from the form (or from wherever you get the data). And after that you put the data in the database.
Check this out:
# Query all Question objects with empty question_title
empties = session.query(Question).filter_by(question_title=None).all()
# Iterate over them
for empty_q in empties:
# Apply the logic
empty_q.question_title = empty_q.question_text[:64] + '...'
session.add(empty_q)
session.commit()
print "Updated Empty Questions!"
I didn't test it, but it should work!