I read about sqlalchemy joinloads like mentioned here and I little confused about the benefits or special usages over simply joining two tables like mentioned here
I would like to know about when to use each method, currently I don't see any benefit for using joinloads for now, can you please explain the difference? And the use cases to prefer joinloads
Sqlalchemy docs says joinedload() is not a replacement for join() and joinedload() doesn't affect the query result :
Query.join()
Query.options(joinedload())
Let's say if you wants to get same date that already related with data you are querying, but when you get this related data it won't change the result of the query it is like an attachment. Better to look sqlalchemy docs joinedload
class User(db.Model):
...
addresses = relationship('Address', backref='user')
class Address(db.Model):
...
user_id = Column(Integer, ForeignKey('users.id'))
The code below query user filter and return that user and optionally you can getting that user addresses.
user = db.session.query(User).options(joinedload(User.addresses)).filter(id==1).one()
Now lets look at join:
user = db.session.query(User).join(Address).filter(User.id==Address.user_id).one()
Conclusion
The query with joinedload() get that user addresses.
Other query, query on both table, check for user id on both table, so the result depend on this. But joinedload() if user doesn't have any address you will have user but no address. in join() if user doesn't have address there will not result.
Related
I have MS Access DB file (.accdb), and inside of file stored photos of persons as attachments. In table constructor I see only one field "photos" with type "Attachment". Actually there three hidden fields with names: photos.FileData, photos.FileName, photos.FileType. For parsing these fields I created following class:
class Person:
__tablename__ = 'persons'
name = Column(String(255), name='name')
photos_data = Column(String, name='photos.FileData', quote=False)
....
If I try to get all attributes of Person in same time, as following:
persons = session.query(Person)
I get error in following generated piece of SQL statement:
SELECT ... [persons].photos.FileData AS persons_photos.FileData ...;
As you can see there dot sign present in alias, which raises ODBC error. I can avoid such behavior to request FileData as separate value:
persons = session.query(Person.photos_data.label('photos_data'))
Or I can use raw SQL without aliases. But This is not normal ORM way that I need, because I have to manually construct Persons object each time after request to DB.
Is it possible to set own label to Column during its declaration or even disable label for selected column?
I saw this great answer, but seems this is not applicable to me. Below statement doesn't work properly:
photos_data = Column(String, name='photos.FileData', quote=False).label('photos_data')
I am newbie in python/sqlachemy world. I am trying to fetch data from PostgreSQL using sqlachemy in flask, where I need to fetch only selected column instead of streaming all the column from database through network. One of the approach is using session (like below) which is working fine,
session.query(User.user_id, User.name).all()
But for some reason I would like to stick with Model.query method instead of using sessions. So I ended up to use something like below,
User.query.options(load_only(User.user_id, User.name)).all()
Above code snippet doesn't filters the selected column, instead it gives back all the column. It looks like sqlachemy doesn't respects the load_only arguments. Why is that behaviour and any solution to achieve my use case in Model.query instead of using sessions?
My user model looks like below,
class User(db.Model):
__tablename__ = 'user_info'
user_id = Column(String(250), primary_key=True)
name = Column(String(250))
email = Column(String(250))
address = Column(String(512))
Version Info
Python - 3.7
sqlachemy - 1.3.11
Edit 1: Though I added load_only attributes, It is generating the following query,
SELECT user.user_id AS user_user_id, user.name AS user_name, user.email AS user_email, user.address AS user_address FROM user_info
I am implementing a search feature for user names. Some names have accented characters, but I want to be able to search for them with the nearest ascii character approximation. For example: Vû Trån would be searchable with Vu Tran.
I found a Python library, called unidecode to handle this conversion. It works as expected and takes my unicode string Vû Trån and returns Vu Tran. Perfect.
The issue arises when I start querying my database – I use SQLAlchemy and Postgres.
Here's my Python query:
Person.query.filter(Person.ascii_name.ilike("%{q}%".format(q=query))).limit(25).all()
ascii_name is the getter for my name column, implemented as such
class Person(Base, PersonUtil):
"""
My abbreviated Person class
"""
__tablename__ = 'person'
id = Column(BigInteger, ForeignKey('user.id'), primary_key=True)
first_name = Column(Unicode, nullable=False)
last_name = Column(Unicode, nullable=False)
name = column_property(first_name + " " + last_name)
ascii_name = synonym('name', descriptor=property(fget=PersonUtil._get_ascii_name))
class PersonUtil(object):
def _get_ascii_name(self):
return unidecode(unicode(self.name))
My intent behind this code is that because I store the unicode version of the first and last names in my database, I need to have a way to call unidecode(unicode(name)) when I retrieve the person's name. Hence, I use the descriptor=property(fget=...) so that whenever I call Person.ascii_name, I retrieve the "unidecoded" name attribute. That way, I can simply write Person.ascii_name.ilike("%{my_query}%")... and match the nearest ascii_name to the search query, which is also just ascii characters.
This doesn't fully work. The ilike method with ascii_name works when I do not have any converted characters in the query. For example, the ilike query will work for the name "Bob Smith", but it will not work for "Bøb Smíth". It fails when it encounters the first converted character, which in the case of "Bøb Smíth" is the letter "ø".
I am not sure why this is happening. The ascii_name getter returns my expected string of "Bob Smith" or "Vu Tran", but when coupled with the ilike method, it doesn't work.
Why is this happening? I've not been able to find anything about this issue.
How can I either fix my existing code to make this work, or is there a better way to do this that will work? I would prefer not to have to change my DB schema.
Thank you.
What you want to do simply won't work because ilike only works on real columns in the database. The column_property and synonym are just syntactic sugars provided by sqlalchemy to help with making the front end easy. If you want to leverage the backend to query with LIKE in the way you intended you need the actual values there. I am afraid you have to generate/store the ascii full name into the database which means you need to change your schema to include ascii_name as a real column, and make sure they are inserted. To verify this yourself, you should dump out the data in the table, and see if your manually constructed queries can work.
Say you the table Users (one): id and activities (many): type, created_on , how do you get all users who's latest activity.type is 'message'?
Heres the query I have so far:
User.objects.filter(activity__type='message').annotate(Max('activity__created_on'))
But its not working since:
User.objects.filter(activity__type='message').annotate(Max('activity__created_on'))[0].activity.latest('created_on').type
is not equal to 'message'.
I believe it does not work because when you filter(activity__type='message') Django is going through all the activities for that User and discarding the User if it has an activity with type other than message, it's not discarding the activities.
If you're using PostgreSQL you can use distinct and get fields from the User model, not actual instances of them though, but that might be enough for you:
Activity.objects.order_by('user__username', 'created_on')\
.distinct('user__username')\
.filter(type='message')\
.values('user__username')
If you are not using PostgreSQL or need full User instances you'll need to do it manually in Python, something like:
users_with_message_activity = []
for u in User.objects.all():
if u.activities_set.latest('created_on').type == 'message':
users_with_message_activity.append(u)
Which is not ideal since it means more queries.
Hope that helps.
Considering my users can save data as "café" or "cafe", I need to be able to search on that fields with an accent-insensitive query.
I've found https://github.com/djcoin/django-unaccent/, but I have no idea if it is possible to implement something similar on sqlalchemy.
I'm using PostgreSQL, so if the solution is specific to this database is good to me. If it is generic solution, it is much much better.
Thanks for your help.
First install the unaccess extension in PostgreSQL: create extension unaccent;
Next, declare the SQL function unaccent in Python:
from sqlalchemy.sql.functions import ReturnTypeFromArgs
class unaccent(ReturnTypeFromArgs):
pass
and use it like this:
for place in session.query(Place).filter(unaccent(Place.name) == "cafe").all():
print place.name
Make sure you have the correct indexes if you have a large table, otherwise this will result in a full table scan.
A simple and database agnostic solution is to write the field(s) that can have accents twice, once with and once without accents. Then you can conduct your searches on the unaccented version.
To generate the unaccented vesrsion of a string you can use Unidecode.
To automatically assign the unaccented version to the database when a record is inserted or updated you can use the default and onupdate clauses in the Column definition. For example, using Flask-SQLAlchemy you could do something like this:
from unidecode import unidecode
def unaccent(context):
return unidecode(context.current_parameters['some_string'])
class MyModel(db.Model):
id = Column(db.Integer, primary_key=True)
some_string = db.Column(db.String(128))
some_string_unaccented = db.Column(db.String(128), default=unaccent, onupdate=unaccent, index=True)
Note how I only indexed the unaccented field, because that is the one on which the searches will be made.
Of course before you can search you also have to unaccent the value you are searching for. For example:
def search(text):
return MyModel.query.filter_by(some_string_unaccented = unaccent(text)).all()
You can apply the same technique to full text search, if necessary.