I have 2 sqlalchemy queries...
dummy = DBSession().query(Dummy).filter(Dummy.c_name) == "foo")
dummy2 = DBSession().query(Dummy2).filter(Dummy2.c_name == "foo").filter(Dummy2.f_name == "bar")
And I have some sort of made up a generic function that combines the two...
def generic(Object, c_name, f_name):
dummy = DBSession().query(Object).filter(Object.c_name == c_name).filter(Object.f_name == f_name)
What's the best way to generically handle this, say if f_name doesn't exist or is not queryable in the Dummy2 table?
To summarise my question:
How do I create a general sqlalchemy query which can query any given table, where in some cases the attributes I am querying varies based on the given object.
I think I need some sort of reflection...maybe? or *args / **kwargs ... I dunno... help?
you should probably pack these in separate helper functions for reusability but here's how I approached this:
check_fields = {'c_name': 'exists', 'f_name': 'doesnt_exist'}
existing_fields = {}
# check if the fields exist in the table
for field in check_fields:
if field in given_table.c:
existing_fields.update({field: check_fields.get(field)})
# construct the query based on existing fields
query = given_table.select()
if existing_fields:
for k, v in existing_fields.items():
query = query.where(getattr(given_table.c, k) == v)
I'm then using session.execute(query) to get the results. Here's a similar answer that doesn't use execute: How to get query in sqlalchemyORM также
Note: All the attributes will be chained with AND
Related
So let's say I want to implement this generic function:
def do_exist(key:str, values:list[any], model: django.db.models.Model) -> bool
Which checks if all the given values exist within a given model's column named key in a SINGLE query.
I've implemented something like this
from django.db.models import Exists
def do_exist(key, values, model):
chained_exists = (Exists(model.objects.filter(F(key)=value)) for value in values)
qs = model.objects.filter(*chained_exists).values("pk")[:1]
# Limit and values used for the sake of less payload
return len(qs) > 0
It generates a pretty valid SQL query, but the thing that scares me is that if I try to append evaluating method call to qs like .first() instead of [:1] or .exists() MYSQL connection drops.
Does anyone have a more elegant way of solving this?
If you know you're passing in N pks, then a count() query filtered by those pks should have exactly N results.
def do_exist(model, pks):
return model.objects.filter(pk__in=pks).count() == len(pks)
qs = MyModel.objects.filter(id__in=pks)
This gives you a queryset that you can apply .all() etc to
res = MyModel.objects.filter(id__in=pks)
In Django, you can use "entry__in" to filter down a queryset based on a list of entries.
results = Model.objects.filter(id__in = pks)
I have a lot of models in my Django project, and many of them need their own queries for specific pages. While working on the seperation of concerns, I feel like there should be a way to make one (or very few) methods that are generic enough and work with input based to work as queries for ALL models.
Say I have model A B and C and I have the following sets of queries:
A.objects.filter(id=object_id)
A.objects.filter(id=object_id).values("id", "name")
A.objects.filter(name="test")
B.objects.filter(relationA=A)
B.objects.filter(id__in=list_of_ids)
C.objects.all()
C.objects.all().exclude('last_name')
Is there any way to create a query by a given:
Model Name (A, B, C)
filter type (possibly, least important if this isn't doable? filter, all)
comparison field (id, name, relation)
comparison type (x=x, x__in=list, x__lte=10)
comparison value (object_id, "test", list_of_ids)
So for example, X being dynamic parts would result in a function (Pseudocode) like:
def query(model, filter_type, comparison_field, comparison_type, comparison_value):
#X.objects.X(X=X)
return model.objects.filter_type(comparison_field + comparison=comparison_value)
When trying it briefly, I immediately ran into the issue of comparison_field not doing what I need it to. Seeing Q objects being mentioned a lot on SO, are they something I should be applying here? and if so, how?
Edit:
As suggested by Klaus D. , I've implemented kwargs with a dynamic dictionary.
query_manager.py
def make_query_kwargs(kwargs_dict, field, value):
kwargs_dict[field] = value
return kwargs_dict
def make_object_query(model, kwargs, columns=[]):
return model.objects.filter(**kwargs).values(*columns)
Which I use by doing the following:
kwargs = {}
id_list = [1, 2, 3, 4]
kwargs = query_manager.make_query_kwargs(kwargs, "id__in", id_list)
query_result = query_manager.make_object_query(A, kwargs)
Looking at the example cited in the Sqlalchemy documentation Loading Inheritance Hierarchies:
engineer_employee = with_polymorphic(
Employee, [Engineer], aliased=True)
manager_employee = with_polymorphic(
Employee, [Manager], aliased=True)
q = s.query(engineer_employee, manager_employee).\
join(
manager_employee,
and_(
engineer_employee.id > manager_employee.id,
engineer_employee.name == manager_employee.name
)
)
q.all()
I'm wondering if there is a way, for example, to filter Engineer table only, keeping Manager data as it is.
I had a problem like this one and the best solution I came with was using the union of the queries instead of with_polymorphic:
q = Manager.query.union(Engineer.query.filter()) # Put your criteria inside filter.
q.all()
In case you want to use order_by on union, just give it the necessary columns:
q = Manager.query.union(Engineer.query.filter()).order_by(Manager.registered_on, Engineer.registered_on)
q.all()
I have an entity in Google App Engine as below:
class HesapKalemi(ndb.Model):
hk=ndb.IntegerProperty(indexed=True)
ha=ndb.StringProperty(indexed=True)
A=ndb.FloatProperty(default=0.00)
B=ndb.FloatProperty(default=0.00)
C=ndb.FloatProperty(default=0.00)
F=ndb.FloatProperty(default=0.00)
G=ndb.FloatProperty(default=0.00)
H=ndb.FloatProperty(default=0.00)
I=ndb.FloatProperty(default=0.00)
J=ndb.FloatProperty(default=0.00)
DG=ndb.FloatProperty(default=0.00)
As known, the normal query can be below:
sektorkodu=self.request.get('sektorkodu')
qall=HesapKalemi.query().order(HesapKalemi.hk)
for hesap in qall:
hesap.ho=hesap.A
Is there any way to fetching A column by writing this way:
hesap.GETTHECOLUMN('A') or
hesap.GETTHECOLUMN(sektorkodu)
I have a very horizontal table and want to query it without if-else structure by the .GETTHECOLUMN('string') method.
Is there this kind of method?
In the NDB world, this is called Projection, or a Projection Query. In that link to the docs, you'll see the following:
Projection queries are similar to SQL queries of the form:
SELECT name, email, phone FROM CUSTOMER
So the .GETTHECOLUMN('A') method you're after would look like either of these:
qall_option_one = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=['A'])
qall_option_two = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=[HasepKalemi.A])
# to access the values
for hesap in qall_option_one:
print hesap
# output:
# HesapKalemi(key=Key('HesapKalemi', 1234567890), A=0.00, _projection=('A',))
# HesapKalemi(key=Key('HesapKalemi', 1234567891), A=0.00, _projection=('A',))
# ...
This is a bit faster than getting the full entities with all of their properties, but you do still have to iterate through them afterwards, even if you want to just generate a list of the 'A' values. Another option you should look at is "Calling a Function For Each Entity (Mapping)", where you define a callback function to be called on each entity as the query runs. So let's say you just want a list of the 'A' values. You could form that list like this:
def callback(hesap):
return hesap.A
a_values = HesapKelami.query().map(callback)
# a_values = [0.00, 0.00, ...]
If you're really after performance, look into asynchronous gets.
Note: instead of projection, you could use GQL, but that would look messier/more confusing than using projection with the regular ndb Query syntax IMO.
Edit: To answer your question in your comment, you can use either projection or mapping to select data from multiple properties.
Projection of multiple properties:
qall_option_one = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=['A', 'B', 'C'])
qall_option_two = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=[HesapKalemi.A, HesapKalemi.B, HesapKalemi.C])
# to access the values
for hesap in qall_option_one:
print hesap
# output:
# HesapKalemi(key=Key('HesapKalemi', 1234567890), A=0.00, B=0.00, C=0.00 _projection=('A', 'B', 'C',))
# HesapKalemi(key=Key('HesapKalemi', 1234567891), A=0.00, B=0.00, C=0.00 _projection=('A', 'B', 'C',))
# ...
Mapping to return multiple properties:
def callback(hesap):
# this returns a tuple of A,B,C values
return hesap.A, hesap.B, hesap.C
values = HesapKelami.query().map(callback)
# values is a list of tuples
# values = [(0.00, 0.00, 0.00), (0.00, 0.00, 0.00), ...]
Edit #2: After rereading the question and comments, I think your question, or at least part of it, may be how to get the property from the model itself using a string, and not how to pull one column out of the datastore. To answer that question, use getattr(hesap, "property_name"), or, and this may be more suited to your needs, turn hesap into a dict with hesap_dict = hesap.to_dict(). Then you could do this:
property_name = 'some_string'
hesap = HesapKelami.query().fetch(1)[0]
hesap_dict = hesap.to_dict()
property_value = hesap_dict.get(property_name, None)
You could pass hesap_dict to your Jinja2 template, and then I think you could accomplish what you asked about in your comments.
Judging by the title this would be the exact same question, but I can't see how any of the answers are applicable to my use case:
I have two classes and a relationship between them:
treatment_association = Table('tr_association', Base.metadata,
Column('chronic_treatments_id', Integer, ForeignKey('chronic_treatments.code')),
Column('animals_id', Integer, ForeignKey('animals.id'))
)
class ChronicTreatment(Base):
__tablename__ = "chronic_treatments"
code = Column(String, primary_key=True)
class Animal(Base):
__tablename__ = "animals"
treatment = relationship("ChronicTreatment", secondary=treatment_association, backref="animals")
I would like to be able to select only the animals which have undergon a treatment which has the code "X". I tried quite a few approaches.
This one fails with an AttributeError:
sql_query = session.query(Animal.treatment).filter(Animal.treatment.code == "chrFlu")
for item in sql_query:
pass
mystring = str(session.query(Animal))
And this one happily returns a list of unfiltered animals:
sql_query = session.query(Animal.treatment).filter(ChronicTreatment.code == "chrFlu")
for item in sql_query:
pass
mystring = str(session.query(Animal))
The closest thing to the example from the aforementioned thread I could put together:
subq = session.query(Animal.id).subquery()
sql_query = session.query(ChronicTreatment).join((subq, subq.c.treatment_id=="chrFlu"))
for item in sql_query:
pass
mystring = str(session.query(Animal))
mydf = pd.read_sql_query(mystring,engine)
Also fails with an AttributeError.
Can you hel me sort this list?
First, there are two issues with table definitions:
1) In the treatment_association you have Integer column pointing to chronic_treatments.code while the code is String column.
I think it's just better to have an integer id in the chronic_treatments, so you don't duplicate the string code in another table and also have a chance to add more fields to chronic_treatments later.
Update: not exactly correct, you still can add more fields, but it will be more complex to change your 'code' if you decide to rename it.
2) In the Animal model you have a relation named treatment. This is confusing because you have many-to-many relation, it should be plural - treatments.
After fixing the above two, it should be clearer why your queries did not work.
This one (I replaced treatment with treatments:
sql_query = session.query(Animal.treatments).filter(
Animal.treatments.code == "chrFlu")
The Animal.treatments represents a many-to-many relation, it is not an SQL Alchemy mode, so you can't pass it to the query nor use in a filter.
Next one can't work for the same reason (you pass Animal.treatments into the query.
The last one is closer, you actually need join to get your results.
I think it is easier to understand the query as SQL (and you anyway need to know SQL to be able to use sqlalchemy):
animals = session.query(Animal).from_statement(text(
"""
select distinct animals.* from animals
left join tr_association assoc on assoc.animals_id = animals.id
left join chronic_treatments on chronic_treatments.id = assoc.chronic_treatments_id
where chronic_treatments.code = :code
""")
).params(code='chrFlu')
It will select animals and join chronic_treatments through the tr_association and filter the result by code.
Having this it is easy to rewrite it using SQL-less syntax:
sql_query = session.query(Animal).join(Animal.treatments).filter(
ChronicTreatment.code == "chrFlu")
That will return what you want - a list of animals who have related chronic treatment with given code.