I have an entity in Google App Engine as below:
class HesapKalemi(ndb.Model):
hk=ndb.IntegerProperty(indexed=True)
ha=ndb.StringProperty(indexed=True)
A=ndb.FloatProperty(default=0.00)
B=ndb.FloatProperty(default=0.00)
C=ndb.FloatProperty(default=0.00)
F=ndb.FloatProperty(default=0.00)
G=ndb.FloatProperty(default=0.00)
H=ndb.FloatProperty(default=0.00)
I=ndb.FloatProperty(default=0.00)
J=ndb.FloatProperty(default=0.00)
DG=ndb.FloatProperty(default=0.00)
As known, the normal query can be below:
sektorkodu=self.request.get('sektorkodu')
qall=HesapKalemi.query().order(HesapKalemi.hk)
for hesap in qall:
hesap.ho=hesap.A
Is there any way to fetching A column by writing this way:
hesap.GETTHECOLUMN('A') or
hesap.GETTHECOLUMN(sektorkodu)
I have a very horizontal table and want to query it without if-else structure by the .GETTHECOLUMN('string') method.
Is there this kind of method?
In the NDB world, this is called Projection, or a Projection Query. In that link to the docs, you'll see the following:
Projection queries are similar to SQL queries of the form:
SELECT name, email, phone FROM CUSTOMER
So the .GETTHECOLUMN('A') method you're after would look like either of these:
qall_option_one = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=['A'])
qall_option_two = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=[HasepKalemi.A])
# to access the values
for hesap in qall_option_one:
print hesap
# output:
# HesapKalemi(key=Key('HesapKalemi', 1234567890), A=0.00, _projection=('A',))
# HesapKalemi(key=Key('HesapKalemi', 1234567891), A=0.00, _projection=('A',))
# ...
This is a bit faster than getting the full entities with all of their properties, but you do still have to iterate through them afterwards, even if you want to just generate a list of the 'A' values. Another option you should look at is "Calling a Function For Each Entity (Mapping)", where you define a callback function to be called on each entity as the query runs. So let's say you just want a list of the 'A' values. You could form that list like this:
def callback(hesap):
return hesap.A
a_values = HesapKelami.query().map(callback)
# a_values = [0.00, 0.00, ...]
If you're really after performance, look into asynchronous gets.
Note: instead of projection, you could use GQL, but that would look messier/more confusing than using projection with the regular ndb Query syntax IMO.
Edit: To answer your question in your comment, you can use either projection or mapping to select data from multiple properties.
Projection of multiple properties:
qall_option_one = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=['A', 'B', 'C'])
qall_option_two = HesapKalemi.query().order(HesapKalemi.hk).fetch(projection=[HesapKalemi.A, HesapKalemi.B, HesapKalemi.C])
# to access the values
for hesap in qall_option_one:
print hesap
# output:
# HesapKalemi(key=Key('HesapKalemi', 1234567890), A=0.00, B=0.00, C=0.00 _projection=('A', 'B', 'C',))
# HesapKalemi(key=Key('HesapKalemi', 1234567891), A=0.00, B=0.00, C=0.00 _projection=('A', 'B', 'C',))
# ...
Mapping to return multiple properties:
def callback(hesap):
# this returns a tuple of A,B,C values
return hesap.A, hesap.B, hesap.C
values = HesapKelami.query().map(callback)
# values is a list of tuples
# values = [(0.00, 0.00, 0.00), (0.00, 0.00, 0.00), ...]
Edit #2: After rereading the question and comments, I think your question, or at least part of it, may be how to get the property from the model itself using a string, and not how to pull one column out of the datastore. To answer that question, use getattr(hesap, "property_name"), or, and this may be more suited to your needs, turn hesap into a dict with hesap_dict = hesap.to_dict(). Then you could do this:
property_name = 'some_string'
hesap = HesapKelami.query().fetch(1)[0]
hesap_dict = hesap.to_dict()
property_value = hesap_dict.get(property_name, None)
You could pass hesap_dict to your Jinja2 template, and then I think you could accomplish what you asked about in your comments.
Related
I have 2 sqlalchemy queries...
dummy = DBSession().query(Dummy).filter(Dummy.c_name) == "foo")
dummy2 = DBSession().query(Dummy2).filter(Dummy2.c_name == "foo").filter(Dummy2.f_name == "bar")
And I have some sort of made up a generic function that combines the two...
def generic(Object, c_name, f_name):
dummy = DBSession().query(Object).filter(Object.c_name == c_name).filter(Object.f_name == f_name)
What's the best way to generically handle this, say if f_name doesn't exist or is not queryable in the Dummy2 table?
To summarise my question:
How do I create a general sqlalchemy query which can query any given table, where in some cases the attributes I am querying varies based on the given object.
I think I need some sort of reflection...maybe? or *args / **kwargs ... I dunno... help?
you should probably pack these in separate helper functions for reusability but here's how I approached this:
check_fields = {'c_name': 'exists', 'f_name': 'doesnt_exist'}
existing_fields = {}
# check if the fields exist in the table
for field in check_fields:
if field in given_table.c:
existing_fields.update({field: check_fields.get(field)})
# construct the query based on existing fields
query = given_table.select()
if existing_fields:
for k, v in existing_fields.items():
query = query.where(getattr(given_table.c, k) == v)
I'm then using session.execute(query) to get the results. Here's a similar answer that doesn't use execute: How to get query in sqlalchemyORM также
Note: All the attributes will be chained with AND
I am new to django so apologies if this is not possible or easy.
I have a view that takes a subset of a model
data = Terms.objects.filter(language = language_id)
The subset is one language. The set has a number of concepts for a language. Some languages might use the same word for multiple concepts, and I want to colour these the same in an SVG image. So I do this next:
for d in data:
if d.term is None:
d.colour = "#D3D3D3"
else:
d.colour = termColours[d.term]
Where termColours is a dictionary with keys as the unique terms and values as the hexadecimal colour I want.
I thought this would add a new colour attribute to my queryset. However, when I convert the queryset to json (in order to pass it to JS) the colour object is not there.
terms_json = serializers.serialize('json', data)
How can I add a new colour element to my queryset?
Convert your Queryset to Dict and then modify values.
Ex:
data = Terms.objects.filter(language = language_id).values()
for d in data:
if d.term is None:
d.colour = "#D3D3D3"
else:
d.colour = termColours[d.term]
If I understand correctly - you need Django ORM annotation. And it might look like that:
from django.db.models import Case, When, Value
data = Terms.objects.filter(language = language_id)
.annotate(color = Case(
When(term__isnull = True, then = "#D3D3D3"),
When(term__isnull = False, then = termColours[Value(term)]),))
Only problem here - I don't exactly know this moment - termColours[Value(term)], you need to test different combinations of that expressions to get the value of field term.
I have a queryset of Products with JSONField attributes containing dict
class Product(models.Model):
attributes = JSONField(
pgettext_lazy('Product field', 'attributes'),
encoder=DjangoJSONEncoder, default={})
I want to filter Products where attributes['12'] == '31'
Following one works:
qs.filter(attributes__contains={'12': '31'})
Following one does not:
qs.filter(attributes__12='31')
Is this something I can achieve with PostgreSQL or should I move it to ES?
EDIT:
Unfortunately I cannot use first solution, as this dict may contain more keys.
First solution works well. Given we have:
product.attributes = {'333': ['6', '1']}
We can filter it out by:
Product.objects.filter(attributes__contains={'333': ['6']}
etc. Totally overlooked it.
You should be able to use the second format, i.e. qs.filter(attributes__key='value').
Your issue in this case, as explained in the docs, is that when using an integer as a key in a JSON query, that key will be used as the index of an array, thus it's interpreted as attributes[12] instead of attributes['12'].
As long as you stick to string keys, you should be fine.
An example:
class MyModel(models.Model)
json = JSONField(default=dict)
p = MyModel.objects.create(json={'0': 'something', 'a': 'something else'})
MyModel.objects.filter(json__0='something') # returns empty queryset
MyModel.objects.filter(json__a='something else') # returns the object created above
I have the following models:
class Member(models.Model):
ref = models.CharField(max_length=200)
# some other stuff
def __str__(self):
return self.ref
class Feature(models.Model):
feature_id = models.BigIntegerField(default=0)
members = models.ManyToManyField(Member)
# some other stuff
A Member is basically just a pointer to a Feature. So let's say I have Features:
feature_id = 2, members = 1, 2
feature_id = 4
feature_id = 3
Then the members would be:
id = 1, ref = 4
id = 2, ref = 3
I want to find all of the Features which contain one or more Members from a list of "ok members." Currently my query looks like this:
# ndtmp is a query set of member-less Features which Members can point to
sids = [str(i) for i in list(ndtmp.values('feature_id'))]
# now make a query set that contains all rels and ways with at least one member with an id in sids
okmems = Member.objects.filter(ref__in=sids)
relsways = Feature.geoobjects.filter(members__in=okmems)
# now combine with nodes
op = relsways | ndtmp
This is enormously slow, and I'm not even sure if it's working. I've tried using print statements to debug, just to make sure anything is actually being parsed, and I get the following:
print(ndtmp.count())
>>> 12747
print(len(sids))
>>> 12747
print(okmems.count())
... and then the code just hangs for minutes, and eventually I quit it. I think that I just overcomplicated the query, but I'm not sure how best to simplify it. Should I:
Migrate Feature to use a CharField instead of a BigIntegerField? There is no real reason for me to use a BigIntegerField, I just did so because I was following a tutorial when I began this project. I tried a simple migration by just changing it in models.py and I got a "numeric" value in the column in PostgreSQL with format 'Decimal:( the id )', but there's probably some way around that that would force it to just shove the id into a string.
Use some feature of Many-To-Many Fields which I don't know abut to more efficiently check for matches
Calculate the bounding box of each Feature and store it in another column so that I don't have to do this calculation every time I query the database (so just the single fixed cost of calculation upon Migration + the cost of calculating whenever I add a new Feature or modify an existing one)?
Or something else? In case it helps, this is for a server-side script for an ongoing OpenStreetMap related project of mine, and you can see the work in progress here.
EDIT - I think a much faster way to get ndids is like this:
ndids = ndtmp.values_list('feature_id', flat=True)
This works, producing a non-empty set of ids.
Unfortunately, I am still at a loss as to how to get okmems. I tried:
okmems = Member.objects.filter(ref__in=str(ndids))
But it returns an empty query set. And I can confirm that the ref points are correct, via the following test:
Member.objects.values('ref')[:1]
>>> [{'ref': '2286047272'}]
Feature.objects.filter(feature_id='2286047272').values('feature_id')[:1]
>>> [{'feature_id': '2286047272'}]
You should take a look at annotate:
okmems = Member.objects.annotate(
feat_count=models.Count('feature')).filter(feat_count__gte=1)
relsways = Feature.geoobjects.filter(members__in=okmems)
Ultimately, I was wrong to set up the database using a numeric id in one table and a text-type id in the other. I am not very familiar with migrations yet, but as some point I'll have to take a deep dive into that world and figure out how to migrate my database to use numerics on both. For now, this works:
# ndtmp is a query set of member-less Features which Members can point to
# get the unique ids from ndtmp as strings
strids = ndtmp.extra({'feature_id_str':"CAST( \
feature_id AS VARCHAR)"}).order_by( \
'-feature_id_str').values_list('feature_id_str',flat=True).distinct()
# find all members whose ref values can be found in stride
okmems = Member.objects.filter(ref__in=strids)
# find all features containing one or more members in the accepted members list
relsways = Feature.geoobjects.filter(members__in=okmems)
# combine that with my existing list of allowed member-less features
op = relsways | ndtmp
# prove that this set is not empty
op.count()
# takes about 10 seconds
>>> 8997148 # looks like it worked!
Basically, I am making a query set of feature_ids (numerics) and casting it to be a query set of text-type (varchar) field values. I am then using values_list to make it only contain these string id values, and then I am finding all of the members whose ref ids are in that list of allowed Features. Now I know which members are allowed, so I can filter out all the Features which contain one or more members in that allowed list. Finally, I combine this query set of allowed Features which contain members with ndtmp, my original query set of allowed Features which do not contain members.
I have a view in Django that will read a get request and its parameters and make a query based on the parameters. Currently my view looks like this:
def getInventory(request):
c = request.GET.get('category', '')
g = request.GET.get('gender', '')
s = request.GET.get('size', '')
available = Item.objects.filter(gender=g,category=c,size=s)
data = serializers.serialize('json',available)
return HttpResponse(data,'json')
Sometimes, though, one of the parameters is not specified. I would like this to result in a value representing 'all' rather than an individual value. Is this possible in the way I've done it? I've tried gender=None but that just results in an empty list.
You need to apply a filter only if you want to filter something. You could do something like this:
# Start off with a base queryset
available = Item.objects.all()
allowed_filters = ['category', 'gender', 'size']
for f in allowed_filters:
if request.GET.get(f):
available = available.filter(**{f: request.GET[f]})
data = serializers.serialize('json',available)
return HttpResponse(data,'json')
That said, you might want to consider using the Forms API to validate the inputs before passing them to your queryset.