So, in my Django.models I have a class as the following:
class Basicorderbook(models.Model):
ordernumber = models.ForeignKey('Idbook', db_column='OrderNumber', related_name='BasicOrder_IDNumber', primary_key=True) # Field name made lowercase.
username = models.ForeignKey('Userbook', db_column='Username', to_field='username', related_name='BasicOrder_Username') # Field name made lowercase.
price = models.FloatField(db_column='Price') # Field name made lowercase.
volume = models.FloatField(db_column='Volume') # Field name made lowercase.
type = models.CharField(db_column='Type', max_length=20) # Field name made lowercase.
...
def __unicode__(self):
return unicode(self.ordernumber.idnumber)
...
and a view as the following to return an object containing multiple "Basic Orders":
class BuyOrdersView(generic.ListView):
model = models.Basicorderbook
template_name = 'Blank.html'
context_object_name = 'buy_order_list'
def get_queryset(self):
"""
Returns the 5 most viable buy orders
"""
base_buy_order_list = models.Basicorderbook.objects.using('exchange').filter(
action = 'Buy',
#price > 520,
#active = 1
)
volume_buy_order_list = base_buy_order_list.order_by('-volume')
#This line here in question (I know this won't work, it's just an example)
type_buy_order_list = volume_buy_order_list.order_by('Limit' in 'type')
price_buy_orders = type_buy_order_list.order_by('-price')
buy_order_list = price_buy_orders[:15]
return buy_order_list
The "type" attribute of the object can be either "Limit", "Liquid", or "Conditional"
As shown, the problem lies in how I intend to sort the order object. I want to make it so that it is ordered with the following hierarchy:
Highest price first
If price is the same, have orders with "conditional" type first, then "limit", then "liquid"
If an order's price and type are the same, have orders sorted by volume
The numerical values (Price, Volume) are easily sortable by the handy order_by function, but when it comes to the string attributes I've come to a stop. It's not even an alphabetical order, so I can't do it that way. I see a few options that are simply annoying and inefficient workarounds:
Use a for loop to take each object and append into a list based on their string attribute
Create another attribute, "TypePriority", which I assign a numerical value (1, 2, 3) based on its priority in regards to the order type
So my TLDR question is, is there a way to sort objects based on a string value attribute in a non-alphabetic order? Also, will my object maintain it's sorting hierarchy with my current model, or should I join the order_by statements in a different way (Not sure if sorting by a certain attribute will "unsort" the new object I just created).
Note: the "_list" suffix is only for clarity, I know it's not a list :)
UPDATE: My current implementation that works is my workaround #1 example, but I'm wondering if there's way to sort it otherwise.
Per your text 'conditional' < 'limit' < 'liquid' so sorting on this should work.
Can you just do a three-way sort such as ('-price', 'type', 'volume') ?
Off course this assumes that the sort is not case sensitive or the input is modified at some point
D.
I forgot to update, but it turns out that there is no way to reorder the object as I wanted. I ended up doing the other reorganizations (by price, volume, etc.), but had to create a new list and check for the "type" attribute and sort them as such. It ended up looking like:
type_buy_order_list = []
for order in volume_buy_order_list:
if order[3] == "limit":
type_buy_order_list.append(order)
if order[3] == "liquid":
type_buy_order_list.append(order)
if order[3] == "conditional":
type_buy_order_list.append(order)
Related
I have three numeric columns in my model that together create a string, which is presented to the user:
class Location(models.Model):
aisle = models.PositiveIntegerField()
rack = models.PositiveIntegerField()
plank = models.PositiveIntegerField()
def __unicode__(self):
return "{a:02}{r:02}{p:02}".format(a=self.aisle, r=self.rack, p=self.plank)
Now I want to filter on (part of) that string, so say I have three locations, 010101, 010102, 010201, and I filter on 0101, I want to select only the first two.
How would I do that, I looked into the Q objects and available database functions, but I don't find a solution.
After a lot of experimenting, I managed to do it using a Func:
class LocationLabel(Func):
function = 'CONCAT'
template = '%(function)s(RIGHT(CONCAT(\'00\',%(expressions)s),2))'
arg_joiner = '),2), RIGHT(CONCAT(\'00\','
models.Location.object.
annotate(locationlabel=
LocationLabel('aisle','rack','plank', output_field=CharField())
).
filter(locationlabel__icontains=query)
You cannot perform a filter on a property, it has to be on fields.
In this case i think this will do what you require because unicode is just a formatted form actual integer value in fields:
Location.objects.filter(aisle=1, rack=1)
I have the following models:
class Member(models.Model):
ref = models.CharField(max_length=200)
# some other stuff
def __str__(self):
return self.ref
class Feature(models.Model):
feature_id = models.BigIntegerField(default=0)
members = models.ManyToManyField(Member)
# some other stuff
A Member is basically just a pointer to a Feature. So let's say I have Features:
feature_id = 2, members = 1, 2
feature_id = 4
feature_id = 3
Then the members would be:
id = 1, ref = 4
id = 2, ref = 3
I want to find all of the Features which contain one or more Members from a list of "ok members." Currently my query looks like this:
# ndtmp is a query set of member-less Features which Members can point to
sids = [str(i) for i in list(ndtmp.values('feature_id'))]
# now make a query set that contains all rels and ways with at least one member with an id in sids
okmems = Member.objects.filter(ref__in=sids)
relsways = Feature.geoobjects.filter(members__in=okmems)
# now combine with nodes
op = relsways | ndtmp
This is enormously slow, and I'm not even sure if it's working. I've tried using print statements to debug, just to make sure anything is actually being parsed, and I get the following:
print(ndtmp.count())
>>> 12747
print(len(sids))
>>> 12747
print(okmems.count())
... and then the code just hangs for minutes, and eventually I quit it. I think that I just overcomplicated the query, but I'm not sure how best to simplify it. Should I:
Migrate Feature to use a CharField instead of a BigIntegerField? There is no real reason for me to use a BigIntegerField, I just did so because I was following a tutorial when I began this project. I tried a simple migration by just changing it in models.py and I got a "numeric" value in the column in PostgreSQL with format 'Decimal:( the id )', but there's probably some way around that that would force it to just shove the id into a string.
Use some feature of Many-To-Many Fields which I don't know abut to more efficiently check for matches
Calculate the bounding box of each Feature and store it in another column so that I don't have to do this calculation every time I query the database (so just the single fixed cost of calculation upon Migration + the cost of calculating whenever I add a new Feature or modify an existing one)?
Or something else? In case it helps, this is for a server-side script for an ongoing OpenStreetMap related project of mine, and you can see the work in progress here.
EDIT - I think a much faster way to get ndids is like this:
ndids = ndtmp.values_list('feature_id', flat=True)
This works, producing a non-empty set of ids.
Unfortunately, I am still at a loss as to how to get okmems. I tried:
okmems = Member.objects.filter(ref__in=str(ndids))
But it returns an empty query set. And I can confirm that the ref points are correct, via the following test:
Member.objects.values('ref')[:1]
>>> [{'ref': '2286047272'}]
Feature.objects.filter(feature_id='2286047272').values('feature_id')[:1]
>>> [{'feature_id': '2286047272'}]
You should take a look at annotate:
okmems = Member.objects.annotate(
feat_count=models.Count('feature')).filter(feat_count__gte=1)
relsways = Feature.geoobjects.filter(members__in=okmems)
Ultimately, I was wrong to set up the database using a numeric id in one table and a text-type id in the other. I am not very familiar with migrations yet, but as some point I'll have to take a deep dive into that world and figure out how to migrate my database to use numerics on both. For now, this works:
# ndtmp is a query set of member-less Features which Members can point to
# get the unique ids from ndtmp as strings
strids = ndtmp.extra({'feature_id_str':"CAST( \
feature_id AS VARCHAR)"}).order_by( \
'-feature_id_str').values_list('feature_id_str',flat=True).distinct()
# find all members whose ref values can be found in stride
okmems = Member.objects.filter(ref__in=strids)
# find all features containing one or more members in the accepted members list
relsways = Feature.geoobjects.filter(members__in=okmems)
# combine that with my existing list of allowed member-less features
op = relsways | ndtmp
# prove that this set is not empty
op.count()
# takes about 10 seconds
>>> 8997148 # looks like it worked!
Basically, I am making a query set of feature_ids (numerics) and casting it to be a query set of text-type (varchar) field values. I am then using values_list to make it only contain these string id values, and then I am finding all of the members whose ref ids are in that list of allowed Features. Now I know which members are allowed, so I can filter out all the Features which contain one or more members in that allowed list. Finally, I combine this query set of allowed Features which contain members with ndtmp, my original query set of allowed Features which do not contain members.
I've hit an interesting problem that I know must have a simple solution, but I'm at a loss. I have a model with a field "priority_score" that changes every day. I want to forecast which model will have the highest priority_score on any given day in the next two weeks.
WEEK = 'WE'
MONTH = 'MO'
QUARTER = 'QU'
TIME_PERIOD_CHOICES = (
(WEEK, 'Week'),
(MONTH, 'Month'),
(QUARTER, 'Quarter'),
)
class Exercise(models.Model):
priority_score = models.DecimalField(max_digits=5, decimal_places=3, editable=False, default = 0)
frequency = models.IntegerField()
time_period = models.CharField(max_length=2, choices=TIME_PERIOD_CHOICES,default=WEEK)
There are a few other factors that affect how any given object's priority_score will change. I'll skip this detail, but in general, I'm iterating 14 times (once per every day over the next two weeks) to determine what the highest priority_score is.
time_period = {
"WE" : 7,
"MO" : 30,
"QU" : 90,
}
days_to_forecast = 14
recommendations = []
exercise_list = Exercise.objects.all()
for i in range(0, days_to_forecast-1):
queryset = sorted(
chain(exercise_list),
key=attrgetter('priority_score'),
reverse=True)
recommendations.append(queryset)
print i
print queryset[0]
print queryset[0].priority_score
print ""
queryset[0].priority_score = 0
for obj in queryset:
priority_iteration = Decimal(float(obj.frequency) / time_period[obj.time_period])
obj.priority_score = obj.priority_score + priority_iteration
The output of the print statements are an accurate reflection of the object with the highest priority_score on each date as well as the priority_score for that object on that date.
What I'm trying to do though is to store the sorted results per day and show that in a template. In each iteration, I'm storing the new queryset in the recommendations list. But when I display the final output of recommendations, it is instead displaying the latest "results" (priority scores) for each object, not the priority scores as of a specific iteration over a two week period.
I believe what's happening is that recommendations is only storing a reference to the object, not the values themselves. So when I fetch the results for recommendations, it is then resolving the query and only showing the end result of each object, not the result at the time I appended the queryset to the recommendations list.
How can I store the result of each iteration of the loop into an object that can then be referenced at the end of the entire loop?
Update
I've isolated the issue to this section:
queryset = sorted(
chain(exercise_list),
key=attrgetter('priority_score'),
reverse=True)
It seems that if I just say queryset = exercise_list, I wouldn't have this issue. But somehow, using sorted(), the output of my recommendations change as querset values change
I'm not sure of the technical aspects of sorted(), but I believe that this function doesn't return an actual object but rather a reference to the object. Therefore, when any updates are made, they are traced back to the original source (in this case, the original exercise_list). So any variable based on the original exercise_list is automatically updated when django resolves the variable to values.
To retain the values that are current as of variable declaration, a copy must be made
import copy
queryset = sorted(
chain(exercise_list),
key=attrgetter('priority_score'),
reverse=True)
newobj = copy.deepcopy(queryset)
recommendations.append(newobj)
These values stored in recommendations will not update when queryset values are changed.
Okay here is the problem. I have this code
list_categories = [None,"mathematics","engineering","science","other"]
class Books(db.Model)
title = db.StringProperty(required=True)
author = db.StringProperty()
isbn = db.StringProperty()
categories = db.StringListProperty(default=None, choices = set(list_categories))
what i want to do here is have my book.categories be a SUBSET of list categories, for example
i have a book whose categories should be 'engineering' and 'mathematics', but when I set
book.categories = ['engineering','mathematics']
it webapp2 gives me an error
BadValueError: Property categories is ['engineering','mathematics']; must be one of set([None,"mathematics","engineering","science","other"])
My initial guess here is that i must set my list_choices to be a POWERSET of [None,"mathematics","engineering","science","other"], but this is too inefficient.
Does anyone know a workaround for this?
The reason for the error (as I'm sure you've guessed) is that StringListProperty does not do any special handling of the choices keyword argument - it simply passes it along to the ListProperty constructor, which in turn passes it to the Property constructor, where it is evaluated:
if self.empty(value):
if self.required:
raise BadValueError('Property %s is required' % self.name)
else:
if self.choices:
match = False
for choice in self.choices:
if choice == value:
match = True
if not match:
raise BadValueError('Property %s is %r; must be one of %r' %
(self.name, value, self.choices))
The issue is that it iterates through each choice individually, but it is comparing it to your entire list (value), which will never result in a match since a string won't equal a list (again, you know this :) ).
My suggestion would be to modify how you assign the list to the property. For instance, instead of:
book.categories = ['engineering','mathematics']
Try something like this:
for category in ['engineering','mathematics']:
book.categories.append(category)
Since the ListProperty contains a list, you can append each item individually so that it passes the test in the aforementioned code. Note that in order to get this to work in my tests, I had to set up the model in a slightly different way - however if you can get to the error you mentioned above, then the append method should work fine.
It makes it a little less straightforward, I agree, but it should circumvent the issue above and hopefully work.
Create a many to many relationship using list of keys. Use the categories property in class Book as a list of keys of class Category.
class Book(db.Model)
title = db.StringProperty(required=True)
author = db.StringProperty()
isbn = db.StringProperty()
# List Of Keys
categories = db.ListProperty(db.Key)
class Category(db.Model)
name = db.StringProperty(choices = ('science', 'engineering', 'math'))
For more info and code samples about modeling check out: https://developers.google.com/appengine/articles/modeling
class Price(models.Model):
date = models.DateField()
price = models.DecimalField(max_digits=6, decimal_places=2)
product = models.ForeignKey("Product")
class Product(models.Model):
name = models.CharField(max_length=256)
price_history = models.ManyToManyField(Price, related_name="product_price", blank=True)
I want to query Product such that I return only those products for whom the price on date x is higher than any earlier date.
Thanks boffins.
As Marcin said in another answer, you can drill down across relationships using the double underscore syntax. However, you can also chain them and sometimes this can be easier to understand logically, even though it leads to more lines of code. In your case though, I might do something that would look this:
first you want to know the price on date x:
a = Product.objects.filter(price_history__date = somedate_x)
you should probably test to see if there are more than one per date:
if a.count() == 1:
pass
else:
do something else here
(or something like that)
Now you have your price and you know your date, so just do this:
b = Product.objects.filter(price_history__date__lt = somedate, price_history__price__gt=a[0].price)
know that the slice will hit the database on its own and return an object. So this query will hit the database three times per function call, once for the count, once for the slice, and once for the actual query. You could forego the count and the slice by doing an aggregate function (like an average across all the returned rows in a day) but those can get expensive in their own right.
for more information, see the queryset api:
https://docs.djangoproject.com/en/dev/ref/models/querysets/
You can perform a query that spans relationships using this syntax:
Product.objects.filter(price_history__price = 3)
However, I'm not sure that it's possible to perform the query you want efficiently in a pure django query.