I am trying to iterate through each instance of a model I have defined.
Say I have the following in models.py, under the people django app:
class foo(models.Model):
name = models.CharField(max_length=20, default="No Name")
age = models.PositiveSmallIntegerField(default=0)
And I have populated the database to have the following data in foo:
name="Charley", age=17
name="Matthew", age=63
name="John", age=34
Now I want to work out the average age of my dataset. In another part of the program (outside the people app, inside the project folder), in a file called bar.py that will be set up to run automatically every day at a specific time, I calculate this average.
from people.models import foo
def findAverageAge():
total = 0
for instance in //What goes here?// :
total += instance.age
length = //What goes here, to find the number of instances?//
average = total / length
return average
print(findAverageAge)
In the code above, the //What goes here?// signifies I don't know what goes there and need help.
You can retrieve all elements with .all() [Django-doc]:
from people.models import foo
def findAverageAge():
total = 0
qs = foo.objects.all()
for instance in :
total += instance.age
length = len(qs)
average = total / length
return average
print(findAverageAge())
But you should not calculate the average at the Django/Python level. This requires retrieving all records, and this can be quite large. A database itself can calculate the average, and this is (normally) done in a more efficient way. You can .aggregate(…) [jango-doc] on the queryset with:
from people.models import foo
def findAverageAge():
return foo.objects.aggregate(
avg_age=Avg('age')
)['avg_age'] or 0
print(findAverageAge())
You should make a management command however: this will load the Django apps, and also makes it more convenient to run command with parameters, etc.
Related
I have 2 columns named Serial and Bag I need them to be auto incremented but based on each other and also based on the user that will update the record, so every bag should have 100 serial and reset the number automatically after reaching 100, then start again with Bag number 2 and put 100 serial in it and reset.
For example:
when user update the first record the Bag will start with number 1 and Serial will be also number 1 the second record Bag will still number 1 and the serial will be changed to number 2 till reach 100 Serial in one Bag, then we will start again with bag number 2 and serial number 1 etc ...
Thanks
The way you explain your example is a bit confusing but I'll try to give you an answer.
I assume the "2 columns named Serial and Bag" are fields of the same model and as you replied in the comments "the record is already existing but it has empty serial and bag", which means the auto-increment begins when the record is updated. Lastly, you mentioned first and second records implying that there are multiple records in this model. Based on these criteria, what you can do is add a save method in your model:
# Sample model
class Record(models.Model):
bag = models.IntegerField(default=0, null=True)
serial = models.IntegerField(default=0, null=True)
created_at = models.DateTimeField(auto_now=True, null=True)
def save(self, *args, **kwargs):
# Ensures the record will only auto-increment during update
if self.created_at:
# Retrieves the object with the highest bag & serial value
latest_record = Record.objects.all().order_by('bag', 'serial').last()
# Incrementing logic
if latest_record.serial_no + 1 <= 100:
self.bag = latest_record.bag if latest_record.bag > 0 else 1
self.serial = latest_record.serial + 1
else:
self.bag = latest_record.bag + 1
self.serial = 1
super(Record, self).save(*args, **kwargs)
Now, each time you write save like:
record = Record()
record.save()
The model save method executes.
Rather than do the incrementing logic in python, where it is subject to race conditions if multiple updates can happen concurrently, it should be possible to push it down into the database.
Something like:
update foop set
bag=vala,
ser=valb
from (
select
case when ser >= 5 then bag+1 else bag end as vala,
case when ser >= 5 then 1 else ser+1 end as valb
from foop
order by bag desc nulls last,
ser desc nulls last
limit 1) as tt
where some_primarykey = %;
It might be possible to translate that into django ORM, but it might also be easier and more readable to just drop into raw SQL or sneak it in via .extra() on a queryset than attempt to shoehorn it in.
I would like to display the percentage mark instead of the total sum of the mark. Right now i have a table that display the student name and their mark attendance. I would like to convert the mark attendance into a percentage. the current implementation is:
Student Name Attendance
Annie 200
Anny 150
But i would like to show the attendance in percentange. for example:
Student Name Attendance
Annie 100%
Anny 85%
i am not sure how to implement the method. But i have tried this:
# models.py:
class MarkAtt(models.Model):
studName = models.ForeignKey(
Namelist, on_delete=models.SET_NULL, blank=True, null=True, default=None,
)
classGrp = models.ForeignKey(GroupInfo, on_delete=models.SET_NULL, null=True)
currentDate = models.DateField(default=now())
week = models.IntegerField(default=1)
attendance = models.IntegerField(default=100) #1 is present
def get_percentage(self):
ttlCount = MarkAtt.objects.filter(studName).count()
perc = ttlCount / 1100 *100
return perc
# views.py:
def attStudName(request):
students = MarkAtt.objects.values('studName__VMSAcc').annotate(mark=Sum('attendance'))
context = {'students' : students}
return render(request,'show-name.html', context)
So you have your numerator but you need your denomenator. I'm not exactly sure what your denomenator should be with your current setup but creating a new field that uses "Count" rather than "Sum" might do the trick for you. Then you would divid the sum field by the count field. I would probably just do this in the view and not mess with the model.
You can use the formatting mini-language to express a percentage in a string
>>> attendence = 20
>>> total = 100
>>> '{:%}'.format(attendence/total)
'20%'
Bear in mind this will return the answer as string instead of an int or float
To use this in your get_percentage, there are a few issues that will need to be addressed:
studName is not defined or passed into the method but is used in the filter query. This will cause a NameError.
Your filter query looks like it is just counting the students in the filter query instead of summing the attendance. You should use a sum annotation like you have done in the view.
You will need to be able to get the value for 100% attendance to be able to divide by it.
To modify your implimentatiom of this on the model you could do something like this.
def get_percentage(self):
student = MarkAtt.objects.filter(studName=self.studName).annotate(mark=Sum('attendance'))
return '{:%}'.format(student.mark / 1100)
However, I don't think the MarkAtt model is the right place to do this as many MarkAtt objects could relate to one NameList object resulting in possibly running the same query several times for each student. I think it would be better to do this on NameList or the view itself.
class NameList(models.Model):
...
def get_percentage(self):
attendance = self.markatt_set().annotate(total=Sum('attendance'))
return '{:%}'.format(attendance.total / 1100)
I have created a custom field in the model mrp.BOM structure as below to compute the total cost of BOM for a product:-
Field Name:- x_bom_total
Field Label:- Total BOM
Field Type:- Float
ReadOnly:- True
Dependency:- bom_line_ids
Dependent Field name 'bom_line_ids' is the field which display all the materials used in the product. It refernces to the model 'mrp.bom.line' in a one2many relationship model. So now in the computation part how to calculate the Total BOM for the product something like this:-
for record in self:
for each_object in record.bom_line_ids:
record['x_bom_total'] += record.bom_line_ids.qty * record.bom_line_ids.list_price
I am using odoo v11. Does anyone have an idea?
You were on the right way, but you should consider child BOMs, too.
First your nearly correct approach without child BOMs:
for record in self:
total = 0.0
for line in record.bom_line_ids:
total += line.product_qty * line.product_id.list_price
record['x_bom_total'] = total
And now with consideration of child BOMs. You're obviously using Odoo Studio App, but i don't know if you can define methods on computed fields, but you can give it a try. A recursive function would be really nice here:
def get_bom_total(lines)
total = 0.0
for line in lines:
if line.child_bom_id:
total += get_bom_total(line.child_bom_ids) # recursive call
else:
total += line.product_qty * line.product_id.list_price
return total
for record in self:
record['x_bom_total'] = get_bom_total(record.bom_line_ids)
I have a model to which I have to record a PositiveSmallIntegerField to the object, that is updated daily with the relevant score.
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
rank = ??
The number of objects with this model will never exceed 100 and the scores/ranks must be retained for a period of 180 days. The database is Postgresql 9.2.
The rank is calculated daily on the score from another app, which I want to store in the database related to the student model, where I'm stuck with the model design, I have no Idea, what should be done for the ranks? Is there a repeating field in Django?
Any clues or experiences will be much appreciated
thanks.
Update:(Adding an example)
The database must look something like this,
+---------+-------+----------+----------+----------+----------+----------+----------+
| Student | Grade | 08-01-15 | 08-02-15 | 08-03-15 | 08-04-15 | 08-05-15 | 08-06-15 |
+---------+-------+----------+----------+----------+----------+----------+----------+
| Alex | 5 | 2 | 1 | 1 | 2 | 3 | 2 |
| John | 5 | 3 | 2 | 3 | 4 | 2 | 4 |
| Susan | 5 | 1 | 4 | 2 | 1 | 1 | 1 |
| Zara | 5 | 4 | 3 | 4 | 3 | 4 | 3 |
+---------+-------+----------+----------+----------+----------+----------+----------+
The rank of the student must be stored for the days like shown here, for the day 1, the ranks must be stored in a column/anything similar, and the number of days must go on for the consecutive 180 days, the ranks for each day must be added to the consecutive days.
I'm not stuck with the save method, but about the field where to save the calculated ranks.
I would suggest something similar to what e4c5 suggested, but I would also:
Generate an index on the date of the ranks so that obtaining all the ranks on any single day can be optimized.
Mark the date and student as unique_together. This prevents the possibility of recording two ranks for the same student on the same date.
The models would look like this:
from django.db import models
class Grade(models.Model):
pass # Whatever you need here...
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
class Rank(models.Model):
class Meta(object):
unique_together = (("date", "student"), )
date = models.DateField(db_index=True)
student = models.ForeignKey(Student)
value = models.IntegerField()
In a full-fledged application I'd also expect to have some uniqueness constraints on Grade and Student but the problem presented in the question does not provide enough details about these models.
You could then run a task every day with cron or whatever task manager you want to use (Celery is also an option), to run a command like the following that would update the ranks according to some computation and purge the old records. The following code is an illustration of how it can be done. The real code should be designed to be generally idempotent (the following code is not because the rank computation is random) so that if the server is rebooted in the middle of an update, the command can just be rerun. Here's the code:
import random
import datetime
from optparse import make_option
from django.utils.timezone import utc
from django.core.management.base import BaseCommand
from school.models import Rank, Student
def utcnow():
return datetime.datetime.utcnow().replace(tzinfo=utc)
class Command(BaseCommand):
help = "Compute ranks and cull the old ones"
option_list = BaseCommand.option_list + (
make_option('--fake-now',
default=None,
help='Fake the now value to X days ago.'),
)
def handle(self, *args, **options):
now = utcnow()
fake_now = options["fake_now"]
if fake_now is not None:
now -= datetime.timedelta(days=int(fake_now))
print "Setting now to: ", now
for student in Student.objects.all():
# This simulates a rank computation for the purpose of
# illustration.
rank_value = random.randint(1, 1000)
try:
rank = Rank.objects.get(student=student, date=now)
except Rank.DoesNotExist:
rank = Rank(
student=student, date=now)
rank.value = rank_value
rank.save()
# Delete all ranks older than 180 days.
Rank.objects.filter(
date__lt=now - datetime.timedelta(days=180)).delete()
Why not pickles?
Multiple reasons:
It is a premature optimization, and overall probably not an optimization at all. Some operations may be faster, but other operations will be slower. If the ranks are pickled into a field on Student then, loading a specific student in memory means loading all the rank information into memory together with that student. This can be mitigated by using .values() or .values_list() but then you are no longer getting Student instances from the database. Why have Student instances in the first place and not just access the raw database?
If I change the fields in Rank, Django's migration facilities easily allow performing the needed changes when I deploy the new version of my application. If the rank information is pickled into a field, I have to manage any structure change by writing custom code.
The database software cannot access values in a pickle and so you have to write custom code to access them. With the model above, if you want to list students by rank today (and the ranks for today have already been computed), then you can do:
for r in Rank.objects.filter(date=utcnow()).order_by("value")\
.prefetch_related():
print r.student.name
If you use pickles, you have to scan all Students and unpickle the ranks to extract the one for the day you want, and then use a Python data structure to order the students by rank. Once this is done, you then have to iterate over this structure to get the names in order.
There is a much better way. Use redis Sorted sets
With sorted sets you can add, remove, or update elements in a very fast way (in a time proportional to the logarithm of the number of elements). Since elements are taken in order and not ordered afterwards, you can also get ranges by score or by rank (position) in a very fast way. Accessing the middle of a sorted set is also very fast, so you can use Sorted Sets as a smart list of non repeating elements where you can quickly access everything you need: elements in order, fast existence test, fast access to elements in the middle!
In short with sorted sets you can do a lot of tasks with great performance that are really hard to model in other kind of databases
How to make use of this feature? Make a sorted set for each of the 180 days that you are interested in. You can either use the string representation of the date or just count them up like day1, day2 etc. Then when you calculate the rank for each user. Add it into the redis set (code borrowed from e4c5's answer).
def save(self, *args, **kwargs):
r = redis.Redis()
super(Exam,self).save(*args,**kwargs)
student = Student.objects.get(id = self.student_id)
rank = some_calculation
student.save()
r.zadd('dayx', self.name, self.rank)
Then you can retrieve rankings for any given day by r.zrange('dayx'). And really other than the imports and whatever logic to calculate the rankings. This is all there is to it.
Updating my answer after you updated the question with an example.
You should not do this with one table, you need two. ONe should be the student model which would look like this.
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
The other would be the rank model which might look like this.
class Rank(models.Model):
data = models.DateField()
student = models.ForeignKey(Student)
The following information is for the original question but parts of it will still be relevant I think.
1) Override save method in the Student model.
def save(self, *args, **kwargs):
super(Exam,self).save(*args,**kwargs)
student = Student.objects.get(id = self.student_id)
student.rank = some_calculation
student.save()
2) Use the post_save signal on the Exam object.
Similar to above
3) Use a trigger.
Since you are using postgresql, you can use the much more elegant solution of creating an AFTER INSERT trigger
If you really want to have just an object field instead of joining two objects you can use something like
https://github.com/shrubberysoft/django-picklefield
serializing into a single text field a dictionary where key/val is date/rank
( this library is quite old, I don't think it will work out of the box with a modern django project )
Anyway, unless you are really forced by some external constraint to use this ugly solution the better (right) way is to just join two meaningful objects like #e4c5 suggested.
I once used such a field in a project where auth_user was a db view shared between multiple projects. That way I asked for just one migrate on the central db to add the "pickled" field and I was then able to add every user option i wished without tampering the original model further.
Anyway:
You can't query for stuff like "average rank on 12 september" with Django ORM.
It's not faster, you are not taking into account the time to depickle the text string into an object, quite sure is slower, probably is way slower.
In my case I had just a couple of options per user without need to query on them, more than that I think you're asking for future troubles mantaining your code.
I have the following models in django
class Job(models.Model):
cost = models.FloatField()
class Account(models.Model):
job = models.ManyToManyField(Job, through='HasJob')
class HasJob(models.Model):
account = models.ForeignKey(Account, related_name='hasjobs')
job = models.ForeignKey(Job, related_name='hasjobs')
quantity = models.IntegerField()
So an Account can have many jobs in different quantities. I want to be able to sum up the total cost of an account. Is that possible in database level or should I do python for it? Like
account = Account.objects.get(pk=1)
sum = 0
for hasjob in account.hasjobs.all():
sum += hasjob.quantity*hasjob.job.cost
I know its a very "starters" way to do that, and I am guessing it includes many hits on the database. So is there a better way?
IFAIK aggregation can't sum by F() expressions so you have to calculate the sum in python code.
But you can reduce the number of db hits to one - just add the select_related() call to the queryset:
total_sum = sum(hasjob.quantity * hasjob.job.cost
for hasjob in account.hasjobs.all().select_related('job'))