I have 2 columns named Serial and Bag I need them to be auto incremented but based on each other and also based on the user that will update the record, so every bag should have 100 serial and reset the number automatically after reaching 100, then start again with Bag number 2 and put 100 serial in it and reset.
For example:
when user update the first record the Bag will start with number 1 and Serial will be also number 1 the second record Bag will still number 1 and the serial will be changed to number 2 till reach 100 Serial in one Bag, then we will start again with bag number 2 and serial number 1 etc ...
Thanks
The way you explain your example is a bit confusing but I'll try to give you an answer.
I assume the "2 columns named Serial and Bag" are fields of the same model and as you replied in the comments "the record is already existing but it has empty serial and bag", which means the auto-increment begins when the record is updated. Lastly, you mentioned first and second records implying that there are multiple records in this model. Based on these criteria, what you can do is add a save method in your model:
# Sample model
class Record(models.Model):
bag = models.IntegerField(default=0, null=True)
serial = models.IntegerField(default=0, null=True)
created_at = models.DateTimeField(auto_now=True, null=True)
def save(self, *args, **kwargs):
# Ensures the record will only auto-increment during update
if self.created_at:
# Retrieves the object with the highest bag & serial value
latest_record = Record.objects.all().order_by('bag', 'serial').last()
# Incrementing logic
if latest_record.serial_no + 1 <= 100:
self.bag = latest_record.bag if latest_record.bag > 0 else 1
self.serial = latest_record.serial + 1
else:
self.bag = latest_record.bag + 1
self.serial = 1
super(Record, self).save(*args, **kwargs)
Now, each time you write save like:
record = Record()
record.save()
The model save method executes.
Rather than do the incrementing logic in python, where it is subject to race conditions if multiple updates can happen concurrently, it should be possible to push it down into the database.
Something like:
update foop set
bag=vala,
ser=valb
from (
select
case when ser >= 5 then bag+1 else bag end as vala,
case when ser >= 5 then 1 else ser+1 end as valb
from foop
order by bag desc nulls last,
ser desc nulls last
limit 1) as tt
where some_primarykey = %;
It might be possible to translate that into django ORM, but it might also be easier and more readable to just drop into raw SQL or sneak it in via .extra() on a queryset than attempt to shoehorn it in.
Related
I am trying to iterate through each instance of a model I have defined.
Say I have the following in models.py, under the people django app:
class foo(models.Model):
name = models.CharField(max_length=20, default="No Name")
age = models.PositiveSmallIntegerField(default=0)
And I have populated the database to have the following data in foo:
name="Charley", age=17
name="Matthew", age=63
name="John", age=34
Now I want to work out the average age of my dataset. In another part of the program (outside the people app, inside the project folder), in a file called bar.py that will be set up to run automatically every day at a specific time, I calculate this average.
from people.models import foo
def findAverageAge():
total = 0
for instance in //What goes here?// :
total += instance.age
length = //What goes here, to find the number of instances?//
average = total / length
return average
print(findAverageAge)
In the code above, the //What goes here?// signifies I don't know what goes there and need help.
You can retrieve all elements with .all() [Django-doc]:
from people.models import foo
def findAverageAge():
total = 0
qs = foo.objects.all()
for instance in :
total += instance.age
length = len(qs)
average = total / length
return average
print(findAverageAge())
But you should not calculate the average at the Django/Python level. This requires retrieving all records, and this can be quite large. A database itself can calculate the average, and this is (normally) done in a more efficient way. You can .aggregate(…) [jango-doc] on the queryset with:
from people.models import foo
def findAverageAge():
return foo.objects.aggregate(
avg_age=Avg('age')
)['avg_age'] or 0
print(findAverageAge())
You should make a management command however: this will load the Django apps, and also makes it more convenient to run command with parameters, etc.
I would like to display the percentage mark instead of the total sum of the mark. Right now i have a table that display the student name and their mark attendance. I would like to convert the mark attendance into a percentage. the current implementation is:
Student Name Attendance
Annie 200
Anny 150
But i would like to show the attendance in percentange. for example:
Student Name Attendance
Annie 100%
Anny 85%
i am not sure how to implement the method. But i have tried this:
# models.py:
class MarkAtt(models.Model):
studName = models.ForeignKey(
Namelist, on_delete=models.SET_NULL, blank=True, null=True, default=None,
)
classGrp = models.ForeignKey(GroupInfo, on_delete=models.SET_NULL, null=True)
currentDate = models.DateField(default=now())
week = models.IntegerField(default=1)
attendance = models.IntegerField(default=100) #1 is present
def get_percentage(self):
ttlCount = MarkAtt.objects.filter(studName).count()
perc = ttlCount / 1100 *100
return perc
# views.py:
def attStudName(request):
students = MarkAtt.objects.values('studName__VMSAcc').annotate(mark=Sum('attendance'))
context = {'students' : students}
return render(request,'show-name.html', context)
So you have your numerator but you need your denomenator. I'm not exactly sure what your denomenator should be with your current setup but creating a new field that uses "Count" rather than "Sum" might do the trick for you. Then you would divid the sum field by the count field. I would probably just do this in the view and not mess with the model.
You can use the formatting mini-language to express a percentage in a string
>>> attendence = 20
>>> total = 100
>>> '{:%}'.format(attendence/total)
'20%'
Bear in mind this will return the answer as string instead of an int or float
To use this in your get_percentage, there are a few issues that will need to be addressed:
studName is not defined or passed into the method but is used in the filter query. This will cause a NameError.
Your filter query looks like it is just counting the students in the filter query instead of summing the attendance. You should use a sum annotation like you have done in the view.
You will need to be able to get the value for 100% attendance to be able to divide by it.
To modify your implimentatiom of this on the model you could do something like this.
def get_percentage(self):
student = MarkAtt.objects.filter(studName=self.studName).annotate(mark=Sum('attendance'))
return '{:%}'.format(student.mark / 1100)
However, I don't think the MarkAtt model is the right place to do this as many MarkAtt objects could relate to one NameList object resulting in possibly running the same query several times for each student. I think it would be better to do this on NameList or the view itself.
class NameList(models.Model):
...
def get_percentage(self):
attendance = self.markatt_set().annotate(total=Sum('attendance'))
return '{:%}'.format(attendance.total / 1100)
I have a model to which I have to record a PositiveSmallIntegerField to the object, that is updated daily with the relevant score.
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
rank = ??
The number of objects with this model will never exceed 100 and the scores/ranks must be retained for a period of 180 days. The database is Postgresql 9.2.
The rank is calculated daily on the score from another app, which I want to store in the database related to the student model, where I'm stuck with the model design, I have no Idea, what should be done for the ranks? Is there a repeating field in Django?
Any clues or experiences will be much appreciated
thanks.
Update:(Adding an example)
The database must look something like this,
+---------+-------+----------+----------+----------+----------+----------+----------+
| Student | Grade | 08-01-15 | 08-02-15 | 08-03-15 | 08-04-15 | 08-05-15 | 08-06-15 |
+---------+-------+----------+----------+----------+----------+----------+----------+
| Alex | 5 | 2 | 1 | 1 | 2 | 3 | 2 |
| John | 5 | 3 | 2 | 3 | 4 | 2 | 4 |
| Susan | 5 | 1 | 4 | 2 | 1 | 1 | 1 |
| Zara | 5 | 4 | 3 | 4 | 3 | 4 | 3 |
+---------+-------+----------+----------+----------+----------+----------+----------+
The rank of the student must be stored for the days like shown here, for the day 1, the ranks must be stored in a column/anything similar, and the number of days must go on for the consecutive 180 days, the ranks for each day must be added to the consecutive days.
I'm not stuck with the save method, but about the field where to save the calculated ranks.
I would suggest something similar to what e4c5 suggested, but I would also:
Generate an index on the date of the ranks so that obtaining all the ranks on any single day can be optimized.
Mark the date and student as unique_together. This prevents the possibility of recording two ranks for the same student on the same date.
The models would look like this:
from django.db import models
class Grade(models.Model):
pass # Whatever you need here...
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
class Rank(models.Model):
class Meta(object):
unique_together = (("date", "student"), )
date = models.DateField(db_index=True)
student = models.ForeignKey(Student)
value = models.IntegerField()
In a full-fledged application I'd also expect to have some uniqueness constraints on Grade and Student but the problem presented in the question does not provide enough details about these models.
You could then run a task every day with cron or whatever task manager you want to use (Celery is also an option), to run a command like the following that would update the ranks according to some computation and purge the old records. The following code is an illustration of how it can be done. The real code should be designed to be generally idempotent (the following code is not because the rank computation is random) so that if the server is rebooted in the middle of an update, the command can just be rerun. Here's the code:
import random
import datetime
from optparse import make_option
from django.utils.timezone import utc
from django.core.management.base import BaseCommand
from school.models import Rank, Student
def utcnow():
return datetime.datetime.utcnow().replace(tzinfo=utc)
class Command(BaseCommand):
help = "Compute ranks and cull the old ones"
option_list = BaseCommand.option_list + (
make_option('--fake-now',
default=None,
help='Fake the now value to X days ago.'),
)
def handle(self, *args, **options):
now = utcnow()
fake_now = options["fake_now"]
if fake_now is not None:
now -= datetime.timedelta(days=int(fake_now))
print "Setting now to: ", now
for student in Student.objects.all():
# This simulates a rank computation for the purpose of
# illustration.
rank_value = random.randint(1, 1000)
try:
rank = Rank.objects.get(student=student, date=now)
except Rank.DoesNotExist:
rank = Rank(
student=student, date=now)
rank.value = rank_value
rank.save()
# Delete all ranks older than 180 days.
Rank.objects.filter(
date__lt=now - datetime.timedelta(days=180)).delete()
Why not pickles?
Multiple reasons:
It is a premature optimization, and overall probably not an optimization at all. Some operations may be faster, but other operations will be slower. If the ranks are pickled into a field on Student then, loading a specific student in memory means loading all the rank information into memory together with that student. This can be mitigated by using .values() or .values_list() but then you are no longer getting Student instances from the database. Why have Student instances in the first place and not just access the raw database?
If I change the fields in Rank, Django's migration facilities easily allow performing the needed changes when I deploy the new version of my application. If the rank information is pickled into a field, I have to manage any structure change by writing custom code.
The database software cannot access values in a pickle and so you have to write custom code to access them. With the model above, if you want to list students by rank today (and the ranks for today have already been computed), then you can do:
for r in Rank.objects.filter(date=utcnow()).order_by("value")\
.prefetch_related():
print r.student.name
If you use pickles, you have to scan all Students and unpickle the ranks to extract the one for the day you want, and then use a Python data structure to order the students by rank. Once this is done, you then have to iterate over this structure to get the names in order.
There is a much better way. Use redis Sorted sets
With sorted sets you can add, remove, or update elements in a very fast way (in a time proportional to the logarithm of the number of elements). Since elements are taken in order and not ordered afterwards, you can also get ranges by score or by rank (position) in a very fast way. Accessing the middle of a sorted set is also very fast, so you can use Sorted Sets as a smart list of non repeating elements where you can quickly access everything you need: elements in order, fast existence test, fast access to elements in the middle!
In short with sorted sets you can do a lot of tasks with great performance that are really hard to model in other kind of databases
How to make use of this feature? Make a sorted set for each of the 180 days that you are interested in. You can either use the string representation of the date or just count them up like day1, day2 etc. Then when you calculate the rank for each user. Add it into the redis set (code borrowed from e4c5's answer).
def save(self, *args, **kwargs):
r = redis.Redis()
super(Exam,self).save(*args,**kwargs)
student = Student.objects.get(id = self.student_id)
rank = some_calculation
student.save()
r.zadd('dayx', self.name, self.rank)
Then you can retrieve rankings for any given day by r.zrange('dayx'). And really other than the imports and whatever logic to calculate the rankings. This is all there is to it.
Updating my answer after you updated the question with an example.
You should not do this with one table, you need two. ONe should be the student model which would look like this.
class Student(models.Model):
name = models.CharField(max_length=20)
grade = models.ForeignKey(Grade)
The other would be the rank model which might look like this.
class Rank(models.Model):
data = models.DateField()
student = models.ForeignKey(Student)
The following information is for the original question but parts of it will still be relevant I think.
1) Override save method in the Student model.
def save(self, *args, **kwargs):
super(Exam,self).save(*args,**kwargs)
student = Student.objects.get(id = self.student_id)
student.rank = some_calculation
student.save()
2) Use the post_save signal on the Exam object.
Similar to above
3) Use a trigger.
Since you are using postgresql, you can use the much more elegant solution of creating an AFTER INSERT trigger
If you really want to have just an object field instead of joining two objects you can use something like
https://github.com/shrubberysoft/django-picklefield
serializing into a single text field a dictionary where key/val is date/rank
( this library is quite old, I don't think it will work out of the box with a modern django project )
Anyway, unless you are really forced by some external constraint to use this ugly solution the better (right) way is to just join two meaningful objects like #e4c5 suggested.
I once used such a field in a project where auth_user was a db view shared between multiple projects. That way I asked for just one migrate on the central db to add the "pickled" field and I was then able to add every user option i wished without tampering the original model further.
Anyway:
You can't query for stuff like "average rank on 12 september" with Django ORM.
It's not faster, you are not taking into account the time to depickle the text string into an object, quite sure is slower, probably is way slower.
In my case I had just a couple of options per user without need to query on them, more than that I think you're asking for future troubles mantaining your code.
I'm trying to write a program that will store the results from a quiz against a player's name. I need to always have a record of the past 3 attempts against the users name. I need to be able to store these results for an indeterminate number of players across 3 class groups (hence the 3 arrays). So far i've got this but am getting pretty stuck now.
I've got 3 arrays with 3 fields. The first field is intended for the name and the following 3 to store the score attempts.
cla_1results = ([],[],[],[])
cla_2results = ([],[],[],[])
cla_3results = ([],[],[],[])
file = open("results.txt"")
if statement determines which array to store the results data in depending on the class code
if class_code == "CL1":
cla_1results[0].append(full_name)
cla_1results[1].append(total)
file.write([cla_1results])
elif class_code == "CL2":
cla_2results[0].append(full_name)
cla_2results[1].append(total)
file.write([cla_2results])
elif class_code == "CL3":
cla_3results[0].append(full_name)
cla_3results[1].append(total)
file.write([cla_3results])
As far as the structure of storing the scores goes, try using something that looks more like this
cla_1results={}
cla_2results={}
cla_3results={}
my_file=open("results.txt"."r+")
if class_code=="CL1":
if full_name in cla_1results:
cla_1results.append(total)
else:
cla_1results[full_name]=[total]
my_file.write(cla_1results[full_name])
elif class_code=="CL2":
if full_name in cla_2results:
cla_2results.append(total)
else:
cla_2results[full_name]=[total]
my_file.write(cla_2results[full_name])
elif class_code=="CL3":
if full_name in cla_3results:
cla_3results.append(total)
else:
cla_3results[full_name]=[total]
my_file.write(cla_3results[full_name])
Also, if you wanted to remove the oldest core if the person had more than 3, you could add in a bit that has cla_1results[full_name][0].remove. Hopefully this helps.
If I should be approaching this problem through a different method, please suggest so. I am creating an item based collaborative filter. I populate the db with the LinkRating2 class and for each link there are more than a 1000 users that I need to call and collect their ratings to perform calculations which I then use to create another table. So I need to call more than 1000 entities for a given link.
For instance lets say there are over a 1000 users rated 'link1' there will be over a 1000 instances of this class for the given link property that I need to call.
How would I complete this example?
class LinkRating2(db.Model):
user = db.StringProperty()
link = db.StringProperty()
rating2 = db.FloatProperty()
query =LinkRating2.all()
link1 = 'link string name'
a = query.filter('link = ', link1)
aa = a.fetch(1000)##how would i get more than 1000 for a given link1 as shown?
##keybased over 1000 in other post example i need method for a subset though not key
class MyModel(db.Expando):
#classmethod
def count_all(cls):
"""
Count *all* of the rows (without maxing out at 1000)
"""
count = 0
query = cls.all().order('__key__')
while count % 1000 == 0:
current_count = query.count()
if current_count == 0:
break
count += current_count
if current_count == 1000:
last_key = query.fetch(1, 999)[0].key()
query = query.filter('__key__ > ', last_key)
return count
The 1000-entity fetch limit was removed recently; you can fetch as many as you need, provided you can do so within the time limits. Your entities look like they'll be fairly small, so you may be able to fetch significantly more than 1000 in a request.
Wooble points out that the 1,000 entity limit is a thing of the past now, so you actually don't need to use cursors to do this - just fetch everything at once (it'll be faster than getting them in 1,000 entity batches too since there will be fewer round-trips to the datastore, etc.)
The removal of the 1000 entity limit was removed in version 1.3.1: http://googleappengine.blogspot.com/2010/02/app-engine-sdk-131-including-major.html
Old solution using cursors:
Use query cursors to fetch results beyond the first 1,000 entities:
# continuing from your code ... get ALL of the query's results:
more = aa
while len(more) == 1000:
a.with_cusor(a.cursor()) # start the query where we left off
more = a.fetch(1000) # get the next 1000 results
aa = aa + more # copy the additional results into aa