I am currently struggling with a topic connected to transactions. I implemented a discount functionality. Whenever a sale is made with a discount code, the counter redeemed_quantity is increased by + 1.
Now I thought about the case. What if one or more users redeem a discount at the same time? Assuming redeemed_quantity is 10. User 1 buys the product and redeemed_quantity increases by +1 = 11. Now User 2 clicked on 'Pay' at the same time and again redeemed_quantity increases by +1 = 11. Even so, it should be 12. I learned about #transaction.atomic but I think the way I implemented them here will not help me with what I am actually trying to prevent. Can anyone help me with that?
view.py
class IndexView(TemplateView):
template_name = 'website/index.html'
initial_price_of_course = 100000 # TODO: Move to settings
def check_discount_and_get_price(self):
discount_code_get = self.request.GET.get('discount')
discount_code = Discount.objects.filter(code=discount_code_get).first()
if discount_code:
discount_available = discount_code.available()
if not discount_available:
messages.add_message(
self.request,
messages.WARNING,
'Discount not available anymore.'
)
if discount_code and discount_available:
return discount_code, self.initial_price_of_course - discount_code.value
else:
return discount_code, self.initial_price_of_course
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['stripe_pub_key'] = settings.STRIPE_PUB_KEY
discount_object, course_price = self.check_discount_and_get_price()
context['course_price'] = course_price
return context
#transaction.atomic
def post(self, request, *args, **kwargs):
stripe.api_key = settings.STRIPE_SECRET_KEY
token = request.POST.get('stripeToken')
email = request.POST.get('stripeEmail')
discount_object, course_price = self.check_discount_and_get_price()
charge = stripe.Charge.create(
amount=course_price,
currency='EUR',
description='My Description',
source=token,
receipt_email=email,
)
if charge.paid:
if discount_object:
discount_object.redeemed_quantity += 1
discount_object.save()
order = Order(
total_gross=course_price,
discount=discount_object
)
order.save()
return redirect('website:index')
models.py
class Discount(TimeStampedModel):
code = models.CharField(max_length=20)
value = models.IntegerField() # Smallest currency unit, as amount charged
max_quantity = models.IntegerField()
redeemed_quantity = models.IntegerField(default=0)
def available(self):
available_quantity = self.max_quantity - self.redeemed_quantity
if available_quantity > 0:
return True
class Order(TimeStampedModel):
total_gross = models.IntegerField()
discount = models.ForeignKey(
Discount,
on_delete=models.PROTECT, # Can't delete discount if used.
related_name='orders',
null=True,
You can pass the handling of the incrementation to the database in order to avoid the race condition in your code by using django's F expression:
from django.db.models import F
# ...
discount_object.redeemed_quantity = F('redeemed_quantity') + 1
discount_object.save()
From the docs with a completely analogous example:
Although reporter.stories_filed = F('stories_filed') + 1 looks like a normal Python assignment of value to an instance attribute, in fact it’s an SQL construct describing an operation on the database.
When Django encounters an instance of F(), it overrides the standard Python operators to create an encapsulated SQL expression; in this case, one which instructs the database to increment the database field represented by reporter.stories_filed.
Django is a piece of a synchronous code. It means that every request you make to the server is processed individually. This problem could arise, when there are multiple server-workers (for example uwsgi workers), but again - it's practically impossible to do this. We run a webshop application with multiple workers and something like this never happend.
But back to the question - if you want to query the database to increase a value by one, see schwobaseggl's answer.
The last thing is that I think you misunderstand what transaction.atomic() does. Simply put it rolls back any queries made to the database in a function if function exits with an error to the state when function was called. See this answer and this piece of documentation. Maybe it will clear some things up.
Related
I want to change this code to a faster query
response = []
for player in Player.objects.all():
total_earn = Credit.objects.filter(player=player).aggregate(Sum('amount')).get('amount__sum', 0)
total_earn += Purchase.objects.filter(player=player).aggregate(Sum('amount')).get('amount__sum', 0)
reponse.append([player.id, player.email, player.phone, total_earn])
I try this for a moment, but now it take a lot of time to execute and it causes timeout on the server.
I want something very fast, like that:
response = Player.objects.annotate(
id='id',
email='email',
phone='phone',
total_earn=(Credit.... + Purchase....)
)
My models:
class Player(AbstractUser):
email = models.EmailField(..)
phone = models.CharField(..)
class Credit(models.Model):
player = models.ForeignKey(Player, ..., CASCADE)
amount = modesl.DecimalField(decimal_places=2, ...)
class Purchase(models.Model):
player = models.ForeignKey(Player, ...)
amount = models.DecimalField(decimal_places=2, ...)
You can make use of subqueries:
from django.db.models import OuterRef, Subquery
credits = Credit.objects.filter(
player=OuterRef('pk')
).values('player').annotate(
total=Sum('amount')
).order_by('player').values('total')
purchases = Purchases.objects.filter(
player=OuterRef('pk')
).values('player').annotate(
total=Sum('amount')
).order_by('player').values('total')
Player.objects.annotate(
total_earn=Subquery(credits)[:1] - Subquery(purchases)[:1]
)
However it looks like there is some bad modeling. It might be better to make a single model for Credits and Purchases and thus use a negative amount for Purchases. If such model is named Earning for example, then one can do that with a simple Player.objects.annotate(total_earn=Sum('earning__amount')).
I have a signal that looks like this:
#receiver([post_save, post_delete], sender=Following)
def increment_follow_count(instance, created=False, **kwargs):
if created:
instance.follower.following_count += 1
instance.target.follower_count += 1
else:
instance.follower.following_count -= 1
instance.target.follower_count -= 1
When a user follows another user, it works correctly. However, when that same user unfollows that user, only the person that the user followed (target) has their follower count decremented, but the user's following count is not decremented. Why is this behavior happening and how can I fix it?
Model:
class Following(models.Model):
target = models.ForeignKey('User', related_name='followers', on_delete=models.CASCADE, null=True)
follower = models.ForeignKey('User', related_name='targets', on_delete=models.CASCADE, null=True)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return '{} is followed by {}'.format(self.target, self.follower)
Code to follow/unfollow user
def follow_unfollow(follower, target):
# Query to see if the following exists or not
following = target.followers.filter(follower=follower)
if following.exists():
following.delete()
else:
target.followers.create(follower=follower)
target.save()
follower.save()
increment_follow_count signal contains increment logic but no save logic and saving is done is another follow_unfollow method?
First, increment is better to be atomic, to make sure no changes are lost.
Atomic increment can be achieved using F() expressions.
from django.db.models import F
#receiver([post_save, post_delete], sender=Following)
def increment_follow_count(instance, created=False, **kwargs):
if created:
User.objects.filter(
pk=instance.follower_id
).update(
following_count=F('following_count') + 1
)
User.objects.filter(
pk=instance.target_id
).update(
following_count=F('following_count') + 1
)
else:
User.objects.filter(
pk=instance.follower_id
).update(
following_count=F('following_count') - 1
)
User.objects.filter(
pk=instance.target_id
).update(
following_count=F('following_count') - 1
)
Here increment is not only atomic, but changes are immediately saved
in the database in the same method.
Also, I would suggest to remove target.save() and follower.save() in follow_unfollow - as it overwrites instance in database with the values in memory, and this should not be the case, at least for following_count as its increment logic is in signal. If in follow_unfollow method some changes to fields, other than followng_count are done - then save() should be called with update_fields list to update only changed fields.
Regular += 1 takes current in-memory instance field value, i.e. count=5, increment (count=6), and later, when save is called, it is being saved as update count=6. And during this time value might have already changed in database many times (and update will set it to 6 regardless), especially under load / simultaneous actions.
With atomic increment logic is moved from python to database - and will increment actual value at the time transaction is made.
I have a Django model called Attendance that has the clock in and clock in times of an employee along with the status of that entry, to see whether it's authorized or not. I then, am making another model called Payroll. I want this to check inside the Attendance entries to see all the Authorized entries and then do some action on them. How do I check all the status fields for all the entries in Attendance?
EDIT: Updated to better elaborate my question.
To better elaborate my question, this is how I've setup my Attendance model:
class CWorkAttendance(models.Model):
AUTO_ATT = "AU"
MANUAL_ATT = "MA"
WORK_ENTRY_TYPES = (
(AUTO_ATT, "Auto-Attendance"),
(MANUAL_ATT, "Manual-Attendance"),
)
AUTHORIZED = "AU"
UNAUTHORIZED = "UA"
WORK_ENTRY_STATUSES = (
(AUTHORIZED, "Athorized"),
(UNAUTHORIZED, "Un-Authorized"),
)
#Thank you motatoes
def face_locations_in(self, instance):
now = datetime.datetime.now()
return "attendance/{}/{}/in".format(instance.work_employee, now.strftime("%Y/%m/%d"))
def face_locations_out(self, instance):
now = datetime.datetime.now()
return "attendance/{}/{}/out".format(instance.work_employee, now.strftime("%Y/%m/%d"))
work_employee = models.ForeignKey('CEmployees', on_delete=models.CASCADE,)
work_start_time = models.DateTimeField()
work_end_time = models.DateTimeField(null=True)
work_duration = models.IntegerField(null=True)
work_entry_type = models.CharField(max_length=2,choices=WORK_ENTRY_TYPES)
work_entry_status = models.CharField(max_length=2, choices=WORK_ENTRY_STATUSES, default=WORK_ENTRY_STATUSES[1][0])
employee_face_captured_in = models.ImageField(upload_to=face_locations_in,)#////////
employee_face_captured_out = models.ImageField(upload_to=face_locations_out,)
If you look closely at the work_entry_status, it's a choice CharField that will contain the status of the entry (UNAUTHORIZED by default).
I want to create a Payroll model that will check for all the rows in the CWorkAttendance model and check their work_entry_status fields to see if they are Authorized, which is what I want to learn how to do.
If those fields are authorized, I want the grab the row's work_employee, work_duration and also some details from the original CEmployees row for the employee.
This is what I want my Payslip/Payroll model to look like:
class Payslip(models.Model):
GENERATED = "GEN"
CONFIRMED = "CON"
PAYSLIP_STATUS = (
(GENERATED, "Generated-UNSAVED"),
(CONFIRMED, "Confirmed-SAVED"),
)
payslip_number = models.IntegerField()#MM/YY/AUTO_GENERATED_NUMBER(AUTO_INCREMENT)
payslip_employee = models.ForeignKey('CEmployees', on_delete=models.CASCADE,)#Choose the employee from the master table CEmployees
payslip_generation_date = models.DateTimeField(default=datetime.datetime.now())#Date of the payroll generation
payslip_total_hours = models.IntegerField()#Total hours that the employee worked
payslip_from_date = models.DateField()"""The date from when the payslip will be made. The payslip will be manual for now, so generate it after choosing a a date to generate from."""
payslip_total_basic_seconds = models.IntegerField()#Total seconds the employee worked
payslip_total_ot_seconds = models.IntegerField()#Total overtime seconds the employee worked
payslip_basic_hourly_rate = models.IntegerField()#The basic hourly rate of the employee mentioned here. Take from the master employees table.
payslip_basic_ot_rate = models.IntegerField()#Taking the basic overtime rate from the master table
payslip_total_amount = models.FloatField()#The total amount of the payslip
payslip_entry_status = models.CharField(max_length=3, default=GENERATED)#The status of the pay slip.
Thanks,
Not sure if I understand your requirements well, so let me know if I misunderstood.
# `employee` is the work_employee in question
# if you don't want to filter by employee, remove `work_employee=employee`
attendances = CWorkAttendance.objects.filter(work_entry_status=CWorkAttendance.AUTHORIZED, work_employee=employee)
for attendances in attendances:
# do things with this attendance record
attendance.work_duration
attendance.employee
# ....
Update
Since you would like to do it manually, I would suggest having a separate view to generate the Payslip. The important thing is to know the date_from and the date_to for this payslip. I imagine that it is the managers who would have access to this view, so you would need the proper access controls set for it. I also think you need to have a payslip_to_date even if you are going to generate it until the current date, which will be useful for record keeping. I assume you have that column in the code below.
views.py:
from django.views import View
class GeneratePayslip(View):
"""
make sure you have the right access controls set to this view
"""
def post(self, request, **kwargs):
employee_id = kwags.POST.get("employee_id")
date_from = kwargs.POST.get("from_date")
date_to = kwargs.POST.get("to_date")
# we fetch all the objects within range
attendances = CWorkAttendance.objects.filter( \
work_entry_status=CWorkAttendance.AUTHORIZED, \
work_employee_id=employee_id, \
work_start_time__gte=date_from, \
work_end_time__lte=date_to \
)
hours = 0
for attendance in attendances:
# perform calculations to compute total sum and rate
pass
# create the payslip object here ..
# redirect to a success page and return
If you wanted to do it automatically later on, you may want to generate payslips automatically, once a month. For that you could use something like Celery to have periodic tasks that run in the background, for each employee. If this is the case you could move the above code to a file such as utils.py. you can create a method which takes employee_id, from_date, to_date, and then generate the payslip object, returning the payslip_id to the calling method
I'm using Django 1.8 and I have a model
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
Now, in my view I want to add a new ModelA object, but only if there are fewer than x entries for that value already.
def my_view(request):
# Using the value of 4 here just as an example
c = ModelA.objects.filter(some_field=4).count()
# Check if fewer than (x=20) objects with this field already
if c < 20:
# Fewer, so create one
new_model = ModelA(4)
new_model.save()
else:
# Return a message saying "too many"
From my understanding, there could be more than one thread running this method and so thread 1 may perform the count and there are fewer than 20 and then the other thread saves a new object, then thread 1 would save its object and there be 20 or more.
Is there some way to have the view be
def my_view(request):
get_a_lock_on_model(ModelA)
c = ModelA.objects....
# Rest of the code the same
release_lock_on_model(ModelA)
Or is there some other way I should be thinking about doing this? There are only ever inserts, never updates or deletes.
Thanks!
In order to do this you need to lock the entire table and how to do that depends on the RDBMS that you are using. It will involve the use of raw sql. An alternative approach is to do the count after you have saved your record
def my_view(request):
new_model = ModelA(4)
new_model.save()
try :
c = ModelA.objects.filter(some_field=4)[20]
if c.pk == new_model.pk:
c.delete()
# Return a message saying "too many"
except IndexError:
pass
This approach does not get in each others way, each thread is responsible for deleting the extra item that it added. Instead of deleting you can use atomic and rollback if the count is greater than 20
Tested on Django 1.10.x and postgres:
models.py:
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
active = models.BooleanField()
And:
from django.db.models.expressions import RawSQL
n = 42
maximum = 3
raw_sql = RawSQL('select (select count(*) from fooapp_modela where some_field=%s) < %s', (n, maximum))
while True:
o = ModelA.objects.create(some_field=n, active=raw_sql)
o.refresh_from_db()
print(o.id, o.active)
if not o.active:
# o.delete()
break
Caveat: By default, while one transaction is active, other transactions on other connections could not "see" inserted rows until the transactions are committed. Try to avoid creating rows in a complex transactions. I believe that this means that this method is not completely bullet proof :-( More info: https://www.postgresql.org/docs/9.6/static/transaction-iso.html .
A more robust solution might include a db constraint (probably unique_together):
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
ordinal = models.IntegerField()
class Meta:
unique_together = (
('some_field', 'ordinal'),
)
#...
raw_sql = RawSQL('select count(*) + 1 from fooapp_modela where some_field=%s', (n,))
o = ModelA.objects.create(some_field=n, ordinal=raw_sql) # retry a few times on IntegrityError
o.refresh_from_db()
print(o.id, o.ordinal)
[Update: Changed question title to be more specific]
Sorry if I didn't make the question very well, I can't figure how to do this:
class WhatEver():
number = model.IntegerField('Just a Field', default=callablefunction)
...
Where callablefunction does this query:
from myproject.app.models import WhatEver
def callablefunction():
no = WhatEver.objects.count()
return no + 1
I want to automatically write the next number, and I don't know how to do it.
I have errors from callablefunction stating that it cannot import the model, and I think there must be an easier way to do this. There's no need even to use this, but I can't figure how to do it with the pk number.
I've googled about this and the only thing I found was to use the save() method for auto incrementing the number... but I wanted to show it in the <textfield> before saving...
What would you do?
Got it! I hope this will help everyone that has any problems making a auto-filled and auto-incrementing field in django. The solution is:
class Cliente(models.Model):
"""This is the client data model, it holds all client information. This
docstring has to be improved."""
def number():
no = Cliente.objects.count()
if no == None:
return 1
else:
return no + 1
clientcode = models.IntegerField(_('Code'), max_length=6, unique=True, \
default=number)
[... here goes the rest of your model ...]
Take in care:
The number function doesn't take any arguments (not even self)
It's written BEFORE everything in the model
This was tested on django 1.2.1
This function will automatically fill the clientcode field with the next number (i.e. If you have 132 clients, when you add the next one the field will be filled with clientcode number 133)
I know that this is absurd for most of the practical situations, since the PK number is also auto-incrementing, but there's no way to autofill or take a practical use for it inside the django admin.
[update: as I stated in my comment, there's a way to use the primary key for this, but it will not fill the field before saving]
Every Django model already has an auto-generated primary key:
id = models.AutoField(primary_key=True)
It seems you are trying to duplicate an already existing behavior, just use the object primary key.
I, too, came across this problem, my instance of it was customer.number which was relative to the customers Store. I was tempted to use something like:
# Don't do this:
class Customer(models.Model):
# store = ...
number = models.IntegerField(default=0)
def save(self, *args, **kwargs):
if self.number == 0:
try:
self.number = self.store.customer_set.count() + 1
else:
self.number = 1
super(Customer, self).save(*args, **kwargs)
The above can cause several problems: Say there were 10 Customers, and I deleted customer number 6. The next customer to be added would be (seemingly) the 10th customer, which would then become a second Customer #10. (This could cause big errors in get() querysets)
What I ended up with was something like:
class Store(models.Model):
customer_number = models.IntegerField(default=1)
class Customer(models.Model):
store = models.ForeignKey(Store)
number = models.IntegerField(default=0)
def save(self, *args, **kwargs):
if self.number == 0:
self.number = self.store.customer_number
self.store.number += 1
self.store.save()
super(Customer, self).save(*args, **kwargs)
PS:
You threw out several times that you wanted this field filled in "before". I imagine you wanted it filled in before saving so that you can access it. To that I would say: this method allows you to access store.customer_number to see the next number to come.
You have errors in code, that's why you can't import it:
from django.db import models
class WhatEver(models.Model):
number = models.IntegerField('Just a Field', default=0)
and Yuval A is right about auto-incrementing: you don't even need to declare such a field. Just use the pk or id, they mean the same unless there's a composite pk in the model:
> w = Whatever(number=10)
> w
<Whatever object>
> w.id
None
> w.save()
> w.id
1
[update] Well, I haven't tried a callable as a default. I think if you fix these errors, it must work.