I have a signal that looks like this:
#receiver([post_save, post_delete], sender=Following)
def increment_follow_count(instance, created=False, **kwargs):
if created:
instance.follower.following_count += 1
instance.target.follower_count += 1
else:
instance.follower.following_count -= 1
instance.target.follower_count -= 1
When a user follows another user, it works correctly. However, when that same user unfollows that user, only the person that the user followed (target) has their follower count decremented, but the user's following count is not decremented. Why is this behavior happening and how can I fix it?
Model:
class Following(models.Model):
target = models.ForeignKey('User', related_name='followers', on_delete=models.CASCADE, null=True)
follower = models.ForeignKey('User', related_name='targets', on_delete=models.CASCADE, null=True)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return '{} is followed by {}'.format(self.target, self.follower)
Code to follow/unfollow user
def follow_unfollow(follower, target):
# Query to see if the following exists or not
following = target.followers.filter(follower=follower)
if following.exists():
following.delete()
else:
target.followers.create(follower=follower)
target.save()
follower.save()
increment_follow_count signal contains increment logic but no save logic and saving is done is another follow_unfollow method?
First, increment is better to be atomic, to make sure no changes are lost.
Atomic increment can be achieved using F() expressions.
from django.db.models import F
#receiver([post_save, post_delete], sender=Following)
def increment_follow_count(instance, created=False, **kwargs):
if created:
User.objects.filter(
pk=instance.follower_id
).update(
following_count=F('following_count') + 1
)
User.objects.filter(
pk=instance.target_id
).update(
following_count=F('following_count') + 1
)
else:
User.objects.filter(
pk=instance.follower_id
).update(
following_count=F('following_count') - 1
)
User.objects.filter(
pk=instance.target_id
).update(
following_count=F('following_count') - 1
)
Here increment is not only atomic, but changes are immediately saved
in the database in the same method.
Also, I would suggest to remove target.save() and follower.save() in follow_unfollow - as it overwrites instance in database with the values in memory, and this should not be the case, at least for following_count as its increment logic is in signal. If in follow_unfollow method some changes to fields, other than followng_count are done - then save() should be called with update_fields list to update only changed fields.
Regular += 1 takes current in-memory instance field value, i.e. count=5, increment (count=6), and later, when save is called, it is being saved as update count=6. And during this time value might have already changed in database many times (and update will set it to 6 regardless), especially under load / simultaneous actions.
With atomic increment logic is moved from python to database - and will increment actual value at the time transaction is made.
Related
I have written a python script in my project. I want to update the value of a field.
Here are my modes
class News_Channel(models.Model):
name = models.TextField(blank=False)
info = models.TextField(blank=False)
image = models.FileField()
website = models.TextField()
total_star = models.PositiveIntegerField(default=0)
total_user = models.IntegerField()
class Meta:
ordering = ["-id"]
def __str__(self):
return self.name
class Count(models.Model):
userId = models.ForeignKey(User, on_delete=models.CASCADE)
channelId = models.ForeignKey(News_Channel, on_delete=models.CASCADE)
rate = models.PositiveIntegerField(default=0)
def __str__(self):
return self.channelId.name
class Meta:
ordering = ["-id"]
This is my python script:
from feed.models import Count, News_Channel
def run():
for i in range(1, 11):
news_channel = Count.objects.filter(channelId=i)
total_rate = 0
for rate in news_channel:
total_rate += rate.rate
print(total_rate)
object = News_Channel.objects.filter(id=i)
print(total_rate)
print("before",object[0].total_star,total_rate)
object[0].total_star = total_rate
print("after", object[0].total_star)
object.update()
After counting the total_rate from the Count table I want to update the total star value in News_Channel table. I am failing to do so and get the data before the update and after the update as zero. Although total_rate has value.
The problem
The reason why this fails is because here object is a QuerySet of News_Channels, yeah that QuerySet might contain exactly one News_Channel, but that is irrelevant.
If you then use object[0] you make a query to the database to fetch the first element and deserialize it into a News_Channel object. Then you set the total_star of that object, but you never save that object. You only call .update() on the entire queryset, resulting in another independent query.
You can fix this with:
objects = News_Channel.objects.filter(id=i)
object = objects[0]
object.total_star = total_rate
object.save()
Or given you do not need any validation, you can boost performance with:
News_Channel.objects.filter(id=i).update(total_star=total_rate)
Updating all News_Channels
If you want to update all News_Channels, you actually better use a Subquery here:
from django.db.models import OuterRef, Sum, Subquery
subq = Subquery(
Count.objects.filter(
channelId=OuterRef('id')
).annotate(
total_rate=Sum('rate')
).order_by('channelId').values('total_rate')[:1]
)
News_Channel.objects.update(total_star=subq)
The reason is that object in your case is a queryset, and after you attempt to update object[0], you don't store the results in the db, and don't refresh the queryset. To get it to work you should pass the field you want to update into the update method.
So, try this:
def run():
for i in range(1, 11):
news_channel = Count.objects.filter(channelId=i)
total_rate = 0
for rate in news_channel:
total_rate += rate.rate
print(total_rate)
object = News_Channel.objects.filter(id=i)
print(total_rate)
print("before",object[0].total_star,total_rate)
object.update(total_star=total_rate)
print("after", object[0].total_star)
News_Channel.total_star can be calculated by using aggregation
news_channel_obj.count_set.aggregate(total_star=Sum('rate'))['total_star']
You can then either use this in your script:
object.total_star = object.count_set.aggregate(total_star=Sum('rate'))['total_star']
Or if you do not need to cache this value because performance is not an issue, you can remove the total_star field and add it as a property on the News_Channel model
#property
def total_star(self):
return self.count_set.aggregate(total_star=Sum('rate'))['total_star']
I am currently struggling with a topic connected to transactions. I implemented a discount functionality. Whenever a sale is made with a discount code, the counter redeemed_quantity is increased by + 1.
Now I thought about the case. What if one or more users redeem a discount at the same time? Assuming redeemed_quantity is 10. User 1 buys the product and redeemed_quantity increases by +1 = 11. Now User 2 clicked on 'Pay' at the same time and again redeemed_quantity increases by +1 = 11. Even so, it should be 12. I learned about #transaction.atomic but I think the way I implemented them here will not help me with what I am actually trying to prevent. Can anyone help me with that?
view.py
class IndexView(TemplateView):
template_name = 'website/index.html'
initial_price_of_course = 100000 # TODO: Move to settings
def check_discount_and_get_price(self):
discount_code_get = self.request.GET.get('discount')
discount_code = Discount.objects.filter(code=discount_code_get).first()
if discount_code:
discount_available = discount_code.available()
if not discount_available:
messages.add_message(
self.request,
messages.WARNING,
'Discount not available anymore.'
)
if discount_code and discount_available:
return discount_code, self.initial_price_of_course - discount_code.value
else:
return discount_code, self.initial_price_of_course
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['stripe_pub_key'] = settings.STRIPE_PUB_KEY
discount_object, course_price = self.check_discount_and_get_price()
context['course_price'] = course_price
return context
#transaction.atomic
def post(self, request, *args, **kwargs):
stripe.api_key = settings.STRIPE_SECRET_KEY
token = request.POST.get('stripeToken')
email = request.POST.get('stripeEmail')
discount_object, course_price = self.check_discount_and_get_price()
charge = stripe.Charge.create(
amount=course_price,
currency='EUR',
description='My Description',
source=token,
receipt_email=email,
)
if charge.paid:
if discount_object:
discount_object.redeemed_quantity += 1
discount_object.save()
order = Order(
total_gross=course_price,
discount=discount_object
)
order.save()
return redirect('website:index')
models.py
class Discount(TimeStampedModel):
code = models.CharField(max_length=20)
value = models.IntegerField() # Smallest currency unit, as amount charged
max_quantity = models.IntegerField()
redeemed_quantity = models.IntegerField(default=0)
def available(self):
available_quantity = self.max_quantity - self.redeemed_quantity
if available_quantity > 0:
return True
class Order(TimeStampedModel):
total_gross = models.IntegerField()
discount = models.ForeignKey(
Discount,
on_delete=models.PROTECT, # Can't delete discount if used.
related_name='orders',
null=True,
You can pass the handling of the incrementation to the database in order to avoid the race condition in your code by using django's F expression:
from django.db.models import F
# ...
discount_object.redeemed_quantity = F('redeemed_quantity') + 1
discount_object.save()
From the docs with a completely analogous example:
Although reporter.stories_filed = F('stories_filed') + 1 looks like a normal Python assignment of value to an instance attribute, in fact it’s an SQL construct describing an operation on the database.
When Django encounters an instance of F(), it overrides the standard Python operators to create an encapsulated SQL expression; in this case, one which instructs the database to increment the database field represented by reporter.stories_filed.
Django is a piece of a synchronous code. It means that every request you make to the server is processed individually. This problem could arise, when there are multiple server-workers (for example uwsgi workers), but again - it's practically impossible to do this. We run a webshop application with multiple workers and something like this never happend.
But back to the question - if you want to query the database to increase a value by one, see schwobaseggl's answer.
The last thing is that I think you misunderstand what transaction.atomic() does. Simply put it rolls back any queries made to the database in a function if function exits with an error to the state when function was called. See this answer and this piece of documentation. Maybe it will clear some things up.
I'm using Django 1.8 and I have a model
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
Now, in my view I want to add a new ModelA object, but only if there are fewer than x entries for that value already.
def my_view(request):
# Using the value of 4 here just as an example
c = ModelA.objects.filter(some_field=4).count()
# Check if fewer than (x=20) objects with this field already
if c < 20:
# Fewer, so create one
new_model = ModelA(4)
new_model.save()
else:
# Return a message saying "too many"
From my understanding, there could be more than one thread running this method and so thread 1 may perform the count and there are fewer than 20 and then the other thread saves a new object, then thread 1 would save its object and there be 20 or more.
Is there some way to have the view be
def my_view(request):
get_a_lock_on_model(ModelA)
c = ModelA.objects....
# Rest of the code the same
release_lock_on_model(ModelA)
Or is there some other way I should be thinking about doing this? There are only ever inserts, never updates or deletes.
Thanks!
In order to do this you need to lock the entire table and how to do that depends on the RDBMS that you are using. It will involve the use of raw sql. An alternative approach is to do the count after you have saved your record
def my_view(request):
new_model = ModelA(4)
new_model.save()
try :
c = ModelA.objects.filter(some_field=4)[20]
if c.pk == new_model.pk:
c.delete()
# Return a message saying "too many"
except IndexError:
pass
This approach does not get in each others way, each thread is responsible for deleting the extra item that it added. Instead of deleting you can use atomic and rollback if the count is greater than 20
Tested on Django 1.10.x and postgres:
models.py:
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
active = models.BooleanField()
And:
from django.db.models.expressions import RawSQL
n = 42
maximum = 3
raw_sql = RawSQL('select (select count(*) from fooapp_modela where some_field=%s) < %s', (n, maximum))
while True:
o = ModelA.objects.create(some_field=n, active=raw_sql)
o.refresh_from_db()
print(o.id, o.active)
if not o.active:
# o.delete()
break
Caveat: By default, while one transaction is active, other transactions on other connections could not "see" inserted rows until the transactions are committed. Try to avoid creating rows in a complex transactions. I believe that this means that this method is not completely bullet proof :-( More info: https://www.postgresql.org/docs/9.6/static/transaction-iso.html .
A more robust solution might include a db constraint (probably unique_together):
class ModelA(models.Model):
some_field = models.PositiveIntegerField()
ordinal = models.IntegerField()
class Meta:
unique_together = (
('some_field', 'ordinal'),
)
#...
raw_sql = RawSQL('select count(*) + 1 from fooapp_modela where some_field=%s', (n,))
o = ModelA.objects.create(some_field=n, ordinal=raw_sql) # retry a few times on IntegrityError
o.refresh_from_db()
print(o.id, o.ordinal)
class UserProfile(models.Model):
user = models.OneToOneField(User)
how_many_new_notifications = models.IntegerField(null=True,blank=True,default=0)
User.profile = property(lambda u: UserProfile.objects.get_or_create(user=u)[0])
In views.py function which is 100% called and whom is present
whom.profile.how_many_new_notifications += 1
whom.save()
Whatever, how_many_new_notifications is still equal zero and not incremented , despite everything else is correct
Also tried something like this:
if whom.profile.how_many_new_notifications == None:
whom.profile.how_many_new_notifications = 1
else:
varible_number_of_notifications = int( whom.profile.how_many_new_notifications)
whom.profile.how_many_new_notifications = varible_number_of_notifications + 1
Get no errors in log, is there any reason why this code wouldn't work, or should I search for issues in other places?
User.profile is a property that gets a new copy of the profile each time it is used.
So when you do
user.profile.how_many_notifications += 1
user.profile.save()
Each line uses its own copy of the profile, the two Python objects are unrelated.
So you need to do
profile = user.profile
profile.how_many_notifications += 1
profile.save()
But using a profile property like that is a bit odd -- you have a OneToOneField, and a related property is already automatically defined as the lower case name of your class. So
user.userprofile.how_many_new_notifications += 1
user.userprofile.save()
Should also work. If you want to change the name userprofile, use related_name:
user = models.OneToOneField(User, related_name='profile')
And then it works with user.profile.
[Update: Changed question title to be more specific]
Sorry if I didn't make the question very well, I can't figure how to do this:
class WhatEver():
number = model.IntegerField('Just a Field', default=callablefunction)
...
Where callablefunction does this query:
from myproject.app.models import WhatEver
def callablefunction():
no = WhatEver.objects.count()
return no + 1
I want to automatically write the next number, and I don't know how to do it.
I have errors from callablefunction stating that it cannot import the model, and I think there must be an easier way to do this. There's no need even to use this, but I can't figure how to do it with the pk number.
I've googled about this and the only thing I found was to use the save() method for auto incrementing the number... but I wanted to show it in the <textfield> before saving...
What would you do?
Got it! I hope this will help everyone that has any problems making a auto-filled and auto-incrementing field in django. The solution is:
class Cliente(models.Model):
"""This is the client data model, it holds all client information. This
docstring has to be improved."""
def number():
no = Cliente.objects.count()
if no == None:
return 1
else:
return no + 1
clientcode = models.IntegerField(_('Code'), max_length=6, unique=True, \
default=number)
[... here goes the rest of your model ...]
Take in care:
The number function doesn't take any arguments (not even self)
It's written BEFORE everything in the model
This was tested on django 1.2.1
This function will automatically fill the clientcode field with the next number (i.e. If you have 132 clients, when you add the next one the field will be filled with clientcode number 133)
I know that this is absurd for most of the practical situations, since the PK number is also auto-incrementing, but there's no way to autofill or take a practical use for it inside the django admin.
[update: as I stated in my comment, there's a way to use the primary key for this, but it will not fill the field before saving]
Every Django model already has an auto-generated primary key:
id = models.AutoField(primary_key=True)
It seems you are trying to duplicate an already existing behavior, just use the object primary key.
I, too, came across this problem, my instance of it was customer.number which was relative to the customers Store. I was tempted to use something like:
# Don't do this:
class Customer(models.Model):
# store = ...
number = models.IntegerField(default=0)
def save(self, *args, **kwargs):
if self.number == 0:
try:
self.number = self.store.customer_set.count() + 1
else:
self.number = 1
super(Customer, self).save(*args, **kwargs)
The above can cause several problems: Say there were 10 Customers, and I deleted customer number 6. The next customer to be added would be (seemingly) the 10th customer, which would then become a second Customer #10. (This could cause big errors in get() querysets)
What I ended up with was something like:
class Store(models.Model):
customer_number = models.IntegerField(default=1)
class Customer(models.Model):
store = models.ForeignKey(Store)
number = models.IntegerField(default=0)
def save(self, *args, **kwargs):
if self.number == 0:
self.number = self.store.customer_number
self.store.number += 1
self.store.save()
super(Customer, self).save(*args, **kwargs)
PS:
You threw out several times that you wanted this field filled in "before". I imagine you wanted it filled in before saving so that you can access it. To that I would say: this method allows you to access store.customer_number to see the next number to come.
You have errors in code, that's why you can't import it:
from django.db import models
class WhatEver(models.Model):
number = models.IntegerField('Just a Field', default=0)
and Yuval A is right about auto-incrementing: you don't even need to declare such a field. Just use the pk or id, they mean the same unless there's a composite pk in the model:
> w = Whatever(number=10)
> w
<Whatever object>
> w.id
None
> w.save()
> w.id
1
[update] Well, I haven't tried a callable as a default. I think if you fix these errors, it must work.