Develop Reference

DRF: how to create custom FilterSet to filter nearest users by distance

I'm trying to create custom FilterSet for filtering nearby users by distance using django-filter
For example if I send
GET /api/list/?distance=300, I want to get all nearby users who are lower or equals to 300m far away
My model has 2 fields:
latitude = models.DecimalField( # [-90.000000, 90.000000]
max_digits=8,
decimal_places=6,
null=True
)
longitude = models.DecimalField( # [-180.000000, 180.000000]
max_digits=9,
decimal_places=6,
null=True
)
objects = ClientManager()
My ClientManager has function for getting coords from model:
def get_geo_coordinates(self, pk):
"""
:param pk: - client id
:return: client's coords
"""
instance = self.get(pk=pk)
data = (instance.latitude, instance.longitude)
return data
My GetListAPIView
class GetClientListAPIView(ListAPIView):
"""
Returns list with filtering capability
Available filter fields:
gender, first_name, last_name, distance
"""
serializer_class = ClientSerializer
queryset = Client.objects.all()
permission_classes = [IsAuthenticated]
filter_backends = [DjangoFilterBackend]
filter_class = ClientFilter
My ClientFilter
class ClientFilter(FilterSet):
distance = filters.NumberFilter(method='get_nearest_clients')
def get_nearest_clients(self, queryset, name, value):
sender_coords = Client.objects.get_geo_coordinates(pk=self.request.user.id)
test_coords = Client.objects.get_geo_coordinates(pk=31)
dist = get_great_circle_distance(sender_coords, test_coords)
class Meta:
model = Client
fields = ['gender', 'first_name', 'last_name']
Here I'm using my function for calculating distance between two clients:
def get_great_circle_distance(first_coords, second_coords):
"""
:param first_coords: (first_client_latitude, first_client_longitude) in degrees
:param second_coords: (second_client_latitude, second_client_longitude) in degrees
:return: distance
"""
earth_radius = 6_400_000 # in metres
la_1, lo_1 = map(radians, first_coords)
la_2, lo_2 = map(radians, second_coords)
coefficient = acos(
cos(la_1) * cos(la_2) * cos(lo_1 - lo_2) +
sin(la_1) * sin(la_2)
)
distance = earth_radius * coefficient
return distance
I do not know how to filter the queryset and do it optimally from the database accesses side.

I would recommend using the existing tools that solved this problem a long time ago. Doing accurate (and efficient) distance calculations on an irregularly shaped sphere is more complicated than this.
https://docs.djangoproject.com/en/4.0/ref/contrib/gis/install/postgis/
GIS field on model:
from django.contrib.gis.db.models import PointField
class Client(models.Model):
location = PointField()
This gives you the proper tools to do distance calculations directly on the queryset and the computation is done on the database side afaik.(https://docs.djangoproject.com/en/4.0/ref/contrib/gis/tutorial/#spatial-queries)
It is a bit more overhead to setup GIS properly but it's worth the effort.
Sidenote: It is possible to do it by hand via queryset annotate() and Q and F expressions but as I said it's tricky to get right. Filtering django-filter on the client-side, as you attempted there, pretty much defeats the purpose of using django-filter in the first place. Hope that helps getting a better understanding.

I fixed this issue
class ClientFilter(FilterSet):
"""
Custom ClientFilter
"""
distance = filters.NumberFilter(method='get_nearest_clients')
def get_nearest_clients(self, queryset: QuerySet, name: str, dist_value: int):
sender_id = self.request.user.id
sender_la, sender_lo = map(radians, Client.objects.get_geo_coordinates(pk=sender_id))
earth_radius = 6_400_000
queryset = (
queryset.exclude(pk=sender_id)
.alias(
rad_lat=Radians('latitude'),
rad_long=Radians('longitude'),
distance=ExpressionWrapper(
ACos(
Cos('rad_lat') * Cos(sender_la) * Cos(F('rad_long') - sender_lo)
+ Sin('rad_lat') * Sin(sender_la)
) * earth_radius,
output_field=FloatField()
)
)
.exclude(distance__gte=dist_value)
.order_by('distance')
)
return queryset

Django database queries

I want to change this code to a faster query
response = []
for player in Player.objects.all():
total_earn = Credit.objects.filter(player=player).aggregate(Sum('amount')).get('amount__sum', 0)
total_earn += Purchase.objects.filter(player=player).aggregate(Sum('amount')).get('amount__sum', 0)
reponse.append([player.id, player.email, player.phone, total_earn])
I try this for a moment, but now it take a lot of time to execute and it causes timeout on the server.
I want something very fast, like that:
response = Player.objects.annotate(
id='id',
email='email',
phone='phone',
total_earn=(Credit.... + Purchase....)
)
My models:
class Player(AbstractUser):
email = models.EmailField(..)
phone = models.CharField(..)
class Credit(models.Model):
player = models.ForeignKey(Player, ..., CASCADE)
amount = modesl.DecimalField(decimal_places=2, ...)
class Purchase(models.Model):
player = models.ForeignKey(Player, ...)
amount = models.DecimalField(decimal_places=2, ...)

You can make use of subqueries:
from django.db.models import OuterRef, Subquery
credits = Credit.objects.filter(
player=OuterRef('pk')
).values('player').annotate(
total=Sum('amount')
).order_by('player').values('total')
purchases = Purchases.objects.filter(
player=OuterRef('pk')
).values('player').annotate(
total=Sum('amount')
).order_by('player').values('total')
Player.objects.annotate(
total_earn=Subquery(credits)[:1] - Subquery(purchases)[:1]
)
However it looks like there is some bad modeling. It might be better to make a single model for Credits and Purchases and thus use a negative amount for Purchases. If such model is named Earning for example, then one can do that with a simple Player.objects.annotate(total_earn=Sum('earning__amount')).

Django perform Annotate , count and Compare

I have following Model.
class Gallery(BaseModel):
company = models.ForeignKey(to=Company, on_delete=models.CASCADE)
image = models.ImageField(
upload_to=upload_company_image_to,
validators=[validate_image]
)
def __str__(self):
return f'{self.company.name}'
I want to allow maximum upto 5 image to be uploaded by one company so I tried my query as
def clean(self):
print(Gallery.objects.values('company').annotate(Count('image')).count())
I don't know how to compare above query with Integer 5. How do I do that?

You can retrieve the Companys that have more than five images with:
from django.db.models import Count
Company.objects.alias(
num_images=Count('gallery')
).filter(num_images__gt=5)
Prior to django-3.2, you can work with .annotate(…) [Django-doc]:
from django.db.models import Count
Company.objects.annotate(
num_images=Count('gallery')
).filter(num_images__gt=5)

Django - Using signals to make changes to other models

say I have two models like so...
class Product(models.Model):
...
overall_rating = models.IntegerField()
...
class Review(models.Model):
...
product = models.ForeignKey(Product, related_name='review', on_delete=models.CASCADE)
rating = models.IntegerField()
...
I want to use the ratings from all of the child Review objects to build an average overall_rating for the parent Product.
Question: I'm wondering how I may be able to achieve something like this using Django signals?
I am a bit of a newbie to this part of Django; have never really understood the signals part before.
This overall_rating value needs to be stored in the database instead of calculated using a method since I plan on ordering the Product objects based on their overall_rating which is done on a DB level. The method may look something like this if I were to implement it (just for reference):
def overall_rating(self):
review_count=self.review.count()
if review_count >= 1:
ratings=self.review.all().values_list('rating',flat=True)
rating_sum = 0
for i in ratings:
rating_sum += int(i)
return rating_sum / review_count
else:
return 0
Thank you

You want to update your Product after each save of Review. So the best and fastest way would be using post save method. For example, after each saved product you can get all reviews and calculate overall rating and then save it to the Product.
#receiver(post_save, sender=Review, dispatch_uid="update_overall_rating")
def update_rating(sender, instance, **kwargs):
parent = instance.product
all_reviews = Review.objects.filter(product=parent)
parent.overall_rating = get_overall_rating(all_reviews)

Django, Model with "Def Self" value, SUM aggregate not working

I have the following model:
class PurchaseOrderLine(models.Model):
productcode = models.ForeignKey(OurProduct, on_delete=models.PROTECT)
price = models.DecimalField (max_digits=6, decimal_places=2)
qty = models.IntegerField()
def linetotal(self):
from decimal import *
total = (self.price * self.qty)
return total
In my VIEWS.PY I am trying to total the linetotal's:
tot=PurchaseOrderLine.objects.aggregate(total=Sum('linetotal'))['total']
return HttpResponse(tot)
But it returns FIELDERROR "Cannot resolve keyword 'linetotal' into field"???
In the query I can replace Sum('linetotal') for Sum('price') and it work fine, but not with the def linetotal(self).

The linetotal property doesn't exist at the database level, so how would the ORM handle it? You need to implement the query using extra:
for purchase_order_line in PurchaseOrderLine.objects.extra(select={'total': 'price * qty'}):
print purchase_order.total