Develop Reference

How to efficiently hit the database for a related field in REST serializer?

I have a complex architecture for my database and I'm having troubles writting the corresponding REST API.
Context: I am requesting various informations on a public API every hour, including data I want to keep history for.
Here's the models I am using
class Player(models.Model):
name = models.CharField(max_length=255)
class PlayerStatsHistory(models.Model):
player = models.ForeignKey('Player', null=True, on_delete=models.CASCADE)
last_refresh = models.DateTimeField(null=True)
...
This way, I can store every change in the stats of each user.
I wrote 2 serializers one for Player and one for PlayerStatsHistory. The easiest one works fine
class PlayerStatsSerializer(HyperlinkedModelSerializer):
class Meta:
model = PlayerStatsHistory
fields = ('last_refresh', ...)
But when I need to request the latest stats for one player I'm getting confused :
class PlayerSerializer(HyperlinkedModelSerializer):
details = SerializerMethodField()
def get_details(self, obj):
return PlayerStatsSerializer(PlayerStatsHistory.objects.filter(player=obj).order_by('-last_refresh').first()).data
class Meta:
model = Player
fields = ('name', 'details')
This works fine, but will hit database at each player provided to the PlayerSerializer and I feel like I'm doing it wrong.
How can I improve this solution ?

I think the best you can do is first get the id of the last PlayerStatsHistory of each Player(using group_by):
latest_stats_history_pks = PlayerStatsHistory.objects.values('player').annotate(max_id=models.Max('id')).values_list('max_id', flat=True)
(it has a problem, it uses all of the players if you are using pagination you don't need all of them, in that case, prefetch without filtering its queryset should be fine)
then only prefetch this values on your Player queryset, so:
queryset = Player.objects.all().prefetch_related(models.Prefetch(
'playerstatshistory_set',
queryset=PlayerStatsHistory.objects.filter(pk__in=latest_stats_history_pks), to_attr='last_stat_list'))
so finally your get_queryset method in your view should be like:
def get_queryset(self):
latest_stats_history_pks = PlayerStatsHistory.objects.values('player').annotate(max_id=models.Max('id')).values_list('max_id', flat=True)
queryset = Player.objects.all().prefetch_related(models.Prefetch(
'playerstatshistory_set',
queryset=PlayerStatsHistory.objects.filter(pk__in=latest_stats_history_pks), to_attr='last_stat_list'))
return queryset
and if you're using FBV, do something like this:
from rest_framework.decorators import api_view
from rest_framework.response import Response
#api_view(['GET'])
def player_list(request, format=None):
if request.method == 'GET':
latest_stats_history_pks = PlayerStatsHistory.objects.values('player').annotate(max_id=models.Max('id')).values_list('max_id', flat=True)
players = Player.objects.all().prefetch_related(models.Prefetch(
'playerstatshistory_set',
queryset=PlayerStatsHistory.objects.filter(pk__in=latest_stats_history_pks), to_attr='last_stat_list'))
serializer = PlayerSerializer(players, many=True)
return Response(serializer.data)
also in your serializer, change that field as below:
class PlayerSerializer(HyperlinkedModelSerializer):
details = SerializerMethodField()
def get_details(self, obj):
return {} if not obj.last_stat_list else PlayerStatsSerializer(obj.last_stat_list[-1]).data
class Meta:
model = Player
fields = ('name', 'details')

How can I create a partial search filter in Django REST framework?

I'm working with the Django REST framework library and I am trying to make a filter that can filter by first_name, last_name, or by both of them.
This is my ContactViewSet.py:
class ContactViewSet(viewsets.ModelViewSet):
queryset = Contact.objects.all()
serializer_class = ContactSerializer
filter_backends = (DjangoFilterBackend, )
filter_fields = ('first_name', 'last_name')
lookup_field = 'idContact'
My DRF's settings.py:
REST_FRAMEWORK = {
'DEFAULT_FILTER_BACKENDS': ('django_filters.rest_framework.DjangoFilterBackend',),
}
My current request URL looks like:
http://localhost:8000/api/v1/contacts/?first_name=Clair&last_name=Test
But I'm looking for something like this:
http://localhost:8000/api/v1/contacts/?first_name=Cl**&last_name=Tes**

I solved my problem by modifying my class ContactFilter like this:
import django_filters
from .models import Contact
class ContactFilter(django_filters.FilterSet):
class Meta:
model = Contact
fields = {
'first_name': ['startswith'],
'last_name': ['startswith'],
}
together = ['first_name', 'last_name']
And in my view I just had to do this:
class ContactViewSet(viewsets.ModelViewSet):
queryset = Contact.objects.all()
serializer_class = ContactSerializer
filter_class = ContactFilter
My request URL looks like this:
http://localhost:8000/api/v1/contact/?first_name__contains=Cl&last_name__contains=Tes
But I still wonder if I can have something like this in Django:
http://localhost:8000/api/v1/contacts/?first_name=Cl**&last_name=Tes**

I think the DjangoFilterBackend is mainly equality-based filtering. But you can customize the filtering method.
Also in DRF, for non exact filtering, there is the SearchFilter which makes case-insensitive partial matches searches by default.

What I do, is write custom FilterBackend. Something like this:
# views.py
from rest_framework import filters
class ObjektFilterBackend(filters.BaseFilterBackend):
allowed_fields = ['objekt', 'naziv', 'kategorija', 'zadnja_sprememba']
def filter_queryset(self, request, queryset, view):
flt = {}
for param in request.query_params:
for fld in self.allowed_fields:
if param.startswith(fld):
flt[param] = request.query_params[param]
return queryset.filter(**flt)
class ObjektiViewSet(mixins.ListModelMixin,
mixins.RetrieveModelMixin,
viewsets.GenericViewSet):
authentication_classes = (
authentication.TokenAuthentication,
authentication.SessionAuthentication)
permission_classes = (IsAuthenticated,)
queryset = models.Objekt.objects.all()
serializer_class = serializers.ObjektSerializer
filter_backends = (ObjektFilterBackend, ObjektOrderBackend,)
....
Besides basic filtering (fieldname=value pairs) I can use any Django queryset Field Lookups (__gt, __gte, __startswith,...) in my URLs like this:
http://localhost:8000/api/v2/objekti/?naziv__startswith=Apartma&zadnja_sprememba__gte=2018-01-01
And ObjektFilterBackend class could be easily adapted to support searching by pattern.
Just a little warning - this approach is potentially dangerous, because it allows end user to filter also by foreign key field. Something like this also works:
http://localhost:8000/api/v2/objekti/?kategorija__naziv__icontains=sobe
So restrict allowed_fields carefully and not include foreign keys that could lead to related User model.

For fuzzy search lookups I recommend using this approach:
filters.py
from django_filters import rest_framework as filters
from django.db.models import Q
from . import models
def filter_name(queryset, name, value):
"""
Split the filter value into separate search terms and construct a set of queries from this. The set of queries
includes an icontains lookup for the lookup fields for each of the search terms. The set of queries is then joined
with the OR operator.
"""
lookups = [name + '__icontains', ]
or_queries = []
search_terms = value.split()
for search_term in search_terms:
or_queries += [Q(**{lookup: search_term}) for lookup in lookups]
return queryset.filter(reduce(operator.or_, or_queries))
class ContactFilter(filters.FilterSet):
first_name = filters.CharFilter(method=filter_name, name='first_name')
last_name = filters.CharFilter(method=filter_name, name='last_name')
class Meta:
model = models.Contact
fields = [
'first_name',
'last_name',
]
api.py
class ContactViewSet(viewsets.ModelViewSet):
queryset = Contact.objects.all()
serializer_class = ContactSerializer
filter_class = ContactFilter
...

If your requests aren't too complicated you can also use:
class YourModelViewSet(viewsets.ModelViewSet):
queryset = YourModel.objects.all()
serializer_class = YourModelSerializer
filter_fields = {'some_field': ['startswith']}
Which will enable '?some_field__starswith=text' sintax support in request query params.
I suppose 'startswith' can be replaced with any django standart queryset filter param.

You should add custom Filter for your viewset.
from django_filters.rest_framework import DjangoFilterBackend
import django_filters
from recipes.models import Ingredient
class MyModelFilter(django_filters.FilterSet):
name = django_filters.CharFilter(
field_name='name', lookup_expr='icontains'
)
class Meta:
model = MyModel
fields = []
class MyViewSet(viewsets.ReadOnlyModelViewSet):
queryset = MyModel.objects.all()
serializer_class = MyModelSerializer
permission_classes = (AllowAny,)
pagination_class = None
filter_backends = (DjangoFilterBackend,) # add this
filterset_class = MyModelFilter # add this

How to filter objects by price range in Django?

I have a model Item with field price.
class Item(models.Model):
title = models.CharField(max_length=200, blank='true')
price = models.IntegerField(default=0)
My query may contain min_price & max_price values. So, my request may be like this: http://example.com/api/items?min_price=50&max_price=500. Can anybody tell me, how can I query items between min & max values? Can I solve it using Django ORM?
Thanks!

Check api reference for range. Like it states
You can use range anywhere you can use BETWEEN in SQL — for dates,
numbers and even characters.
So, in your case:
Item.objects.filter(price__range=(min_price, max_price))

Specifying a FilterSet
For more advanced filtering requirements you can specify a FilterSet class that should be used by the view. For example:
import django_filters
from myapp.models import Item
from myapp.serializers import ItemSerializer
from rest_framework import generics
class ItemListFilter(django_filters.FilterSet):
min_price = django_filters.NumberFilter(name="price", lookup_type='gte')
max_price = django_filters.NumberFilter(name="price", lookup_type='lte')
class Meta:
model = Item
fields = ['min_price', 'max_price']
class ItemList(generics.ListAPIView):
queryset = Item.objects.all()
serializer_class = GameSerializer
filter_class = ItemListFilter
Which will allow you to make requests such as:
http://example.com/api/games?min_price=1.20&max_price=8.00

How to get Custom List in a Viewset

How can we write a function in a ModelViewSet that get a list of distinct record in the database?
Supposed that we have this model.
class Animal(models.Model):
this_id = models.CharField(max_length=25)
name = models.CharField(max_length=25)
species_type = models.CharField(max_length=25)
...
and serializer
class AnimalSerializer(serializers.ModelSerializer):
class Meta:
model = Animal
fields = (
'this_id',
'name',
'species_type',
...,
)
read_only_fields = ('id', 'created_at', 'updated_at')
and ViewSet.
class AnimalViewSet(viewsets.ModelViewSet):
"""
This viewset automatically provides `list`, `create`, `retrieve`,
`update` and `destroy` actions.
"""
queryset = Animal.objects.all()
serializer_class = AnimalSerializer
I found this link useful such as decorators like #list_route()
but i can't understand it well.
I would like to get list of distinct Animal.species_type record from the ViewSet. Please help.

There are several different options in filtering. You can send the species type via your request /animals?species_type=MusMusculus and reference it when you over ride the get_queryset() method in the view.
In your view
def get_queryset(self):
species = self.request.query_params.get('species_type', None)
if species is not None:
queryset = Animals.objects.all().distinct('species_type')
species = SpeciesSerializer(data=queryset)
return queryset
Serializer
from rest_framework import serializers
class Species(serializers.Serializer):
species_type = serializers.Charfield()
alternatively, you can adopt a django filter framework http://www.django-rest-framework.org/api-guide/filtering/#djangofilterbackend

Django REST framework foreign keys and filtering

I have following models in django app:
models.py:
class Make(BaseModel):
slug = models.CharField(max_length=32) #alfa-romeo
name = models.CharField(max_length=32) #Alfa Romeo
def __unicode__(self):
return self.name
class Model(BaseModel):
make = models.ForeignKey(Make) #Alfa Romeo
name = models.CharField(max_length=64) # line[2]
engine_capacity = models.IntegerField()
trim = models.CharField(max_length=128) # line[4]
And serializers.py:
from .models import Make,Model
from rest_framework import serializers
class MakeSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Make
fields = ('url', 'slug', 'name')
class ModelSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Model
fields = ('url', 'make', 'name', 'trim', 'engine_capacity')
and also views.py:
from rest_framework import viewsets
from rest_framework import filters
from rest_framework import generics
from .models import Make, Model
from .serializers import MakeSerializer, ModelSerializer
class MakeViewSet(viewsets.ModelViewSet):
queryset = Make.objects.all()
serializer_class = MakeSerializer
filter_backends = (filters.DjangoFilterBackend,)
class ModelViewSet(viewsets.ModelViewSet):
make = MakeSerializer
queryset = Model.objects.all()
serializer_class = ModelSerializer
filter_backends = (filters.DjangoFilterBackend,)
What I need to to, I want to fetch all Models manufactured by specific make.
How can I get all models with particular make foreign key using query params? And my 2nd question - can I filter results using queryparams to get models with specific engine_capacity?
One comment: It would be perfect, if I can to query results using something like this in url: /api/models/?make=ford where make is slug field in Make model

You can specify filter_fields = ('make__slug', ) in your view set. Don't forget to include filter_backends = (DjangoFilterBackend, ) as well. Also you will need to add django-filter dependency.
class ModelViewSet(viewsets.ModelViewSet):
queryset = Model.objects.all()
serializer_class = ModelSerializer
filter_backends = (filters.DjangoFilterBackend,)
filter_fields = ('make__slug',)
Then you query like /api/models/?make__slug=ford. Note double underscore symbol.
Docs.
If you don't like make__slug keyword argument in the URL, then you can create a filter class:
import django_filters
from myapp.models import Make
class ModelFilter(django_filters.FilterSet):
make = django_filters.ModelChoiceFilter(field_name="make__slug",
queryset=Make.objects.all())
class Meta:
model = Model
fields = ('make',)
and then
class ModelViewSet(viewsets.ModelViewSet):
make = MakeSerializer
queryset = Model.objects.all()
serializer_class = ModelSerializer
filter_backends = (filters.DjangoFilterBackend,)
filter_class = ModelFilter
/api/models/?make=ford should work.

urls.py
url('^model/by/(?P<make>\w+)/$', ModelByMakerList.as_view()),
views.py
class ModelByMakerList(generics.ListAPIView):
serializer_class = ModelSerializer
def get_queryset(self):
"""
This view should return a list of all models by
the maker passed in the URL
"""
maker = self.kwargs['make']
return Model.objects.filter(make=maker)
For more info checkout the docs.
You can also use filtering with QUERY_PARAMS, but IMHO this looks better.

To expand on #vladimir-prudnikov's answer:
Things changed a bit in recent versions of django-filter. You probably want:
class ModelFilter(django_filters.FilterSet):
make = django_filters.ModelChoiceFilter(field_name='make__slug',
to_field_name='slug',
queryset=Make.objects.all())
class Meta:
model = Model
fields = ('make',)
See https://django-filter.readthedocs.io/en/master/ref/filters.html#field-name and https://django-filter.readthedocs.io/en/master/ref/filters.html#to-field-name

What you need to do in your view is something like this:
It is called "Lookups that span relationships"
queryset = Model.objects.filter(make__name__exact='Alfa Romeo')
the filtering of models with specific engine capacity is similar
queryset = Model.objects.filter(engine_capacity__exact=5)
if you want both filters combined, you can chain them:
queryset = Model.objects.filter(make__name__exact='Alfa Romeo').filter(engine_capacity__exact=5)
more examples can be found here django query making