Develop Reference

Is there a way to combine view functions of similar structures into one? Django

I'm a beginner at Django. Recently, I started writing a web app for inventory management and I realised that when i was writing the views, there were a lot of them with similar structures. For instance:
def invoices(request):
"""The page for displaying invoices."""
invoice_list = Document.objects.filter(type_name__name='Invoice')
page = request.GET.get('page', 1)
paginator = Paginator(invoice_list, 10)
try:
invoices = paginator.page(page)
except PageNotAnInteger:
invoices = paginator.page(1)
except EmptyPage:
invoices = paginator.page(paginator.num_pages)
context = {'invoices':invoices}
return render(request, 'imsapp/invoices.html', context)
and this one:
def credit_notes(request):
"""The page for displaying credit notes."""
credit_notes_list = Document.objects.filter(type_name__name='Credit Note')
page = request.GET.get('page', 1)
paginator = Paginator(credit_notes_list, 10)
try:
credit_notes = paginator.page(page)
except PageNotAnInteger:
credit_notes = paginator.page(1)
except EmptyPage:
credit_notes = paginator.page(paginator.num_pages)
context = {'credit_notes':credit_notes}
return render(request, 'imsapp/credit_notes.html', context)
So, I'm just thinking if there is a more elegant way to represent the above function definitions. Is Class-based view what I'm looking for?

You could refactor the logic such that a function handles the common logic or you can use a class based view.
def paginate(request, queryset):
page_index = request.GET.get('page', 1)
paginator = Paginator(queryset, 10)
try:
page = paginator.page(page_index)
except PageNotAnInteger:
page = paginator.page(1)
except EmptyPage:
page = paginator.page(paginator.num_pages)
return page
def credit_notes(request):
"""The page for displaying credit notes."""
credit_notes_list = Document.objects.filter(type_name__name='Credit Note')
credit_notes = paginate(request, credit_notes_list)
context = {'credit_notes':credit_notes}
return render(request, 'imsapp/credit_notes.html', context)
Or you can use ListView
from django.views.generic.list import ListView
class CreditNoteListView(ListView):
queryset = Document.objects.filter(type_name__name='Credit Note')
paginate_by = 10
template_name = 'imsapp/credit_notes.html'
You'll need to change your template in that case though since the template context will be slightly different.

Views are just Python functions - and as such, tehy can call further functions.
In the above code, it looks like the only things that change are the type object name and the template.
Simply create a new function that takes these two as parameters
def paged_view(request, type_name, template, name_in_context):
"""The page for displaying various items in a paginated way."""
item_list = Document.objects.filter(type_name__name=time_name)
page = request.GET.get('page', 1)
paginator = Paginator(item_list, 10)
try:
items = paginator.page(page)
except PageNotAnInteger:
items = paginator.page(1)
except EmptyPage:
items = paginator.page(paginator.num_pages)
context = {name_in_context:items}
return render(request, template, context)
def invoices(request):
"""The page for displaying invoices."""
return paged_view(requests, 'imsapp/invoices.html', 'Invoice', 'invoices')
def credit_notes(request):
"""The page for displaying credit notes."""
return paged_view(requests, 'imsapp/credit_notes.html', 'Credit Note', 'credit_notes')

Django, python, redis sorting

I used redis to sort out the most viewed items in a project I am working on. I would like to sort out only the most viewed for the day and out the overall most viewed. In other words, viewing count should restart everyday. Here is my code
#login_required
def image_ranking(request):
image_ranking = r.zrange('image_ranking', 0, -1, desc=True[:110000]
image_ranking_ids = [int(id) for id in image_ranking]
most_viewed = list(Image.objects.filter(id__in=image_ranking_ids))
most_viewed.sort(key=lambda x: image_ranking_ids.index(x.id))
paginator = Paginator(most_viewed, 24)
page = request.GET.get('page')
try:
most_viewed = paginator.page(page)
except PageNotAnInteger:
most_viewed = paginator.page(1)
except EmptyPage:
if request.is_ajax():
return HttpResponse('')
most_viewed = paginator.page(paginator.num_pages)
if request.is_ajax():
return render(request, 'images/image/ranking_ajax', {'section': 'images', 'most_viewed': most_viewed})
return render(request, 'images/image/ranking.html', {'section': 'images', 'most_viewed': most_viewed})

How to implement pagination in Search results in django?

I found a resource where it has code like this:
from django.contrib.auth.models import User
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
def index(request):
user_list = User.objects.all()
page = request.GET.get('page', 1)
paginator = Paginator(user_list, 10)
try:
users = paginator.page(page)
except PageNotAnInteger:
users = paginator.page(1)
except EmptyPage:
users = paginator.page(paginator.num_pages)
return render(request, 'core/user_list.html', { 'users': users })
I have a code like this:
def product_list(request, category_slug=None):
category = None
categories = Category.objects.all().order_by("-rating")
#paginator = Paginator(categories, 10)
products = Product.objects.all().order_by("-number")
users = User.objects.exclude(id=request.user.id)
query = request.GET.get('q')
if query=='':
return HttpResponseRedirect('/')
if query:
categories = Category.objects.filter(Q(slug__icontains=query)| Q(url__icontains=query)).order_by("-rating")
products = Product.objects.filter(Q(slug__icontains=query) | Q(name__icontains=query) | Q(description__icontains=query)).order_by("number")
if category_slug:
category = get_object_or_404(Category, slug=category_slug)
products = Product.objects.filter(category=category)
categories_counter = products.annotate(Count('id'))
categories_count = len(categories_counter)
#contacts = paginator.get_page(query)
context = {
'category': category,
'categories': categories,
'products': products,
'categories_count':categories_count,
'query':query,
'users':users,
#'contacts':contacts,
}
return render(request, 'shop/product/list.html', context)
I have retrieved objects from two models, Category, and Product. How do I implement the pagination code in this view? It's not a normal pagination but pagination in search results.

user_list = User.objects.all() # this is the full queryset, contains all objects
page = request.GET.get('page', 1) # this is the page number of page whose data you want to retrieve, you need to pass page value as query params
paginator = Paginator(user_list, 10) # this will paginate the full queryset in pages of 10 objects
try:
users = paginator.page(page) # this will return data of that particular page
except PageNotAnInteger:
users = paginator.page(1) # if the passed page is not Integer then first page is returned, you can customize this
except EmptyPage:
users = paginator.page(paginator.num_pages) # if that page contains no element, then last page is returned, you can customize this also
You can apply same logic to retrieve categories in category_page and other data.

Django filters not applying when going to next page

I am using Django filters and when I apply the filters they work but when I go to the next page of results they reset. I have looked at this post django-filter use paginations but they seem to be doing the same things that I am. What am I doing wrong?
Url
url(r'^relations/$', views.annotation_views.relations, name="relations")
returns a url like this when the filters are applied:
/relations/?createdBy=&occursIn=&createdAfter=&createdBefore=&terminal_nodes=&project=
Filters.py
class RelationSetFilter(django_filters.FilterSet):
occursIn = django_filters.CharFilter('occursIn__uri', method='filter_occursIn')
createdBefore = django_filters.DateTimeFilter('created', lookup_expr='lt')
createdAfter = django_filters.DateTimeFilter('created', lookup_expr='gt')
terminal_nodes = django_filters.CharFilter('terminal_nodes__uri')
def filter_occursIn(self, queryset, name, value):
if not value:
return queryset
return queryset.filter(Q(occursIn__uri=value) | Q(occursIn__part_of__uri=value) | Q(occursIn__part_of__part_of__uri=value))
class Meta:
model = RelationSet
fields = ['createdBy', 'project', 'occursIn', 'terminal_nodes']
View
def relations(request):
from annotations.filters import RelationSetFilter
qs = RelationSet.objects.all()
filtered = RelationSetFilter(request.GET, queryset=qs)
qs = filtered.qs
for r in qs:
print r.__dict__
paginator = Paginator(qs, 40)
page = request.GET.get('page')
try:
relations = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page.
relations = paginator.page(1)
except EmptyPage:
# If page is out of range (e.g. 9999), deliver last page of results.
relations = paginator.page(paginator.num_pages)
context = {
'paginator': paginator,
'relations': relations,
'params': request.GET.urlencode(),
'filter': filtered,
}
return render(request, 'annotations/relations.html', context)

I was able to solve this by copying the url and passing the params to the template. I had to delete the page param due to it being duplicated. In order to do this I added the following:
View
def relations(request):
from annotations.filters import RelationSetFilter
qs = RelationSet.objects.all()
filtered = RelationSetFilter(request.GET, queryset=qs)
qs = filtered.qs
for r in qs:
print r.__dict__
paginator = Paginator(qs, 40)
page = request.GET.get('page')
gt = request.GET.copy()
if 'page' in gt:
del gt['page']
try:
relations = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page.
relations = paginator.page(1)
except EmptyPage:
# If page is out of range (e.g. 9999), deliver last page of results.
relations = paginator.page(paginator.num_pages)
context = {
'paginator': paginator,
'relations': relations,
'params': urlencode(gt),
'filter': filtered,
}
return render(request, 'annotations/relations.html', context)
Template
<div class="col-xs-4 clearfix text-center">
{% if relations.has_next %}
<div class="pull-right">
Next »
</div>
{% endif %}
</div>
Update
I found a more secure fix for this by getting the data directly for the filter object
project = filtered.data.get('project')
then adding this to the view
&project={{ project }}

Python elasticsearch-dsl django pagination

How can i use django pagination on elasticsearch dsl.
My code:
query = MultiMatch(query=q, fields=['title', 'body'], fuzziness='AUTO')
s = Search(using=elastic_client, index='post').query(query).sort('-created_at')
response = s.execute()
// this always returns page count 1
paginator = Paginator(response, 100)
page = request.GET.get('page')
try:
posts = paginator.page(page)
except PageNotAnInteger:
posts = paginator.page(1)
except EmptyPage:
posts = paginator.page(paginator.num_pages)
Any solution for this?

I found this paginator on this link:
from django.core.paginator import Paginator, Page
class DSEPaginator(Paginator):
"""
Override Django's built-in Paginator class to take in a count/total number of items;
Elasticsearch provides the total as a part of the query results, so we can minimize hits.
"""
def __init__(self, *args, **kwargs):
super(DSEPaginator, self).__init__(*args, **kwargs)
self._count = self.object_list.hits.total
def page(self, number):
# this is overridden to prevent any slicing of the object_list - Elasticsearch has
# returned the sliced data already.
number = self.validate_number(number)
return Page(self.object_list, number, self)
and then in view i use:
q = request.GET.get('q', None)
page = int(request.GET.get('page', '1'))
start = (page-1) * 10
end = start + 10
query = MultiMatch(query=q, fields=['title', 'body'], fuzziness='AUTO')
s = Search(using=elastic_client, index='post').query(query)[start:end]
response = s.execute()
paginator = DSEPaginator(response, settings.POSTS_PER_PAGE)
try:
posts = paginator.page(page)
except PageNotAnInteger:
posts = paginator.page(1)
except EmptyPage:
posts = paginator.page(paginator.num_pages)
this way it works perfectly..

Following the advice from Danielle Madeley, I also created a proxy to search results which works well with the latest version of django-elasticsearch-dsl==0.4.4.
from django.utils.functional import LazyObject
class SearchResults(LazyObject):
def __init__(self, search_object):
self._wrapped = search_object
def __len__(self):
return self._wrapped.count()
def __getitem__(self, index):
search_results = self._wrapped[index]
if isinstance(index, slice):
search_results = list(search_results)
return search_results
Then you can use it in your search view like this:
paginate_by = 20
search = MyModelDocument.search()
# ... do some filtering ...
search_results = SearchResults(search)
paginator = Paginator(search_results, paginate_by)
page_number = request.GET.get("page")
try:
page = paginator.page(page_number)
except PageNotAnInteger:
# If page parameter is not an integer, show first page.
page = paginator.page(1)
except EmptyPage:
# If page parameter is out of range, show last existing page.
page = paginator.page(paginator.num_pages)
Django's LazyObject proxies all attributes and methods from the object assigned to the _wrapped attribute. I am overriding a couple of methods that are required by Django's paginator, but don't work out of the box with the Search() instances.

A very simple solution is to use MultipleObjectMixin and extract your Elastic results in get_queryset() by overriding it. In this case Django will take care of the pagination itself if you add the paginate_by attribute.
It should look like that:
class MyView(MultipleObjectMixin, ListView):
paginate_by = 10
def get_queryset(self):
object_list = []
""" Query Elastic here and return the response data in `object_list`.
If you wish to add filters when querying Elastic,
you can use self.request.GET params here. """
return object_list
Note: The code above is broad and different from my own case so I can not guarantee it works. I used similar solution by inheriting other Mixins, overriding get_queryset() and taking advantage of Django's built in pagination - it worked great for me. As it was an easy fix I decided to post it here with a similar example.

Another way forward is to create a proxy between the Paginator and the Elasticsearch query. Paginator requires two things, __len__ (or count) and __getitem__ (that takes a slice). A rough version of the proxy works like this:
class ResultsProxy(object):
"""
A proxy object for returning Elasticsearch results that is able to be
passed to a Paginator.
"""
def __init__(self, es, index=None, body=None):
self.es = es
self.index = index
self.body = body
def __len__(self):
result = self.es.count(index=self.index,
body=self.body)
return result['count']
def __getitem__(self, item):
assert isinstance(item, slice)
results = self.es.search(
index=self.index,
body=self.body,
from_=item.start,
size=item.stop - item.start,
)
return results['hits']['hits']
A proxy instance can be passed to Paginator and will make requests to ES as needed.