Develop Reference

Django pagination: EmptyPage: That page contains no results

When using Django CBV ListView with pagination:
class Proposals(ListView):
model = Proposal
ordering = "id"
paginate_by = 10
In the browser, if I provide a page that is out of range, I get an error:
I would like to have a different behaviour: to fallback to the last existing page if the provided page is out of range.
I dug into Django source code paginator.py file and was surprised to find some code that does exactly this:
So using paginator.get_page(page) (and not paginator.page(page)) would be the way to go. However, ListView does not use it as you can see here:
What is the best way to deal with this?
Thanks.

The only solution I found is by overriding the paginate_queryset method.
However I don't like it as I'm forced to rewrite the whole logic while I just want to change a single line.
Open to any better suggestion.
class PermissivePaginationListView(ListView):
def paginate_queryset(self, queryset, page_size):
"""
This is an exact copy of the original method, jut changing `page` to `get_page` method to prevent errors with out of range pages.
This is useful with HTMX, when the last row of the table is deleted, as the current page in URL is not valid anymore because there is no result in it.
"""
paginator = self.get_paginator(
queryset,
page_size,
orphans=self.get_paginate_orphans(),
allow_empty_first_page=self.get_allow_empty(),
)
page_kwarg = self.page_kwarg
page = self.kwargs.get(page_kwarg) or self.request.GET.get(page_kwarg) or 1
try:
page_number = int(page)
except ValueError:
if page == "last":
page_number = paginator.num_pages
else:
raise Http404(_("Page is not “last”, nor can it be converted to an int."))
try:
page = paginator.get_page(page_number)
return (paginator, page, page.object_list, page.has_other_pages())
except InvalidPage as e:
raise Http404(
_("Invalid page (%(page_number)s): %(message)s")
% {"page_number": page_number, "message": str(e)}
)

Why does my django pagination keeps returning the same items infinitely?

I am using django and ajax to create a infinite scroll on my website. I would like it to load more items from the database (that are in the next page) when the user scrolls to the bottom of the page. Everything is working perfectly except that it keeps returning the first page's items infinitely. For example if a user scrolls down to the end of a page instead of adding page 2's elements it just add the same elements from the first page to the bottom again. I am very sure this issue is from my views.py. I can't seem to figure out what's wrong.
def feed(request):
queryset2 = Store_detail.objects.filter(store_lat__gte=lat1, store_lat__lte=lat2)\
.filter(store_lng__gte=lng1, store_lng__lte=lng2)
queryset3 = Paginator(queryset2, 4)
page = request.GET.get('page',1)
try:
queryset = queryset3.page(1)
except PageNotAnInteger:
queryset = queryset3.page(page)
except EmptyPage:
queryset = ""
context = {
"location":location,
"queryset":queryset,
}
# return HttpResponse(template.render(context,request))
return render(request, 'main/feed.html', {'queryset': queryset,
'location':location,})
So basically I would like to load the next page when a user scrolls to the end of the screen and if there are no more items in the next page or the next page does not exist then stop adding items.

The pagination logic is a bit off. You paginate with:
try:
# you first try to retrieve page 1
queryset = queryset3.page(1)
except PageNotAnInteger:
queryset = queryset3.page(page)
except EmptyPage:
queryset = ""
This thus means that you first aim to fetch the first page, and only if 1 is not an integer, you will fetch a page with the page querystring parameter. You should swap these, like:
try:
queryset = queryset3.page(page)
except PageNotAnInteger:
queryset = queryset3.page(1)
except EmptyPage:
queryset = Store_detail.objects.none()

Django; Tips for making views thin

I'm a beginner and I have been working on project using Django.
I am wondering if there's a good way to avoid repeating the same code.
Also if there are similar logic in some functions, how can I decide if the logic is organized or not.
for example,
def entry_list(request):
entry_list = Entry.objects.all()
#this part is repeated
page = request.GET.get('page', 1)
paginator = Paginator(entry_list, 10)
try:
entries = paginator.page(page)
except PageNotAnInteger:
entries = paginator.page(1)
except EmptyPage:
entries = paginator.page(paginator.num_pages)
return render(request, 'blog/entry_list.html', {'entries': entries})
The logic to paginate is repeated in some other functions as well.
Where should I put the repeated code and how can I decide if I should organize code?

Using function-based views
You could encapsulate it in another function (for example construct such function in a file named utils.py):
# in app/utils.py
def get_page_entries(entry_list, page, per_page=10):
paginator = Paginator(entry_list, per_page)
try:
return paginator.page(page)
except PageNotAnInteger:
return paginator.page(1)
except EmptyPage:
return paginator.page(paginator.num_pages)
You can then use it like:
# app/views.py
from app.utils import get_page_entries
def entry_list(request):
entry_list = Entry.objects.all()
entries= get_page_entries(entry_list, request.GET.get('page', 1))
return render(request, 'blog/entry_list.html', {'entries': entries})
You can provide an optional third parameter with the number of elements per page. If not provided, it will by default be 10.
Or we can encapsulate the request.GET.get(..) logic as well, like:
# in app/utils.py
def get_page_entries(entry_list, querydict, per_page=10, key='page', default_page=1):
page = querydict.get(key, default_page)
paginator = Paginator(entry_list, per_page)
try:
return paginator.page(page)
except PageNotAnInteger:
return paginator.page(default_page)
except EmptyPage:
return paginator.page(paginator.num_pages)
and thus call it with:
# app/views.py
from app.utils import get_page_entries
def entry_list(request):
return render(request, 'blog/entry_list.html', {
'entries': get_page_entries(Entry.objects.all(), request.GET)
})
Using class-based views
You however do not need to use function-based views. This use case is covered by the ListView class [Django-doc]:
class EntryListView(ListView):
model = Entry
template_name = 'blog/entry_list.html'
context_object_name = 'entries'
paginate_by = 10
and then in the urls.py, use EntryListView.as_view() instead of the function, so:
# app/urls.py
from django.urls import path
from app.views import EntryListView
urlpatterns = [
path('entries', EntryListView.as_view(), name='entry-list'),
]
Note only did we reduce the number of lines of code, this is also a more declarative way of developing: instead of specifying how we want to do something, we specify what we want to do. How it is done is more up to Django's implementation of a ListView. Furthermore by providing the settings as attributes of the class, we can easily develop more tooling that take these parameters into account.

I asked myself the same questions for several things including:
functions I might use many times in my views,
messages I would use many times in my views but also tests (in order to test behavior of my apps),
exceptions that could be used in many models or views,
choices when defining Charfields which value can be only specific values,
context values that could be used in many templates,
etc.
Here how I work for instance:
functions: a utils.py file as Willem said does the trick,
messages: I've got a customMessages.py file used for that, where I define simple messages (e.g.:SUCCESS_M_CREATE_OBJECT=_("You created this object successfully."), then used by calling customMessages.SUCCESS_M_CREATE_OBJECT in my models, tests or views) or more complexe ones with variable fields for instance, defined by a function and called with lambda in my tests,
context: by defining dedicated functions in my context_processors.py returning dictionaries of context vars, registering them in my settings.py and then I'm fine if want to call any in my templates,
etc.
Well, there is always a way to not repeat yourself for anything you do in Python and Django, and you are just right to ask yourself those questions at anytime in your development, because your futur self will thank you for that!

How does django's pagination work in a django view [duplicate]

This question already has answers here:
Django lazy QuerySet and pagination
(2 answers)
Closed 7 years ago.
So I was reading about pagination, I have done it quite a few times writing this app but I was wondering how does pagination in django work at sql level.
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
def listing(request):
contact_list = Contacts.objects.all()
paginator = Paginator(contact_list, 25) # Show 25 contacts per page
page = request.GET.get('page')
try:
contacts = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page.
contacts = paginator.page(1)
except EmptyPage:
# If page is out of range (e.g. 9999), deliver last page of results.
contacts = paginator.page(paginator.num_pages)
return render_to_response('list.html', {"contacts": contacts})
Does Contacts.objects.all() get called evertime I am hitting the view ?
Or does Paginator maintains a state somehow ? What would the sql query look like for paginator = Paginator(contact_list, 25) Are there generators being used behind the scenes and things are lazily evaluated?
Sorry if this is a dumb question, there is a lot of abstraction in Django and I seem to miss all the action behind.
Thanks in advance

I am posting this duplicate link as answer, because title of your question is very readable and there is answer for all your question
Django lazy QuerySet and pagination

Django Pagination

I need to make real pagination instead of paginating on all retreived data. The example in Django documentation site, is like;
def listing(request):
contact_list = Contacts.objects.all()
paginator = Paginator(contact_list, 25) # Show 25 contacts per page
page = request.GET.get('page')
try:
contacts = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page.
contacts = paginator.page(1)
except EmptyPage:
# If page is out of range (e.g. 9999), deliver last page of results.
contacts = paginator.page(paginator.num_pages)
return render_to_response('list.html', {"contacts": contacts})
This code is paginating records on all retreived records. But there is a trouble. Trying to retreive all record takes many time if there are so many records. I need a solution to retrieve the records page by page from database.
Is there another solution to do this in Django?

You make a false assumption. Django does not retrieve all objects when paginating: it slices the queryset appropriately, which uses LIMIT and COUNT on the SQL.

A QuerySet is a lazy object. When you assign contact_list = Contacts.objects.all(), Django will NOT hit the database. Until you call the methods such as count(), filter(), first(),..., or contact_list[1:5], Django will really retrieve data from the database. SQL statements that Django generate will correlate to each method and these SQL statments will be sent to the DB.
E.g: contact_list[1:5] generate a SQL statement have LIMIT and OFFSET clauses.
In Paginator class, the QuerySet will passed to the its constructor
paginator = Paginator(contact_list, 25)
When you call paginator.page(page), the statement is called:
return self._get_page(self.object_list[bottom:top], number, self)

Look inside Paginator class (django/core/paginator.py), it fetches only required pages. There is only one problem on big tables: if you want to show total page numbers you must make count(*) on entire table which can took a long time in some databases (i.e. postgresql, mysql with innodb).
BTW, try to use generic views in django, ListView would be fine here.

Here we using get_page() to get page wise data(1 page contain 25 data).I would suggest for this like :
def listing(request):
contact_list = Contacts.objects.all()
paginator = Paginator(contact_list, 25) # Show 25 contacts per page
page = request.GET.get('page')
contacts = paginator.get_page(page)
return render_to_response('list.html', {"contacts": contacts})