Paginating the results of a Django forms POST request - python

I'm using Django Forms to do a filtered/faceted search via POST, and I would like to Django's paginator class to organize the results. How do I preserve the original request when passing the client between the various pages? In other words, it seems that I lose the POST data as soon as I pass the GET request for another page back to my views. I've seen some recommendations to use AJAX to refresh only the results block of the page, but I'm wondering if there is a Django-native mechanism for doing this.
Thanks.

If you want to access the store data in later request, you would have to store it somewhere. Django provides several ways to archive this:
1) You can use sessions to store the query: Every visitor who visits your site will get an empty session object and you can store whatever you want inside this object, which acts like a dict. Drawback: A single visitor can't do multiple searches with pagination concurrently.
2) Use cookies: If you set a cookie which is stored on the client side, the browser will append the data of the cookie to each request where you can access it. Cookies are more server friendly, because you don't need a session manager for them on the server, but the data stored in cookies is visible (and editable) to the client. Drawback: same as before.
3) Use hidden fields: You can add a form with some hidden fields on your search-result page and store the query inside them. Then, the client will resend the query whenever you submit the form. Drawback: You must use a form with submit buttons for the pagination on your page (simple links wont work).
4) Create Links which contain the query: Instead of using POST, you can also use GET. For example, you could have a link like "/search/hello+world/?order=votes" and "paginated links" like "/search/hello+world/2/?order-votes". Then the query can be easily retrieved from the URL. Drawback: The maximum amount of data you can send via GET is limited (But that shouldn't be a problem for a simple search).
5) Use a combination: You might want to store all the data in a session or a database and access them via a generated key which you can put in the URL. URLs might then look like "/search/029af239ccd23/2" (for the 2nd page) and you can use the key to access a huge amount of data which you have stored before. This eliminates the drawback of solution 1 as well as that of solution 4. New drawback: much work :)
6) Use AJAX: With ajax you can store the data inside some js-variables on the client side, which can then passed to the other requests. And since ajax will only update your result list, the variables aren't getting lost.

Reading the very nice answer from tux21b I decided to implement the first option, i.e., to use the session to store the query. This is an application that searches real estate databases. Here is the view code (using django 1.5):
def main_search(request):
search_form = UserSearchForm()
return render(request, 'search/busca_inicial.html', {'search_form': search_form})
def result(request):
if request.method == 'POST':
search_form = UserSearchForm(request.POST)
if search_form.is_valid():
# Loads the values entered by the user on the form. The first and the second
# are MultiChoiceFields. The third and fourth are Integer fields
location_query_list = search_form.cleaned_data['location']
realty_type_query_list = search_form.cleaned_data['realty_type']
price_min = search_form.cleaned_data['price_min']
price_max = search_form.cleaned_data['price_max']
# Those ifs here populate the fields with convenient values if the user
# left them blank. Basically the idea is to populate them with values
# that correspond to the broadest search possible.
if location_query_list == []:
location_query_list = [l for l in range(483)]
if realty_type_query_list == []:
realty_type_query_list = [r for r in range(20)]
if price_min == None:
price_min = 0
if price_max == None:
price_max = 100000000
# Saving the search parameters on the session
request.session['location_query_list'] = location_query_list
request.session['price_min'] = price_min
request.session['price_max'] = price_max
request.session['realty_type_query_lyst'] = realty_type_query_list
# making a query outside the if method == POST. This logic makes the pagination possible.
# If the user has made a new search, the session values would be updated. If not,
# the session values will be from the former search. Of course, that is what we want because
# we want the 'next' and 'previous' pages correspond to the original search
realty_list_result = FctRealtyOffer.objects.filter(location__in=request.session['location_query_list']
).filter(price__range=(request.session['price_min'], request.session['price_max'])
).filter(realty_type__in=request.session['realty_type_query_lyst'])
# Here we pass the list to a table created using django-tables2 that handles sorting
# and pagination for us
table = FctRealtyOfferTable(realty_list_result)
# django-tables2 pagination configuration
RequestConfig(request, paginate={'per_page': 10}).configure(table)
return render(request, 'search/search_result.html', {'realty_list_size': len(realty_list_result),
'table': table})
Hope it helps!If anyone has any improvement to suggest, be welcome.

As #rvnovaes, a way to use session to solve the matter.
The drawback of his solution is that if there are many search fields you have to write many lines of code, and also if you show the search form in the result page, all the fields will be blank, while they should keep their values.
So I'd rather save all the post data in session, and at the beginning of the view force the value of request.POST and request.method if a session is defined:
""" ... """
if not request.method == 'POST':
if 'search-persons-post' in request.session:
request.POST = request.session['search-persons-post']
request.method = 'POST'
if request.method == 'POST':
form = PersonForm(request.POST)
request.session['search-persons-post'] = request.POST
if form.is_valid():
id = form.cleaned_data['id']
""" ... """
More info here

I did this in my web application with get parameters Maybe i can help you :
Views.py
class HomeView(ListView):
model = Hotel
template_name = 'index.html'
paginate_by = 10 # if pagination is desired
def get_queryset(self):
qs = super().get_queryset()
kwargs = {}
if 'title' in self.request.GET:
title = self.request.GET.get('title')
if title != '':
kwargs['title__icontains'] = title
if 'category' in self.request.GET:
category = self.request.GET.get('category')
if category:
kwargs['category_id'] = category
if 'size' in self.request.GET:
size = self.request.GET.get('size')
if size:
kwargs['size_id'] = size
if 'service' in self.request.GET:
service = self.request.GET.get('service')
if service:
kwargs['service_id'] = service
if 'ownership' in self.request.GET:
ownership = self.request.GET.get('ownership')
if ownership:
kwargs['ownership_id'] = ownership
qs = qs.filter(**kwargs)
return qs
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
form_init = {}
form = SearchForm()
if self.request.GET.items():
try:
parameters = self.request.GET.items()
except KeyError:
parameters = {}
for key, value in parameters:
for field in form.fields:
if key == field:
form_init[key] = value
form.initial = form_init
if 'title' in self.request.GET:
title = self.request.GET.get('title')
if title != '':
context.update({
'title': title
})
if 'category' in self.request.GET:
category = self.request.GET.get('category')
context.update({
'category': category
})
if 'size' in self.request.GET:
size = self.request.GET.get('size')
context.update({
'size': size
})
if 'service' in self.request.GET:
service = self.request.GET.get('service')
context.update({
'service': service
})
if 'ownership' in self.request.GET:
ownership = self.request.GET.get('ownership')
context.update({
'ownership': ownership
})
context.update({
'search_form': form
})
return context
Pagination file html
<div class="row">
{% if is_paginated %}
<nav aria-label="...">
<ul class="pagination">
{% if page_obj.has_previous %}
<li class="page-item"><a class="page-link" href="?category={{category}}&size={{size}}&service={{service}}&ownership={{ownership}}&page={{ page_obj.previous_page_number }}">Previous</a></li>
{% else %}
<li class="page-item disabled"><span class="page-link">Previous</span></li>
{% endif %}
<span class="page-current">
Page {{ page_obj.number }} of {{ page_obj.paginator.num_pages }}.
</span>
{% if page_obj.has_next %}
<li class="page-item"><a class="page-link" href="?category={{category}}&size={{size}}&service={{service}}&ownership={{ownership}}&page={{ page_obj.next_page_number }}">Next</a></li>
{% else %}
<li class="page-item disabled"><span class="page-link">Next</span></li>
{% endif %}
</ul>
</nav>
{% endif %}
</div>

You can ask request object if it's ajax, simply request.is_ajax. This way you can detect, whether it's first post request or further questions about the next pages.

Have the search form and the results display on one single django template. Initially, use css to hide the results display area. On POSTing the form, you could check to see if the search returned any results and hide the search form with css if results exist. If results do not exist, use css to hide the results display area like before. In your pagination links, use javascript to submit the form, this could be as simple as document.forms[0].submit(); return false;
You will need to handle how to pass the page number to django's paging engine.

My suggestion would be to store the post request using a session or a cookie. In case the post data is sensitive, you should use session to store it. The code below contains my logic to implement it using session.
def index(request):
is_cookie_set = 0
# Check if the session has already been created. If created, get their values and store it.
if 'age' in request.session and 'sex' in request.session:
age = request.session['age']
sex = request.session['sex']
is_cookie_set = 1
else:
# Store the data in the session object which can be used later
request.session['age'] = age
request.session['sex'] = sex
if(request.method == 'POST'):
if(is_cookie_set == 0): # form submission by the user
form = EmployeeForm(request.POST)
sex = form.cleaned_data['sex']
age = form.cleaned_data['age']
if form.is_valid():
result = Employee.objects.all(sex=sex,age_gte=age) # filter all employees based on sex and age
else: # When the session has been created
result = Employee.objects.all(sex=sex,age_gte=age)
paginator = Paginator(result, 20) # Show 20 results per page
page = request.GET.get('page')
r = paginator.get_page(page)
response = render(request, 'app/result.html',{'result':result})
return response
else:
form = EmployeeForm()
return render(request,'app/home.html',{'form':form})
You should also check if the post fields are empty or not and change the logic according to it. You can also store the whole post request in the session as suggested by #abidibo.
You can also use cookies for the same. I have explained it here

The Below code is working, the first request is a GET request, which accesses the form, will go directly to the else block.
Once the user puts up a search query, results will be shown, which will be a post request, and 2nd if block will be activated, this request we will store in a session.
When the user accesses 2nd search page, it will be a GET request, but we are checking whether there is an active pagination session, and also checking whether it's not a page request of GET. At this point 1st if block will be trigerred.
def search(request):
if not request.method == "POST" and 'page' in request.GET:
if 'search-query' in request.session:
request.POST = request.session['search-query']
request.method = 'POST'
if request.method == 'POST':
form = Search_form(request.POST)
request.session['search-query'] = request.POST
if form.is_valid():
search_query = form.cleaned_data.get('search_query')
search_parameter = form.cleaned_data.get('search_parameter')
print(search_query, search_parameter)
queryset_list = CompanyRecords.objects.filter(**{f'{search_parameter}__icontains': search_query}).exclude(
company_name__isnull=True).exclude(description__isnull=True).exclude(phones__isnull=True).exclude(
emails__isnull=True)[:5]
page = request.GET.get('page', 1)
paginator = Paginator(queryset_list, 2)
try:
queryset = paginator.page(page)
except PageNotAnInteger:
queryset = paginator.page(1)
except EmptyPage:
queryset = paginator.page(paginator.num_pages)
return render(request, 'search/search_results.html', {'queryset': queryset})
else:
context = {
'form': Search_form()
}
return render(request, 'search/search.html', context)

Related

Django objects.get matching query does not exist

I'm trying to get all attributes of a single object. I keep getting a "Devices matching query does not exist." I just cannot figure out my issue.
Models.py
`class Devices(models.Model):
category_id = models.ForeignKey(Category, on_delete=models.CASCADE)
device_description = models.CharField(max_length=100)
device_status = models.CharField(max_length=50)
device_date = models.DateTimeField()
device_user = models.CharField(max_length=50)`
Views.py
def view_status(request, pk=None):
device = Devices.objects.get(pk=pk)
return render(request, 'homesite/device_status.html', device)
urls.py
url(r'^viewstatus/$', views.view_status, name='ViewStatus'),
here is the url I use to call http://localhost:8000/homesite/viewstatus/?pk=1
device_satus.html
{% extends "base.html" %}
{% block head %}
<title>Device Status</title>
{% endblock%}
{% block body %}
<h3>Device Status Detail</h3>
{{ devices.device_description }}
{{ devices.device_status }}
{{devices.device_date|date:"Y-m-d H:m:s"}}
{% endblock %}
There are 4 records in my able so I know there is a match for PK=1.
Note, that this is not the usual way to build an URL for accessing a specific object. Below I present first the approach that integrates pk in the URI and second the one passing pk as a parameter.
1. Approach
Here you put the pk in the URI and request something like http://localhost:8000/homesite/viewstatus/1/. If you do so, you need to adapt your urls.py by specifying what part of the URI is the pk you want:
# urls.py
url(r'^viewstatus/(?P<pk>\d+)/$', views.view_status, name='ViewStatus'),
The way you wrote the view is fine:
def view_status(request, pk=None):
if pk is not None:
device = Devices.objects.get(pk=pk)
else:
device = None
return render(request, 'homesite/device_status.html', {'device' : device})
Now, views.view_status will be called with both the request object and the pk as arguments and objects.get will behave as you expected, if the pk you put in the URI exists in you database.
Note that this is the preferred way to get an object.
2. Approach
In this case you pass the pk as a parameter, so call http://localhost:8000/homesite/viewstatus/?pk=1, for example. Now pk is a parameter of a GET request. In this case:
# urls.py
url(r'^viewstatus/$', views.view_status, name='ViewStatus'),
And the view only takes the request object as argument. Within the view you can get the pk as follows:
def view_status(request):
pk = request.GET.get('pk', None)
if pk is not None:
device = Devices.objects.get(pk=int(pk))
else:
device = None
return render(request, 'homesite/device_status.html', {'device' : device})
So in this case your view does not take any arguments other than the request object.
Another issue is in your view function: Django's shortcut render takes a dict object for the optional argument context. Currently you directly pass a Devices object. You need to update your return statement in view_status:
return render(request, 'homesite/device_status.html', {'device' : device})
Hope that helps!
I get an error 'Devices' object is not iterable
urls.py
this is how the url is set up.
url(r'^viewstatus/$', views.view_status, name='ViewStatus'),
but is should be like this
url(r'^viewstatus/(?P<pk>\d+)/$', views.view_status, name='ViewStatus'),
so that I can call like this correct? http://localhost:8000/homesite/viewstatus/1/
views.py
def view_status(request):
pk = request.GET['pk']
device = Devices.objects.get(pk=pk)
return render(request, 'homesite/device_status.html', device
so i need the corresponding views.py code to work with
http://localhost:8000/homesite/viewstatus/1/
I've stared at this for hours so I know I'm missing something simple.
Try changing your view function:
def view_status(request):
pk = request.GET['pk']
device = Devices.objects.get(pk=pk)
return render(request, 'homesite/device_status.html', device)
Let me know if it helps :)

How to make an edit view using django python?

I have edit_client view, Client model and a ClientForm. What I need is to edit an existing record from Client but display it as an editable form, and save the updated record. What should be seen in my views.py and my edit_client.html?
You can create a function named : edit_client into your view file.
As an example, you can use a link in your html like this :
<a href="{% "edit_client" client.pk %}> {{ client.name }} </a>
And your function can be :
def edit_client(request, client_id):
client = Client.objects.get(pk=client_id)
clients = Client.objects.all()
if request.method = "POST":
# what you want to edit (name, age etc ...)
client.save()
return render_to_response('index.html', {"clients":clients}, context_instance=RequestContext(request))
else:
return render_to_response('edit_client.html', {"client":client}, context_instance=RequestContext(request))
Note that it will be different if you want to use a form.

Django: Delete model object using template

I am currently using models to have users enter data, using templates (not admin) that is then stored, at which point the users can then see all the data they entered. I would like to also give users the ability to delete specific entries, this would be done using object ids to identify and delete specific objects.
Here is my views.py:
#login_required(login_url='/login/')
def fav(request):
context = RequestContext(request)
#This returns all of the data the user has entered
favorites_list = StockTickerSymbol.objects.filter(user=request.user).order_by('-added_date')
`
#This is to try to get the ID of every object in favorites_list and append it to a list
for obj in favorites_list:
stock_id = []
stock_id.append(obj.id)
#Here is where the form is processed to save the data the user has entered
if request.method == 'POST':
form = FavoritesForm(request.POST)
if form.is_valid():
stock = form.save(commit=False)
stock.user = request.user
stock.save()
return redirect(fav)
else:
print form.errors
else:
form = FavoritesForm()
context_dict = {'favorites': favorites_list, 'form':form, 'stock_id':stock_id}
return render_to_response('favorites/favorites.html', context_dict, context)
def delete(request, id):
stock_to_delete = get_object_or_404(StockTickerSymbol, pk=id).delete()
return redirect(fav)
Here is my urls.py:
url(r'^favorites/$', views.fav, name='favorites'),
url(r'^add_favorites/$', views.add_fav, name='add favorites'),
url(r'^delete/(?P<id>\d+)/$', views.delete, name='delete')
And this is the part of my template file responsible for deleting
{% for id in stock_id %}
<div align="right">Delete</div>
{% endfor %}
My problem with this code, is that the delete link in my template only gives the first object ID for all the links. For example if there are three submissions for the user, and there id's are 1,2,3. The delete link will read "/delete/1" for all the submissions, thus only allowing users to delete their first submission. Any idea on how I can solve this?
Your problem is here:
for obj in favorites_list:
stock_id = []
stock_id.append(obj.id)
You are reinitializing inside the loop.
Try this
stock_id = []
for obj in favorites_list:
stock_id.append(obj.id)
Note that you can also do:
favorites_list = StockTickerSymbol.objects.filter(user=request.user).order_by('-added_date')
stock_ids = list(facorites_list.values_list('id', flat=True)) #IMO - It is a good idea to name a list with plural for readability
Also, in your delete method - See if the user does have permission to delete the object. If not, anyone can hit this url with some random id and start deleting the objects in the database.
I would start off by adding the login_required decorator, followed by adding a created_by or attaching a group associated with the model, which need to be verified before allowing the user to delete the object.
EDIT
{% for fav in favorite_list %}
<div class="fav">
{{fav.name}}
</div>
Delete me
{% endfor %}
Now you can do away with the id list.

flask-wtf form validation not working for my new app

I've used flask before and I've had working form validation, but for some reason it's not working for my new app. Here is the basic code of the form.
from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators,ValidationError
class subReddit(Form):
subreddit = TextField('subreddit', [validators.Required('enter valid subreddit')])
next = SubmitField('next')
change = SubmitField('change')
user = TextField('user', [validators.Required('enter valid user')])
fetch = SubmitField('fetch comments')
I have subreddit as the validation field, so if it's empty, I want it to throw an error and reload the page.
The HTML:
<form class='sub' action="{{ url_for('sr') }}" method='post'>
{{ form.hidden_tag() }}
<p>
if you want to enter more than one subreddit, use the + symbol, like this:
funny+pics+cringepics
<p>
<br/>
{% for error in form.subreddit.errors %}
<p>{{error}}</p>
{% endfor %}
{{form.subreddit.label}}
{{form.subreddit}}
{{form.change}}
</form>
I have CSRF_ENABLED=True in my routes.py as well. What am I missing? When I leave the subredditfield empty and click change, it just reloads the page, no errors. This is an issue because whatever is in the field will get recorded in my database, and it can't be empty.
EDIT
#app.route('/index',methods=['GET','POST'])
#app.route('/',methods=['GET','POST'])
def index():
form = subReddit()
rand = random.randint(0,99)
sr = g.db.execute('select sr from subreddit')
srr = sr.fetchone()[0]
r = requests.get('http://www.reddit.com/r/{subreddit}.json?limit=100'.format(subreddit=srr))
j = json.loads(r.content)
pic = j['data']['children'][rand]['data']['url']
title = None
if form.validate_on_submit():
g.db.execute("UPDATE subreddit SET sr=(?)", [form.subreddit.data])
print 'validate '
if j['data']['children'][rand]['data']['url']:
print 'pic real'
sr = g.db.execute('select sr from subreddit')
srr = sr.fetchone()[0]
r = requests.get('http://www.reddit.com/r/{subreddit}.json?limit=100'.format(subreddit=srr))
pic = j['data']['children'][rand]['data']['url']
title = str(j['data']['children'][rand]['data']['title']).decode('utf-8')
return render_template('index.html',form=form,srr=srr,pic=pic,title=title)
else:
print 'not valid pic'
return render_template('index.html',form=form,srr=srr,pic=pic)
else:
print 'not valid submit'
return render_template('index.html',form=form,srr=srr,pic=pic)
return render_template('index.html',form=form,srr=srr,pic=pic)
You have a number of problems.
The most important is that validation occurs in the POST request view function. In your example this is function sr. That function should create the form object and validate it before adding stuff to the database.
Another problem in your code (assuming the above problem is fixed) is that after validate fails you redirect. The correct thing to do is to render the template right there without redirecting, because the error messages that resulted from validation are loaded in that form instance. If you redirect you lose the validation results.
Also, use validate_on_submit instead of validate as that saves you from having to check that request.method == 'POST'.
Example:
#app.route('/sr', methods=['POST'])
def sr():
form = subReddit()
if not form.validate_on_submit():
return render_template('index.html',form=form)
g.db.execute("UPDATE subreddit SET sr=(?)", [form.subreddit.data])
return redirect(url_for('index'))
Additional suggestions:
it is common practice to start your class names with an upper case character. SubReddit is better than subReddit.
it is also common to have the GET and POST request handlers for a form based page in the same view function, because that keep the URLs clean when validation fails without having to jump through hoops to get redirects working. Instead of having the sr function separately you can just combine it with index() and have the action in the form go to url_for('index').
Flask-WTF adds a new method onto the form called validate_on_submit(). This is like the WTForms validate() method, but hooks into the Flask framework to access the post data. The example given on the Flask site is:
form = MyForm()
if form.validate_on_submit():
flash("Success")
return redirect(url_for("index"))
return render_template("index.html", form=form)
Because you're just using validate(), the form is trying to validate without any data (which, of course, will fail). Then you're redirecting. Try to use validate_on_submit() as shown above.

Will this cost me two queries for the same thing?

I have the normal pagination like this in my view:
paginator = Paginator(book_list, 100)
And then in my view I am passing the values to my template:
return render(request,
...
'paginator': paginator,
...
And I have a custom tag for my pagination, which I am loading like this:
{% if paginator.count > paginator.per_page %}
{% load paginator %}
{% paginator 3 %}
{% endif %}
In my custom template pagination tag, I have the following along the code:
def paginator(context, adjacent_pages=2):
page_obj = context['paginator'].page(context['object_list'].number)
...
'hits': context['paginator'].count,
...
Everything is working as expected but I am worried about context['paginator'].page(context['object_list'].number), is Django fetching the data from DB with this bit or it's using the same data that was fetched from my main view?
Please advise. Thanks.
The paginator keeps the query_set as object_list, in django 1.3.4, the page method, is
def page(self, number):
"Returns a Page object for the given 1-based page number."
number = self.validate_number(number)
bottom = (number - 1) * self.per_page
top = bottom + self.per_page
if top + self.orphans >= self.count:
top = self.count
return Page(self.object_list[bottom:top], number, self)
Only the last line related to db,
self.object_list[bottom:top]
The object_list is just a QuerySet, so the problems comes to if you invoke query_set[x:y] more times, whether there exists multiple queries.
Django's query_set is lazy, if you don't iterate through it, no sql will be triggered. Otherwise, there will be db queries.
You can use check queries in django.db.connection.queries for following code,
from django.db import connection
original = XXXX.objects.filter(...)
res1 = original[x:y]
for item in res1:
print item
print len(connection.queries), connection.queries[-1]
res2 = original[x:y]
for item in res2:
print item
print len(connection.queries), connection.queries[-1]
You'll find that the query length grows.
My understanding here is that it's simply using whatever object you passed it in your main view. context['paginator'] is going to return the object stored in the paginator variable that you passed to the context, an instance of the Paginator class.
The question of whether or not it's going back to the database is simply about the .page(...) method. If calling Paginator.page(...) issues a database query, then it will be going back to the database--it wouldn't cache that value. However, if that information is already available locally in the paginator variable and that is what is called up by the .page method, then you're not refetching the data from the database.

Categories

Resources