Develop Reference

Why is my django core.serializers so slow

I have my a serializer from core.serializers in my django view. It does work, but it sometimes takes over 1 minute to show my results table. Any ideas how to get it faster?
# views.py
from django.core import serializers
def search_institution(request):
form = SearchInstitutionsForm()
qs = Institution.objects.all()
if request.method == "POST":
form = SearchInstitutionsForm(request.POST)
if form.is_valid():
cd = form.cleaned_data
if cd['name']:
qs = Institution.objects.filter(name__contains=cd['name'])
print(f"Before requesting from db: {datetime.now()}")
print(f"After requesting from db, before serializing: {datetime.now()}")
context = {
"result_data": SafeString(serializers.serialize("json", qs)),
'form': form
}
print(f"After serializing, before rendering: {datetime.now()}")
return render(request, "landing/result_table.html", context)
else:
context = {
"form": SearchInstitutionsForm
}
return render(request, "stakeholders/institution_form.html", context)

Your print statements don't help much because Querysets are lazy. It means they are not evaluated at time of filtering qs = Institution.objects.filter(name__contains=cd['name']) but rather when the actual value is used.
So I'm not going much into serialization there are really nice ways of doing that, and Django on it's own provides good tools already.
Look into Queryset and database performance, it's the most essential topic, where I suspect your performance issue comes from. Also do more research on select_related and prefetch_related, it will help you reduce 200+ queries to 2
You can use django Debug toolbar in your local environment to easily monitor your application. But there are other profiling tools and tips too.

Using request.session to populate a django form

I want to save a form in session so that a user doesn't have to reapply filters when going to a different url. Here is the error I'm getting:
'unicode' object has no attribute 'get'
This is the code I have:
views.py
def someview(request):
if request.method == 'POST':
form = Form(request.POST, initial=request.session.get('save_form', None))
if form.is_valid():
clean_form = json.dumps(form.cleaned_data, indent=4, sort_keys=True, default=str)
request.session['save_form'] = clean_form
else:
pass
else:
form = Form()
forms.py
class Form(forms.Form):
startdate = forms.DateField(
label='Start date',
initial= '2018-01-01',
widget=forms.TextInput(attrs={'class':'datepicker'})
)
enddate = forms.DateField(
label='End date',
initial= datetime.date.today() - datetime.timedelta(days=1),
widget=forms.TextInput(attrs={'class':'datepicker'})
)
manager = forms.MultipleChoiceField(
required=False,
choices=managers,
widget=forms.SelectMultiple(),
)
Being a beginner in Django, I'm not sure where to go from here. I'm pretty sure the error is in the initial argument. But, printing it, it seems print what I would expect.
TIA

My best guess is that you JSONfify the form values (json.dumps(form.cleaned_data, ...)), but you do not un-JSONify it when reloading from the session.
When reading the session you need to do json.loads(...) at some point:
saved_form = json.loads(request.session.get('save_form', ''))
The default value '' might not be the best, I haven't tried. If that's giving you an error, you might want to try replacing it with {}.
Hope that helps.
EDIT:
Note that loading content from the session should probably be done in the last else branch. You generally don't want to provide an initial value when the user is sending data, like in a POST request:
def someview(request):
if request.method == 'POST':
form = Form(request.POST)
if form.is_valid():
clean_form = json.dumps(form.cleaned_data, default=str)
request.session['save_form'] = clean_form
else:
pass
else:
saved_form = json.loads(request.session.get('save_form', ''))
form = Form(initial=saved_form)
EDIT 2:
Generally, what you want to do in a form view is often very similar, and you might find yourself repeating the same boilerplate. Luckily, Django has something to help with that, and it called class-based views, and I recommend ccbv.co.uk if you're interested to dive into that, for example FormView.
However, they can be a bit confusing and hard to get your head around when you're just starting. Maybe better to stick with function-based views (as you do) for now, but I thought I linked to these 2 resources which I wish I had when I started learning class-based views.

Django: Modify model field value when form is submitted

I have a model field for each use that keeps track of their score on my website. Every time a user does some I want to add or subtract from their score.
In this particular case, I would like to change the users score when they publish a post on my site. I am able to access the user's score, but not modify it. Here is what I have so far:
def createPost(request):
user = UserProfile.objects.get(user=request.user)
current_score = user.user_score
if request.method == 'POST':
form = PostForm(request.POST)
if form.is_valid():
...
user.user_score = current_score - 1
...
else:
form = PostForm()
return render(request,'feed/userpost_form.html',{'form':form})
I am not getting any errors and publishing the post works fine, just not modifying the user's score. Also, I am using Django 1.11

you were missing user.save()
btw another approach could be:
from django.db.models import F
UserProfile.objects.filter(user=request.user).update(user_score=F('user_score')+1)
important feature of above approach in mentioned context is:
Another useful
benefit of F() is that having the database - rather than Python -
update a field’s value avoids a race condition.
https://docs.djangoproject.com/en/1.8/ref/models/expressions/#avoiding-race-conditions-using-f

django-mptt: dealing with concurrent inserts

I have a threaded comment system which works fine 99.9% of the time, but very occasionally the tree breaks down and left/right values get duplicated.
I have discovered that this happens when two posts happen at the same time (within a second of each other), and presumably what is happening is that the second post is updating the left/right values of the tree before the first has completed doing so.
My comment insert code from views.py is the following:
#login_required
#transaction.autocommit
def comment(request, post_id):
parent = get_object_or_404(Post, pk=post_id)
if request.method == 'POST':
form = PostForm(request.POST)
form.parent = post_id
if form.is_valid():
new_post = newPost(request.user, form.cleaned_data['subject'], form.cleaned_data['body'])
new_post.insert_at(parent, 'last-child', save=True)
return HttpResponseRedirect('/posts/')
else:
form = PostForm()
return render_to_response('posts/reply.html', {'requestPost': request.POST, 'form': form, 'parent': parent}, context_instance=RequestContext(request))
What is the correct approach to dealing with this? Is there a django way to ensure that the second view does not get called until the first database transaction is complete? Or should I rebuild the tree after each insert to ensure integrity? Or is there a better insert method to be using?
Thanks!
edit: I'm using MySQL.

transaction.autocommit() is a standard django behavior. You decorator does nothing, if global transaction behavior was not redefined.
Use should use commit_on_success() decorator. All db operations in view will be in one transaction.
You can read more on https://docs.djangoproject.com/en/1.5/topics/db/transactions/
PS: In django 1.6 transaction management will be updated, be attentive.

POST in django. How does a user update?

I'm brand new to django and fairly new to programming in general. I've done the django tutorial and searched the web for an answer to this question, but to no avail, so now I'm here. I am confused how post works with django. All of the tutorials I've looked at how have a return function in views that displays the webpage. I get that. But then how does a user update data if the page is being rendered from that return statement? After the return there can't be any more updates because the function stops, right? What am I missing here? Any help would be greatly appreciated, I'm getting fairly desperate here.

One pattern for Django views (by no means the only pattern) is to check the request method (GET or POST) at the beginning of the view. If it is POST, then handle the incoming data (before the view returns), and then return either a rendered template, or a redirect.
def view_function(request):
if request.method == 'POST':
if data_is_valid(request.POST):
save_data(request.POST)
return HttpResponseRedirect('/somewhere/good')
else:
return render('template', {'errors': what_went_wrong}
else:
return render('template')

The user updates data in the logic of the view function. That is to say, if the user wishes to update something, you place the update logic in the view function before the return. For example, you would do this:
def update(request):
item = <some model>.objects.get(<something>)
<more code>
return <something>

Usually an edit view function contains two parts -- one for updating data, and the other for displaying the update form. For example,
def user_edit(request):
if request.method == 'POST': # is this a save action?
# save the user data
user_id = request.POST.get('user_id')
username = request.POST.get('username')
description = request.POST.get('description')
user = User.objects.get(id=user_id)
user.username = username
user.description = description
user.save()
return HttpResponseRedirect('/user/') # redirect to index
else:
# show the edit form
user_id = request.GET.get('user_id')
user = User.object.get(id=user_id)
return render_to_response('/user/edit.html', { 'user': user })
There are many different choices for the if request.method == 'POST' line. You can also use if request.POST.get('user_id') to check if specified field is set, to determine if this is a save action.