Django - iterating a queryset produces hundreds of queries - python

I have a model where a "variable template" is defined, and attached to every Location model in the database, producing a 2-dimensional array of instances of these variables.
class Location(models.Model):
name = models.CharField(max_length=4, blank=False, unique=True)
class LocationVariableDefinition(models.Model):
name = models.CharField(max_length=32, blank=False, unique=True)
type = models.CharField(max_length=60, default="text")
default_value = models.CharField(max_length=250, blank=True, default="[UNSET]")
class LocationBasedVariable(models.Model):
location = models.ForeignKey(Location)
definition = models.ForeignKey(LocationVariableDefinition, default=None)
value = models.CharField(max_length=250, blank=True)
To edit these variables, I have a single page with a table, variable names down the side, and locations along the top. I added a new SimpleTag to allow me to access values from a dict-within-a-dict, and the view produces this dict:
class VarListView(TemplateView):
template_name = "lbv-list.html"
def locations(self):
return Location.objects.all()
def locvars(self):
dict = {}
for v in LocationBasedVariable.objects.all():
dict.setdefault(v.location.id, {})
dict[v.location.id][v.definition.id] = v.value
return dict
def locvarnames(self):
return LocationVariableDefinition.objects.all()
and then in the template:
<table class="table table-striped table-condensed striped">
<thead>
<tr>
<th> </th>
{% for loc in view.locations %}
<th>{{ loc.name }}</th>
{% endfor %}
</tr>
</thead>
<tbody>
{% for lv in view.locvarnames %}
<tr>
<th>{{ lv.name }}</th>
{% for loc in view.locations %}
<td>{% get_dictitem2 view.locvars loc.id lv.id%}</td>
{% endfor %}
</tr>
{% endfor %}
</tbody>
</table>
Which all works, except that with 3 locations and 5 variables, I get 475 SQL queries, taking about 3 seconds to run on my laptop.
What can I do to improve this? I always thought that the queryset would be cached, but the SQL logged in the console seems to be fetching the same sets over and over. Is there a hint I can give Django that the result won't have changed, or a way to do each query only once?
Or should I be just passing a sorted queryset to the template, and figuring out the rows and columns in there? (the way I would normally do that would involve keeping state in the template for the last location to compare with the current, and I don't think I can do that in a template)

The usual approach would be to evaluate the methods once, and add the results to the template context in get_context_data.
class VarListView(TemplateView):
def get_context_data(self, **kwargs):
# Call the base implementation first to get a context
context = super(VarListView, self).get_context_data(**kwargs)
context['locations'] = self.locations()
context['locvarnames'] = self.locavarnames()
context['locvars'] = self.locvars()
return context
Then, in your template, remove the view prefix e.g. locations instead of view.locations.

Aside from changing .id to _id, you should also be looking at what you actually need to return, since your for loop only ever uses the id you could just use values to filter this to start with
query_gives_locations.values('id')
You have to keep getting .locvars every time so you can move this outside with with
{% with locvars=view.locvars %}
{% for loc in view.locations %}
<td>{% get_dictitem2 locvars loc_id lv_id%}</td>
{% endfor %}
{% endwith %}

You have a couple of options here, but I think the best is to use the _id property for your foreign key relations. Django stores the id of the foreign keys on the model in question, so in your case you could replace your loop with:
for v in LocationBasedVariable.objects.all():
dict.setdefault(v.location_id, {})
dict[v.location_id][v.definition_id] = v.value
This means that you won't have to query the related tables just to get the ID.
Also, it looks like you are invoking {% get_dictitem2 view.locvars loc.id lv.id%} for each item in view.locvarnames which in turn will go to the database, load all of your LocationBasedVariable and populate that dict.
I might be tempted to cache that in the view so you only populate it once:
class VarListView(TemplateView):
template_name = "lbv-list.html"
_location_based_variable_cache = None
def locations(self):
return Location.objects.all()
def locvars(self):
if not self._location_based_variable_cache:
self._location_based_variable_cache = {}
for v in LocationBasedVariable.objects.all():
self._location_based_variable_cache.setdefault(v.location_id, {})
self._location_based_variable_cache[v.location_id][v.definition_id] = v.value
return self._location_based_variable_cache
def locvarnames(self):
return LocationVariableDefinition.objects.all()

Related

Accessing items in a list through template tags in Django template tags

Template tag screenshot and possible solution options
First time, posting to stack-overflow. Please excuse if formatting is not ideal.
html
<tbody>
{% for list in todolist %}
<tr>
<td>
{{ list.name }}
</td>
<td>
{{ list.items.all|length }}
</td>
<td>
{% comment %}need this to be allCompleted[list.id - 1]
#allCompleted.0 works. allCompleted[0] or allCompleted['0'] does not.{% endcomment %}
{% if allCompleted == True %}
{{ allCompleted|getindex:list.id }}
Yes
{% else %}
No
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
Views.py:
class TodoListListView(ListView):
model = TodoList
context_object_name = "todolist"
template_name = "todos/list.html"
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
allCompletedList = []
for list in TodoList.objects.all():
allCompleted = True
for item in list.items.all():
if item.is_completed == False:
allCompleted = False
allCompletedList.append(allCompleted)
context['allCompleted'] = allCompletedList
print ('context: ', context)
return context
Models.py
class TodoList(models.Model):
name = models.CharField(max_length=100, blank=False)
created_on = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.name
class TodoItem(models.Model):
task = models.CharField(max_length=100)
due_date = models.DateTimeField(null=True, blank=True)
is_completed = models.BooleanField(default=False)
list = models.ForeignKey("Todolist", related_name="items", on_delete=models.CASCADE)
def __str__(self):
return self.task
When printing context:
I get 'allCompleted': [False, True]
This is accurate as I have some items in housing chores not completed but I triple checked to make sure all my coding projects are completed.
As seen from the HTML screenshot, I need something like:
{{ allCompleted[list.id - 1] }} to match with the corresponding list in each row.
But it seems Django does not like that. I've tried many combos like allCompleted['list.id-1']. Strangely, allCompleted.0 = False but allCompleted[0] gets a parse error. I have also tried to create a custom template tag in my app/templatetag folder under a file I made (getindex.py)
from django import template
from todos.models import TodoItem, TodoList
register = template.Library()
def getindex(lst, idx):
return lst[idx]
register.filter(getindex)
For my template tag, I did {{ allCompleted|getindex:list.id-1 }} and it says getindex is not a valid filter so maybe I am registering it incorrectly?
If there is no way to access allCompleted[list.id - 1], I thought of other solutions explained in my HTMl screenshot.
Instead of using template tags for this purpose, it is best to give to the template data in the form it can easily use. The most efficient way to do this would be to leave the task to the database itself and write a query that will give you allCompleted as a column in the result. You can use Exists() subqueries to do this:
from django.db.models import Exists, OuterRef
class TodoListListView(ListView):
model = TodoList
context_object_name = "todolist"
template_name = "todos/list.html"
def get_queryset(self):
queryset = super().get_queryset()
subquery = TodoItem.objects.filter(
list=OuterRef('pk'),
is_completed=False
)
queryset = queryset.annotate(all_completed=~Exists(subquery))
return queryset
Then in your template you can simply write {{ list.all_completed }}:
<tbody>
{% for list in todolist %}
<tr>
<td>
{{ list.name }}
</td>
<td>
{{ list.items.all|length }}
</td>
<td>
{% if list.all_completed == True %}
Yes
{% else %}
No
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>

Django: Add result from second queryset to first queryset

I have two models from two different databases (one read-only) without ForeignKey between the two models (did not get that working, as far i found it isn't possible).
In the main model I store the ID from the second model (read-only DB).
I want to display multiple records/rows on one view (like al table)
There for I want to get the content of the second model with the id from the main model.
and combine it to one row.
Normal you can get it by the ForeignKey but did won't work with 2 different databases.
What i got (simplified):
model.py
class Overeenkomst(models.Model):
park = models.IntegerField(blank=False, null=False, default='0')
object = models.IntegerField(blank=False, null=False, default='0') # ID from model second database
date_start = models.DateField()
date_end = models.DateField()
class Object(models.Model):
id = models.IntegerField(db_column='ID', primary_key=True) # Field name made lowercase.
nummer = models.IntegerField(db_column='NUMMER', blank=True, null=True) # Field name made lowercase.
omschrijving = models.CharField(db_column='OMSCHRIJVING', max_length=50, blank=True, null=True) # Field name made lowercase.
idobjectsoort = models.IntegerField(db_column='IDOBJECTSOORT', blank=True, null=True) # Field name made lowercase.
idobjecttype = models.IntegerField(db_column='IDOBJECTTYPE', blank=True, null=True) # Field name made lowercase.
(.....)
class Meta:
managed = False
db_table = 'OBJECT'
unique_together = (('nummer', 'idpark', 'id'), ('id', 'idpark', 'idobjecttype', 'idobjectsoort', 'dubbelboeking'), ('code', 'id'),)
def __str__(self):
return self.omschrijving
view.py
def ovk_overview(request):
ctx={}
overeenkomsten =models.Overeenkomst.objects.filter(park=request.session['park_id'])
ovk = []
for overeenkomst in overeenkomsten:
obj = models.Object.objects.using('database2').filter(pk=overeenkomst.object).values('omschrijving')
##### Here I Missing a part #####
ctx['overeenkomsten'] = ovk
return render(request, 'overeenkomsten/overzicht.html', context=ctx)
overzicht.html
{% extends "base.html" %}
{% load static %}
{% block content %}
<table class='table table-sm'>
<tr>
<th>#</th>
<th>Object</th>
<th>Start datum</th>
<th>Eind datum</th>
<th> </th>
</tr>
{% for ovk in overeenkomsten %}
{{ ovk }}::
<tr>
<td>{% now 'Y' %}{{ ovk.park }}{{ovk.object}}{{ovk.id}}</td>
<td>{{ ovk.object.omschrijving}}</td>
<td>{{ ovk.date_start|date:"d-m-Y" }}</td>
<td>{{ ovk.date_end|date:"d-m-Y" }}</td>
<td><button class="btn btn-primary">Download pdf</button></td>
</tr>
{% endfor %}
</table>
{% endblock %}
I have tried to use list() and chain() (as answer https://stackoverflow.com/a/8171434) but then i get only the values of the Object model and nothing from the Overeenkomsten model.
I hope someone has a answer/idea for me.
You could do something like this.
views.py
from django.forms.models import model_to_dict
def ovk_overview(request):
ctx={}
overeenkomsten =models.Overeenkomst.objects.filter(park=request.session['park_id'])
ovk = []
for overeenkomst in overeenkomsten:
overeenkomst_dict = model_to_dict(overeenkomst)
obj = models.Object.objects.using('database2').get(pk=overeenkomst.object) #Assumed there is always an obj. If not, change accordingly.
overeenkomst_dict['omschrijving'] = obj.omschrijving
ovk.append(overeenkomst_dict)
ctx['overeenkomsten'] = ovk
return render(request, 'overeenkomsten/overzicht.html', context=ctx)
overzicht.html
{% extends "base.html" %}
{% load static %}
{% block content %}
<table class='table table-sm'>
<tr>
<th>#</th>
<th>Object</th>
<th>Start datum</th>
<th>Eind datum</th>
<th> </th>
</tr>
{% for ovk in overeenkomsten %}
{{ ovk }}::
<tr>
<td>{% now 'Y' %}{{ ovk.park }}{{ovk.object}}{{ovk.id}}</td>
<td>{{ ovk.omschrijving }}</td>
<td>{{ ovk.date_start|date:"d-m-Y" }}</td>
<td>{{ ovk.date_end|date:"d-m-Y" }}</td>
<td><button class="btn btn-primary">Download pdf</button></td>
</tr>
{% endfor %}
</table>
{% endblock %}
First, a warning. Don't call fields object or id. They aren't Python reserved words, but using them overlays the meanings normally supplied by Python. It's confusing as hell, and in the case of object in particular, is also likely to cause you a world of pain at a later date if you start using Class-based views and Mixins. So call them something_id, or just obj.
OK. An idea ... yes. It depends on having enough memory to convert the main queryset into a list of objects. Then you "annotate" the objects in the first list, with the data from the corresponding objects in the second database.
I've replaced object with other_id in the following, because I simply couldn't think about it with the original name. It was like BLUE
printed in RED ink.
# query the first DB. You might want to chheck that the length
# of the query is sensible, and/or slice it with a maximum length
overeenkomsten = models.Overeenkomst.objects.filter(park=request.session['park_id'])
if overeenkomsten.count() > MAX_OBJECTS: # shouldn't ever happen
# do something to save our server!
overeenkomsten = overeenkomsten[:MAX_OBJECTS]
# fetch all the data
overeenkomsten = list( overeenkomsten )
# get the required data from the other DB.
# One query, retaining pk to tie the two together.
# avoids N queries on second DB
other_db_ids = [ x.other_id for x in overeenkomsten ] # was x.object
data_from_other_db = models.Object.objects.using('database2'
).filter(pk__in = other_db_ids
).values_list('pk', 'omschrijving'
)
# convert to a dict. This way for clarity and because I can't remember the dict method
#for converting a list of key/value pairs into a dict.
omschrivings = {}
for k,v in data_from_other_db:
omschrivings[k] = v
# "Annotate" the objects from the first query with the data from the second.
# It's read-only so no need to worry about saving it should somebody update it.
# (but you could subclass the save method if it wasn't)
for obj in overeenkomsten:
setattr( obj, 'omschriving', omschrivings[ obj[x.other_id] ] )
And in the template, just
<td>{{ ovk.omschrijving}}</td>

Django Filter Choice Field displays correct values but wont filter

I'm using the Django filter app: https://github.com/carltongibson/django-filter
I'm making a web-comic app and what I'm trying to accomplish is a drop-down filter field that displays the distinct values for that field. In this case, the series field.
After reading their docs. And stack overflow questions like this one or this one . I can make the filter form display the correct drop down values. But when I attempt to filter by them, my result queryset returns no values!
There must be something in the docs I'm not understanding. Here's my hopefully relevant code.
models.py
class ComicPanel(models.Model):
#... Other fields
series = models.CharField(max_length = 255, null = False, blank = False, default = getMainSeriesName)
# Other Filter Fields
chapter = models.FloatField(null = True, blank = True)
episode = models.IntegerField(null = True, blank = True)
class Meta:
constraints = [
models.UniqueConstraint(fields=['series','chapter', 'episode'])
]
def __str__(self):
return self.title
def save(self):
super(ComicPanel, self).save()
filters.py
import django_filters
from .models import ComicPanel
# Grab Distinct values for series
def getUniqueSeries():
series_dicts = ComicPanel.objects.all().values_list('series').distinct()
series_list = []
i = 0
for item in series_dicts:
series_list.append((i,item[0]))
i = i+1
return series_list
class ComicPanelFilter(django_filters.FilterSet):
series = django_filters.ChoiceFilter(choices = getUniqueSeries())
class Meta:
model = ComicPanel
fields = ['chapter', 'episode']
views.py
def view_archive(request):
comic_list = ComicPanel.objects.all()
comic_filter = ComicPanelFilter(request.GET, queryset=comic_list)
paginator = Paginator(comic_filter.qs, 8)
page = request.GET.get('page', 1)
try:
comics = paginator.page(page)
except (PageNotAnInteger, TypeError):
comics = paginator.page(1)
except EmptyPage:
comics = paginator.page(paginator.num_pages)
return render(request, 'comics/comic_archive.html', context = {'filter': comic_filter, 'comics': comics})
template.html
...
<form class="form" method="GET">
{% csrf_token %}
<table class="my_classes">
<thead>
<tr>
<th scope="col">Filter Comics: </th>
<th scope="col"></th>
</tr>
</thead>
<tbody>
{% for field in filter.form %}
<tr>
<th scope="row"> {{ field.name }}</th>
<td>{{ field }}</td>
</tr>
{% endfor %}
</tbody>
</table>
<button type="submit" class="btn btn-outline-dark">Filter</button>
</form>
...
{% if comics %}
{% for comic in comics %}
...
{% endfor %}
<!-- if no comics -->
{% else %}
<p class = "my_class"> Looks like you don't have any comics! Upload some? </p>
{% endif %}
Results:
My filter form displays the correct, distinct series for the drop down items. But When I filter by any value, the queryset returns no comics
I've also tried using the ModelChoiceFilter in my filters class with a similar result (but the values show up in a Tuple format):
series=django_filters.ModelChoiceFilter(queryset=ComicPanel.objects.all().values_list('series').distinct())
Can someone please tell me what I'm doing wrong?
My mistake was pretty silly actually,
If we look at the Django docs for field.choices:
The first element in each tuple is the actual value to be set on the model, and the second element is the human-readable name.
So this method:
# Grab Distinct values for series
def getUniqueSeries():
series_dicts = ComicPanel.objects.all().values_list('series').distinct()
series_list = []
i = 0
for item in series_dicts:
series_list.append((i,item[0]))
i = i+1
return series_list
returns a list that looks like: [(0,'series1'), (1, 'series2'), ...]
For my model, the first value of the tuple needs to be the series name.
So the list should look like:
[('series1','series1'), ('series2', 'series2'), ...]
So the simple fix is to change this line:
series_list.append((i,item[0]))
to this:
series_list.append((item[0],item[0]))

Remove leading zeros in Django

I'm displaying data from my MySQL database. I would like to remove the leading zeros in a certain fields(the ops and obp fields), only when the value to the left of the decimal point is a 0. If the digit(s) to the left of the decimal point is not 0, then I obviously want that displayed.
For instance,
0.750 becomes .750
1.000 stays as is
I suspect this can be done in when calling the data using django template or in the views.py. How would I do either one?
Any help on this would be greatly appreicated! Thank You.
views.py
from django.shortcuts import render
from .models import BattingRegStd
# Create your views here.
def batting(request):
battingregstd2018 = BattingRegStd.objects.filter(year=2018)
return render(request, 'playerstats/battingReg2018.html', {'battingregstd2018':battingregstd2018})
models.py
class BattingRegStd(models.Model):
id = models.IntegerField(db_column='ID', primary_key=True) # Field name made lowercase.
obp = models.FloatField(db_column='OBP', blank=True, null=True) # Field name made lowercase.
ops = models.FloatField(db_column='OPS', blank=True, null=True) # Field name made lowercase.
tb = models.IntegerField(db_column='TB', blank=True, null=True) # Field name made lowercase.
HTML
{% extends "base.html" %}
{% block contents %}
<div>
<table>
<thead>
<tr>
<th>OBP</th>
<th>OPS</th>
<th>TB</th>
</tr>
</thead>
<tbody>
{% for index in battingregstd2018%}
<td>{{ index.obp}}</td>
<td>{{ index.ops}}</td>
<td>{{ index.tb}}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}
You can use
def batting_avg_format(num):
numstr = str(num)
if numstr[0]!='0':
return numstr
else:
return numstr[1:]
Either incorporate it as a method on your BattingRegStd Model and apply it, or directly do a list comprehension after you have filtered the model but before your render your HTML. The advantage of incorporating it as a model method is that you can now call it from the template, as long as the only parameter in your method is self.
I assume that value is greater than zero.
views.py
def batting(request):
battingregstd2018 = BattingRegStd.objects.filter(year=2018)
for i in range(0, len(battingregstd2018)):
# if need to edit value
if battingregstd2018[i].obp is not None and battingregstd2018[i].obp < 1:
# data edit as string
# data type does not matter in template
battingregstd2018[i].obp = str(battingregstd2018[i].obp)[1:]
if battingregstd2018[i].ops is not None and battingregstd2018[i].ops < 1:
battingregstd2018[i].ops = str(battingregstd2018[i].ops)[1:]
return render(request, 'playerstats/battingReg2018.html', {'battingregstd2018':battingregstd2018})
than second attempt,
let's just fix template this time. use the original view.py
{% for index in battingregstd2018%}
<td>
{% if index.obp is None %}
-
{% elif index.obp < 1 %}
.{{index.obp|slugify|slice:"1:"}}
{% else %}
{{index.obp}}
{% endif %}
</td>
{% endfor %}
.{{index.obp|slugify|slice:"1:"}}
|slice can only be used in string data type.
|slugify converts to ASCII for use |slice. but decimal point is deleted.
so i added . to the front.
ps)
maybe, float type will store 0.750 as 0.75. if you want to 0.750 than use it .{{index.obp|floatformat:"3"|slugify|slice:"1:"}}
Examples you can confirm in a Python shell, tested with Python 3.5.3:
x = "0.750"
x.lstrip("0")
> '.750'
# Included because a normal floating point type won't print trailing zeroes:
from decimal import Decimal
x = Decimal("0.750")
x.to_eng_string().lstrip("0")
> '.750'
str(x).lstrip("0")
> '.750'
While you won't be able to do this in Django templates unless you've swapped out the engine for jinja2 or something else that allows parens in method calls, you can either do this in the view (possibly ugly depending on how you're pulling this data) or in a simple custom template filter.

Iterate over model instance field names and values in template

I'm trying to create a basic template to display the selected instance's field values, along with their names. Think of it as just a standard output of the values of that instance in table format, with the field name (verbose_name specifically if specified on the field) in the first column and the value of that field in the second column.
For example, let's say we have the following model definition:
class Client(Model):
name = CharField(max_length=150)
email = EmailField(max_length=100, verbose_name="E-mail")
I would want it to be output in the template like so (assume an instance with the given values):
Field Name Field Value
---------- -----------
Name Wayne Koorts
E-mail waynes#email.com
What I'm trying to achieve is being able to pass an instance of the model to a template and be able to iterate over it dynamically in the template, something like this:
<table>
{% for field in fields %}
<tr>
<td>{{ field.name }}</td>
<td>{{ field.value }}</td>
</tr>
{% endfor %}
</table>
Is there a neat, "Django-approved" way to do this? It seems like a very common task, and I will need to do it often for this particular project.
model._meta.get_all_field_names() will give you all the model's field names, then you can use model._meta.get_field() to work your way to the verbose name, and getattr(model_instance, 'field_name') to get the value from the model.
NOTE: model._meta.get_all_field_names() is deprecated in django 1.9. Instead use model._meta.get_fields() to get the model's fields and field.name to get each field name.
You can use Django's to-python queryset serializer.
Just put the following code in your view:
from django.core import serializers
data = serializers.serialize( "python", SomeModel.objects.all() )
And then in the template:
{% for instance in data %}
{% for field, value in instance.fields.items %}
{{ field }}: {{ value }}
{% endfor %}
{% endfor %}
Its great advantage is the fact that it handles relation fields.
For the subset of fields try:
data = serializers.serialize('python', SomeModel.objects.all(), fields=('name','size'))
Finally found a good solution to this on the dev mailing list:
In the view add:
from django.forms.models import model_to_dict
def show(request, object_id):
object = FooForm(data=model_to_dict(Foo.objects.get(pk=object_id)))
return render_to_response('foo/foo_detail.html', {'object': object})
in the template add:
{% for field in object %}
<li><b>{{ field.label }}:</b> {{ field.data }}</li>
{% endfor %}
Here's another approach using a model method. This version resolves picklist/choice fields, skips empty fields, and lets you exclude specific fields.
def get_all_fields(self):
"""Returns a list of all field names on the instance."""
fields = []
for f in self._meta.fields:
fname = f.name
# resolve picklists/choices, with get_xyz_display() function
get_choice = 'get_'+fname+'_display'
if hasattr(self, get_choice):
value = getattr(self, get_choice)()
else:
try:
value = getattr(self, fname)
except AttributeError:
value = None
# only display fields with values and skip some fields entirely
if f.editable and value and f.name not in ('id', 'status', 'workshop', 'user', 'complete') :
fields.append(
{
'label':f.verbose_name,
'name':f.name,
'value':value,
}
)
return fields
Then in your template:
{% for f in app.get_all_fields %}
<dt>{{f.label|capfirst}}</dt>
<dd>
{{f.value|escape|urlize|linebreaks}}
</dd>
{% endfor %}
In light of Django 1.8's release (and the formalization of the Model _meta API, I figured I would update this with a more recent answer.
Assuming the same model:
class Client(Model):
name = CharField(max_length=150)
email = EmailField(max_length=100, verbose_name="E-mail")
Django <= 1.7
fields = [(f.verbose_name, f.name) for f in Client._meta.fields]
>>> fields
[(u'ID', u'id'), (u'name', u'name'), (u'E-mail', u'email')]
Django 1.8+ (formalized Model _meta API)
Changed in Django 1.8:
The Model _meta API has always existed as a Django internal, but wasn’t formally documented and supported. As part of the effort to make this API public, some of the already existing API entry points have changed slightly. A migration guide has been provided to assist in converting your code to use the new, official API.
In the below example, we will utilize the formalized method for retrieving all field instances of a model via Client._meta.get_fields():
fields = [(f.verbose_name, f.name) for f in Client._meta.get_fields()]
>>> fields
[(u'ID', u'id'), (u'name', u'name'), (u'E-mail', u'email')]
Actually, it has been brought to my attention that the above is slightly overboard for what was needed (I agree!). Simple is better than complex. I am leaving the above for reference. However, to display in the template, the best method would be to use a ModelForm and pass in an instance. You can iterate over the form (equivalent of iterating over each of the form's fields) and use the label attribute to retrieve the verbose_name of the model field, and use the value method to retrieve the value:
from django.forms import ModelForm
from django.shortcuts import get_object_or_404, render
from .models import Client
def my_view(request, pk):
instance = get_object_or_404(Client, pk=pk)
class ClientForm(ModelForm):
class Meta:
model = Client
fields = ('name', 'email')
form = ClientForm(instance=instance)
return render(
request,
template_name='template.html',
{'form': form}
)
Now, we render the fields in the template:
<table>
<thead>
{% for field in form %}
<th>{{ field.label }}</th>
{% endfor %}
</thead>
<tbody>
<tr>
{% for field in form %}
<td>{{ field.value|default_if_none:'' }}</td>
{% endfor %}
</tr>
</tbody>
</table>
Ok, I know this is a bit late, but since I stumbled upon this before finding the correct answer so might someone else.
From the django docs:
# This list contains a Blog object.
>>> Blog.objects.filter(name__startswith='Beatles')
[<Blog: Beatles Blog>]
# This list contains a dictionary.
>>> Blog.objects.filter(name__startswith='Beatles').values()
[{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}]
You can use the values() method of a queryset, which returns a dictionary. Further, this method accepts a list of fields to subset on. The values() method will not work with get(), so you must use filter() (refer to the QuerySet API).
In view...
def show(request, object_id):
object = Foo.objects.filter(id=object_id).values()[0]
return render_to_response('detail.html', {'object': object})
In detail.html...
<ul>
{% for key, value in object.items %}
<li><b>{{ key }}:</b> {{ value }}</li>
{% endfor %}
</ul>
For a collection of instances returned by filter:
object = Foo.objects.filter(id=object_id).values() # no [0]
In detail.html...
{% for instance in object %}
<h1>{{ instance.id }}</h1>
<ul>
{% for key, value in instance.items %}
<li><b>{{ key }}:</b> {{ value }}</li>
{% endfor %}
</ul>
{% endfor %}
I used https://stackoverflow.com/a/3431104/2022534 but replaced Django's model_to_dict() with this to be able to handle ForeignKey:
def model_to_dict(instance):
data = {}
for field in instance._meta.fields:
data[field.name] = field.value_from_object(instance)
if isinstance(field, ForeignKey):
data[field.name] = field.rel.to.objects.get(pk=data[field.name])
return data
Please note that I have simplified it quite a bit by removing the parts of the original I didn't need. You might want to put those back.
You can have a form do the work for you.
def my_model_view(request, mymodel_id):
class MyModelForm(forms.ModelForm):
class Meta:
model = MyModel
model = get_object_or_404(MyModel, pk=mymodel_id)
form = MyModelForm(instance=model)
return render(request, 'model.html', { 'form': form})
Then in the template:
<table>
{% for field in form %}
<tr>
<td>{{ field.name }}</td>
<td>{{ field.value }}</td>
</tr>
{% endfor %}
</table>
Below is mine, inspired by shacker's get_all_fields.
It gets a dict of one model instance, if encounter relation field, then asign the field value a dict recursively.
def to_dict(obj, exclude=[]):
"""生成一个 dict, 递归包含一个 model instance 数据.
"""
tree = {}
for field in obj._meta.fields + obj._meta.many_to_many:
if field.name in exclude or \
'%s.%s' % (type(obj).__name__, field.name) in exclude:
continue
try :
value = getattr(obj, field.name)
except obj.DoesNotExist:
value = None
if type(field) in [ForeignKey, OneToOneField]:
tree[field.name] = to_dict(value, exclude=exclude)
elif isinstance(field, ManyToManyField):
vs = []
for v in value.all():
vs.append(to_dict(v, exclude=exclude))
tree[field.name] = vs
elif isinstance(field, DateTimeField):
tree[field.name] = str(value)
elif isinstance(field, FileField):
tree[field.name] = {'url': value.url}
else:
tree[field.name] = value
return tree
This function is mainly used to dump a model instance to json data:
def to_json(self):
tree = to_dict(self, exclude=('id', 'User.password'))
return json.dumps(tree, ensure_ascii=False)
There should really be a built-in way to do this. I wrote this utility build_pretty_data_view that takes a model object and form instance (a form based on your model) and returns a SortedDict.
Benefits to this solution include:
It preserves order using Django's built-in SortedDict.
When tries to get the label/verbose_name, but falls back to the field name if one is not defined.
It will also optionally take an exclude() list of field names to exclude certain fields.
If your form class includes a Meta: exclude(), but you still want to return the values, then add those fields to the optional append() list.
To use this solution, first add this file/function somewhere, then import it into your views.py.
utils.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ai ts=4 sts=4 et sw=4
from django.utils.datastructures import SortedDict
def build_pretty_data_view(form_instance, model_object, exclude=(), append=()):
i=0
sd=SortedDict()
for j in append:
try:
sdvalue={'label':j.capitalize(),
'fieldvalue':model_object.__getattribute__(j)}
sd.insert(i, j, sdvalue)
i+=1
except(AttributeError):
pass
for k,v in form_instance.fields.items():
sdvalue={'label':"", 'fieldvalue':""}
if not exclude.__contains__(k):
if v.label is not None:
sdvalue = {'label':v.label,
'fieldvalue': model_object.__getattribute__(k)}
else:
sdvalue = {'label':k,
'fieldvalue': model_object.__getattribute__(k)}
sd.insert(i, k, sdvalue)
i+=1
return sd
So now in your views.py you might do something like this
from django.shortcuts import render_to_response
from django.template import RequestContext
from utils import build_pretty_data_view
from models import Blog
from forms import BlogForm
.
.
def my_view(request):
b=Blog.objects.get(pk=1)
bf=BlogForm(instance=b)
data=build_pretty_data_view(form_instance=bf, model_object=b,
exclude=('number_of_comments', 'number_of_likes'),
append=('user',))
return render_to_response('my-template.html',
RequestContext(request,
{'data':data,}))
Now in your my-template.html template you can iterate over the data like so...
{% for field,value in data.items %}
<p>{{ field }} : {{value.label}}: {{value.fieldvalue}}</p>
{% endfor %}
Good Luck. Hope this helps someone!
Instead of editing every model I would recommend to write one template tag which will return all field of any model given.
Every object has list of fields ._meta.fields.
Every field object has attribute name that will return it's name and method value_to_string() that supplied with your model object will return its value.
The rest is as simple as it's said in Django documentation.
Here is my example how this templatetag might look like:
from django.conf import settings
from django import template
if not getattr(settings, 'DEBUG', False):
raise template.TemplateSyntaxError('get_fields is available only when DEBUG = True')
register = template.Library()
class GetFieldsNode(template.Node):
def __init__(self, object, context_name=None):
self.object = template.Variable(object)
self.context_name = context_name
def render(self, context):
object = self.object.resolve(context)
fields = [(field.name, field.value_to_string(object)) for field in object._meta.fields]
if self.context_name:
context[self.context_name] = fields
return ''
else:
return fields
#register.tag
def get_fields(parser, token):
bits = token.split_contents()
if len(bits) == 4 and bits[2] == 'as':
return GetFieldsNode(bits[1], context_name=bits[3])
elif len(bits) == 2:
return GetFieldsNode(bits[1])
else:
raise template.TemplateSyntaxError("get_fields expects a syntax of "
"{% get_fields <object> [as <context_name>] %}")
Yeah it's not pretty, you'll have to make your own wrapper. Take a look at builtin databrowse app, which has all the functionality you need really.
This may be considered a hack but I've done this before using modelform_factory to turn a model instance into a form.
The Form class has a lot more information inside that's super easy to iterate over and it will serve the same purpose at the expense of slightly more overhead. If your set sizes are relatively small I think the performance impact would be negligible.
The one advantage besides convenience of course is that you can easily turn the table into an editable datagrid at a later date.
I've come up with the following method, which works for me because in every case the model will have a ModelForm associated with it.
def GetModelData(form, fields):
"""
Extract data from the bound form model instance and return a
dictionary that is easily usable in templates with the actual
field verbose name as the label, e.g.
model_data{"Address line 1": "32 Memory lane",
"Address line 2": "Brainville",
"Phone": "0212378492"}
This way, the template has an ordered list that can be easily
presented in tabular form.
"""
model_data = {}
for field in fields:
model_data[form[field].label] = eval("form.data.%s" % form[field].name)
return model_data
#login_required
def clients_view(request, client_id):
client = Client.objects.get(id=client_id)
form = AddClientForm(client)
fields = ("address1", "address2", "address3", "address4",
"phone", "fax", "mobile", "email")
model_data = GetModelData(form, fields)
template_vars = RequestContext(request,
{
"client": client,
"model_data": model_data
}
)
return render_to_response("clients-view.html", template_vars)
Here is an extract from the template I am using for this particular view:
<table class="client-view">
<tbody>
{% for field, value in model_data.items %}
<tr>
<td class="field-name">{{ field }}</td><td>{{ value }}</td>
</tr>
{% endfor %}
</tbody>
</table>
The nice thing about this method is that I can choose on a template-by-template basis the order in which I would like to display the field labels, using the tuple passed in to GetModelData and specifying the field names. This also allows me to exclude certain fields (e.g. a User foreign key) as only the field names passed in via the tuple are built into the final dictionary.
I'm not going to accept this as the answer because I'm sure someone can come up with something more "Djangonic" :-)
Update: I'm choosing this as the final answer because it is the simplest out of those given that does what I need. Thanks to everyone who contributed answers.
Django 1.7 solution for me:
There variables are exact to the question, but you should definitely be able to dissect this example
The key here is to pretty much use the .__dict__ of the model
views.py:
def display_specific(request, key):
context = {
'question_id':question_id,
'client':Client.objects.get(pk=key).__dict__,
}
return render(request, "general_household/view_specific.html", context)
template:
{% for field in gen_house %}
{% if field != '_state' %}
{{ gen_house|getattribute:field }}
{% endif %}
{% endfor %}
in the template I used a filter to access the field in the dict
filters.py:
#register.filter(name='getattribute')
def getattribute(value, arg):
if value is None or arg is None:
return ""
try:
return value[arg]
except KeyError:
return ""
except TypeError:
return ""
I'm using this, https://github.com/miracle2k/django-tables.
<table>
<tr>
{% for column in table.columns %}
<th>{{ column }}</th>
{% endfor %}
</tr>
{% for row in table.rows %}
<tr>
{% for value in row %}
<td>{{ value }}</td>
{% endfor %}
</tr>
{% endfor %}
</table>
This approach shows how to use a class like django's ModelForm and a template tag like {{ form.as_table }}, but have all the table look like data output, not a form.
The first step was to subclass django's TextInput widget:
from django import forms
from django.utils.safestring import mark_safe
from django.forms.util import flatatt
class PlainText(forms.TextInput):
def render(self, name, value, attrs=None):
if value is None:
value = ''
final_attrs = self.build_attrs(attrs)
return mark_safe(u'<p %s>%s</p>' % (flatatt(final_attrs),value))
Then I subclassed django's ModelForm to swap out the default widgets for readonly versions:
from django.forms import ModelForm
class ReadOnlyModelForm(ModelForm):
def __init__(self,*args,**kwrds):
super(ReadOnlyModelForm,self).__init__(*args,**kwrds)
for field in self.fields:
if isinstance(self.fields[field].widget,forms.TextInput) or \
isinstance(self.fields[field].widget,forms.Textarea):
self.fields[field].widget=PlainText()
elif isinstance(self.fields[field].widget,forms.CheckboxInput):
self.fields[field].widget.attrs['disabled']="disabled"
Those were the only widgets I needed. But it should not be difficult to extend this idea to other widgets.
Just an edit of #wonder
def to_dict(obj, exclude=[]):
tree = {}
for field in obj._meta.fields + obj._meta.many_to_many:
if field.name in exclude or \
'%s.%s' % (type(obj).__name__, field.name) in exclude:
continue
try :
value = getattr(obj, field.name)
except obj.DoesNotExist as e:
value = None
except ObjectDoesNotExist as e:
value = None
continue
if type(field) in [ForeignKey, OneToOneField]:
tree[field.name] = to_dict(value, exclude=exclude)
elif isinstance(field, ManyToManyField):
vs = []
for v in value.all():
vs.append(to_dict(v, exclude=exclude))
tree[field.name] = vs
else:
tree[field.name] = obj.serializable_value(field.name)
return tree
Let Django handle all the other fields other than the related fields. I feel that is more stable
Take a look at django-etc application. It has model_field_verbose_name template tag to get field verbose name from templates: http://django-etc.rtfd.org/en/latest/models.html#model-field-template-tags
I just tested something like this in shell and seems to do it's job:
my_object_mapped = {attr.name: str(getattr(my_object, attr.name)) for attr in MyModel._meta.fields}
Note that if you want str() representation for foreign objects you should define it in their str method. From that you have dict of values for object. Then you can render some kind of template or whatever.
Django >= 2.0
Add get_fields() to your models.py:
class Client(Model):
name = CharField(max_length=150)
email = EmailField(max_length=100, verbose_name="E-mail")
def get_fields(self):
return [(field.verbose_name, field.value_from_object(self)) for field in self.__class__._meta.fields]
Then call it as object.get_fields on your template.html:
<table>
{% for label, value in object.get_fields %}
<tr>
<td>{{ label }}</td>
<td>{{ value }}</td>
</tr>
{% endfor %}
</table>
If you model name is Client and you are getting client object by id then proceed like this
client = Client.objects.get(id=id)
fields = Client._meta.get_fields()
for field in fields:
value = getattr(client, field.name)
print(field.name)
print(value)
<table border='1'>
<tr>
{% for mfild in fields%}
<td>{{mfild}}</td>
{% endfor%}
</tr>
{%for v in records%}
<tr>
<td>{{v.id}}</td>
<td>{{v.title}}</td>
<td class="">{{v.desc}}</td>
</tr>
{% endfor%}
</table>
enter code here

Categories

Resources