I'm trying to learn django so while I have a current solution I'm not sure if it follows best practices in django. I would like to display information from a web api on my website. Let's say the api url is as follows:
http://api.example.com/books?author=edwards&year=2009
Thsis would return a list of books by Edwards written in the year 2009. Returned in the following format:
{'results':
[
{
'title':'Book 1',
'Author':'Edwards Man',
'Year':2009
},
{
'title':'Book 2',
'Author':'Edwards Man',
'Year':2009}
]
}
Currently I am consuming the API in my views file as follows:
class BooksPage(generic.TemplateView):
def get(self,request):
r = requests.get('http://api.example.com/books?author=edwards&year=2009')
books = r.json()
books_list = {'books':books['results']}
return render(request,'books.html',books_list)
Normally, we grab data from the database in the models.py file, but I am unsure if I should be grabbing this API data in models.py or views.py. If it should be in models.py, can someone provide an example of how to do this? I wrote the above example sepecifically for stackoverflow, so any bugs are purely a result of writing it here.
I like the approach of putting that kind of logic in a separate service layer (services.py); the data you are rendering is quite not a "model" in the Django ORM sense, and it's more than simple "view" logic. A clean encapsulation ensures you can do things like control the interface to the backing service (i.e., make it look like a Python API vs. URL with parameters), add enhancements such as caching, as #sobolevn mentioned, test the API in isolation, etc.
So I'd suggest a simple services.py, that looks something like this:
def get_books(year, author):
url = 'http://api.example.com/books'
params = {'year': year, 'author': author}
r = requests.get(url, params=params)
books = r.json()
books_list = {'books':books['results']}
return books_list
Note how the parameters get passed (using a capability of the requests package).
Then in views.py:
import services
class BooksPage(generic.TemplateView):
def get(self,request):
books_list = services.get_books('2009', 'edwards')
return render(request,'books.html',books_list)
See also:
Separation of business logic and data access in django
Use the serializer instead of .json, as it gives flexibility to return in a number of formats.As while using rest-api , the provided serializer use is preferred.
Also keep the data handling and get data requests in view.py.The forms are used for templating not as the business logic.
Well, there are several things to keep in mind. First of all, in this case your data is not changing so often. So it is a good practice to cache this kind of responces. There are a lot of caching tools around, but redis is a popular option. Alternatevly, you can choose additional NoSQL database just for caching.
Secondly, what is the purpose of displaying this data? Are you expecting your users to interact with books or authors, etc? If it is just an information, there is no need in forms and models. If not, you must provide proper views, forms and models for both books and authors, etc.
And considering the place where you should call an API request, I would say it dependes heavily on the second question. Choices are:
views.py for just displaying data.
forms.py or still views.py for ineractivity.
<tbody>
{% if libros %}
{% for libro in libros %}
<tr>
<td>{{ libro.id }}</td>
<td>{{ libro.titulo }}</td>
<td>{{ libro.autor }}</td>
<td>{{ libro.eiditorial }}</td>
<td>{{ libro.descripcion }}</td>
<td>{{ libro.cantidad }}</td>
<td>{{ libro.Bodega.nombre }}</td>
<td>
{% if libro.imagen %}
<td><img src= "{{ libro.imagen.url }} "align="center" width="50px" ></td>
{% else %}
<h6>no hay imagen de libros</h6>
{% endif %}
</td>
<td><a class ="btn btn-primary "href="/">Editar</a></td>
<td><a class ="btn btn-danger red"href="/">Eliminar</a></td>
</tr>
{% endfor %}
{% else %}
<h1>no hay registros de libros</h1>
{% endif%}
</tbody>
as I can call it in an html when I already have a list of my local application greetings
Related
This bounty has ended. Answers to this question are eligible for a +50 reputation bounty. Bounty grace period ends in 20 hours.
Joao Daniel is looking for a canonical answer:
Please I need based on my detailed question one which I can test and implement
Currently I have this code that shows me the normal data table but if I have millions of records it becomes very slow, I understand that serverside of datatable can be used but I don't know how I can implement it
list.html
<table class="table table-striped table-hover responsive" style="width:100%" id="buyer">
<thead>
<th>#id</th>
<th>Fecha</th>
<th>Cod Cliente</th>
<th>Cliente</th>
<th>Dirección</th>
<th>Comentario</th>
<th>Tipo Retiro</th>
<th>Total</th>
<th>Guias a Generar</th>
</thead>
<tbody>
{% for item in ResulOrdenes %}
<tr>
<td>{{item.DOCNUM}}</td>
<td>{{item.DOC_DATE|date:"Y-m-d"}}</td>
<td>{{ item.CARDCODE }}</td>
<td>{{ item.CARDNAME }}</td>
<td>{{ item.ADDRESS }}</td>
<td>{{ item.COMMENTS }}</td>
<td>{{ item.URETIRO }}</td>
<td>{{ item.DOCTOTAL }}</td>
<td>{{ item.NGUIAS }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}
{% block content_wrapper %}
<p></p>
{% endblock content_wrapper %}
{% block js_page %}
<script>
$(document).ready(function() {
$('.table').DataTable({
dom: 'Bfrtip',
});
</script>
{% endblock %}
view.py
def Listado_guia(request):
template_name='app_listguia/lista.html'
conn = dbapi.connect(..)
cursorSol2 = conn.cursor()
sql_command = 'select a."DocNum" as "DOCNUM", a."DocDate" as "DOC_DATE", a."CardCode" as "CARDCODE", a."CardName" as "CARDNAME", a."DocTotal" as "DOCTOTAL", CEILING(a."DocTotal"/20000) as "NGUIAS", a."DocStatus" as "DOCSTATUS", a."Address" as "ADDRESS", a."Address2" as "ADDRESS2", a."Comments" as "COMMENTS", a."U_Retiro_mercaderia" as "URETIRO" '
sql_command = sql_command + 'from "'+ connections.databases[ConfigBaseHana]["NAME"] +'"."ODLN" as a '
cursorSol2.execute(sql_command)
RES_ORDENES = cursorSol2.fetchall()
cursorSol2.close()
conn.close()
dnow = today.strftime("%Y-%m-%d")
contexto = { "ResulOrdenes":RES_ORDENES, "dnow":dnow }
return render(request,template_name,contexto)
urls.py
from django.urls import path
from django.contrib.auth import views as auth_views
from app_guia.views import *
urlpatterns = [
path('guiaList',Listado_guia,name='urlListGuia_list'),
]
Can you please give me some light on how to proceed? Thanks
The approach you are using is not suitable for millions of records. First of all, if this is your complete sql, you should stick with normal Django QuerySets and switch your view to use a (generic) class-based view. This gives you a proper paginator, which will reduce the load on the database, django and the resulting html significantly.
If you do not want to have pagination, but rather a continuous list, you need a javascript component to virtually render your table. This way, only a few hundred DOM nodes are present in the clients javascript, and will be populated from server, on the fly, while scrolling. The term to search for is "virtualized table", an examplatory candidate would be react-window.
On a sidenote: if the code shown is your "real" production code (and not a boiled-down demo version for this post), I highly recommend working through a django tutorial, e.g. here or here, and sticking to the default django way as documented in the djangoproject as much as possible. You will save yourself hours of programming and huge problems with future enhancements.
To improve the performance of your data table, you can use the server-side processing mode of the DataTables library.
Add the serverSide option to your DataTable initialization code and
set it to true:
$(document).ready(function() {
$('.table').DataTable({
serverSide: true,
processing: true,
ajax: '/guiaList',
dom: 'Bfrtip',
});
});
In your view function, you need to handle the AJAX request and return
the data in the expected format. You can use the JsonResponse class
from Django to return the data in JSON format
from django.http import JsonResponse
from django.core.paginator import Paginator
def Listado_guia(request):
draw = int(request.GET.get('draw', 1))
start = int(request.GET.get('start', 0))
length = int(request.GET.get('length', 10))
conn = dbapi.connect(...)
cursorSol2 = conn.cursor()
sql_command = 'SELECT ...'
cursorSol2.execute(sql_command)
total_items = cursorSol2.rowcount
results = cursorSol2.fetchall()
cursorSol2.close()
conn.close()
paginator = Paginator(results, length)
page_results = paginator.get_page(int(start/length)+1)
data = []
for item in page_results:
data.append({
'DOCNUM': item[0],
'DOC_DATE': item[1].strftime("%Y-%m-%d"),
'CARDCODE': item[2],
'CARDNAME': item[3],
'DOCTOTAL': item[4],
'NGUIAS': item[5],
'DOCSTATUS': item[6],
'ADDRESS': item[7],
'ADDRESS2': item[8],
'COMMENTS': item[9],
'URETIRO': item[10]
})
response = {
'draw': draw,
'recordsTotal': total_items,
'recordsFiltered': total_items,
'data': data
}
return JsonResponse(response)
The start, length, and draw parameters are sent automatically by DataTables to the server when requesting data. The start parameter indicates the start index of the current page, the length parameter indicates the number of rows to display on the current page, and the draw parameter is used to ensure that the response matches the current state of the table.
The JsonResponse returns the data in the format expected by DataTables, which includes the draw parameter, the total number of records (recordsTotal), the number of filtered records (recordsFiltered), and the data array.
I have created a simple Django application to display individual articles. These articles have a number of fields that users can edit. I am using the package 'django-auditlog' to log changes to these article models. So far, I have simply followed the auditlog installation doc to setup model history tracking (as well as enabling the middleware to allow 'actor_id' to be tracked). I have also added the example code that displays the most recent changes on the individual model pages as such:
<!-- History display -->
<div class="table-responsive">
<table id="history" class="table table-striped table-bordered">
<thead>
<tr>
<th>Actor</th>
<th>Field</th>
<th>From</th>
<th>To</th>
</tr>
</thead>
<tbody>
<!-- Human readable - change to '.changes_dict.' for proper logs -->
{% for key, value in article.history.latest.changes_display_dict.items %}
<tr>
<td>{{ article.history.latest.author_id }}</td>
<td>{{ key }}</td>
<td>{{ value.0|default:"None"|striptags|safe }}</td>
<td>{{ value.1|default:"None"|striptags|safe }}</td>
</tr>
{% empty %}
<p>No history for this item has been logged yet.</p>
{% endfor %}
</tbody>
</table>
</div>
As my code may suggest, I am trying to add an additional column to the history table to show who made the changes that are being displayed.
Is there an easy way to do this through auditlog, or will I have to create some kind of sql query to my sqlite auditlog db table to retrieve the 'author_id' field?
Thank you!
I figured out the answer after looking through the models file for Django AuditLog. It is not possible to pull the actor out directly from the history field of the model if you have created the history field using the AuditlogHistoryField() method as described in the django-auditlog tutorial.
Instead, I did the following:
In the views.py file
from auditlog.models import LogEntry
...
dal_log = LogEntry.objects.get_for_object(article)
...
context = {'article': article, 'logs': dal_log}
return render(request, "detail.html", context)
Then in my template, I was able to work with the log entries for the specified object (in my case, these were 'article' models). There may be a cleaner way, but this worked for me.
I'm trying to use the Reddit API via PRAW (https://praw.readthedocs.io/en/stable/) with Django, and am thinking of trying to use functools lru_cache decorator to implement some kind of caching so I can cache the results of similar API calls to reduce overall calls being made. I've never done anything like this so I've been mainly following examples of implementing the #lru_cache decorator.
I have 3 files that are primarily involved with the API calls / display here. I have:
account.html
{% extends 'myapp/base.html' %}
<h1> {{ name }} </h1>
<h3> {{ comment_karma }} </h3>
<h5> Top Posts </h5>
<table>
<tr>
<th> Post </th>
<th> Subreddit </th>
</tr>
{% for s in top_submissions %}
<tr>
<td> {{ s.title }} </td>
<td> {{ s.subreddit }} </td>
</tr>
{% endfor %}
</table>
views.py
from . import reddit
reddit = reddit.Reddit()
def account_page():
context = reddit.details(request.user.reddit_username)
return render(request, 'stats/dashboard.html', context)
reddit.py
from functools import lru_cache
class Reddit:
def __init__(self):
self.praw = praw.Reddit(
client_id = CLIENT_ID,
client_secret = CLIENT_SECRET,
user_agent = USER_AGENT
)
#lru_cache(maxsize = 256)
def details(self, redditor):
redditor = self.praw.redditor(redditor)
overview = {
'name': redditor.name,
'comment_karma': redditor.comment_karma,
'top_submissions': redditor.submissions.top(limit=10),
}
return overview
Here's the problem: when I don't have the lru_cache, then everything works fine and all the data comes in as always. However, when I do put the lru_cache option, then only the name and comment_karma come, while the submissions (an iterable list) just does not display on my page (so I assume it doesn't end up having any values).
Am I using lru_cache wrong? Essentially, my goal is if a redditor is passed into the function overview, I don't want to keep making the same API calls over and over, but instead want to put it into the cache and pull that same data if it's in the cache.
PRAW returns generator objects that are lazily evaluated. You want to evaluate them inside your cached function. Otherwise, after the generator is exhausted, you can't get the results again.
So the working version should look like this:
#lru_cache(maxsize = 256)
def details(self, redditor):
redditor = self.praw.redditor(redditor)
overview = {
'name': redditor.name,
'comment_karma': redditor.comment_karma,
'top_submissions': list(redditor.submissions.top(limit=10)),
}
return overview
list(redditor.submissions.top(limit=10)) will consume the generator and the cached result will contain the list, rather than the generator object which can only be consumed once.
I currently try to figure out how to display multiple documents from my MongoDB collection "one by one". My recent code to fetch the data looks like this:
#app.route('/shops', methods=['GET', 'POST'])
#login_required
def certified_shops():
shopdata = col.find({ "go": True})
return render_template("tables.html", shopdata=shopdata)
My HTML Jinja injection like this:
<tr>
<td>
{% for data in shopdata: %}
{{ data.url_name }}
{% endfor %}
</td>
</tr>
The code itself works fine - but literally all URLs are displayed in one row of course.
Right now I can't find a good way to handle this.
I just tried to displaying the data with a different loop or some "in range", but this doesn't worked so far. Everything i found here or in the documentation for mongodb and jinja didn't went well for me.
Might there be an easy trick with my html handling or the db query to display one url for each td/tr easy and flexible ?
Since the collection will grow, the list should automatically expand in my web application.
Would love to here some ideas since I've been racking my brain for days now without any good solutions and I've been thinking that there must be an easy way and I just can't see it because it's that simple.
Did I understand your question well? This will create a new row for every url_name:
{% for data in shopdata %}
<tr>
<td>
{{ data.url_name }}
</td>
</tr>
{% endfor %}
Notice. No colon (:) used in the jinja for loop after 'shopdata'.
I have a template which has a code like
<td>{{ contact.Tag_ID }}</td>
to list the Tag_ID of items in a table. Now when I click on ot it goes to as os page which is a form with the Tag_ID field. I some how want to catch the Tag_ID when I click on it and pass it to the view ans set it as the initial value. I know how to set the initial value but cannot figure how to catch the Tag_ID or its associated data.
A simple solution would be to pass the value in the url:
<td>{{ contact.Tag_ID }}</td>
or
<td>{{ contact.Tag_ID }}</td>
In the second case, it is just an old-style query string, which you can use javascript to read. But, the better (django) way would be to use the first method, and do a pretty url.
In urls.py:
(r'^os/(?P<tag_id>[a-zA-Z0-9]*)/*', 'os_view_handler'),
Then in views.py:
def os_view_handler(request, tag_id=None):
...
url code:
<a class="item" href="{% url "your_views_funct" contact.Tag_ID %}">{{ contact.Tag_ID }}</a>
and add this to your urls.py:
(r'^os/(?P<tag_id>[a-zA-Z0-9]*)/*', 'your_views_funct'),
Without knowing too much about your app, I'm guessing you want a URL like /os/<ID>/
First of all, you should declare this URL in your app's url.py. Second of all, use the {% url %} tag in the template. https://docs.djangoproject.com/en/dev/ref/templates/builtins/ And thirdly, your view should be able to accept this ID as a parameter and act accordingly.