Develop Reference

Sqlite format output in python

I've got a simple flask web-app set up where the user can add and delete tasks from a database. All the entries in the database are displayed in a template, sorted by the type they're assigned. I can't format the output to be at least somewhat readable, though. How do i do that?
The actual database uses different values and whatnot, so not all of them may make sense right now.
This is the function that gets all the entries from my sqlite database:
def get_tsk_by_type(type):
c.execute("SELECT * FROM tasks WHERE type=:type", {'type': type})
result = c.fetchall()
return result
The database:
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
type text,
description text,
amount integer,
id integer
)""")
And here's how i'm returning all the entries that are then displayed in in a template. There is also a function to delete tasks if you input their id.
#app.route('/', methods = ['GET', 'POST'])
def index():
form = DeleteForm()
curr_homework = str(get_tsk_by_type("homework"))
curr_cleaning = str(get_tsk_by_type("cleaning"))
curr_cooking = str(get_tsk_by_type("cooking"))
if form.validate_on_submit():
try:
conn = sqlite3.connect('tasks.db', check_same_thread=False)
c = conn.cursor()
delete = request.form['delete']
if (delete):
remove_tsk(delete)
return redirect('/')
conn.close()
except:
return "Something went wrong while submitting the form"
return render_template('index.html', curr_homework = curr_homwork, curr_cleaning = curr_cleaning, curr_cooking = curr_cooking, form = form)
The relevant parts of my index.html look like this:
{% block content %}
<div>
<p>
<span>Currently registered homework: {{ curr_homework }}</span><br />
<span>Currently registered cleaning tasks: {{ curr_cleaning }}</span><br />
<span>Currently registered cooking tasks {{ curr_cooking }}</span>
</p>
</div>
{% endblock content %}
However, the output i'm getting looks like this:
Currently registered homework: [('homework', 'math', 1, 'df19c0b1-a2128-431274-2e32-3a2f901b1b26')]
Currently registered cleaning tasks: [('cleaning', 'kitchen', 1, 'df19c0b1-aa18-4874-9e32-3a2f901b1b26')]
Currently registered cooking tasks: [('cooking', 'lunch', 1, '0697c139-0299-4c93-88ac-c07d77377796')]
I've tried for-loops and that kinda thing but it only ever returns the first tuple inside the list that get_tsk_by_type() returns. I also tried panda but i couldn't get that to output the way i want it to, either. How do i prettify it to the point that it's easily readable? Without brackets etc.? I later want to display each individual task separately, preferably in divs.

I recommend using a dict cursor so you can access the result elements by name.
You can do that like this (from: https://stackoverflow.com/a/3300514/42346):
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
d[col[0]] = row[idx]
return d
conn.row_factory = dict_factory
Then you will get this kind of result:
result = c.fetchall()
result
# [{'type':'homework','description':'math',
# 'amount':1,'id':'df19c0b1-a2128-431274-2e32-3a2f901b1b26'}]
Then in your template you can do something like:
{% for homework in curr_homework %}
<div>
<h6>{{ homework['type'] }}</h6>
<div>{{ homework['description'] }}</div>
</div>
{% if not loop.last %}
<hr>
{% endif %}
{% endfor %}
You may benefit from a little bit of re-organization of the database-specific code.
You can do this:
from flask import g
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
d[col[0]] = row[idx]
return d
def get_db():
if 'db' not in g:
conn = sqlite3.connect('tasks.db', check_same_thread=False)
conn.row_factory = dict_factory
g.db = conn.cursor()
return g.db
And then in your view do this:
db = get_db()
db.execute('your query here')

Python Flask Jinja SQL: No value being passed to template

In my GET portion of the Flask function I'm working on I have some very simple code written in Python 3. The data I'm trying to pass in is never displayed on my HTML render.
#app.route("/sellselected", methods=["GET", "POST"])
#login_required
def sellselected(order_num):
if request.method == "POST":
#not done, just have a redirect to index
else:
stock_to_sell = db.execute("SELECT * FROM purchases WHERE order_num = :order_num", order_num=order_num)
#stock_to_sell = ['fish']
return render_template("sellselected.html", stock_to_sell=stock_to_sell, order_num=order_num)
The SQL statement seems to passing in nothing, it's just blank on the HTML render. But as a test I also used 'fish' and it's None/empty too.
Jinja looks like:
{% block main %}
<list>
<ul>{{ stock_to_sell }}</ul>
<ul>{{ order_num }}</ul>
</list>
{% endblock %}
So the body of the page has the order number, but stock_to_sell is always empty.

Your problem is not related to jinja anymore. Your route is wrong, it should be: #app.route("/sellselected/<order_num>", methods=["GET", "POST"]).
Because you pass order_num to sellselected function so you need to declare it on route.

You say you like to pass the param order_num as GET right? Because the code say POST is redirect to index.
So, you pass the param as GET. You need to get it first
current_order = request.args.get('order_num')
Your code should be:
#app.route("/sellselected", methods=["GET", "POST"])
#login_required
def sellselected(order_num):
if request.method == "POST":
# not done, just have a redirect to index
else:
order_num = request.args.get('order_num')
stock_to_sell = db.execute("SELECT * FROM purchases WHERE order_num = :order_num", order_num=order_num)
return render_template("sellselected.html", stock_to_sell=stock_to_sell, order_num=order_num)

Beware if you iterate over an object then it's contents will comeback empty. This is because the result is a generator, and it's values can be accessed only once.
Won't work as gcp_certs has been iterated over:
gcp_certs = connection.execute(query)
for cert in gcp_certs:
print(cert.name)
return render_template('certifications.html',
gcpcerts=gcp_certs,
now=datetime.utcnow(),
cookie=token)
Works:
gcp_certs = connection.execute(query)
return render_template('certifications.html',
gcpcerts=gcp_certs,
now=datetime.utcnow(),
cookie=token)
One solution:
imports copy
gcp_certs = list(connection.execute(query))
sql_select = ''
for cert in copy.copy(gcp_certs):
print(cert.name)
return render_template('certifications.html',
gcpcerts=gcp_certs,
now=datetime.utcnow(),
cookie=token)

Create in python a dictionary from JSON url

I want to create a list in HTML of locations from JSON API url in python.
#app.route('/api')
def api():
url = urlopen('https://api.openaq.org/v1/locations?country=GB').read()
#encoded we do not need it
#encoded_data = json.dumps(url)
#create variables
array = []
data = {}
#decoded
decoded_data = json.loads(url)
#we search for result each entry
for i in decoded_data["results"]:
#append every entry to an array
array.append(i["location"])
#we create a dictionary from that array created which has a variable (for jinja2) called location
data = [dict(location=array)]
return render_template('api.html', data=data)
But instead of receiving each element, I get this:
[u'Aberdeen', u'Aberdeen Union Street Roadside', u'Aberdeen Wellington Road', u'Armagh Roadside', u'Aston Hill', u'Auchencorth Moss', u'Ballymena Ballykeel', u'Barnsley Gawber', u'Barnstaple A39', u'Bath Roadside', u'Belfast Centre', u"Belfast Stockman's Lane", u'Billingham', u'Birkenhead Borough Road', u'Birmingham A4540 Roads...
Edit: Template
{% if data %}
<ul>
{% for d in data %}
<li>{{ d.location }}</li>
{% endfor %}
</ul>
{% else %}
<p class="lead">
You should not see this msg, otherwise, check the code again.
</p>
{% endif %}

I broke my answer down a bit because I didn't want to activate flask.
import requests
def api():
res = requests.get('https://api.openaq.org/v1/locations?country=GB')
data = res.json()['results']
return data
#app.route('/api')
def api():
res = requests.get('https://api.openaq.org/v1/locations?country=GB')
try:
data = res.json()['results']
except KeyError:
data = None
# this is the logic that you use in your template, I moved it out here
# cause i don't want to start a flask instance
for d in data:
print d['location']
return render_template('api.html', data=data)
api()
Basically I use the requests module which can return a json. I pass the results to the data varible. I used a for loop to demo how it would work in your template. Basically pass in the data as a dictionary and get the location via iteration d['location']
So the code to use is
import requests
#app.route('/api')
def api():
res = requests.get('https://api.openaq.org/v1/locations?country=GB')
try:
data = res.json()['results']
except KeyError:
data = None
return render_template('api.html', data=data)

You are converting the array to a dict, but then you are putting the dict inside an array of length 1, with the only object being the dict. The issue is, your template is then expecting each element in the array to be a dictionary, with a "location" field.
You either can remove the square brackets from the conversion data = dict(location=array) and then update your template to just do for d in data.location, or you can update your append call to append a dictionary item instead of a string: array.append({"location": i["location"]})

Couple of things:
url is a bytes object, which will not work with json.loads(str). So you'll have to convert it to a string either by doing json.loads(str(url,'utf-8')) or the method suggested by #Mattia
#louhoula is correct. But, in case you are expecting data to be a list of dictionaries each containing a location key (that's the idea I get by looking at your template), then you should change d.location in your template to :
{% if 'location' in d.keys(): %}
{{ d['location'] }}
{% else %}
<p class="lead">
You should not see this msg, otherwise, check the code again.
</p>
{% endif %}

Try this:
import urllib.request, json
url = 'https://api.openaq.org/v1/locations?country=GB'
response = urllib.request.urlopen(url);
decoded_data = json.loads(response.read().decode("utf-8"))

Django form, view, and forms.py not working together

My first foray into Django, allows the user to input a day of the week, and searches my database for restaurants open on that given day. Right now, the restaurant objects (Resto) have a days_open attribute, with each day separated by a comma (Monday, Tuesday, etc...).
When I input a day, the resulting page only displays the title and '[]' and I can't seem to get it to return the list of Resto objects. What is displaying the square brackets, and how do I go about fixing it to display the results of the search?
The code is below- my apologies if I neglected to include any relevant bits.
my forms.py:
from django import forms
from .models import Resto
class RestoSearch(forms.ModelForm):
class Meta:
model = Resto
fields = ('title', 'description', 'opening_hour', 'closing_hour')
models.py:
from django.db import models
class Resto(models.Model):
title = models.CharField(max_length=300)
description = models.TextField()
opening_hour = models.TimeField(auto_now=False, auto_now_add=False, null=True)
closing_hour = models.TimeField(auto_now=False, auto_now_add=False, null=True)
days_open = models.TextField(blank=True)
views.py:
from django.shortcuts import render
from django.http import Http404
from django.shortcuts import HttpResponse
from belize.models import Resto
from django.core.exceptions import *
from .forms import RestoSearch
def index(request):
return render(request, 'form.html')
def search(request):
form = RestoSearch()
if request.method == 'POST':
search_id=request.POST.get('textfield', None)
try:
#I think this is where my problem is
available = Resto.objects.filter(days_open = search_id)
html = ("<H1>Hello</H1>", available)
return HttpResponse(html)
except Resto.DoesNotExist:
return HttpResponse("Please try another day")
else:
return render(request, 'belize/form.html')
def restaurant_detail(request, id):
try:
restaurant = Resto.objects.get(id=id)
except Resto.DoesNotExist:
raise Http404('This restaurant does not exist')
return render(request, 'belize/restaurant_detail.html', {
'restaurant': restaurant,
})
template form.html:
<form method="POST" action="/search/">
{% csrf_token %}
<input type="text" name="textfield">
<button type="submit">Enter a day of the week</button>
</form>

I presume what you are trying to show is the RestoForm in that case the index method is not correct. It should be
def index(request):
form = RestoForm()
return render(request, 'form.html', {'form': form })
And then your template should change as
<form method="POST" action="/search/">
{% csrf_token %}
{{ form }}
<button type="submit">Enter a day of the week</button>
</form>
For additional details please see the examples at https://docs.djangoproject.com/en/1.9/topics/forms/#the-template

The [] means that your .filter() returned no results, its not surprising as you have a few issues with your code, lets start from the top:
You are declaring a form that you never use.
You are trying to catch an exception that is never raised by .filter()
You filter condition will only work for exact matches.
I've annotated your code as well:
def search(request):
form = RestoSearch() # You aren't using this form anywhere in your code?
if request.method == 'POST':
# Where is 'textfield' coming from?
search_id = request.POST.get('textfield', None)
try:
# If search id is "Tuesday", and a restaurant is
# open on monday and tuesday, so it has "Monday,Tuesday"
# in the days_open field, then this search will not
# return any results, because its looking for an exact
# match
available = Resto.objects.filter(days_open=search_id)
html = ('<H1>Hello World</H1>', available)
return HttpResponse(html)
except Resto.DoesNotExist:
# This exception is not raised by .filter(),
# .filter() will return an empty list, [] if no results are found
# so this entire try/except is not doing anything
return HttpResponse("Please try another day")
else: # in your code, this else is not indented correctly
return render(request, 'belize/form.html')
So there is a lot going on here, lets try something simple, starting with the template:
{% if results %}
{% for restaurant in results %}
{{ restaurant }}
{% endfor %}
{% else %}
Sorry, no results for your search. Try again.
{% endif %}
<form>
{{ form }}
<input type="submit" name="Search" />
</form>
Next, the search form:
class SearchForm(forms.Form):
search_field = forms.CharField('Search', strip=True)
Finally the view:
from django.db.models import Q
def search(request):
form = SearchForm(request.GET)
results = [] # Set results to an empty list
if form.is_valid():
needle = form.cleaned_data['search_field'].capitalize()
results = Resto.objects.filter(Q(days_open__startswith='{},'.format(needle)) |
Q(days_open__endswith=',{}'.format(needle)) |
Q(days_open__contains=',{},'.format(needle)) |
Q(days_open='{},'.format(needle)) |
Q(days_open='{}'.format(needle)))
return render(request, 'search.html', {'results': results, 'form': form})
Lets assume the user entered 'Thursday' as a search field. In this view you are searching for all restaurants whose days_open field:
Either starts with Thursday, or
Ends with ,Thursday or
Contains ,Thursday, in the middle or
Has the value Thursday, or
Has the value Thursday
Your template will then only show the results if there are any values to display; since an empty list [] is false, then the {% if results %} condition will fail, so on empty lists the template will display the error notice instead.
In your view, you only do the database check if someone enters something in the search field, that's what if form.is_valid(): does. In django, by default all form fields are required - so a blank form will return an error. Using this trick, we make sure we only search if someone enters a value in the search box.
The main action happens with all the Q() calls. A Q object is a way to do multiple queries and chain them together. It is used whenever you want to do an "or" or "and" type query. Our search is an "or" type query, because we want to return any results if the value in days_open matches any number of conditions.

pagination is applied not working in django

in my custom templates pagination is applied on it. the list that show on page is much bigger than i used pagination on it. the limit to display the list is wroking properly but when i click on the next button than it go on the else condition.
views.py :-
#csrf_exempt
def search(request):
if request.method == 'POST':
getchoice = request.POST['userchoice']
getfirstdate = request.POST['firstdate']
getseconddate = request.POST['seconddate']
if getchoice == '0':
getdata = applicationform.objects.filter(date__gte=getfirstdate , date__lte=getseconddate)
##### PAGINATION
searchpagination = Paginator(getdata ,5)
page = request.GET.get('searchpage')
try:
searchcontacts = searchpagination.page(page)
except PageNotAnInteger:
searchcontacts = searchpagination.page(1)
except EmptyPage:
searchcontacts = searchpagination.page(searchpagination.num_pages)
if getdata:
return render_to_response('registration/search_page.html', {'getdata':getdata ,'getchoice':getchoice ,'searchcontacts': searchcontacts})
else:
return HttpResponse('NO ITEMS FOUND ON THIS DATE')
else :
return render_to_response('registration/search_page.html')
custom template:-
{% if searchcontacts.has_previous %}
PREVIOUS
{% endif %}
{% if searchcontacts.has_next %}
NEXT
{% endif %}

Click on NEXT will be GET request and hence it is going in else section.
To continue search for next page, either you need to store search parameters in the session or pass it through url. And then use that in filter() query.