I am creating an app guided by CS50's web series, which requires me to ONLY use raw SQL queries not ORM.
I am trying to make a search function where a user can look up the list of books that's stored in the database. I want to able them to query ISBN, title, author column in the table called 'books'
Currently, it does shoot a 'GET' request with no problem but it is not returning any data and I think the problem is at the SQL line I've scripted.
Here's the route:
#app.route("/", methods=['GET','POST'])
def index():
# search function for books
if request.method == "GET":
searchQuery = request.form.get("searchQuery")
# return value from the search
searchResult = db.execute("SELECT isbn, author, title FROM books WHERE isbn LIKE '%"+searchQuery+"%' OR author LIKE '%"+searchQuery+"%' OR title LIKE '%"+searchQuery+"%'").fetchall()
# add search result to the list
session["books"] = []
# add the each result to the list
for i in searchResult:
session["books"].append(i)
return render_template("index.html", books=session["books"])
return render_template("index.html")
and here's my template.
<form method="GET">
<input type="text" name="searchQuery" class="searchTerm" placeholder="What are you looking for?">
<button type="submit" class="searchButton">submit</button>
</form>
<div>
<h3>
{% for book in books %}
{{ book }}
{% endfor %}
</h3>
</div>
Can anyone spot the problem, please? Please note that I am supposed to utilize the raw SQL queries and session.
I created a github with full solution for you :)
https://github.com/researcher2/stackoverflow_57120430
A couple of things:
Avoiding SQL Injection
I recommend using bindings when doing raw sql statements, my code reflects that. I spent ages trying to get this working with your statement before stumbling upon this:
Python SQLite parameter substitution with wildcards in LIKE
Basically you can't put bindings inside the LIKE '%?%' because the quotes cause the replacement token to be ignored.
Instead you just have to do LIKE ? and build the replacement manually.
Using Session
All session information is JSON serialized and then sent to the client. In this case the row records weren't JSON serializable. This showed up as an error for me:
TypeError: Object of type 'RowProxy' is not JSON serializable
I probably wouldn't use the session here as there is no need for the client to be aware of this, as you're going to build them a nice html page with the information anyway. Just use a python dictionary and pass it to the template engine. My code did use the session because this is what you started with.
In case github ever goes down:
from flask import request, render_template, session
from app import app, db
#app.route("/", methods=['GET','POST'])
def index():
if request.method == "POST":
searchQuery = request.form.get("searchQuery")
print(searchQuery)
# Avoid SQL Injection Using Bindings
sql = "SELECT isbn, author, title \
FROM book \
WHERE isbn LIKE :x \
OR author LIKE :y \
OR title LIKE :z"
# I spent an hour wondering why I couldnt put the bindings inside the wildcard string...
# https://stackoverflow.com/questions/3105249/python-sqlite-parameter-substitution-with-wildcards-in-like
matchString = "%{}%".format(searchQuery)
stmt = db.text(sql).bindparams(x=matchString, y=matchString, z=matchString)
results = db.session.execute(stmt).fetchall()
print(results)
session["books"] = []
for row in results:
# A row is not JSON serializable so we pull out the pieces
book = dict()
book["isbn"] = row[0]
book["author"] = row[1]
book["title"] = row[2]
session["books"].append(book)
return render_template("index.html", searchedFor=searchQuery, books=session["books"])
return render_template("index.html")
Thanks for the chosen answer.
The problem was solved with one simple change.
I needed to swap 'LIKE' to 'ILIKE' in the SQL query line.
searchResult = db.execute(
"""
SELECT isbn, author, title
FROM books
WHERE isbn ILIKE :search OR author ILIKE :search OR title ILIKE :search
""",
{"search": "%"+searchQuery+"%"}
).fetchall()
This may not be an elegant solution but it did solve a temporary problem. Again, kudos for the guy who answered and create a github repo for everyone else who would encounter the problem while doing CS50's web series!
Related
I have a Flask application with a GET handler that given a recipe id as a URL parameter, it retrieves the recipe from the database and then renders it:
#app.route('/recipe/<int:id>', methods=['GET'])
def get(id):
recipe = get_recipe_from_db(id)
return render_template('recipe.html', recipe=recipe)
This results in a URL like /recipe/5. Rather than showing just the id in the URL, I would like the recipe title to be part of the resulting URL, for example recipe/5/lemon-cake. On the first request only the id is known.
I'm not sure what a neat way to do this is. So far I've come up with the following:
#app.route('/recipe/<int:id>', methods=['GET'])
def get(id):
recipe = get_recipe_from_db(id)
return redirect(url_for('get_with_title', id=id, title=urlify(recipe.title)))
#app.route('/recipe/<int:id>/<title>', methods=['GET'])
def get_with_title(id, title=None):
recipe = get_recipe_from_db(id)
return render_template('recipe.html', recipe=recipe)
This works (i.e. when user visits /recipe/5 it redirects to /recipe/5/lemon-cake) but suffers from the fact that the same recipe is retrieved from the database twice.
Is there a better way to go about this?
Note: the recipe object is large containing multiple fields and I do not want to pass it over the network unecessarily.
Option 1
The easiest solution would be to modify the URL on client side, as soon as the response is received; hence, no need for redirection and/or querying the databse twice. This could be achieved by using either history.pushState() or history.replaceState().
client side
<!DOCTYPE html>
<html>
<head>
<script>
function modify_url() {
var title = {{recipe.title|tojson}};
var id = {{recipe.id|tojson}};
window.history.pushState('', '', id + "/" + title);
}
</script>
</head>
<h2>Recipe for {{recipe.title}}</h2>
<body onload="modify_url()"></body>
</html>
server side
#app.route('/recipe/<int:id>', methods=['GET'])
def get(id):
recipe = get_recipe_from_db(id)
return render_template('recipe.html', recipe=recipe)
You may also would like to retain the get_with_title() route as well, in case users bookmark/share the URL (including the title) and want it to be accessible (otherwise, "Not Found" error would be returned when accessing it).
Option 2
If you wouldn't like to query the database every time a new request arrives - not even jsut for retrieving the title (not every single column) of a recipe entry - and you have sufficient amount of memory to hold the data, I would suggest to query the database once at startup (selecting only id and title) and create a dictionary, so that you can quickly look up for a title from the recipe id. Please note, in this way, every time an INSERT/DELETE/etc operation is performed on the table, that dictionary has to be updated accordingly. Thus, if you have such operations frequently on that table, might not be the best approach to the problem, and better keep with querying the table just for retrieving the title.
recipes = dict((row[0], row[1]) for row in result) # where row[0] is id and row[1] is title
Then in your endpoint:
#app.route('/recipe/<int:id>', methods=['GET'])
def get(id):
title = recipes.get(id)
return redirect(url_for('get_with_title', id=id, title=urlify(title)))
You can look into only changing URL with javascript, as shown here. This way, you don't do any redirects or reloading.
If you don't want to mess with javascript, let's think about backend solutions:
Is loading twice from a database significant performance overhead to be worthy of a complicated solution? If not, your solution is good.
If you really want to avoid loading the whole recipe, you can load only the title in your get method, instead of the whole object.
You can write custom get_recipe_title(id) which does (roughly) SELECT title FROM recipes WHERE id={id}.
If using SQLAlchemy, you may probably use load_only - here
If you want the 'recipe title' to appear in the URL, your solution is not suitable for that. You should do this before the 'request'. There is probably an HTML page where you submit the recipe data to list all the recipes.
Try the following in your HTML page:
<ul>
{% for recipe in recipes %}
<li><a href="/postdetail/{{recipe.id}}/{{recipe.title}}"></li>
{% endfor %}
</ul>
After that, when you click on that recipe you can also see the title.
Just pass loaded recipe as argument:
#app.route('/recipe/<int:id>', methods=['GET'])
def get(id):
recipe = get_recipe_from_db(id)
return redirect(
url_for(
'get_with_title',
id=id,
title=urlify(recipe.title),
recipe=recipe,
))
#app.route('/recipe/<int:id>/<title>', methods=['GET'])
def get_with_title(id, title=None, recipe=None):
if recipe is None:
recipe = get_recipe_from_db(id)
return render_template('recipe.html', recipe=recipe)
I am trying to make a ecommerce site using flask and raw mysql i.e. using flask_mysqldb to write my own mysql query. I am trying to use dynamic routing to display data in a single template based on their id to avoid creating multiple templates for different products. But i am stuck as i cannot get the id of the products to pass it as a parameter in the mysql query. How to get the html tags attribute values to pass to the sql query as a parameter??
My route:
#app.route('/products/')
def product_details():
cursor = mysql.connection.cursor(MySQLdb.cursors.DictCursor)
productID = request.args.get('id')
print(productID)
cursor.execute('SELECT productCode, productName FROM Products WHERE productCode = %s', (productID, )
)
products = cursor.fetchone()
return render_template('product-details.html', products=products)
My html files
products.html:
<div>
{% for product in products %}
<h1><a href={{url_for('product_details', pk=product.productCode) }} id={{product.productCode}}>{{product.productName}}</a></h1>
<p>Rs. {{product.price}}</p>
{{product.productCode}}
{% endfor %}
product-details.html:
<h1 id={{products.productCode}}>{{products.productName}}</h1>
The problem is caused by a mismatch between the query parameters you are sending to, and expecting in product_details.
You are sending pk in your url_for call but are looking for id in productID = request.args.get('id').
The solution is to look for pk in product_details: productID = request.args.get('pk').
Note that anything information outside the url_for call is not visible to Flask i.e. Flask does not have access to the <a> tag's id attribute.
I have a query like this:
session.query(System).filter_by(id_system = s.id_system).join(Command).filter_by(id_command=c.id_command).first()
I'd like to do this query in a template page (I'm using mako), but it doesn't work:
% for c in s.commands:
code = session.query(System).filter....
% endfor
What is best way to do a query in pages? Or is it not possible?
Template is not a right place to do such operations. All business logic needs to be included into controllers according to MVC (or RV as Pyramid) paradigm.
Do query in your view function:
results = DBSession.query(model.Articles).all()
return dict(status=True, results=results)
or even better create module which contains common database operations (then import function and call it in view function) and return results to template by
import your_project.lib.dbfunctions
results = dbfunctions.get_articles()
return dict(status=True, results=results)
than use them into template:
<div>
% for r in results:
${r}
% endfor
</div>
I hope it will help you.
While it may be possible by injecting the session and other needed functions into the template context, this is not the right way to do things. The data, for the most part, should be handled outside the template, then passed to the context when rendering.
The view/function that is calling render for this template should be making this query and passing it to the template.
I would like to display all results which match selected facets even though a search query has not been inserted. Similar to how some shop applications work e.g. Amazon
e.g. Show all products which are "blue" and between $10-$100.
Haystack does not return any values if a search query is not specified.
Any ideas how I can get around it?
Thanks!
If anyone is still looking, there's a simple solution suggested in haystack code:
https://github.com/toastdriven/django-haystack/blob/master/haystack/forms.py#L34
class SearchForm(forms.Form):
def no_query_found(self):
"""
Determines the behavior when no query was found.
By default, no results are returned (``EmptySearchQuerySet``).
Should you want to show all results, override this method in your
own ``SearchForm`` subclass and do ``return self.searchqueryset.all()``.
"""
return EmptySearchQuerySet()
Why No Results?
I imagine you're using a search template similar to the one in the haystack getting started documentation. This view doesn't display anything if there is no query:
{% if query %}
{# Display the results #}
{% else %}
{# Show some example queries to run, maybe query syntax, something else? #}
{% endif %}
The second problem is that the default search form's search() method doesn't actually search for anything unless there's a query.
Getting Results
To get around this, I'm using a custom search form. Here's an abbreviated sample:
class CustomSearchForm(SearchForm):
...
def search(self):
# First, store the SearchQuerySet received from other processing.
sqs = super(CustomSearchForm, self).search()
if not self.is_valid():
return sqs
filts = []
# Check to see if a start_date was chosen.
if self.cleaned_data['start_date']:
filts.append(SQ(created_date__gte=self.cleaned_data['start_date']))
# Check to see if an end_date was chosen.
if self.cleaned_data['end_date']:
filts.append(SQ(created_date__lte=self.cleaned_data['end_date']))
# Etc., for any other things you add
# If we started without a query, we'd have no search
# results (which is fine, normally). However, if we
# had no query but we DID have other parameters, then
# we'd like to filter starting from everything rather
# than nothing (i.e., q='' and tags='bear' should
# return everything with a tag 'bear'.)
if len(filts) > 0 and not self.cleaned_data['q']:
sqs = SearchQuerySet().order_by('-created_date')
# Apply the filters
for filt in filts:
sqs = sqs.filter(filt)
return sqs
Also, don't forget to change the view:
{% if query or page.object_list %}
{# Display the results #}
{% else %}
{# Show some example queries to run, maybe query syntax, something else? #}
{% endif %}
Actually, the view code is a little hackish. It doesn't distinguish query-less searches with no results from search with no parameters.
Cheers!
Look at SearchQuerySet.
This should be possible if color and price has been defined in your SearchIndex:
sqs = SearchQuerySet().filter(color="blue", price__range=(10,100))
You can limit the query to certain models by adding models(Model) to the SearchQuerySet. So if you want to limit your query to the model Item use:
sqs = SearchQuerySet().filter(color="blue", price__range=(10,100)).models(Item)
Following form display all the result if not query string is present. Now you can add custom filters.
from your_app.forms import NonEmptySearchForm
url(r'^your_url$',
SearchView(template='search.html',searchqueryset=sqs,form_class=NonEmptySearchForm), name='haystack_search'),
forms.py
#Overridding because the default sqs is always none if no query string is present
class NonEmptySearchForm(SearchForm):
def search(self):
if not self.is_valid():
return self.no_query_found()
sqs = self.searchqueryset.auto_query(self.cleaned_data['q'])
if self.load_all:
sqs = sqs.load_all()
return sqs
Stumpy Joe Pete's answer is pretty spot on, but as he mentioned, the template if query or page.object_list check is a little hacked. A better way of solving this would be to create your own SearchForm which would still find something if q is empty - will not repost that - AND to customize the SearchView with something like:
class MySearchView(SearchView):
def get_query(self):
query = []
if self.form.is_valid():
for field in self.form:
if field.name in self.form.cleaned_data and self.form.cleaned_data[field.name]:
query.append(field.name+'='+str(self.form.cleaned_data[field.name]))
return ' AND '.join(query)
In most cases, you won't even be using the query value, so you could just as well do a quick check if any of the fields is set and return True or something like that.. or of course you can modify the output any way you want (I'm not even 100% sure my solution would work for all field types, but you get the idea).
I'm using SQLAlchemy with Flask and the Flask-SQLAlchemy extension. What I'm confused about is how to optimize the updating of a record. As a proof of concept I wrote the following endpoints:
#bp.route('/books', methods=['GET', 'PUT'], endpoint="books_index")
#bp.route('/books/<book_id>', methods=['GET', 'POST'], endpoint="books_show")
def books_index(book_id=None):
book = Book.query.get_or_404(book_id) if book_id else Book()
form = Book.model_form(request.form, book if book_id else None)
verb = "updated" if book_id else "created"
if form.validate_on_submit():
form.populate_obj(book)
db.session.add(book)
db.session.commit()
flash('Book %s successfully' % verb, 'info')
return dict(
books=[] if book_id else Book.query.all(),
book=book,
form=form
)
Everything works as I want it to, but when updating an existing book record, SQLAlchemy runs three queries, they are:
SELECT book.id AS book_id, book.title AS book_title FROM book WHERE book.id = ?
UPDATE book SET title=? WHERE book.id = ?
SELECT book.id AS book_id, book.title AS book_title FROM book WHERE book.id = ?
I understand why SQLAlchemy makes all three queries, but it seems a bit excessive for simply updating one record. Is there a way to optimize the amount of queries happening here?
(as a note, I've got some wrapping on the endpoints to render templates based on a convention which is why you don't see render_template)