I'm trying to build a query in python, I'm looking for an elegant way to append a WHERE condition in the middle of a string:
def get_raw_queryset(frequency=None, where_condition=None):
qs = "SELECT id, COUNT(*) AS count FROM user_transaction_log " \
# I WANT TO APPEND A WHERE ONLY IF not None
if where_condition:
"WHERE .... = 1" \
"GROUP BY type , strftime('{0}', datetime) ORDER BY id" \
.format(frequency)
return qs
This will work as long as you can safely evaluate the WHERE string even if where_condition is not a string:
"SELECT ..." + bool(where_condition) * ("WHERE ...") + "GROUP ..."
I hope you're being extremely careful to avoid SQL injection.
Related
Trying to use parameters in my query for Postgres, using Psycopg2.
When I google search, I see people using %s but I want to give more descriptive names like you see me doing below, so that later I can potentially add more columns into the WHERE clause and it be more easily readable. Is that possible? What am I doing wrong?
q = ""
q += "SELECT"
q += " id"
q += " FROM tbl_users "
q += " WHERE "
q += "("
q += " b_enabled = (%enabled)"
q += ")"
enabled = "True"
db_cursor.execute(q,enabled)
FYI: I realize I could in this case just put "True" as the second parameter in the execute method but this is a cut-down version of my code to focus on the issue. I want to know how to put a variable in the execute, rather than hard code "True" in that spot.
Error message:
psycopg2.errors.SyntaxError: syntax error at or near "$"
LINE 1: ...d_visit_last FROM tbl_users WHERE ( b_enabled = ($t_enabled...
You might want to use a multiline string to define your SQL query, to provide more meaningful names you can use a dictionary to pass values to psycopg2.execute():
import psycopg2
conn = psycopg2.connect("dbname=mf port=5959 host=localhost user=mf_usr")
cur = conn.cursor()
sql = """
SELECT
id
FROM tbl_users
WHERE
b_enabled = %(enabled)s
"""
print (cur.mogrify(sql, {'enabled': "True"}).decode('utf-8'))
# cur.execute(sql, {'enabled': "True"})
Output:
SELECT
id
FROM tbl_users
WHERE
b_enabled = 'True'
Please have a look at the official docs for further information.
I need to query data from a mysqldatabase with the table name containing hyphens.
current_table = "tw3-10_1"
sql2 = "SELECT * FROM " + str(current_table )
cursor.execute(sql2)
Unfortunately I get:
1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '-10_1' at line 1")
Is there any way to work around that issue? Unfortunately I cannot change the names of the tables.....
You can normally use backticks to quote a table name or column name, in case it contains unhelpful characters.
current_table = "`tw3-10_1`"
sql2 = "SELECT * FROM " + current_table
or if you prefer
current_table = "tw3-10_1"
sql2 = "SELECT * FROM `{}`".format(current_table)
try like this, I don't know about MariaDB but quotes should work in SQL
sql2 = """
SELECT
*
FROM "{table_name}"
""".format(
table_name='table-with-hyphens'
)
print(sql2)
# SELECT
# *
# FROM "table-with-hyphens"
This question already has an answer here:
How do I escape % from python mysql query
(1 answer)
Closed 6 years ago.
I use PyMySQL to query from a MySQL database in python:
filter = "Pe"
connection = pymysql.connect(host="X", user="X", password="X", db="X", port=3306, cursorclass=pymysql.cursors.SSCursor)
cursor = connection.cursor()
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%%s%'"
cursor.execute(sql, (filter))
response = cursor.fetchall()
connection.close()
This returns nothing.
I could write:
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%" + filter +"%'"
and execute: cursor.execute(sql), but then I lose the escaping, which makes the program vulnerable for injection attacks, right?
Is there way I could insert the value into the LIKE without losing the escape?
...WHERE name LIKE '%%%s%%'" does not work. I think %s adds ' on both sides of the replaced escaped string as a part of its function within PyMySQL.
You need to pass the whole pattern as a query parameter, and use a tuple:
filter = "%Pe%"
sql = "SELECT * FROM usertable WHERE name LIKE %s"
cursor.execute(sql, (filter,))
Double the % you want to keep.
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%%%s%%'"
Good day all. Lurking here has been a great help - thanks in advance.
What I'd like to do is accept an input from the user, then search both the 'type' and the 'count' columns of the 'mytable' table for anything that matches the user's input.
Here's my code:
import sys
import sqlite3 as lite
for arg in sys.argv:
print arg
var = raw_input("What are you looking for: ")
print "Standby; looking for : ", var
vart = '%'+var+'%' # to add wildcards to the var string
con = lite.connect('test.db')
print
print "Ok. Here's what I found."
print
with con:
cur=con.cursor()
cur.execute( "SELECT * FROM mytable" )
# cur.execute( "SELECT * FROM mytable WHERE type LIKE ( ? )", [vart]) # this actually works - but only searches type column
# cur.execute( "SELECT * FROM mytable WHERE type LIKE ( ? ) OR WHERE count like ( ? )", [vart], [vart] ) fails
# cur.execute( "SELECT * FROM mytable WHERE type LIKE ( ? ) UNION ALL SELECT * FROM mytable WHERE count LIKE ( ?)", [vart], [vart])
rows = cur.fetchall()
# now row has each line to deal with
#print len(rows) #prints the number of lines in the db
for row in rows:
#print len(row) # prints the number of items in the list
# if var in row[0]... then print
mstr=row[0]
print mstr.encode('ascii'), row[1]
Here is the puny database:
type : count
fox|23
dog|34
cat|99
bird|123
rat|201
mouse|23
hedgehog|44
gnu|666
I was successful at searching one column only for the input string, but when I try to do both columns at once, it fails. There's got to be a way that using the sqlite3 functions and not rely on the python ones.
A valid un-nested SQL SELECT statement only has one WHERE statement; also, if you're passing multiple parameters to a sqlite cursor, they must be contained in a single list. The correct syntax for your code would be:
cur.execute('SELECT * FROM mytable WHERE type LIKE ( ? ) OR count like ( ? )', [vart, vart])
just a little syntax fix, and I spruced up your python to be more pep8 friendly (and closer to python3 support, although raw_input is not native in python3). From that you should be able to expand....
import sys
import sqlite3 as lite
'''
made your code more pep8 python like
note comments in python are reserved for
why not what..e.g. code is self descriptive
of what, but why is what is important
'''
print('{}'.format(sys.argv)) # debug
var = raw_input("What are you looking for: ")
print("Standby; looking for :{}".format(var))
vart = '%{}%'.format(var)
con = lite.connect('test.db')
print("\nOk. Here's what I found.\n")
with con:
cur = con.cursor()
sql_query = 'SELECT * FROM mytable WHERE type LIKE ? or count LIKE ?'
cur.execute(sql_query, ['%{0}%'.format(var), '%{0}%'.format(var)])
try:
rows = cur.fetchall()
except Exception as err:
print(err)
for row in rows:
mstr = row[0]
print('Found: {} : {}'.format(mstr.encode('ascii'), row[1]))
output examples
host-wifi:java user$ python /tmp/p.py
['/tmp/p.py']
What are you looking for: 99
Standby; looking for :99
Ok. Here's what I found.
Found: cat : 99
host-wifi:java user$ python /tmp/p.py
['/tmp/p.py']
What are you looking for: 3
Standby; looking for :3
Ok. Here's what I found.
Found: fox : 23
Found: dog : 34
Found: bird : 123
I am new to programming and working on a homework assignment. I am trying to search a database by comparing a user's search term with any matching values from a selected column. If a user searches "Smith" and clicks on the "Smith" radio button in my GUI, all the records containing "Smith" as its author should appear. I am able to print all the records in the database, but not the records that relate to my search.
db = None
colNum = None
def search_db(self):
global db
global colNum
self.searchTerm = self.searchvalue.get()
dbname = 'books.db'
if os.path.exists(dbname):
db = sqlite3.connect(dbname)
cursor = db.cursor()
sql = 'SELECT * FROM BOOKS'
cursor.execute(sql)
rows = cursor.fetchall()
for record in rows:
sql_search = 'SELECT * FROM BOOKS WHERE' + ' ' + record[colNum] + ' ' + 'LIKE "%' + ' ' + self.searchTerm + '%"'
cursor.execute(sql_search)
searched_rows = cursor.fetchall()
print(searched_rows)
The error I'm receiving is "sqlite3.OperationalError: no such column:"
There isn't enough information in your question to be sure, but this certainly is fishy:
sql_search = 'SELECT * FROM BOOKS WHERE' + ' ' + record[colNum] + ' ' + 'LIKE "%' + ' ' + self.searchTerm + '%"'
That record[colNum] is the value in a row for your column, not the name of the column. For example, if the column you wanted is Title, you're going to treat every title of every book as if it were a column name.
So, you end up running queries like this:
SELECT * FROM BOOKS WHERE The Meaning of Life: The Script like %Spam%
Even if that were valid SQL (quoted properly), The Meaning of Life: The Script is probably not a column in the BOOKS table.
Meanwhile, SELECT * returns the columns in an arbitrary order, so using colNum isn't really guaranteed to do anything useful. But, if you really want to do what you're trying to do, I think it's this:
sql = 'SELECT * FROM BOOKS'
cursor.execute(sql)
colName = cursor.description[colNum][0]
sql_search = 'SELECT * FROM BOOKS WHERE ' + colName + ' LIKE "%' + ' ' + self.searchTerm + '%"'
However, you really shouldn't be wanting to do that…
You need to get the column name from the fields of the table, or from somewhere else. Your query uses record[colNum] but record contains rows of data. Instead, to get the field names, use something like this:
fields = []
for field in cursor.description:
fields.append(field[0])
When you use rows = cursor.fetchall(), you are only getting data (and not the column headers).
It looks to me like you are just not forming the SQL correctly. Remember that whatever you put after the LIKE clause needs to be quoted. Your SQL needs to look like
SELECT * FROM BOOKS WHERE Title like '%Spam%'
So you need another set of single quotes in there so that's why I would use double quotes to surround your Python string:
sql_search = "SELECT * FROM BOOKS WHERE " + record[colNum] + " LIKE '%" + self.searchTerm + "%'"