I am trying to search my SQLite3 database using a pythonic variable as a search term. The term I'm searching for is a part of the contents of the cell in the database (e.g. Smith in a cell: [Harrison GB, Smith JH]) and is often in the middle of the string in a cell.
I have tried to code it as shown below:
def read_from_db():
c.execute("SELECT authors, year, title, abstract FROM usertable WHERE authors LIKE (?)",(var1,))
data = c.fetchall()
print(data)
for row in data:
searchlist.append(row)
var1="Smith"
read_from_db()
This should show the results row after row. However, I get 0 results when var1 = "Smith". When I change its value to "Harrison GB, Smith JH", I get all the results.
When I try to solve it by changing the SQLite3 execute query I yield an error.
ERROR
c.execute("SELECT authors, year, title, abstract FROM usertable WHERE authors LIKE '%?%'",(var1,))
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 0, and there are 1 supplied.
I get syntax errors if I change the endings with: $?$, (%?%) etc. I tried this with:
...authors="%?%"
But this doesn't work either. There is a few similar questions on SO, but they don't exactly solve my issue...
I am running Python 3.4 and SQLite3 on Windows 10.
Consider concatenating the % wildcards to the binded value, var1:
c.execute("SELECT authors, year, title, abstract" +
" FROM usertable WHERE authors LIKE (?)", ('%'+var1+'%',))
The reason you need to do so is the ? placeholder substitutes a string literal in parameterized queries and for LIKE expressions, wildcards with values together are string literals as denoted by their enclosed single quotes:
SELECT authors, year, title, abstract FROM usertable WHERE authors LIKE '%Smith%'
Your initial attempt failed because you wrap ? with single quotes and the cursor cannot bind the param to prepared statement properly and so the symbol is taken literally as question mark.
Related
I'm to link my code to a MySQL database using pymysql. In general everything has gone smoothly but I'm having difficulty with the following function to find the minimum of a variable column.
def findmin(column):
cur = db.cursor()
sql = "SELECT MIN(%s) FROM table"
cur.execute(sql,column)
mintup = cur.fetchone()
if everything went smoothly this would return me a tuple with the minimum, e.g. (1,).
However, if I run the function:
findmin(column_name)
I have to put column name in "" (i.e. "column_name"), else Python sees it as an unknown variable. But if I put the quotation marks around column_name then SQL sees
SELECT MIN("column_name") FROM table
which just returns the column header, not the value.
How can I get around this?
The issue is likely the use of %s for the column name. That means the SQL Driver will try to escape that variable when interpolating it, including quoting, which is not what you want for things like column names, table names, etc.
When using a value in SELECT, WHERE, etc. then you do want to use %s to prevent SQL injections and enable quoting, among other things.
Here, you just want to interpolate using pure Python (assuming a trusted value; please see below for more information). That also means no bindings tuple passed to the execute method.
def findmin(column):
cur = db.cursor()
sql = "SELECT MIN({0}) FROM table".format(column)
cur.execute(sql)
mintup = cur.fetchone()
SQL fiddle showing the SQL working:
http://sqlfiddle.com/#!2/e70a41/1
In response to the Jul 15, 2014 comment from Colin Phipps (September 2022):
The relatively recent edit on this post by another community member brought it to my attention, and I wanted to respond to Colin's comment from many years ago.
I totally agree re: being careful about one's input if one interpolates like this. Certainly one needs to know exactly what is being interpolated. In this case, I would say a defined value within a trusted internal script or one supplied by a trusted internal source would be fine. But if, as Colin mentioned, there is any external input, then that is much different and additional precautions should be taken.
I have a python script to retrieve a value (table ID) from a PostgreSQL database. The column name contains a colon though and I believe this is stopping it working. I've tested this on columns without colons and it does get the ID correctly.
The line in question is
cur.execute("SELECT tID from titles where name like 'METEOROLOG:WINDSPEED_F' order by structure, comp1, comp2")
rowswind=cur.fetchall()
When I print rowswind nothing is returned (just empty brackets)
I have also tried..
cur.execute('SELECT tID from titles where name like "METEOROLOGY:WINDSPEED_F" order by structure, comp1, comp2')
But that comes back with the error
psycopg2.ProgrammingError: column "METEOROLOGY:WINDSPEED_F" does not
exist
(it definitely does).
I've also tried escaping the colon any way I can think of (i.e. back slash) but nothing works, I just get syntax errors.
Any advice would be welcome. Thanks.
ADDITION 20190429
I've now tried parameterizing the query but also with no success.
wind=('METEOROLOGY:WINDSPEED_F')
sql="SELECT tID from titles where name like '{0}' order by structure, comp1, comp2".format(wind)
I've tried many different combinations of double and single quotes to try and escape the colon with no success.
psycopg2.ProgrammingError: column "METEOROLOGY:WINDSPEED_F" does not exist
You're getting this error because you're using double quotes around the targeted value in your query's WHERE statement, here:
cur.execute('SELECT tID from titles where name like "METEOROLOGY:WINDSPEED_F" order by structure, comp1, comp2')
You're getting 0 results back here:
cur.execute("SELECT tID from titles where name like 'METEOROLOG:WINDSPEED_F' order by structure, comp1, comp2")
because 0 rows exist with the value "METEOROLOG:WINDSPEED_F" in the name column. This might just be because you're spelling METEOROLOGY wrong.
The way you're using LIKE, you might as well be using =. LIKE is great if you're going to use % to find other values like that value.
Example:
SELECT *
FROM
TABLE
WHERE
UPPER(NAME) LIKE 'JOSH%'
This would return results for these values in name: JOSHUA, JoShUa, joshua, josh, JOSH. If I straight up did NAME LIKE 'JOSH' then I would only find results for the exact value of JOSH.
Since you are making the value all caps in your WHERE, try this adding an UPPER() to your query like this:
cur.execute("SELECT tID from titles where UPPER(name) like 'METEOROLOG:WINDSPEED_F' order by structure, comp1, comp2")
I'm trying to insert a record into an sqlite database using named parameters in python (with the sqlite3 module).
The values I want to insert are in a dictionary, but the dictionary keys might contain dashes, for example {'request-id': 100, 'year': '2015'}.
I'm trying to execute the following:
import sqlite3
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
cursor.execute('''CREATE TABLE IF NOT EXISTS requests (request_id text, year text)''')
query = '''INSERT INTO requests (request_id, year) VALUES (:request-id, :year)'''
cursor.execute(query, {'request-id': 100, 'year': '2015'})
conn.commit()
conn.close()
I get this error during the insert statement:
sqlite3.OperationalError: no such column: id
It seems like dashes are not well accepted as named parameters.
There are many workarounds for this, like creating a new dictionary where dashes in the keys are replaced by underscores, but I'd like to know if I could use some escaping technique or something else to avoid that.
Thanks for your help
The documentation for sqlite3_bind_* states that parameter names must be composed of alphanumeric characters, and doesn't mention a way of escaping them.
Your query is probably being parsed as :request - id, i.e. :request minus id, and since there's no such column id, SQLite throws an error.
(Also, as Prerak Sola points out, you create the table with a date column but try to insert to a year column which doesn't exist.)
SQL parameter names have no quoting or escaping mechanism; you have to use the same rules as for an unquoted identifier.
I'm to link my code to a MySQL database using pymysql. In general everything has gone smoothly but I'm having difficulty with the following function to find the minimum of a variable column.
def findmin(column):
cur = db.cursor()
sql = "SELECT MIN(%s) FROM table"
cur.execute(sql,column)
mintup = cur.fetchone()
if everything went smoothly this would return me a tuple with the minimum, e.g. (1,).
However, if I run the function:
findmin(column_name)
I have to put column name in "" (i.e. "column_name"), else Python sees it as an unknown variable. But if I put the quotation marks around column_name then SQL sees
SELECT MIN("column_name") FROM table
which just returns the column header, not the value.
How can I get around this?
The issue is likely the use of %s for the column name. That means the SQL Driver will try to escape that variable when interpolating it, including quoting, which is not what you want for things like column names, table names, etc.
When using a value in SELECT, WHERE, etc. then you do want to use %s to prevent SQL injections and enable quoting, among other things.
Here, you just want to interpolate using pure Python (assuming a trusted value; please see below for more information). That also means no bindings tuple passed to the execute method.
def findmin(column):
cur = db.cursor()
sql = "SELECT MIN({0}) FROM table".format(column)
cur.execute(sql)
mintup = cur.fetchone()
SQL fiddle showing the SQL working:
http://sqlfiddle.com/#!2/e70a41/1
In response to the Jul 15, 2014 comment from Colin Phipps (September 2022):
The relatively recent edit on this post by another community member brought it to my attention, and I wanted to respond to Colin's comment from many years ago.
I totally agree re: being careful about one's input if one interpolates like this. Certainly one needs to know exactly what is being interpolated. In this case, I would say a defined value within a trusted internal script or one supplied by a trusted internal source would be fine. But if, as Colin mentioned, there is any external input, then that is much different and additional precautions should be taken.
Using SQLite3 with Python 2.5, I'm trying to iterate through a list and pull the weight of an item from the database based on the item's name.
I tried using the "?" parameter substitution suggested to prevent SQL injections but it doesn't work. For example, when I use:
for item in self.inventory_names:
self.cursor.execute("SELECT weight FROM Equipment WHERE name = ?", item)
self.cursor.close()
I get the error:
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 8 supplied.
I believe this is somehow caused by the initial creation of the database; the module I made that actually creates the DB does have 8 bindings.
cursor.execute("""CREATE TABLE Equipment
(id INTEGER PRIMARY KEY,
name TEXT,
price INTEGER,
weight REAL,
info TEXT,
ammo_cap INTEGER,
availability_west TEXT,
availability_east TEXT)""")
However, when I use the less-secure "%s" substitution for each item name, it works just fine. Like so:
for item in self.inventory_names:
self.cursor.execute("SELECT weight FROM Equipment WHERE name = '%s'" % item)
self.cursor.close()
I can't figure out why it thinks I have 8 bindins when I'm only calling one. How can I fix it?
The Cursor.execute() method expects a sequence as second parameter. You are supplying a string which happens to be 8 characters long.
Use the following form instead:
self.cursor.execute("SELECT weight FROM Equipment WHERE name = ?", [item])
Python library reference: sqlite3 Cursor Objects.
I have spent half a day trying to figure out why something like this would give me an error:
cursor.execute("SELECT * from ? WHERE name = ?", (table_name, name))
only to find out that table names cannot be parametrized. Hope this will help other people save some time.
The argument of cursor.execute that represents the values you need inserted in the database should be a tuple (sequence). However consider this example and see what's happening:
>>> ('jason')
'jason'
>>> ('jason',)
('jason',)
The first example evaluates to a string instead; so the correct way of representing single valued tuple is as in the second evaluation. Anyhow, the code below to fix your error.
self.cursor.execute("SELECT weight FROM Equipment WHERE name = ?", (item,))
Also giving the cursor.execute value arguments as strings,(which is what you are doing) results in the first evaluation in the example and results into the error you are getting.
The sqlite3 module supports two kinds of placeholders for parameters:
qmark style
Use one or more ? to mark the position of each parameter, and supply a list or tuple of parameters. E.g.:
curs.execute(
"SELECT weight FROM Equipment WHERE name = ? AND price = ?",
["lead", 24],
)
named style
Use :par placeholders for each named parameter, and supply a dict. E.g.:
curs.execute(
"SELECT weight FROM Equipment WHERE name = :name AND price = :price",
{"name": "lead", "price": 24},
)
Advantages of named style parameters is that you don't need to worry about the order of parameters, and each :par can be used multiple times in large/complex SQL queries.
have You tried this ? :
for item in self.inventory_names:
t = (item,)
self.cursor.execute("SELECT weight FROM Equipment WHERE name = ?", t)
self.cursor.close()
cursor.execute() expects a sequence (list,tuple) as second parameter. (-> ddaa )
Quoting (is that what the parens mean?) the ? with parens seems to work for me. I kept trying with (literally) '?' but I kept getting
ProgrammingError: Incorrect number of bindings supplied. The current statement uses 0, and there are 1 supplied.
When I did:
SELECT fact FROM factoids WHERE key LIKE (?)
instead of:
SELECT fact FROM factoids WHERE key LIKE '?'
It worked.
Is this some python 2.6 thing?
each element of items has to be a tuple.
assuming names looks something like this:
names = ['Joe', 'Bob', 'Mary']
you should do the following:
for item in self.inventory_names:
self.cursor.execute("SELECT weight FROM Equipment WHERE name = ?", (item, ))
by using (item, ) you are making it a tuple instead of a string.
Try
execute("select fact from factoids where key like ?", "%%s%" % val)
You don't wrap anything around the ? at all, Python sqlite will correctly convert it into a quoted entity.