I need to look up for a string that contains multiple single quotes from my psql table. My current solution is replacing the single quotes with double single quotes like this:
sql_query = f"""SELECT exists (SELECT 1 FROM {table_name} WHERE my_column = '{my_string.replace("'","''")}' LIMIT 1);"""
cursor = connection.cursor()
cursor.execute(sql_query)
Is there a nicer solution for this kind of formatting ?
I solved this by making the query like this
sql_query = f"""SELECT exists (SELECT 1 FROM {table_name} WHERE my_column = %s LIMIT 1);"""
cursor = connection.cursor()
cursor.execute(sql_query, (mystring,))
In this case it formats mystring in such way that it is not executed as sql query and doesn't cause the problems mentioned in the question
Related
I have a sqlite database named StudentDB which has 3 columns Roll number, Name, Marks. Now I want to fetch only the columns that user selects in the IDE. User can select one column or two or all the three. How can I alter the query accordingly using Python?
I tried:
import sqlite3
sel={"Roll Number":12}
query = 'select * from StudentDB Where({seq})'.format(seq=','.join(['?']*len(sel))),[i for k,i in sel.items()]
con = sqlite3.connect(database)
cur = con.cursor()
cur.execute(query)
all_data = cur.fetchall()
all_data
I am getting:
operation parameter must be str
You should control the text of the query. The where clause shall allways be in the form WHERE colname=value [AND colname2=...] or (better) WHERE colname=? [AND ...] if you want to build a parameterized query.
So you want:
query = 'select * from StudentDB Where ' + ' AND '.join('"{}"=?'.format(col)
for col in sel.keys())
...
cur.execute(query, tuple(sel.values()))
In your code, the query is now a tuple instead of str and that is why the error.
I assume you want to execute a query like below -
select * from StudentDB Where "Roll number"=?
Then you can change the sql query like this (assuming you want and and not or) -
query = "select * from StudentDB Where {seq}".format(seq=" and ".join('"{}"=?'.format(k) for k in sel.keys()))
and execute the query like -
cur.execute(query, tuple(sel.values()))
Please make sure in your code the provided database is defined and contains the database name and studentDB is indeed the table name and not database name.
Assuming I have a database with table users with a row:
ID = 0, name = 'myName'
I can get the ID value (assuming I'm connected) by using either:
cursor.execute("""SELECT ID FROM users WHERE %s = %s""" % ('name', 'myName'))
print(cursor.fetchone())
or
cursor.execute("""SELECT ID FROM users WHERE {} = {}""".format('name', 'myName'))
print(cursor.fetchone())
which both work perfectly fine but are bad practice according to the documentation.
The documentation suggest using either qmark or named style for queries with variable input. The problem arises when using the suggested styles in a SELECT * FROM myDatabase WHERE ? = ? query.
qmark style:
cursor.execute("""SELECT ID FROM users WHERE ? = ?""", ('name', 'myName'))
print(cursor.fetchone())
named style
cursor.execute("""SELECT ID FROM users WHERE :column = :cell""", {'column': 'name', 'cell':'myName'})
print(cursor.fetchone())
Using either of the latter styles results in a None returned. When using the qmark or named style in the context of INSERT such as used in the example in the documentation it does work as expected.
(To clarify, with the qmark style, adding parentheses to each ? or the entire ? = ? does not change the outcome. Using parentheses with each ? and using an additional argument causes execute() to malfunction due to too many arguments given.)
What causes this behaviour? Is it possible to use qmark or named style in a SELECT...WHERE ? = ? query?
Parameter substitution is for values, not identifiers (the names of columns and tables etc). RDBMS have different rules for quoting values and identifiers. Using the parameter substitution placeholder for identifiers leads to the identifier being incorrectly quoted, for example
cur.execute('SELECT * FROM tbl WHERE ? = ?', ('col1', 42))
ends up as
SELECT * FROM tbl WHERE 'col1' = 42
note the single quotes around col1, which cause it to evaluated as a string, not a column name.
If you want to have dynamic identifiers as well as values in your query then use string formatting for the identifiers and parameter substitution for the values. For example, using double quotes for the identifier
cur.execute('SELECT * FROM tbl WHERE "{}" = ?'.format('col1'), (42,))
Here's an example of string formatting resulting in an error
>>> conn = sqlite3.connect(':memory:')
>>> conn.execute('create table tbl (col1 date)')
<sqlite3.Cursor object at 0x7f56abcf1ce0>
>>> cur = conn.cursor()
>>> cur.execute('INSERT INTO tbl (col1) VALUES(?)', ('2021-05-01',))
<sqlite3.Cursor object at 0x7f56abc8f030>
>>> cur.execute('INSERT INTO tbl (col1) VALUES(%s)' % '2021-05-01')
<sqlite3.Cursor object at 0x7f56abc8f030>
>>> conn.commit()
>>> cur.execute('SELECT col1 FROM tbl WHERE %s = %s' % ('col1', '2021-05-01'))
<sqlite3.Cursor object at 0x7f56abc8f030>
>>> for row in cur:print(row)
...
(2015,)
When string formatting is used in the INSERT and SELECT statements, the date is evaluated as an arithmetic expression, resulting in the wrong value being stored and retrieved. Errors like this are annoying, but using string formatting can also leave your application to SQL injection attacks, which could have more serious consequences.
I have a strange issue where I can't get an SQL query with parameter to work with a string comparison in the where clause - I don't get a row back. when i connect to the MySQL db via bash, the query works.
python 3.7.3
mysql-connector-python==8.0.11
mysql 5.7
works (getting my row):
select * from my_table where my_column = 'my_string';
also works (getting my row):
cursor.execute(
"""
select *
from my_table
where my_column = 'my_string'
"""
)
doesn't work (cursor.fetchall() is []):
cursor.execute(
"""
select *
from my_table
where my_column = '%s'
""",
('my_string')
)
Be careful with tuples. I think you need ('my_string',).
FYI I wrote the original comment mentioned by #tscherg in his comment below the question.
Remove the quotes:
cursor.execute(
"""
select *
from my_table
where my_column = %s
""",
('my_string')
)
I know how to use MySQL pattern matching, for example:
SELECT * FROM table WHERE col LIKE '%mid%'
I also know how to bind parameters into a sql query in python, for example:
import MySQLdb
s = 'something'
db = MySQLdb.connect(host=blablabla...)
cur = db.cursor()
sql = "SELECT * FROM table WHERE col = %s"
cur.execute(sql, s)
data = cur.fetchall()
db.close()
But I can't find a method to combine these together in one query, like
sql = "SELECT * FROM table WHERE col LIKE '%%s%'"
cur.execute(sql, s)
where the first and the third '%' are pattern character and the middle '%s' is used to bind parameter s.
Anyone have an idea?
Alright, I will answer myself..
#dsgdfg inspired me and here is my code:
sql = "SELECT * FROM table WHERE col LIKE %s"
cur.execute(sql, "%"+s+"%")
sql = "SELECT * FROM table WHERE col LIKE CONCAT('%', %s, '%')"
cur.execute(sql, s)
(I am assuming that execute deals with escaping, thereby preventing SQL injection.)
for the people using py charm
you can do this
qq=input("enter the author name:")
print(pd.read_sql_query("select Book_Id,Book_Name,book_author from bookd where book_author like '%s'" %("%"+qq+"%",), conn2))
I am trying to access PostgreSQL using psycopg2:
sql = """
SELECT
%s
FROM
table;
"""
cur = con.cursor()
input = (['id', 'name'], )
cur.execute(sql, input)
data = pd.DataFrame.from_records(cur.fetchall())
However, the returned result is:
0
0 [id, name]
1 [id, name]
2 [id, name]
3 [id, name]
4 [id, name]
If I try to access single column, it looks like:
0
0 id
1 id
2 id
3 id
4 id
It looks like something is wrong with the quoting around column name (single quote which should not be there):
In [49]: print cur.mogrify(sql, input)
SELECT
'id'
FROM
table;
but I am following doc: http://initd.org/psycopg/docs/usage.html#
Anyone can tell me what is going on here? Thanks a lot!!!
Use the AsIs extension
import psycopg2
from psycopg2.extensions import AsIs
column_list = ['id','name']
columns = ', '.join(column_list)
cursor.execute("SELECT %s FROM table", (AsIs(columns),))
And mogrify will show that it is not quoting the column names and passing them in as is.
Nowadays, you can use sql.Identifier to do this in a clean and secure way :
from psycopg2 import sql
statement = """
SELECT
{id}, {name}
FROM
table;
"""
with con.cursor() as cur:
cur.execute(sql.SQL(statement).format(
id=sql.SQL.Identifier("id"),
name=sql.SQL.Identifier("name")
))
data = pd.DataFrame.from_records(cur.fetchall())
More information on query composition here : https://www.psycopg.org/docs/sql.html
The reason was that you were passing the string representation of the array ['id', 'name'] as SQL query parameter but not as the column names. So the resulting query was similar to
SELECT 'id, name' FROM table
Looks your table had 5 rows so the returned result was just this literal for each row.
Column names cannot be the SQL query parameters but can be just the usual string parameters which you can prepare before executing the query-
sql = """
SELECT
%s
FROM
table;
"""
input = 'id, name'
sql = sql % input
print(sql)
cur = con.cursor()
cur.execute(sql)
data = pd.DataFrame.from_records(cur.fetchall())
In this case the resulting query is
SELECT
id, name
FROM
table;