Develop Reference

Using variables in redshift insert command - Python lambda function

I am trying to insert records to a redshift table using a lambda function. I am using
boto3.client('redshift-data')
for the same. Now I have the query as below.
query1 = "insert into dbname.tablename values('aaaa','bbbb','cccc')"
response = rsclient.execute_statement(
ClusterIdentifier='xxxxx',
Database='yyyy',
DbUser='zzzz',
Sql= query1,
StatementName='examplestatement'
)
This works fine. But I want to pass variables here instead of values. For instance,
var1 = 'aaaa'
var2 = 'bbbb'
var3 = 'cccc'
Then try the query as below but it it doesn't work, I think it something silly to do with quotes.
query1 = "insert into dbname.tablename values(var1,var2,var3)"
How can I achieve this. I write lambda function using python3. Any help is appreciated.

You can use f-strings:
query1 = f"insert into dbname.tablename values('{var1}','{var2}','{var3}')"

Alter query according to user selection in sqlite python

I have a sqlite database named StudentDB which has 3 columns Roll number, Name, Marks. Now I want to fetch only the columns that user selects in the IDE. User can select one column or two or all the three. How can I alter the query accordingly using Python?
I tried:
import sqlite3
sel={"Roll Number":12}
query = 'select * from StudentDB Where({seq})'.format(seq=','.join(['?']*len(sel))),[i for k,i in sel.items()]
con = sqlite3.connect(database)
cur = con.cursor()
cur.execute(query)
all_data = cur.fetchall()
all_data
I am getting:
operation parameter must be str

You should control the text of the query. The where clause shall allways be in the form WHERE colname=value [AND colname2=...] or (better) WHERE colname=? [AND ...] if you want to build a parameterized query.
So you want:
query = 'select * from StudentDB Where ' + ' AND '.join('"{}"=?'.format(col)
for col in sel.keys())
...
cur.execute(query, tuple(sel.values()))

In your code, the query is now a tuple instead of str and that is why the error.
I assume you want to execute a query like below -
select * from StudentDB Where "Roll number"=?
Then you can change the sql query like this (assuming you want and and not or) -
query = "select * from StudentDB Where {seq}".format(seq=" and ".join('"{}"=?'.format(k) for k in sel.keys()))
and execute the query like -
cur.execute(query, tuple(sel.values()))
Please make sure in your code the provided database is defined and contains the database name and studentDB is indeed the table name and not database name.

how to using MySQL pattern matching and binding parameters in sql query in python?

I know how to use MySQL pattern matching, for example:
SELECT * FROM table WHERE col LIKE '%mid%'
I also know how to bind parameters into a sql query in python, for example:
import MySQLdb
s = 'something'
db = MySQLdb.connect(host=blablabla...)
cur = db.cursor()
sql = "SELECT * FROM table WHERE col = %s"
cur.execute(sql, s)
data = cur.fetchall()
db.close()
But I can't find a method to combine these together in one query, like
sql = "SELECT * FROM table WHERE col LIKE '%%s%'"
cur.execute(sql, s)
where the first and the third '%' are pattern character and the middle '%s' is used to bind parameter s.
Anyone have an idea?

Alright, I will answer myself..
#dsgdfg inspired me and here is my code:
sql = "SELECT * FROM table WHERE col LIKE %s"
cur.execute(sql, "%"+s+"%")

sql = "SELECT * FROM table WHERE col LIKE CONCAT('%', %s, '%')"
cur.execute(sql, s)
(I am assuming that execute deals with escaping, thereby preventing SQL injection.)

for the people using py charm
you can do this
qq=input("enter the author name:")
print(pd.read_sql_query("select Book_Id,Book_Name,book_author from bookd where book_author like '%s'" %("%"+qq+"%",), conn2))

Passing parameter in psycopg2

I am trying to access PostgreSQL using psycopg2:
sql = """
SELECT
%s
FROM
table;
"""
cur = con.cursor()
input = (['id', 'name'], )
cur.execute(sql, input)
data = pd.DataFrame.from_records(cur.fetchall())
However, the returned result is:
0
0 [id, name]
1 [id, name]
2 [id, name]
3 [id, name]
4 [id, name]
If I try to access single column, it looks like:
0
0 id
1 id
2 id
3 id
4 id
It looks like something is wrong with the quoting around column name (single quote which should not be there):
In [49]: print cur.mogrify(sql, input)
SELECT
'id'
FROM
table;
but I am following doc: http://initd.org/psycopg/docs/usage.html#
Anyone can tell me what is going on here? Thanks a lot!!!

Use the AsIs extension
import psycopg2
from psycopg2.extensions import AsIs
column_list = ['id','name']
columns = ', '.join(column_list)
cursor.execute("SELECT %s FROM table", (AsIs(columns),))
And mogrify will show that it is not quoting the column names and passing them in as is.

Nowadays, you can use sql.Identifier to do this in a clean and secure way :
from psycopg2 import sql
statement = """
SELECT
{id}, {name}
FROM
table;
"""
with con.cursor() as cur:
cur.execute(sql.SQL(statement).format(
id=sql.SQL.Identifier("id"),
name=sql.SQL.Identifier("name")
))
data = pd.DataFrame.from_records(cur.fetchall())
More information on query composition here : https://www.psycopg.org/docs/sql.html

The reason was that you were passing the string representation of the array ['id', 'name'] as SQL query parameter but not as the column names. So the resulting query was similar to
SELECT 'id, name' FROM table
Looks your table had 5 rows so the returned result was just this literal for each row.
Column names cannot be the SQL query parameters but can be just the usual string parameters which you can prepare before executing the query-
sql = """
SELECT
%s
FROM
table;
"""
input = 'id, name'
sql = sql % input
print(sql)
cur = con.cursor()
cur.execute(sql)
data = pd.DataFrame.from_records(cur.fetchall())
In this case the resulting query is
SELECT
id, name
FROM
table;

Passing a column name in a SELECT statement in Python

if count == 1:
cursor.execute("SELECT * FROM PacketManager WHERE ? = ?", filters[0], parameters[0])
all_rows = cursor.fetchall()
elif count == 2:
cursor.execute("SELECT * FROM PacketManager WHERE ? = ? AND ? = ?", filters[0], parameters[0], filters[1], parameters[1])
all_rows = cursor.fetchall()
elif count == 3 :
cursor.execute("SELECT * FROM PacketManager WHERE ? = ? AND ? = ? AND ? = ?", filters[0], parameters[0], filters[1], parameters[1], filters[2], parameters[2])
all_rows = cursor.fetchall()
This is a code snippet in my program. What I'm planning to do is pass the column name and the parameter in the query.
The filters array contains the columnnames, the parameter array contains the parameters. The count is the number of filters set by the user. The filters and paramters array are already ready and have no problem. I just need to pass it to the query for it to execute. This give me an error of "TypeError: function takes at most 2 arguments"

You cannot use SQL parameters to interpolate column names. You'll have to use classic string formatting for those parts. That's the point of SQL parameters; they quote values so they cannot possibly be interpreted as SQL statements or object names.
The following, using string formatting for the column name works, but be 100% certain that the filters[0] value doesn't come from user input:
cursor.execute("SELECT * FROM PacketManager WHERE {} = ?".format(filters[0]), (parameters[0],))
You probably want to validate the column name against a set of permissible column names, to ensure no injection can take place.

You can only set parameters using ?, not table or column names.
You could build a dict with predefined queries.
queries = {
"foo": "SELECT * FROM PacketManager WHERE foo = ?",
"bar": "SELECT * FROM PacketManager WHERE bar = ?",
"foo_bar": "SELECT * FROM PacketManager WHERE foo = ? AND bar = ?",
}
# count == 1
cursor.execute(queries[filters[0], parameters[0])
# count == 2
cursor.execute(queries[filters[0] + "_" + queries[filters[1], parameters[0])
This approach will make you save from SQL injection in filters[0].