I'm trying to pass the same parameters to an oracle query in two separate places in the SQL code.
My code works if I hard code the criteria for table2 like this:
# define parameters
years = ['2018','2019']
placeholder= ':d'
placeholders= ', '.join(placeholder for unused in years)
placeholders
# create cursor
cursor = connection.cursor()
# query
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in ['2018','2019'] --here's where I say I'm hard coding criteria
""" % placeholders
data = cursor.execute(qry, years)
df = pd.DataFrame(data.fetchall(), columns = [column[0] for column in cursor.description])
# close database connection
connection.close()
If I try to use the parameter for table2 like this:
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in (%s) --part of code I'm having issues with
""" % placeholders
I get the following error:
TypeError: not enough arguments for format string
I can't simply rewrite the SQL because I frequently have to use someone else's code and it wouldn't be feasible to rewrite all of it.
If you want to fill multiple placeholders, you have to supply the same number of parameters.
"one meal: %s" % "sandwich" # works
"two meals: %s, %s" % "sandwich" # not working
"two meals: %s, %s" % ("sandwich", "sandwich") # works
NOTE: It is a bad/dangerous thing to use string formatting for the assembly of SQL queries (lookup "SQL Injection"). In your case it is fine, but in general you should use parameterized queries, especially when dealing with input from untrusted sources like user input. You don't want a user to input "2018; DROP TABLE table1;".
Related
I am struggling with generating the delete query where parameters for the query is actually a set of values.
So I need to delete rows where parameters are a pair values for example:
delete from table where col1 = %s and col2 = %s
which can be executed in Python like:
cur = conn.cursor()
cur.execute(query, (col1_value, col2_value))
Now I would like to run a query:
delete from table where (col1, col2) in ( (col1_value1, col2_value1), (col1_value2, col2_value2) );
I can generate the queries and values and execute the exact SQL but I can't quite generate prepared statement.
I tried:
delete from table where (col1, col2) in %s
and
delete from table where (col1, col2) in (%s)
But when I try to execute:
cur.execute(query, list_of_col_value_tuples)
or
cur.execute(query, tuple_of_col_value_tuples)
I get an exception that indicates that psycopg2 cannot convert arguments to strings.
Is there any way to use psycopg2 to execute a query like this?
You could dynamically add %s placeholders to your query:
cur = con.cursor()
query = "delete from table where (role, username) in (%s)"
options = [('admin', 'foo'), ('user', 'bar')]
placeholders = '%s,' * len(options)
query = query % placeholders[:-1] # remove last comma
print(query)
print(cur.mogrify(query, options).decode('utf-8'))
Out:
delete from table where (role, user) in (%s,%s)
delete from table where (role, user) in (('admin', 'foo'),('user', 'bar'))
Alternatively, build the query using psycopg2.sql as answered there.
Actually the resolution is quite easy if carefully constructed.
In the miscellaneous goodies of psycopg2 there is a function execute_values.
While all the examples that are given by psycopg2 deal with inserts as the function basically converts the list of arguments into a VALUES list if the call to delete is formatted like so:
qry = "delete from table where (col1, col2) in (%s)"
The call:
execute_values(cur=cur, qry=qry, argslist=<list of value tuples>)
will make the delete perform exactly as required.
Need to pass 2 variables to sql query body
The official documentation says that parameters are passed through a variable as follows, but this only works with one parameter:
SQL = "INSERT INTO authors (name) VALUES (%s);" # Note: no quotes
data = ("O'Reilly", )
cur.execute(SQL, data)
how it would be possible to pass two variables, that is, something like this:
SQL = "INSERT INTO authors (%s) VALUES (%s);"
data_1 = ("name",)
data_2 = ("O'Reilly", )
cur.execute(SQL, data_1,data_2)
I want to pass list values along with other parameter values. following is my scenario. I want to pass multiple values for column "Code" and want to pass single value to "Continent" column.
param = [('AFG', 'IND'),'Asia']
query = "select * from country where Code in (%s) AND Continent = %s"
cursor.execute(query,param)
while executing in Python, I am getting following error.
Failed to execute Query: Failed processing format-parameters; Python
'tuple' cannot be converted to a MySQL type
The trick here is the WHERE IN clause, which isn't really amenable to being parameterized. One option generates an IN clause with the exact number of placeholders in your list:
codes = ('AFG', 'IND')
continent = 'Asia'
params = codes + (continent,)
where_in = ','.join(['%s'] * len(codes))
sql = "SELECT * FROM country WHERE Code IN (%s) AND Continent = %s" % (where_in, '%s')
cursor.execute(sql, params)
To see what the above script actually did, lets look at the various parts:
print(where_in)
print(sql)
%s,%s
SELECT * FROM country WHERE Code IN (%s,%s) AND Continent = %s
The trick here is that we actually use a %s placeholder twice, once for the Python string, and a second time for the SQL query string. Also, we bind a single level tuple containing all bound values:
('AFG', 'IND', 'ASIA')
first you split the list and then you split the tuple.
param = [('AFG', 'IND'),'Asia']
p1,p2=param[0]
query = "select * from country where Code in ('%s','%s') AND Continent = %s" % (p1,p2,param[1])
cursor.execute(query)
I am really struggling with updating many rows in python using SAP HANA as my database and PyHDB for establishing the interface between both applications. Its working when I "hardcode" the columns, but I need to dynamically switch the columns by defining them inside of an array for example.
I am able to update in a hardcoded way the necessary columns by performing the following SQL-query:
sql = """UPDATE "ARE"."EMPLOYEES" SET "LIKELIHOOD_NO" = %s, "LIKELIHOOD_YES"= %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
What I want to achieve is the following scenario:
dynamic_columns = ["LIKELIHOOD_NO", "LIKELIHOOD_Yes"]
sql = """UPDATE "ARE"."EMPLOYEES" SET dynamic_column = %s, "LIKELIHOOD_YES" = %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
I am always getting the error that the relevant column / columns could not be found, but I cant figure out a way to solve this.
You can use normal string interpolation (.format()) to add the dynamic column name. You can see in the code here that pyHDB supports "numeric" paramstyle:
for col in ['LIKELIHOOD_YES', 'LIKELIHOOD_NO']:
sql = ('UPDATE "ARE"."EMPLOYEES" SET "{some_col}" = :1, "CLASS" = :2 '
'WHERE "EmployeeNumber" = :3;').format(some_col=col)
cursor.executemany(sql, list_of_tuples)
This code will run for both columns 'LIKELIHOOD_YES' and 'LIKELIHOOD_NO'. Adapt it as you need. It would work with a list of tuples like this:
list_of_tuples = [
(value1, class_1, employee_no_1),
(value2, class_2, employee_no_2),
(value3, class_3, employee_no_3),
(value4, class_4, employee_no_4),
]
The code in your question seems to be using the 'format' paramstyle instead, but that doesn't seem what pyHDB is using. See PEP 249 for more information on paramstyles.
I know how to use MySQL pattern matching, for example:
SELECT * FROM table WHERE col LIKE '%mid%'
I also know how to bind parameters into a sql query in python, for example:
import MySQLdb
s = 'something'
db = MySQLdb.connect(host=blablabla...)
cur = db.cursor()
sql = "SELECT * FROM table WHERE col = %s"
cur.execute(sql, s)
data = cur.fetchall()
db.close()
But I can't find a method to combine these together in one query, like
sql = "SELECT * FROM table WHERE col LIKE '%%s%'"
cur.execute(sql, s)
where the first and the third '%' are pattern character and the middle '%s' is used to bind parameter s.
Anyone have an idea?
Alright, I will answer myself..
#dsgdfg inspired me and here is my code:
sql = "SELECT * FROM table WHERE col LIKE %s"
cur.execute(sql, "%"+s+"%")
sql = "SELECT * FROM table WHERE col LIKE CONCAT('%', %s, '%')"
cur.execute(sql, s)
(I am assuming that execute deals with escaping, thereby preventing SQL injection.)
for the people using py charm
you can do this
qq=input("enter the author name:")
print(pd.read_sql_query("select Book_Id,Book_Name,book_author from bookd where book_author like '%s'" %("%"+qq+"%",), conn2))