I want to pass list values along with other parameter values. following is my scenario. I want to pass multiple values for column "Code" and want to pass single value to "Continent" column.
param = [('AFG', 'IND'),'Asia']
query = "select * from country where Code in (%s) AND Continent = %s"
cursor.execute(query,param)
while executing in Python, I am getting following error.
Failed to execute Query: Failed processing format-parameters; Python
'tuple' cannot be converted to a MySQL type
The trick here is the WHERE IN clause, which isn't really amenable to being parameterized. One option generates an IN clause with the exact number of placeholders in your list:
codes = ('AFG', 'IND')
continent = 'Asia'
params = codes + (continent,)
where_in = ','.join(['%s'] * len(codes))
sql = "SELECT * FROM country WHERE Code IN (%s) AND Continent = %s" % (where_in, '%s')
cursor.execute(sql, params)
To see what the above script actually did, lets look at the various parts:
print(where_in)
print(sql)
%s,%s
SELECT * FROM country WHERE Code IN (%s,%s) AND Continent = %s
The trick here is that we actually use a %s placeholder twice, once for the Python string, and a second time for the SQL query string. Also, we bind a single level tuple containing all bound values:
('AFG', 'IND', 'ASIA')
first you split the list and then you split the tuple.
param = [('AFG', 'IND'),'Asia']
p1,p2=param[0]
query = "select * from country where Code in ('%s','%s') AND Continent = %s" % (p1,p2,param[1])
cursor.execute(query)
Related
I'm trying to run SQL statements through Python on a list.
By passing in a list, in this case date. Since i want to run multiple SELECT SQL queries and return them.
I've tested this by passing in integers, however when trying to pass in a date I am getting ORA-01036 error. Illegal variable name/number. I'm using an Oracle DB.
cursor = connection.cursor()
date = ["'01-DEC-21'", "'02-DEC-21'"]
sql = "select * from table1 where datestamp = :date"
for item in date:
cursor.execute(sql,id=item)
res=cursor.fetchall()
print(res)
Any suggestions to make this run?
You can't name a bind variable date, it's an illegal name. Also your named variable in cursor.execute should match the bind variable name. Try something like:
sql = "select * from table1 where datestamp = :date_input"
for item in date:
cursor.execute(sql,date_input=item)
res=cursor.fetchall()
print(res)
Some recommendation and warnings to your approach:
you should not depend on your default NLS date setting, while binding a String (e.g. "'01-DEC-21'") to a DATE column. (You probably need also remone one of the quotes).
You should ommit to fetch data in a loop if you can fetch them in one query (using an IN list)
use prepared statement
Example
date = ['01-DEC-21', '02-DEC-21']
This generates the query that uses bind variables for your input list
in_list = ','.join([f" TO_DATE(:d{ind},'DD-MON-RR','NLS_DATE_LANGUAGE = American')" for ind, d in enumerate(date)])
sql_query = "select * from table1 where datestamp in ( " + in_list + " )"
The sql_query generate is
select * from table1 where datestamp in
( TO_DATE(:d0,'DD-MON-RR','NLS_DATE_LANGUAGE = American'), TO_DATE(:d1,'DD-MON-RR','NLS_DATE_LANGUAGE = American') )
Note that the INlist contains one bind variable for each member of your input list.
Note also the usage of to_date with explicite mask and fixing the language to avoid problems with interpretation of the month abbreviation. (e.g. ORA-01843: not a valid month)
Now you can use the query to fetch the data in one pass
cur.prepare(sql_query)
cur.execute(None, date)
res = cur.fetchall()
I have a list of IDs
L1=['A1','A14','B43']
I am trying to use a SQL script to extract information from a table where the ID is in the above list.
sqlquery= "select * from table where ID in " + L1
cur.execute(sqlquery)
I've connected to vertica using vertica_python and sqlalchemy_vertica. But I'm not sure how to incorporate my variable (the list L1) into the sql query.
Updated Code:
data = ['A1', 'A14', 'B43', ...]
placeholders = ','.join('?' * len(data)) # this gives you e.g. '?,?,?'
sqlquery = 'SELECT * FROM table WHERE id IN (%s)' % placeholders
cur.execute(sqlquery, tuple(data))
The docs on https://github.com/vertica/vertica-python shows that the Vertica DBAPI implementation uses ? for positional placeholders, so you can use a parametrized query.
Unfortunately lists cannot be passed nicely and need once parameter per element, so you need to generate this part dynamically:
data = ['A1', 'A14', 'B43', ...]
placeholders = ','.join('?' * len(data)) # this gives you e.g. '?,?,?'
sqlquery = 'SELECT * FROM table WHERE id IN (%s)' % placeholders
cur.execute(sqlquery, data)
But you still keep data and SQL separate that way, so there's no risk of SQL injection!
I'm trying to pass the same parameters to an oracle query in two separate places in the SQL code.
My code works if I hard code the criteria for table2 like this:
# define parameters
years = ['2018','2019']
placeholder= ':d'
placeholders= ', '.join(placeholder for unused in years)
placeholders
# create cursor
cursor = connection.cursor()
# query
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in ['2018','2019'] --here's where I say I'm hard coding criteria
""" % placeholders
data = cursor.execute(qry, years)
df = pd.DataFrame(data.fetchall(), columns = [column[0] for column in cursor.description])
# close database connection
connection.close()
If I try to use the parameter for table2 like this:
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in (%s) --part of code I'm having issues with
""" % placeholders
I get the following error:
TypeError: not enough arguments for format string
I can't simply rewrite the SQL because I frequently have to use someone else's code and it wouldn't be feasible to rewrite all of it.
If you want to fill multiple placeholders, you have to supply the same number of parameters.
"one meal: %s" % "sandwich" # works
"two meals: %s, %s" % "sandwich" # not working
"two meals: %s, %s" % ("sandwich", "sandwich") # works
NOTE: It is a bad/dangerous thing to use string formatting for the assembly of SQL queries (lookup "SQL Injection"). In your case it is fine, but in general you should use parameterized queries, especially when dealing with input from untrusted sources like user input. You don't want a user to input "2018; DROP TABLE table1;".
I am really struggling with updating many rows in python using SAP HANA as my database and PyHDB for establishing the interface between both applications. Its working when I "hardcode" the columns, but I need to dynamically switch the columns by defining them inside of an array for example.
I am able to update in a hardcoded way the necessary columns by performing the following SQL-query:
sql = """UPDATE "ARE"."EMPLOYEES" SET "LIKELIHOOD_NO" = %s, "LIKELIHOOD_YES"= %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
What I want to achieve is the following scenario:
dynamic_columns = ["LIKELIHOOD_NO", "LIKELIHOOD_Yes"]
sql = """UPDATE "ARE"."EMPLOYEES" SET dynamic_column = %s, "LIKELIHOOD_YES" = %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
I am always getting the error that the relevant column / columns could not be found, but I cant figure out a way to solve this.
You can use normal string interpolation (.format()) to add the dynamic column name. You can see in the code here that pyHDB supports "numeric" paramstyle:
for col in ['LIKELIHOOD_YES', 'LIKELIHOOD_NO']:
sql = ('UPDATE "ARE"."EMPLOYEES" SET "{some_col}" = :1, "CLASS" = :2 '
'WHERE "EmployeeNumber" = :3;').format(some_col=col)
cursor.executemany(sql, list_of_tuples)
This code will run for both columns 'LIKELIHOOD_YES' and 'LIKELIHOOD_NO'. Adapt it as you need. It would work with a list of tuples like this:
list_of_tuples = [
(value1, class_1, employee_no_1),
(value2, class_2, employee_no_2),
(value3, class_3, employee_no_3),
(value4, class_4, employee_no_4),
]
The code in your question seems to be using the 'format' paramstyle instead, but that doesn't seem what pyHDB is using. See PEP 249 for more information on paramstyles.
Normally, if i want to insert values into a table, i will do something like this (assuming that i know which columns that the values i want to insert belong to):
conn = sqlite3.connect('mydatabase.db')
conn.execute("INSERT INTO MYTABLE (ID,COLUMN1,COLUMN2)\
VALUES(?,?,?)",[myid,value1,value2])
But now i have a list of columns (the length of list may vary) and a list of values for each columns in the list.
For example, if i have a table with 10 columns (Namely, column1, column2...,column10 etc). I have a list of columns that i want to update.Let's say [column3,column4]. And i have a list of values for those columns. [value for column3,value for column4].
How do i insert the values in the list to the individual columns that each belong?
As far as I know the parameter list in conn.execute works only for values, so we have to use string formatting like this:
import sqlite3
conn = sqlite3.connect(':memory:')
conn.execute('CREATE TABLE t (a integer, b integer, c integer)')
col_names = ['a', 'b', 'c']
values = [0, 1, 2]
conn.execute('INSERT INTO t (%s, %s, %s) values(?,?,?)'%tuple(col_names), values)
Please notice this is a very bad attempt since strings passed to the database shall always be checked for injection attack. However you could pass the list of column names to some injection function before insertion.
EDITED:
For variables with various length you could try something like
exec_text = 'INSERT INTO t (' + ','.join(col_names) +') values(' + ','.join(['?'] * len(values)) + ')'
conn.exec(exec_text, values)
# as long as len(col_names) == len(values)
Of course string formatting will work, you just need to be a bit cleverer about it.
col_names = ','.join(col_list)
col_spaces = ','.join(['?'] * len(col_list))
sql = 'INSERT INTO t (%s) values(%s)' % (col_list, col_spaces)
conn.execute(sql, values)
I was looking for a solution to create columns based on a list of unknown / variable length and found this question. However, I managed to find a nicer solution (for me anyway), that's also a bit more modern, so thought I'd include it in case it helps someone:
import sqlite3
def create_sql_db(my_list):
file = 'my_sql.db'
table_name = 'table_1'
init_col = 'id'
col_type = 'TEXT'
conn = sqlite3.connect(file)
c = conn.cursor()
# CREATE TABLE (IF IT DOESN'T ALREADY EXIST)
c.execute('CREATE TABLE IF NOT EXISTS {tn} ({nf} {ft})'.format(
tn=table_name, nf=init_col, ft=col_type))
# CREATE A COLUMN FOR EACH ITEM IN THE LIST
for new_column in my_list:
c.execute('ALTER TABLE {tn} ADD COLUMN "{cn}" {ct}'.format(
tn=table_name, cn=new_column, ct=col_type))
conn.close()
my_list = ["Col1", "Col2", "Col3"]
create_sql_db(my_list)
All my data is of the type text, so I just have a single variable "col_type" - but you could for example feed in a list of tuples (or a tuple of tuples, if that's what you're into):
my_other_list = [("ColA", "TEXT"), ("ColB", "INTEGER"), ("ColC", "BLOB")]
and change the CREATE A COLUMN step to:
for tupl in my_other_list:
new_column = tupl[0] # "ColA", "ColB", "ColC"
col_type = tupl[1] # "TEXT", "INTEGER", "BLOB"
c.execute('ALTER TABLE {tn} ADD COLUMN "{cn}" {ct}'.format(
tn=table_name, cn=new_column, ct=col_type))
As a noob, I can't comment on the very succinct, updated solution #ron_g offered. While testing, though I had to frequently delete the sample database itself, so for any other noobs using this to test, I would advise adding in:
c.execute('DROP TABLE IF EXISTS {tn}'.format(
tn=table_name))
Prior the the 'CREATE TABLE ...' portion.
It appears there are multiple instances of
.format(
tn=table_name ....)
in both 'CREATE TABLE ...' and 'ALTER TABLE ...' so trying to figure out if it's possible to create a single instance (similar to, or including in, the def section).