I am really struggling with updating many rows in python using SAP HANA as my database and PyHDB for establishing the interface between both applications. Its working when I "hardcode" the columns, but I need to dynamically switch the columns by defining them inside of an array for example.
I am able to update in a hardcoded way the necessary columns by performing the following SQL-query:
sql = """UPDATE "ARE"."EMPLOYEES" SET "LIKELIHOOD_NO" = %s, "LIKELIHOOD_YES"= %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
What I want to achieve is the following scenario:
dynamic_columns = ["LIKELIHOOD_NO", "LIKELIHOOD_Yes"]
sql = """UPDATE "ARE"."EMPLOYEES" SET dynamic_column = %s, "LIKELIHOOD_YES" = %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
I am always getting the error that the relevant column / columns could not be found, but I cant figure out a way to solve this.
You can use normal string interpolation (.format()) to add the dynamic column name. You can see in the code here that pyHDB supports "numeric" paramstyle:
for col in ['LIKELIHOOD_YES', 'LIKELIHOOD_NO']:
sql = ('UPDATE "ARE"."EMPLOYEES" SET "{some_col}" = :1, "CLASS" = :2 '
'WHERE "EmployeeNumber" = :3;').format(some_col=col)
cursor.executemany(sql, list_of_tuples)
This code will run for both columns 'LIKELIHOOD_YES' and 'LIKELIHOOD_NO'. Adapt it as you need. It would work with a list of tuples like this:
list_of_tuples = [
(value1, class_1, employee_no_1),
(value2, class_2, employee_no_2),
(value3, class_3, employee_no_3),
(value4, class_4, employee_no_4),
]
The code in your question seems to be using the 'format' paramstyle instead, but that doesn't seem what pyHDB is using. See PEP 249 for more information on paramstyles.
Related
I want to pass list values along with other parameter values. following is my scenario. I want to pass multiple values for column "Code" and want to pass single value to "Continent" column.
param = [('AFG', 'IND'),'Asia']
query = "select * from country where Code in (%s) AND Continent = %s"
cursor.execute(query,param)
while executing in Python, I am getting following error.
Failed to execute Query: Failed processing format-parameters; Python
'tuple' cannot be converted to a MySQL type
The trick here is the WHERE IN clause, which isn't really amenable to being parameterized. One option generates an IN clause with the exact number of placeholders in your list:
codes = ('AFG', 'IND')
continent = 'Asia'
params = codes + (continent,)
where_in = ','.join(['%s'] * len(codes))
sql = "SELECT * FROM country WHERE Code IN (%s) AND Continent = %s" % (where_in, '%s')
cursor.execute(sql, params)
To see what the above script actually did, lets look at the various parts:
print(where_in)
print(sql)
%s,%s
SELECT * FROM country WHERE Code IN (%s,%s) AND Continent = %s
The trick here is that we actually use a %s placeholder twice, once for the Python string, and a second time for the SQL query string. Also, we bind a single level tuple containing all bound values:
('AFG', 'IND', 'ASIA')
first you split the list and then you split the tuple.
param = [('AFG', 'IND'),'Asia']
p1,p2=param[0]
query = "select * from country where Code in ('%s','%s') AND Continent = %s" % (p1,p2,param[1])
cursor.execute(query)
I'm trying to pass the same parameters to an oracle query in two separate places in the SQL code.
My code works if I hard code the criteria for table2 like this:
# define parameters
years = ['2018','2019']
placeholder= ':d'
placeholders= ', '.join(placeholder for unused in years)
placeholders
# create cursor
cursor = connection.cursor()
# query
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in ['2018','2019'] --here's where I say I'm hard coding criteria
""" % placeholders
data = cursor.execute(qry, years)
df = pd.DataFrame(data.fetchall(), columns = [column[0] for column in cursor.description])
# close database connection
connection.close()
If I try to use the parameter for table2 like this:
qry = """
select * from table1
INNER
JOIN table2
ON table1_id = table2_id
where table1_year in (%s)
and table2_year in (%s) --part of code I'm having issues with
""" % placeholders
I get the following error:
TypeError: not enough arguments for format string
I can't simply rewrite the SQL because I frequently have to use someone else's code and it wouldn't be feasible to rewrite all of it.
If you want to fill multiple placeholders, you have to supply the same number of parameters.
"one meal: %s" % "sandwich" # works
"two meals: %s, %s" % "sandwich" # not working
"two meals: %s, %s" % ("sandwich", "sandwich") # works
NOTE: It is a bad/dangerous thing to use string formatting for the assembly of SQL queries (lookup "SQL Injection"). In your case it is fine, but in general you should use parameterized queries, especially when dealing with input from untrusted sources like user input. You don't want a user to input "2018; DROP TABLE table1;".
I am working on Heroku Connect (integrated with Salesforce). One of my columns has the type of "Multipicklist".
I am working in Python, so using psycopg2.
What is the correct way for querying, matching a single variable value in a multipicklist type column?
An example is:
Say custom__c has column called multipicklistcol__c.
I have some custom__c row with multipicklistcol__c values of {'A','B','C'}.
If I have a variable param = 'A', how do I select my custom__c row using param?
I have tried a few different approaches, none of which work for me:
cur.execute("""SELECT name, sfid FROM salesforce.custom__c WHERE (%s) <# multipicklistcol__c;""", (param, ))
cur.execute("""SELECT name, sfid FROM salesforce.custom__c WHERE (%s) IN multipicklistcol__c;""", (param, ))
cur.execute("""SELECT name, sfid FROM salesforce.custom__c WHERE multipicklistcol__c includes (%s);""", (param, ))
cur.execute("""SELECT name, sfid FROM salesforce.custom__c WHERE (%s) = ANY(multipicklistcol__c)""", (param, ))
You have to write your own typecaster. See the docs about type casting. Maybe you can get away with just creating a new array type caster on a base type already handled.
Normally, if i want to insert values into a table, i will do something like this (assuming that i know which columns that the values i want to insert belong to):
conn = sqlite3.connect('mydatabase.db')
conn.execute("INSERT INTO MYTABLE (ID,COLUMN1,COLUMN2)\
VALUES(?,?,?)",[myid,value1,value2])
But now i have a list of columns (the length of list may vary) and a list of values for each columns in the list.
For example, if i have a table with 10 columns (Namely, column1, column2...,column10 etc). I have a list of columns that i want to update.Let's say [column3,column4]. And i have a list of values for those columns. [value for column3,value for column4].
How do i insert the values in the list to the individual columns that each belong?
As far as I know the parameter list in conn.execute works only for values, so we have to use string formatting like this:
import sqlite3
conn = sqlite3.connect(':memory:')
conn.execute('CREATE TABLE t (a integer, b integer, c integer)')
col_names = ['a', 'b', 'c']
values = [0, 1, 2]
conn.execute('INSERT INTO t (%s, %s, %s) values(?,?,?)'%tuple(col_names), values)
Please notice this is a very bad attempt since strings passed to the database shall always be checked for injection attack. However you could pass the list of column names to some injection function before insertion.
EDITED:
For variables with various length you could try something like
exec_text = 'INSERT INTO t (' + ','.join(col_names) +') values(' + ','.join(['?'] * len(values)) + ')'
conn.exec(exec_text, values)
# as long as len(col_names) == len(values)
Of course string formatting will work, you just need to be a bit cleverer about it.
col_names = ','.join(col_list)
col_spaces = ','.join(['?'] * len(col_list))
sql = 'INSERT INTO t (%s) values(%s)' % (col_list, col_spaces)
conn.execute(sql, values)
I was looking for a solution to create columns based on a list of unknown / variable length and found this question. However, I managed to find a nicer solution (for me anyway), that's also a bit more modern, so thought I'd include it in case it helps someone:
import sqlite3
def create_sql_db(my_list):
file = 'my_sql.db'
table_name = 'table_1'
init_col = 'id'
col_type = 'TEXT'
conn = sqlite3.connect(file)
c = conn.cursor()
# CREATE TABLE (IF IT DOESN'T ALREADY EXIST)
c.execute('CREATE TABLE IF NOT EXISTS {tn} ({nf} {ft})'.format(
tn=table_name, nf=init_col, ft=col_type))
# CREATE A COLUMN FOR EACH ITEM IN THE LIST
for new_column in my_list:
c.execute('ALTER TABLE {tn} ADD COLUMN "{cn}" {ct}'.format(
tn=table_name, cn=new_column, ct=col_type))
conn.close()
my_list = ["Col1", "Col2", "Col3"]
create_sql_db(my_list)
All my data is of the type text, so I just have a single variable "col_type" - but you could for example feed in a list of tuples (or a tuple of tuples, if that's what you're into):
my_other_list = [("ColA", "TEXT"), ("ColB", "INTEGER"), ("ColC", "BLOB")]
and change the CREATE A COLUMN step to:
for tupl in my_other_list:
new_column = tupl[0] # "ColA", "ColB", "ColC"
col_type = tupl[1] # "TEXT", "INTEGER", "BLOB"
c.execute('ALTER TABLE {tn} ADD COLUMN "{cn}" {ct}'.format(
tn=table_name, cn=new_column, ct=col_type))
As a noob, I can't comment on the very succinct, updated solution #ron_g offered. While testing, though I had to frequently delete the sample database itself, so for any other noobs using this to test, I would advise adding in:
c.execute('DROP TABLE IF EXISTS {tn}'.format(
tn=table_name))
Prior the the 'CREATE TABLE ...' portion.
It appears there are multiple instances of
.format(
tn=table_name ....)
in both 'CREATE TABLE ...' and 'ALTER TABLE ...' so trying to figure out if it's possible to create a single instance (similar to, or including in, the def section).
I'm trying to insert dict values into database. But i'm getting error .here is my dictionary. I'm using this code .
`cols = filter_dict.keys()
vals = filter_dict.values()
("INSERT INTO table (%s) VALUES (%s)", [cols, vals])
`
When I only print this query ("INSERT INTO table (%s) VALUES (%s)", [cols, vals]).
I'm getting following output.
('INSERT INTO table (%s) VALUES (%s)', [['cinematography', 'name', 'producer', 'caption', 'studio', 'editing'], ['Venkat Prasad', '100% Love', 'Bunny Vasu', '', '100% Love Poster.jpg', 'Chandra Sekhar T Ramesh,Hari Prasad']])
But When I execute this query I"m getting strange output.
the manual that corresponds to your MySQL server version for the right syntax to use near \'table (("\'cinematography\'", "\'name\
Why execute query adding `\' to each columns?? Can any one help me? thanks
The backslashes are likely due to your debugger. Are you running your code in the interactive prompt?
Anyway, the actual problems are:
table is a reserved word. You should put table in backticks.
If your table contains more than one column, you need to have multiple %s and multiple values.
For example:
"INSERT INTO `table` (%s, %s, &s) VALUES (%s, %s, %s)"
You will also need to change (cols, vals) to list the individual values.
(cols[0], cols[1], cols[2] , vals[0], vals[1], vals[2])
I'd also strongly suggest that you try to find a better name for your table, preferably one that:
1) describes what sort of data the table contains and
2) isn't a reserved word