Develop Reference

Using variables in redshift insert command - Python lambda function

I am trying to insert records to a redshift table using a lambda function. I am using
boto3.client('redshift-data')
for the same. Now I have the query as below.
query1 = "insert into dbname.tablename values('aaaa','bbbb','cccc')"
response = rsclient.execute_statement(
ClusterIdentifier='xxxxx',
Database='yyyy',
DbUser='zzzz',
Sql= query1,
StatementName='examplestatement'
)
This works fine. But I want to pass variables here instead of values. For instance,
var1 = 'aaaa'
var2 = 'bbbb'
var3 = 'cccc'
Then try the query as below but it it doesn't work, I think it something silly to do with quotes.
query1 = "insert into dbname.tablename values(var1,var2,var3)"
How can I achieve this. I write lambda function using python3. Any help is appreciated.

You can use f-strings:
query1 = f"insert into dbname.tablename values('{var1}','{var2}','{var3}')"

Illegal Variable Name/Number when Passing in Python List

I'm trying to run SQL statements through Python on a list.
By passing in a list, in this case date. Since i want to run multiple SELECT SQL queries and return them.
I've tested this by passing in integers, however when trying to pass in a date I am getting ORA-01036 error. Illegal variable name/number. I'm using an Oracle DB.
cursor = connection.cursor()
date = ["'01-DEC-21'", "'02-DEC-21'"]
sql = "select * from table1 where datestamp = :date"
for item in date:
cursor.execute(sql,id=item)
res=cursor.fetchall()
print(res)
Any suggestions to make this run?

You can't name a bind variable date, it's an illegal name. Also your named variable in cursor.execute should match the bind variable name. Try something like:
sql = "select * from table1 where datestamp = :date_input"
for item in date:
cursor.execute(sql,date_input=item)
res=cursor.fetchall()
print(res)

Some recommendation and warnings to your approach:
you should not depend on your default NLS date setting, while binding a String (e.g. "'01-DEC-21'") to a DATE column. (You probably need also remone one of the quotes).
You should ommit to fetch data in a loop if you can fetch them in one query (using an IN list)
use prepared statement
Example
date = ['01-DEC-21', '02-DEC-21']
This generates the query that uses bind variables for your input list
in_list = ','.join([f" TO_DATE(:d{ind},'DD-MON-RR','NLS_DATE_LANGUAGE = American')" for ind, d in enumerate(date)])
sql_query = "select * from table1 where datestamp in ( " + in_list + " )"
The sql_query generate is
select * from table1 where datestamp in
( TO_DATE(:d0,'DD-MON-RR','NLS_DATE_LANGUAGE = American'), TO_DATE(:d1,'DD-MON-RR','NLS_DATE_LANGUAGE = American') )
Note that the INlist contains one bind variable for each member of your input list.
Note also the usage of to_date with explicite mask and fixing the language to avoid problems with interpretation of the month abbreviation. (e.g. ORA-01843: not a valid month)
Now you can use the query to fetch the data in one pass
cur.prepare(sql_query)
cur.execute(None, date)
res = cur.fetchall()

How to insert NULL value in SQL nvarchar column using python

I am trying to insert None from python in SQL table, but it is inserting as string 'null'
I am using below set of code.
query_update = f'''update table set Name = '{name}',Key = '{Key}' where Id = {id} '''
stmt.execute(query_update)
conn.commit()
I am getting values in python for variable 'Key' these values I am trying to update in column "Key". These two columns are nvarchar columns.
Now sometime we get None values Null in variable 'Key' as below
Key = 'Null'
So when I insert above value in SQL it is getting updated as string instead of NULL, as I had to put quote in script while updating from Python.
Can anyone help me here, How can I avoid inserting string while inserting Null in SQL from Python

The problem that your Null values are actually a string and not "real" Null.
If you want to insert Null, your key should be equal to None.
You can can convert it as follows:
Key = Key if Key != 'Null' else None

I guess the problem here is that you are placing quotes around the placeholders. Have a look how I think the same can be done.
query_update = 'update table set Name = {name}, Key = {Key} where Id = {id}'
query_update.format(name='myname', Key = 'mykey', Id = 123)
stmt.execute(query_update)
conn.commit()

Don't use dynamic SQL. Use a proper parameterized query:
# test data
name = "Gord"
key = None # not a string
id = 1
query_update = "update table_name set Name = ?, Key = ? where Id = ?"
stmt.execute(query_update, name, key, id)
conn.commit()

Executemany() SQL-Update Statement with variables for column names

I am really struggling with updating many rows in python using SAP HANA as my database and PyHDB for establishing the interface between both applications. Its working when I "hardcode" the columns, but I need to dynamically switch the columns by defining them inside of an array for example.
I am able to update in a hardcoded way the necessary columns by performing the following SQL-query:
sql = """UPDATE "ARE"."EMPLOYEES" SET "LIKELIHOOD_NO" = %s, "LIKELIHOOD_YES"= %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
What I want to achieve is the following scenario:
dynamic_columns = ["LIKELIHOOD_NO", "LIKELIHOOD_Yes"]
sql = """UPDATE "ARE"."EMPLOYEES" SET dynamic_column = %s, "LIKELIHOOD_YES" = %s, "CLASS" = %s WHERE "EmployeeNumber" = %s;"""
cursor.executemany(sql, list)
connection.commit()
I am always getting the error that the relevant column / columns could not be found, but I cant figure out a way to solve this.

You can use normal string interpolation (.format()) to add the dynamic column name. You can see in the code here that pyHDB supports "numeric" paramstyle:
for col in ['LIKELIHOOD_YES', 'LIKELIHOOD_NO']:
sql = ('UPDATE "ARE"."EMPLOYEES" SET "{some_col}" = :1, "CLASS" = :2 '
'WHERE "EmployeeNumber" = :3;').format(some_col=col)
cursor.executemany(sql, list_of_tuples)
This code will run for both columns 'LIKELIHOOD_YES' and 'LIKELIHOOD_NO'. Adapt it as you need. It would work with a list of tuples like this:
list_of_tuples = [
(value1, class_1, employee_no_1),
(value2, class_2, employee_no_2),
(value3, class_3, employee_no_3),
(value4, class_4, employee_no_4),
]
The code in your question seems to be using the 'format' paramstyle instead, but that doesn't seem what pyHDB is using. See PEP 249 for more information on paramstyles.

How to use python to execute mysql and use replace into - with more than 255 variables ?

Here below is my code that I am using
con.execute("""
REPLACE INTO T(var1,var2,...,var300)VALUES(?,?,....?)""",(var1,var2,...,var300)
This statement works just fine if I have var1-var255 , once I have more than that it gave me an error...
So far, I was able to split T into 2 different times
con.execute("""
REPLACE INTO T(var1,var2,...,var150)VALUES(?,?,....?)""",(var1,var2,...,var150)
con.execute("""
REPLACE INTO T(var151,var152,...,var300)VALUES(?,?,....?)""",(var151,var152,...,var300)
This gave me no error , but my final value in table "T" would only values in the second execute statement , all of var1, var2, ... var 150 got replace with null

Have you tried using update instead?
MySQL documentation tells the following:
"REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted"

There does not seem to be any inherent problem using more than 255 columns in MySQL, interfaced with MySQLdb:
import MySQLdb
import config
connection = MySQLdb.connect(
host = config.HOST, user = config.USER,
passwd = config.PASS, db = 'test')
cursor = connection.cursor()
cols = ['col{i:d}'.format(i =i) for i in range(300)]
types = ['int(11)']*len(cols)
columns = ','.join('{f} {t}'.format(f = f, t = t) for f, t in zip(cols, types))
sql = '''CREATE TABLE IF NOT EXISTS test (
id INT(11) NOT NULL AUTO_INCREMENT,
{c},
PRIMARY KEY (id)
)'''.format(c = columns)
cursor.execute(sql)
sql = '''REPLACE INTO test({c}) VALUES ({v})'''.format(
c = ','.join(cols),
v = ','.join(['%s']*len(cols)))
cursor.execute(sql, range(300))
result = cursor.fetchall()
This adds rows to test.test without a problem.