I want to delete rows in a snowflake table based on a list I convert from df. However, when I execute the code below, the error shows TypeError: not all arguments converted during string formatting
Here is some mock up code similar to my setup
data = ['5003s00000gUnEqAAK','5003s00000gUnEqAAK']
id_df = pd.DataFrame(data,columns=['ID'])
l = id_df['ID'].to_list()
delete_query = f"""delete from xx.table1 where id in (%s)""" % ','.join('?' * len(l))
conn.cursor().execute(delete_query.l)
What did I do wrong so the engine cannot format the variable to the correct way?
I expect the engine will plug the list in where clause correctly
I am automating a task through Python that will run an SQL statement to insert into an existing table in a DB.
My CSV headers look as such:
ID,ACCOUNTID,CATEGORY,SUBCATEGORY,CREATION_DATE,CREATED_BY,REMARK,ISIMPORTANT,TYPE,ENTITY_TYPE
My values:
seq_addnoteid.nextval,123456,TEST,ADMN_TEST,sysdate,ME,This is a test,Y,1,A
NOTE: Currently, seq_addnote works from DB but in my code i added a small snippet to get the max ID and the rows will increase this by one for each iteration.
Sysdate could also be passed as format '19-MAY-22'
If i was to run from DB, this would work:
insert into notes values(seq_addnoteid.nextval,'123456','TEST','ADMN_TEST',sysdate,'ME','This is a test','Y',1,'A');
# Snippet to get function
cursor.execute("SELECT MAX(ID) from NOTES")
max = cursor.fetchone()
max = int(max[0])
with open ('sample.csv', 'r') as f:
reader = csv.reader(f)
columns = next(reader)
query = 'INSERT INTO NOTES({0}) values ({1})'
query = query.format(','.join(columns), ','.join('?' * len(columns)))
cursor = conn.cursor()
for data in reader:
cursor.execute(query, data)
conn.commit()
print("Records inserted successfully")
cursor.close()
conn.close()
Currently, i'm getting Oracle-Error-Message: ORA-01036: illegal variable name/number and i think its because of my query.format line. However, I'm looking for help to get this code to handle the data types properly.
Thanks!
Try printing your query before you execute it. I think you'll find that it's printing this:
INSERT INTO NOTES(ID,ACCOUNTID,CATEGORY,SUBCATEGORY,CREATION_DATE,CREATED_BY,REMARK,ISIMPORTANT,TYPE,ENTITY_TYPE)
values(seq_addnoteid.nextval,123456,TEST,ADMN_TEST,sysdate,ME,This is a test,Y,1,A);
Which will also give you a ORA-01036 if you try to run it manually.
The problem is that you want some of your column values to be literal values, and some of them to be strings escaped in single-quotes, and your code doesn't do that. I don't think there's an easy to way to do it with ','.join(), so you'll either need to modify your CSVs to quote the strings, like:
seq_addnoteid.nextval,"'123456'","'TEST'","'ADMN_TEST'",sysdate,"'ME'","'This is a test'","'Y'",1,"'A'"
Or modify your query.format to add the quotes around the parameters that you want to treat as strings:
query.format(','.join(columns), "?,'?','?','?',?,'?','?','?',?,'?'")
As the commenters mentioned, pandas does handle this all very nicely.
EDIT: I see what you're saying. I'm not sure pandas will help with the literal functions you want to pass to the insert. But yes, you should be able to change your CSV and then do:
query.format(','.join(columns) + ',ID,CREATION_DATE', "'?','?','?','?','?','?',?,'?',seq_addnoteid.nextval,sysdate")
As a side note, a lot of people do this sort of thing on the database side in a BEFORE INSERT trigger, e.g.:
create or replace trigger NOTES_INS_TRG
before insert on NOTES
for each row
begin
:NEW.ID := seq_addnoteid.nextval;
:NEW.CREATION_DATE := sysdate;
end;
/
Then you could leave those columns out of your insert entirely.
Edit again:
I'm not sure you can use ? for bind/substitution variables in cx_oracle (see documentation ). So where your raw query is currently:
INSERT INTO NOTES(ACCOUNTID,CATEGORY,SUBCATEGORY,CREATED_BY,REMARK,ISIMPORTANT,TYPE,ENTITY_TYPE,ID,CREATION_DATE)
values (seq_addnoteid.nextval,sysdate,'?','?','?','?','?','?',?,'?')
You'd need something like:
INSERT INTO NOTES(ACCOUNTID,CATEGORY,SUBCATEGORY,CREATED_BY,REMARK,ISIMPORTANT,TYPE,ENTITY_TYPE,ID,CREATION_DATE)
values (seq_addnoteid.nextval,sysdate,:1,:2,:3,:4,:5,:6,:7,:8)
We can probably do that by modifying the format string again to generate some bind variables:
query.format('ID,CREATION_DATE,' + ','.join(columns),
"seq_addnoteid.nextval,sysdate," + ','.join([':'+c for c in columns])
Again, try printing the query before executing it to make sure the column names and values are lining up correctly.
as mentioned in the title i get this error when i try to execute a prepared statement. The full error is:
RuntimeError: No matching overloads found for prepareStatement in find. at native\common\jp_method.cpp:127
As far as i can understand is, that propably that since i am trying to use a prepared statement, that the compiler can not find something to overload the ? placeholder.
Code snippet:
curs = self.dbconn.cursor()
sqlLogin = ("SELECT name,email FROM BENUTZER where name=? and email=?", ( benutzerObjekt.name,benutzerObjekt.email))
curs.execute(sqlLogin)
The error seems to happen at curs.execute(sqlLogin), which is shown to me in the traceback when debugging.
I'm trying to use the input of an html input, which is stored in benutzerObjekt.name and benutzerObjekt.email as input for the select SQL statement. So most probably something is wrong with either my SQL statement, or the execution of the statement, which is underlined when debugging. I am using db2.
Thanks in advance!
You need to pass parameters as second argument in cursor.execute. Right now your query is a nested tuple of two items with first being a string and second item being a tuple of two values.
Consider separating the arguments for function call:
curs = self.dbconn.cursor()
# SINGLE STRING VALUE
sqlLogin = "SELECT name,email FROM BENUTZER WHERE name=? AND email=?"
# TUPLE OF TWO VALUES
vals = (benutzerObjekt.name, benutzerObjekt.email)
# PASS SQL AND PARAMS SEPARATELY
curs.execute(sqlLogin, vals)
Alternatively, you can unpack your nested tuple using asterisk, *:
sqlLogin = (
"SELECT name,email FROM BENUTZER where name=? and email=?",
(benutzerObjekt.name, benutzerObjekt.email)
)
curs.execute(*sqlLogin)
I'm trying to return a hard coded value in my SQL query, but when running the query using pyodbc, random records return '\x0e' instead of the hard coded value (in this case '16'). If I run the query on the server (MS SQL Server 2008), the query returns all the correct results and values.
The beginning of the query looks like this:
My SQL Code:
Select '"16","' + S.ShipNum + '","'
My python code:
cursor.execute("""Select '\"16\",\"' + SS.ShipNum + '\",\"'
Is there another way to guarantee a value is returned from a query?
\016 is the oct representation of \x0e
So I would think that it has more to do with the way in which you are escaping your double quotes. In your python you are actually geting \16 and not "16" as you desire.
You should try a prepared statment maybe.
ps = db.prepare("SELECT 16")
ps()
returns:
[(16,)]
Addtional examples can be seen here:
[http://python.projects.pgfoundry.org/docs/0.8/driver.html#parameterized-statements]
You can see all of the ascii and other character sets here
[http://donsnotes.com/tech/charsets/ascii.html]
It looks like you're trying to create a comma-delimited, quoted, string representation of the row. Don't try to do this in the database query, string formatting isn't one of T-SQL's strengths.
Pass the static value using a parameter, then join the row values. Using sys.databases for the example:
params = ("Some value",)
sql = "SELECT ?, name, user_access_desc FROM sys.databases"
for row in cursor.execute(sql):
print(','.join('"{0}"'.format(column) for column in row))
I'm trying execute a simple query, but getting this error no matter how I pass the parameters.
Here is the query (I'm using Trac db object to connect to a DB):
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
schema and each['id'] both are simple strings
print("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
Result:
SELECT name FROM "Planing".customer WHERE firm_id='135'
There is on error is a remove quote after firm_id=, but that way parameter is treated a an integer and ::text leads to the very same error.
In my case I didn't realize that you had to pass a tuple to cursor.execute. I had this:
cursor.execute(query, (id))
But I needed to pass a tuple instead
cursor.execute(query, (id,))
I got this same error and couldn't for the life of me work out how to fix, in the end it was my mistake because I didn't have enough parameters matching the number of elements in the tuple:
con.execute("INSERT INTO table VALUES (%s,%s,%s,%s,%s)",(1,2,3,4,5,6))
Note that I have 5 elements in the values to be inserted into the table, but 6 in the tuple.
It is recommended to not use string interpolation for passing variables in database queries, but using string interpolation to set the table name is fine as long as it's not an external input or you restrict the allowed value. Try:
cursor.execute("""
SELECT name FROM %s.customer WHERE firm_id=%%s
""" % schema, (each['id'],))
Rules for DB API usage provides guidance for programming against the database.
Use AsIs
from psycopg2.extensions import AsIs
cursor.execute("""
select name
from %s.customer
where firm_id = %s
""",
(AsIs(schema), each['id'])
)
You could try this:
cursor.execute("INSERT INTO table_name (key) VALUES(%s)",(value1,))
You will get an error if you are missing a (,) after value1.
The correct way to pass variables in a SQL command is using the second argument of the execute() method. And i think you should remove single quotes from second parameter, read about it here - http://initd.org/psycopg/docs/usage.html#the-problem-with-the-query-parameters.
Note that you cant pass table name as parameter to execute and it considered as bad practice but there is some workarounds:
Passing table name as a parameter in psycopg2
psycopg2 cursor.execute() with SQL query parameter causes syntax error
To pass table name try this:
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id=%s""" % (schema, '%s'), (each['id'],))
Every time I have this kind of error, I am passing the wrong amount of values. Try check it