I am connecting to MS Access using ODBC (Python ODBC module ). There is one part of the code which put some values into DB. Looks similar to this:
for item in changes:
format_str = """INSERT INTO changes (short_description) VALUES ('{short_description}');"""
sql_command = format_str.format(short_description =item.short_description)
cursor.execute(sql_command)
cursor.commit()
The problem is that it returns syttaxt error:
pypyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error (missing operator) in query expression
I found that it is because for one the cases I have short_description like this one:
The Service Request status is not changing to \"OPEN', once dispatched
to another group
the problem is for " ' " after OPEN.
to provide you full picture here. Actually the string I see is this:
The Service Request status is not changing to "OPEN', once dispatched
to another group
The string with "\" I get from the API for the application which serves the data. It adds "\" to escape string, but not everywhere.
The question is - what will be the simplest way to solve it?
Theoretically I could replace\remove unwanted signs, but what in case I want to keep it as it is?
For any other cases all works fine.
You should to use parameters to avoid sql injection:
for item in changes:
sql_command = """INSERT INTO changes (short_description) VALUES (?);"""
cursor.execute(sql_command, (item.short_description,) )
cursor.commit()
Related
I am using SQL server and need to run the following SQL via Python script
SELECT DISTINCT LEN(Wav)-CHARINDEX('.', Wav) FROM <>;
I have tried to play with the String but couldn’t figure out how to work around the dot character.
sql = 'SELECT DISTINCT LEN(Wav)-CHARINDEX({}, Wav) FROM xxx'.format('.')
print(sql)
cursor = conn.cursor()
cursor.execute(sql)
Any idea how to resolve this
Thank you
'.' is the string ., you want "'.'", the string '.'
>>> print("{}".format('.'))
.
>>> print("{}".format("'.'"))
'.'
As #Justin Ezequiel's answer notes, do beware of SQL injections here!
Specifically, unfiltered user inputs can and will cause an SQL injection where unanticipated commands can be run against the target database by breaking out of the raw string. These can do anything your connection has permission to do, such as retrieving, modifying, or deleting arbitrary data.
A traditional approach is to use prepared statements
In Python, you can also use a regex or other test to explicitly error for statements with control characters (if not re.match(r"^[a-zA-Z\d _+-]+$"), s):raise_) or use (trust) an escaping library to do it for you if you must take arbitrary strings.
Use parameters to avoid SQL-injection attacks.
sql = 'SELECT DISTINCT LEN(Wav)-CHARINDEX(?, Wav) FROM xxx' # note placeholder (?)
print(sql)
params = ('.',) # tuple
cursor = conn.cursor()
cursor.execute(sql, params)
I am trying to create an 'upsert' function for pypyodbc SQL Server. I have validated that the query built up will run in SSMS with the desired outcome, but when trying to execute and commit with pypyodbc I receive the following error: 'HY007', '[HY007] [Microsoft][ODBC SQL Server Driver]Associated statement is not prepared'.
Here is the upsert function:
def sql_upsert(sql_connection, table, key_field, key_value, **kwargs):
keys = ["{key}".format(key=k) for k in kwargs]
values = ["'{value}'".format(value=v) for v in kwargs.values()]
update_columns = ["{key} = '{value}'".format(key=k, value=v) for k, v in kwargs.items()]
sql = list()
#update
sql.append("UPDATE {table} SET ".format(table=table))
sql.append(", ".join(update_columns))
sql.append(" WHERE {} = '{}'".format(key_field, key_value))
sql.append(" IF ##ROWCOUNT=0 BEGIN ")
# insert
sql.append("INSERT INTO {table} (".format(table=table))
sql.append(", ".join(keys))
sql.append(") VALUES (")
sql.append(", ".join(values))
sql.append(")")
sql.append(" END")
query = "".join(sql)
print(query)
The function builds up a query string in a format based on this other thread How to insert or update using single query?
Here is an example of the output:
UPDATE test SET name='john' WHERE id=3012
IF ##ROWCOUNT=0 BEGIN
INSERT INTO test(name) VALUES('john')
END
The error message you cited is produced by the ancient "SQL Server" ODBC driver that ships as part of Windows. A more up-to-date driver version like "ODBC Driver 17 for SQL Server" should produce a meaningful error message.
If you look here or here you'll see people complaining about this over a decade ago.
Apparently SQL Server's ODBC driver returns that error when you're executing two statements that fail due to a field value being too long, or perhaps due to foreign key violations.
Using SSMS to see which statement causes this problem, or better - stop using ODBC and use pymssql
This error may also come when you don't give correct permissions to stored procedure
Go the SQL server --> Right click on your sp-->properties-->permissions
Add required users and roles which are going to execute this sp
This may help resolving the issue
Quick rundown:
The idea here is to read some data in from a csv file, and use that as the list in the NOT IN part of my sql query. I'm connecting to the db (.mdb) with the code below.
Note LP is the tuple/list I'm trying to pass, IRdb is the path to the db
constr = r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=' + IRdb
conn = pyodbc.connect(constr, autocommit=True)
cur = conn.cursor()
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN ?'''
cur.execute(IRsql,('Total',r'Conn%',LP))
The issue:
Everything works fine except for the execute statement (which did work before i added the NOT IN part). I've tried passing LP as string, tuple, and list, but nothing seems to be working. I get the following error
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] In operator without () in query expression 'IRRPResults.PointName<>Pa_RaM000 AND IRRPResults.RRPName NOT LIKE Pa_RaM001 AND IRRPResults.PointName NOT IN Pa_RaM002'. (-3100) (SQLExecDirectW)")
Any help would be greatly appreciated.
For anyone interested, or for that guy reading this 2 years from now with the same issue, or my future self when I forget what i did, I've figured out a solution or two.
The first work around was to simply use .format on the sql string to insert LP directly before it gets passed to the execute statement.
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN {}'''.format(LP)
cur.execute(IRsql,('Total',r'Conn%'))
The other solution, I got from this question, is a little more elegant and clever in that it builds a string of '?' markers for each element in LP. Then LP gets passed as a tuple/list to the execute statement.
placeholders=','.join('?'*len(LP))
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN ({})'''.format(placeholders)
cur.execute(IRsql,('Total',r'Conn%',*LP))
When running this minimal code:
import pypyodbc
conn = pypyodbc.connect(r'Driver={{Microsoft Access Driver (*.mdb, *.accdb)}}; Dbq=C:\temp\example.accdb;'
cur = conn.cursor()
cur.execute('SELECT [Pass#] FROM [Companies]')
I get the following error:
pypyodbc.DatabaseError: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 1.')
The cause of the error appears to be the '#' character, which is a special wildcard character for MS Access. However, I can't figure out any way to escape it. Similar errors suggest the the square brackets ([]) are the way to escape, but it doesn't seem to work.
I have tried these variations like these with no success:
cur.execute('SELECT [Pass[#]] FROM [Companies]')
cur.execute('SELECT Pass[#] FROM [Companies]')
cur.execute('SELECT [Pass\\#] FROM [Companies]')
cur.execute('SELECT Pass# FROM [Companies]')
I should also mention the Access DB is not controlled by me or my company, so I am unable to rename the column.
Thanks to Gord Thompson for pointing me in the right direction. Long story short, the schema I was working against was old, and the field [Pass#] had been renamed to [PassID].
As it turns out however, the behavior was still confusing for other fields in my DB. For anyone who finds this answer in the future, it would appear that the Access ODBC driver invoked this way will give the error Too few parameters. Expected N. whenever you have wrong column names, instead of a more helpful Column name not recognized or similar. # characters are allowed in the column names, as long as the name is square-bracketed.
I am trying to execute SQL query within Access database using PYODBC and I get following error:
pyodbc.Error: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver]
Too few parameters. Expected 1. (-3010) (SQLExecDirectW)')
The problem is that I am not using any additional parameters. Here is the code:
access_con_string = r"Driver={};Dbq={};".format(driver, base)
cnn = pyodbc.connect(access_con_string)
db_cursor = cnn.cursor()
expression = """SELECT F_ARODES.ARODES_INT_NUM, F_ARODES.TEMP_ADRESS_FOREST,F_AROD_LAND_USE.ARODES_INT_NUM, F_ARODES.ARODES_TYP_CD
FROM F_ARODES LEFT JOIN F_AROD_LAND_USE ON F_ARODES.ARODES_INT_NUM = F_AROD_LAND_USE.ARODES_INT_NUM
WHERE (((F_AROD_LAND_USE.ARODES_INT_NUM) Is Null) AND ((F_ARODES.ARODES_TYP_CD)="wydziel") AND ((F_ARODES.TEMP_ACT_ADRESS)=True));"""
db_cursor.execute(expression)
Query itself, if used inside MS-Access works fine. Also, connection is OK, as other queries are executed properly.
What am I doing wrong?
Constants in such queries are problematic - you never know the exact underlying syntax for booleans, strings etc. - even if it works in MS-Access, it can be different inside the intermediary library you're using.
The safest way is to extract them as parameters anyway:
expression = """SELECT F_ARODES.ARODES_INT_NUM, F_ARODES.TEMP_ADRESS_FOREST,F_AROD_LAND_USE.ARODES_INT_NUM, F_ARODES.ARODES_TYP_CD FROM F_ARODES LEFT JOIN F_AROD_LAND_USE ON F_ARODES.ARODES_INT_NUM = F_AROD_LAND_USE.ARODES_INT_NUM WHERE (((F_AROD_LAND_USE.ARODES_INT_NUM) Is Null)
AND ((F_ARODES.ARODES_TYP_CD)=?) AND ((F_ARODES.TEMP_ACT_ADRESS)=?));"""
db_cursor.execute(expression, "wydziel", True)
I had a similar problem, with an update I was trying to perform with pyodbc. When executed in Access, the query worked fine, same for when using the application (it allows some queries from within the app). But when ran in python with pyodbc the same text would throw errors. I determined the problem is the double quote (OP's query has a set of them as well). The query began to work when I replaced them with single quotes.
This does not work:
Update ApplicationStandards Set ShortCutKey = "I" Where ShortName = "ISO"
This does:
Update ApplicationStandards Set ShortCutKey = 'I' Where ShortName = 'ISO'