Develop Reference

Pyodbc can't parse query from iql file

I have a docker image with a script which reads an iql file and executes it in impala. Removing the file reference and running a simple script works fine (ie SELECT 1;), but whenever I try to execute the script from a file I get an error
pyodbc.Error: ('HY000', '[HY000] [Cloudera][ImpalaODBC] (110) Error while executing a query in Impala: [HY000] : ParseException: Syntax error in line 2:\n\n^\nEncountered: EOF\nExpected: ALTER, COMMENT, COMPUTE, COPY, CREATE, DELETE, DESCRIBE, DROP, EXPLAIN, GRANT, INSERT, INVALIDATE, LOAD, REFRESH, REVOKE, SELECT, SET, SHOW, TRUNCATE, UPDATE, UPSERT, USE, VALUES, WITH\n\nCAUSED BY: Exception: Syntax error\n (110) (SQLExecDirectW)'
I have tried removing DriverManagerEncoding=UTF-16 from the cloudera.impalaodbc.ini file and various types of sql query (including changing the file extension to isql, iusql) but always get the same error.
Any ideas?
Python function:
def process_data_to_impala():
conn = pyodbc.connect(DSN='some_DSN', autocommit=True)
crsr = conn.cursor()
with open('/path/to/sql/file.iql','r') as inserts:
for statement in inserts:
crsr.execute(statement)
print(crsr.fetchall())

The problem relates to the way the python code is looping through the statement and the way ';' ends the query.
A better way to do it for a single query in a file is
with open('file/location.iql') as script:
statement = script.read()
print(statement)
crsr = conn.cursor()
crsr.execute(statement)
And when you have multiple queries in a file and you need to loop:
with open('file/location.iql') as script:
sqlscript = script.read()
for statement in sqlscript.split(';'):
print(statement)
crsr = conn.cursor()
crsr.execute(statement)
When reading multiple queries from one script (ie Drop table, create table) you must remove ';' from the last query. This is because the split creates a final blank query after the last ';'. See this question for more info.
I hope this helps somebody in the future!

Invalid Cursor State Python SQL Statement

I am writing a simple python program to retrieve information from a localhost database i set up.
I am encountering the following error:
('24000', '[24000] [Microsoft][ODBC Driver 17 for SQL Server]Invalid cursor state')
My code:
try:
cursor = conn.cursor()
cursor.execute(getSQLStatement('SelectRecipe'),[item])
recipe = cursor.fetchone()[0]
cursor.close()
except Exception as e:
print(e)
return
The query i am running works fine in the database client, the query isnt the issue, it is a simple SELECT statement:
SELECT
Recipe
FROM recipes
WHERE Name = ? --(item)
My desired result is just the most basic statement just so i can get it running, and then be able to expand the complexity later, but something is wrong at this level. I am just trying to get the value of one record and one column i believe with this code. When i searched other similar issues people had with this the answer was always related to multiple result sets being attempted to be returned? My query is just one result set and i am only running it once? I am kind of scratching my head here to find what i am missing?
The error seems to be related to the:
recipe = cursor.fetchone()[0]
...line as it runs fine without throwing the error without this line, however then i am not sure how to get the data as a variable if i dont do it that way.
Wondering if anyone could help me out here, if there is a better way to get my data or if there is something wrong with my implementation of this way. Thanks.

Python - special characters in SQL statement

I am connecting to MS Access using ODBC (Python ODBC module ). There is one part of the code which put some values into DB. Looks similar to this:
for item in changes:
format_str = """INSERT INTO changes (short_description) VALUES ('{short_description}');"""
sql_command = format_str.format(short_description =item.short_description)
cursor.execute(sql_command)
cursor.commit()
The problem is that it returns syttaxt error:
pypyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error (missing operator) in query expression
I found that it is because for one the cases I have short_description like this one:
The Service Request status is not changing to \"OPEN', once dispatched
to another group
the problem is for " ' " after OPEN.
to provide you full picture here. Actually the string I see is this:
The Service Request status is not changing to "OPEN', once dispatched
to another group
The string with "\" I get from the API for the application which serves the data. It adds "\" to escape string, but not everywhere.
The question is - what will be the simplest way to solve it?
Theoretically I could replace\remove unwanted signs, but what in case I want to keep it as it is?
For any other cases all works fine.

You should to use parameters to avoid sql injection:
for item in changes:
sql_command = """INSERT INTO changes (short_description) VALUES (?);"""
cursor.execute(sql_command, (item.short_description,) )
cursor.commit()

Error in MS Access SELECT statement when number symbol (#) present

When running this minimal code:
import pypyodbc
conn = pypyodbc.connect(r'Driver={{Microsoft Access Driver (*.mdb, *.accdb)}}; Dbq=C:\temp\example.accdb;'
cur = conn.cursor()
cur.execute('SELECT [Pass#] FROM [Companies]')
I get the following error:
pypyodbc.DatabaseError: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 1.')
The cause of the error appears to be the '#' character, which is a special wildcard character for MS Access. However, I can't figure out any way to escape it. Similar errors suggest the the square brackets ([]) are the way to escape, but it doesn't seem to work.
I have tried these variations like these with no success:
cur.execute('SELECT [Pass[#]] FROM [Companies]')
cur.execute('SELECT Pass[#] FROM [Companies]')
cur.execute('SELECT [Pass\\#] FROM [Companies]')
cur.execute('SELECT Pass# FROM [Companies]')
I should also mention the Access DB is not controlled by me or my company, so I am unable to rename the column.

Thanks to Gord Thompson for pointing me in the right direction. Long story short, the schema I was working against was old, and the field [Pass#] had been renamed to [PassID].
As it turns out however, the behavior was still confusing for other fields in my DB. For anyone who finds this answer in the future, it would appear that the Access ODBC driver invoked this way will give the error Too few parameters. Expected N. whenever you have wrong column names, instead of a more helpful Column name not recognized or similar. # characters are allowed in the column names, as long as the name is square-bracketed.

Too few parameters error, while no parameters placeholders used

I am trying to execute SQL query within Access database using PYODBC and I get following error:
pyodbc.Error: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver]
Too few parameters. Expected 1. (-3010) (SQLExecDirectW)')
The problem is that I am not using any additional parameters. Here is the code:
access_con_string = r"Driver={};Dbq={};".format(driver, base)
cnn = pyodbc.connect(access_con_string)
db_cursor = cnn.cursor()
expression = """SELECT F_ARODES.ARODES_INT_NUM, F_ARODES.TEMP_ADRESS_FOREST,F_AROD_LAND_USE.ARODES_INT_NUM, F_ARODES.ARODES_TYP_CD
FROM F_ARODES LEFT JOIN F_AROD_LAND_USE ON F_ARODES.ARODES_INT_NUM = F_AROD_LAND_USE.ARODES_INT_NUM
WHERE (((F_AROD_LAND_USE.ARODES_INT_NUM) Is Null) AND ((F_ARODES.ARODES_TYP_CD)="wydziel") AND ((F_ARODES.TEMP_ACT_ADRESS)=True));"""
db_cursor.execute(expression)
Query itself, if used inside MS-Access works fine. Also, connection is OK, as other queries are executed properly.
What am I doing wrong?

Constants in such queries are problematic - you never know the exact underlying syntax for booleans, strings etc. - even if it works in MS-Access, it can be different inside the intermediary library you're using.
The safest way is to extract them as parameters anyway:
expression = """SELECT F_ARODES.ARODES_INT_NUM, F_ARODES.TEMP_ADRESS_FOREST,F_AROD_LAND_USE.ARODES_INT_NUM, F_ARODES.ARODES_TYP_CD FROM F_ARODES LEFT JOIN F_AROD_LAND_USE ON F_ARODES.ARODES_INT_NUM = F_AROD_LAND_USE.ARODES_INT_NUM WHERE (((F_AROD_LAND_USE.ARODES_INT_NUM) Is Null)
AND ((F_ARODES.ARODES_TYP_CD)=?) AND ((F_ARODES.TEMP_ACT_ADRESS)=?));"""
db_cursor.execute(expression, "wydziel", True)

I had a similar problem, with an update I was trying to perform with pyodbc. When executed in Access, the query worked fine, same for when using the application (it allows some queries from within the app). But when ran in python with pyodbc the same text would throw errors. I determined the problem is the double quote (OP's query has a set of them as well). The query began to work when I replaced them with single quotes.
This does not work:
Update ApplicationStandards Set ShortCutKey = "I" Where ShortName = "ISO"
This does:
Update ApplicationStandards Set ShortCutKey = 'I' Where ShortName = 'ISO'