Develop Reference

pyodbc parameterized insert for Impala ODBC Driver

I am attempting to do parameterized insert query using pyodbc and Cloudera ODBC Driver for Impala but it is not working.
I am trying to execute the following query.
import pyodbc
insert_query = "insert into table_name(id,name,price) values(?,?,?)"
cursor.execute(insert_query,1,"Apples","20 dollars")
Here is the error I receive:
The SQL contains 0 parameter markers but 3 parameters were supplied, 'HY000'
If I execute the query using Python formatting like this it WORKS:
query_format = "insert into table_name(id,name,price) values({},\"{}\",\"{}\")".format(1,"Apples","20 dollars")
cursor.execute(query_format)
But I really want to specify parameters separately because I would want to do cursor.executemany() to do batch inserts later on and using the formatting approach forces me to use for loop.
Instead of sending values separately , I used tuple too , a list too. Nothing works. Why is it not recognizing "?" as a parameter marker?
Python - 3.7.4,
Pyodbc - 4.0.27,
Anaconda - 1.7.2,
OS - Windows 10

As resolved on GitHub here, this is an issue with the ODBC driver in question, not a problem with pyodbc.

String data, right truncation while using fast executemany with pyodbc

I am struggling with
pyodbc.ProgrammingError: ('String data, right truncation: length 636 buffer 510', 'HY000')
while using executeMany() with __crsr.fast_executemany = True. When I remove this line everything works fine.
I am using pyodbc (4.0.30) and MSSQL with ODBC Driver 17 for SQL Server.
My database table looks like 4 columns and each of them is varchar(255).
I already to add this line: crsr.setinputsizes([(pyodbc.SQL_WVARCHAR, 50, 0)]) and add UseFMTOnly=yes to connection string but it didn't work.
Could you guys help me, please? I am already tired of that.

Check the ODBC Driver you are using in your pyodbc connection string, if it is an old version, errors could be misleading, use for example:
driver="{ODBC Driver 17 for SQL Server}"
instead of:
driver="{SQL Server}"

pypyodbc error 'Associated statement is not prepared'

I am trying to create an 'upsert' function for pypyodbc SQL Server. I have validated that the query built up will run in SSMS with the desired outcome, but when trying to execute and commit with pypyodbc I receive the following error: 'HY007', '[HY007] [Microsoft][ODBC SQL Server Driver]Associated statement is not prepared'.
Here is the upsert function:
def sql_upsert(sql_connection, table, key_field, key_value, **kwargs):
keys = ["{key}".format(key=k) for k in kwargs]
values = ["'{value}'".format(value=v) for v in kwargs.values()]
update_columns = ["{key} = '{value}'".format(key=k, value=v) for k, v in kwargs.items()]
sql = list()
#update
sql.append("UPDATE {table} SET ".format(table=table))
sql.append(", ".join(update_columns))
sql.append(" WHERE {} = '{}'".format(key_field, key_value))
sql.append(" IF ##ROWCOUNT=0 BEGIN ")
# insert
sql.append("INSERT INTO {table} (".format(table=table))
sql.append(", ".join(keys))
sql.append(") VALUES (")
sql.append(", ".join(values))
sql.append(")")
sql.append(" END")
query = "".join(sql)
print(query)
The function builds up a query string in a format based on this other thread How to insert or update using single query?
Here is an example of the output:
UPDATE test SET name='john' WHERE id=3012
IF ##ROWCOUNT=0 BEGIN
INSERT INTO test(name) VALUES('john')
END

The error message you cited is produced by the ancient "SQL Server" ODBC driver that ships as part of Windows. A more up-to-date driver version like "ODBC Driver 17 for SQL Server" should produce a meaningful error message.

If you look here or here you'll see people complaining about this over a decade ago.
Apparently SQL Server's ODBC driver returns that error when you're executing two statements that fail due to a field value being too long, or perhaps due to foreign key violations.
Using SSMS to see which statement causes this problem, or better - stop using ODBC and use pymssql

This error may also come when you don't give correct permissions to stored procedure
Go the SQL server --> Right click on your sp-->properties-->permissions
Add required users and roles which are going to execute this sp
This may help resolving the issue

SQLALCHEMY: how to connect to a database when one only knows the DSN?

I have this simple connectivity problem:
Someone installed the right ODBC drivers on my computer, and so far I am able to connect to my database using the following code
import pyodbc
import pandas as pd
import numpy as np
cnxn = pyodbc.connect('DSN=MYDSN')
cnxn.timeout = 3600
cursor = cnxn.cursor()
However, for many reasons I want to use SQLAlchemy and using the naive
import sqlalchemy
engine = sqlalchemy.create_engine('DSN=MYDSN')
fails miserably.
ArgumentError: Could not parse rfc1738 URL from string 'DSN=MYDSN'
I cannot find what is the right syntax here. I dont know the url of my database either. Am I lost without hope here?
Thanks!

Having used a "raw" pyodbc connection to get some idea of what the DSN might expect to be talking to ...
>>> cnxn = pyodbc.connect('DSN=MYDSN')
>>> cnxn.getinfo(pyodbc.SQL_DRIVER_NAME)
'Cisco Information Server 7.0.3 ODBC Driver'
it seems somewhat unlikely that there will be a specific SQLAlchemy dialect for that product. Further testing revealed that both
engine = sqlalchemy.create_engine('mssql+pyodbc://MYDSN')
and
engine = sqlalchemy.create_engine('mysql+pyodbc://MYDSN')
can connect, but attempts to use SQLAlchemy features like
print(engine.table_names())
fail because CIS does not "speak" native T-SQL or MySQL.
You may still be able to use some SQLAlchemy features, e.g., for running basic queries (that rely on fairly plain SQL syntax). For the SQLAlchemy features that don't work you could fall back to native pyodbc. For example, instead of
print(engine.table_names())
you could use
cnxn = engine.connect()
crsr = cnxn.cursor()
table_names = [x.table_name for x in crsr.tables(tableType='TABLE')]
print(table_names)

pypyodbc: OPENJSON incorrect syntax near keyword "WITH"

I'm trying to use OPENJSON in a Python script to import some basic JSON into a SQL database. I initially tried with a more complex JSON file, but simplified it for the sake of this post. Here's what I have:
sql_statement = "declare #json nvarchar(max) = '{\"name\":\"James\"}'; SELECT * FROM OPENJSON(#json) WITH (name nvarchar(20))"
cursor.execute(sql_statement)
cursor.commit()
connection.close()
The error I receive:
pypyodbc.ProgrammingError: (u'42000', u"[42000] [Microsoft][ODBC SQL
Server Driver][SQL Server]Incorrect syntax near the keyword 'with'. If
this statement is a common table expression, an xmlnamespaces clause
or a change tracking context clause, the previous statement must be
terminated with a semicolon.")
Any thoughts on why I'm seeing this error? I was successfully able to execute other SQL queries with the same pypyodbc / database configuration.

The problem could be that your database is running in an older compatibility level, where OPEN JSON is not available.
To find the compatibility level of your database, run following SQL statement:
SELECT compatibility_level FROM sys.databases WHERE name = 'your_db_name';
If the result is 120 or lower, you'll need to update your compatibility level to 130, by running:
ALTER DATABASE your_db_name SET COMPATIBILITY_LEVEL = 130;
Note: In case your database is actually Azure SQL DB, you should check the version as well, as OPEN JSON is not available for versions prior to 12.x