I am struggling with
pyodbc.ProgrammingError: ('String data, right truncation: length 636 buffer 510', 'HY000')
while using executeMany() with __crsr.fast_executemany = True. When I remove this line everything works fine.
I am using pyodbc (4.0.30) and MSSQL with ODBC Driver 17 for SQL Server.
My database table looks like 4 columns and each of them is varchar(255).
I already to add this line: crsr.setinputsizes([(pyodbc.SQL_WVARCHAR, 50, 0)]) and add UseFMTOnly=yes to connection string but it didn't work.
Could you guys help me, please? I am already tired of that.
Check the ODBC Driver you are using in your pyodbc connection string, if it is an old version, errors could be misleading, use for example:
driver="{ODBC Driver 17 for SQL Server}"
instead of:
driver="{SQL Server}"
Related
I am attempting to do parameterized insert query using pyodbc and Cloudera ODBC Driver for Impala but it is not working.
I am trying to execute the following query.
import pyodbc
insert_query = "insert into table_name(id,name,price) values(?,?,?)"
cursor.execute(insert_query,1,"Apples","20 dollars")
Here is the error I receive:
The SQL contains 0 parameter markers but 3 parameters were supplied, 'HY000'
If I execute the query using Python formatting like this it WORKS:
query_format = "insert into table_name(id,name,price) values({},\"{}\",\"{}\")".format(1,"Apples","20 dollars")
cursor.execute(query_format)
But I really want to specify parameters separately because I would want to do cursor.executemany() to do batch inserts later on and using the formatting approach forces me to use for loop.
Instead of sending values separately , I used tuple too , a list too. Nothing works. Why is it not recognizing "?" as a parameter marker?
Python - 3.7.4,
Pyodbc - 4.0.27,
Anaconda - 1.7.2,
OS - Windows 10
As resolved on GitHub here, this is an issue with the ODBC driver in question, not a problem with pyodbc.
Help me to understand if I can parameterize my connection string for a pyodbc connection to this qodbc interface for QuickBooks:
pyodbc.connect(r'DSN=qremote_dsn;IPAddress=192.168.0.50;Port=4500;RemoteDSN=login_dsn;OpenMode=F;OLE DB Services=-2;', autocommit=True)
I have several different DSNs, RemoteDSNs and servers which I'd like to loop over. Several SO posts (here and here) point to this code.google documentation suggesting I can use strings, keywords, or both with pyodbc's connect function.
pyodbc works with many different odbc APIs, so how do determine if any of the pyodbc keywords will map to my required qodbc keywords?
My search for the qodbc keywords in the pyodbc documentation returns no results. Must I conclude f-strings are my only option?
Must I conclude f-strings are my only option?
Not at all.
pyodbc is built to deal with any ODBC driver, so it does not identify which keywords are "legal" and which ones arent. As explained here there are a few keywords that are specified by the DBAPI spec, and a few keywords reserved for pyodbc internal use, and they "are not passed to the odbc driver" implying that other keywords are passed to the ODBC driver.
Example: When I use this connect call ...
cnxn = pyodbc.connect(
driver="ODBC Driver 17 for SQL Server",
server="192.168.0.179,49242",
database="myDb",
uid="sa", pwd="_whatever_",
trusted_connection="no"
)
... the ODBC trace shows that this is the connection string passed to the driver
[ODBC][2404][1589493655.363466][SQLDriverConnectW.c][290]
Entry:
Connection = 0xf7d9c0
Window Hdl = (nil)
Str In = [driver=ODBC Driver 17 for SQL Server;server=192.168.0.179,49242;database=myDb;uid=sa;pwd=_whatever_;trusted_connection=no;][length = 122 (SQL_NTS)]
Note that trusted_connection is specific to SQL Server.
I am trying to create an 'upsert' function for pypyodbc SQL Server. I have validated that the query built up will run in SSMS with the desired outcome, but when trying to execute and commit with pypyodbc I receive the following error: 'HY007', '[HY007] [Microsoft][ODBC SQL Server Driver]Associated statement is not prepared'.
Here is the upsert function:
def sql_upsert(sql_connection, table, key_field, key_value, **kwargs):
keys = ["{key}".format(key=k) for k in kwargs]
values = ["'{value}'".format(value=v) for v in kwargs.values()]
update_columns = ["{key} = '{value}'".format(key=k, value=v) for k, v in kwargs.items()]
sql = list()
#update
sql.append("UPDATE {table} SET ".format(table=table))
sql.append(", ".join(update_columns))
sql.append(" WHERE {} = '{}'".format(key_field, key_value))
sql.append(" IF ##ROWCOUNT=0 BEGIN ")
# insert
sql.append("INSERT INTO {table} (".format(table=table))
sql.append(", ".join(keys))
sql.append(") VALUES (")
sql.append(", ".join(values))
sql.append(")")
sql.append(" END")
query = "".join(sql)
print(query)
The function builds up a query string in a format based on this other thread How to insert or update using single query?
Here is an example of the output:
UPDATE test SET name='john' WHERE id=3012
IF ##ROWCOUNT=0 BEGIN
INSERT INTO test(name) VALUES('john')
END
The error message you cited is produced by the ancient "SQL Server" ODBC driver that ships as part of Windows. A more up-to-date driver version like "ODBC Driver 17 for SQL Server" should produce a meaningful error message.
If you look here or here you'll see people complaining about this over a decade ago.
Apparently SQL Server's ODBC driver returns that error when you're executing two statements that fail due to a field value being too long, or perhaps due to foreign key violations.
Using SSMS to see which statement causes this problem, or better - stop using ODBC and use pymssql
This error may also come when you don't give correct permissions to stored procedure
Go the SQL server --> Right click on your sp-->properties-->permissions
Add required users and roles which are going to execute this sp
This may help resolving the issue
Quick rundown:
The idea here is to read some data in from a csv file, and use that as the list in the NOT IN part of my sql query. I'm connecting to the db (.mdb) with the code below.
Note LP is the tuple/list I'm trying to pass, IRdb is the path to the db
constr = r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=' + IRdb
conn = pyodbc.connect(constr, autocommit=True)
cur = conn.cursor()
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN ?'''
cur.execute(IRsql,('Total',r'Conn%',LP))
The issue:
Everything works fine except for the execute statement (which did work before i added the NOT IN part). I've tried passing LP as string, tuple, and list, but nothing seems to be working. I get the following error
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] In operator without () in query expression 'IRRPResults.PointName<>Pa_RaM000 AND IRRPResults.RRPName NOT LIKE Pa_RaM001 AND IRRPResults.PointName NOT IN Pa_RaM002'. (-3100) (SQLExecDirectW)")
Any help would be greatly appreciated.
For anyone interested, or for that guy reading this 2 years from now with the same issue, or my future self when I forget what i did, I've figured out a solution or two.
The first work around was to simply use .format on the sql string to insert LP directly before it gets passed to the execute statement.
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN {}'''.format(LP)
cur.execute(IRsql,('Total',r'Conn%'))
The other solution, I got from this question, is a little more elegant and clever in that it builds a string of '?' markers for each element in LP. Then LP gets passed as a tuple/list to the execute statement.
placeholders=','.join('?'*len(LP))
IRsql='''SELECT IRRPResults.RRPName, IRRPResults.PointName, IRRPResults.RiskPerTime FROM IRRPResults
WHERE IRRPResults.PointName<>?
AND IRRPResults.RRPName NOT LIKE ? AND IRRPResults.PointName NOT IN ({})'''.format(placeholders)
cur.execute(IRsql,('Total',r'Conn%',*LP))
In SQL Server Management Studio I can run this query:
select a.foo, colX, colY, colZ from tblA a
join tblB b
on (a.foo = b.foo)
Running the same query through pyodbc in python script, I get:
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][
SQL Server]"b" is not a recognized table hints option. If it is intended as a p
arameter to a table-valued function or to the CHANGETABLE function, ensure that
your database compatibility mode is set to 90. (321) (SQLExecDirectW)')
Can anyone explain why this query works directly against SQL Server but there seems to be a problem with the table alias when using pyodbc?
This is what I've been able to find out:
SQL Server running on Microsoft Windows NT 6.1 (7601)
SQL Server version is 11.0.5532.0
pyodbc library: pyodbc-3.0.7.win32-py2.7
1) I do not know how to produce a reproducible example and I'm not a DBA; I can just about get by in SQL. 2) I cannot create tables on this database. However: I can do the full query with 3 joins / 3 aliases works fine using the RODBC r package and the sqlQuery() call against the same db. Therefore the culprit is pyodbc so I will try to find a python dev to try reproduce and post back the findings.
UPDATE + FIX
The issue is caused by concatenation errors:
Got python developer help to find and print raw sql before it was executed in python:
select a.foo, colX, colY, colZ from tblA a join tblB bon (a.foo = b.foo)
...and similar issue at each join, caused by missing whitespace within the sql string:
sql = "select a.foo, colX, colY, colZ from tblA a "
sql += "join tblB" #no whitespace at end of string
sql += "on (a.foo = b.foo)" #runs into next segment.
Worked in r with RODBC because string in r built like:
sql = "
...full query...
"
...with no opportunity for concatenation errors.