I have the following sql
select * from table
where "column" = 'ĚČÍ'
which works fine when committed directly to the database (in my case Microsoft SQL Server 2016).
If I use (in python)
import pyodbc
ex_str= """select * from table
where "column" = (?)"""
str2insert= ('ĚČÍ')
conn = pyodbc.connect(cstring)
cur = conn.cursor()
cur.execute(ex_str,str2insert)
content = cur.fetchall()
I get no result although ĚČÍ is definetly an entry in the database column
I added
conn.setencoding("utf-8")
encoded and decoded the str2insert with latin utf-8 etc. and added
cstring = cstring + ";convert_unicode=True"
but nothing worked.
There seems to be an issue with pyodbc and special characters. Has anyone an idea?
Related
I am trying to create a temporary table from a pandas df and then use it in a sql statement
import snowflake.connector
from snowflake.connector.pandas_tools import write_pandas
with snowflake.connector.connect(
account='snoflakewebsite',
user='username',
authenticator='externalbrowser',
database='db',
schema='schema'
) as con:
success, nchunks, nrows, _ = write_pandas(
conn=con,
df=df,
table_name='temp_table',
auto_create_table = True,
table_type='temporary',
overwrite = True,
database='db',
schema='schema'
)
cur = con.cursor()
cur.execute('select * from temp_table')
The error I get:
ProgrammingError: 002003 (42S02): SQL compilation error:
Object 'TEMP_TABLE' does not exist or not authorized.
write_pandas() creates a table using the letter case exactly how it is passed in table_name=, while the query submitted in cur.execute() passes the entire string with the query to Snowflake SQL, and Snowflake SQL capitalizes the object names unless they are written in double quotes.
Therefore, either you create a table using capital letters table_name='TEMP_TABLE',
or you query it using double quotes:
cur.execute('select * from "temp_table"')
In this case, you will get your table created in small letters, and you always need to add double quotes to refer to its name.
I'm trying to get query as a xml data from mssql server with pyodbc. After query, im writing data to new xml file with unique name. Everything works fine to this point with small data. Problem is when i try to read data over 2037 character, i cant get all of them. Its gives me just first 2037 character.
SQL Server version 15.0.2000.5(SQL Server 2019 Express)
Driver is ODBC Driver 17 for SQL Server
Python version 3.11.1
pyodbc version 4.0.35
Code is running on Windows Server 2016 Standard
SQL Query For XML Data
SELECT
C.BLKODU AS "BLKODU",
C.CARIKODU AS "CARIKODU",
C.TICARI_UNVANI AS "TICARI_UNVANI",
C.ADI_SOYADI AS "ADI_SOYADI",
C.VERGI_NO AS "VERGI_NO",
C.TC_KIMLIK_NO AS "TC_KIMLIK_NO",
C.VERGI_DAIRESI AS "VERGI_DAIRESI",
C.CEP_TEL AS "CEP_TEL",
C.ILI AS "ILI",
C.ILCESI AS "ILCESI",
C.ADRESI_1 AS "ADRESI",
(SELECT
CHR.BLKODU AS "BLKODU",
CHR.EVRAK_NO AS "EVRAK_NO",
CHR.MAKBUZNO AS "MAKBUZ_NO",
CAST(CHR.TARIHI AS DATE) AS "TARIHI",
CAST(CHR.VADESI AS DATE) AS "VADESI",
CHR.MUH_DURUM AS "MUH_DURUM",
CAST(CHR.KPB_ATUT AS DECIMAL(10, 2)) AS "KPB_ATUT",
CAST(CHR.KPB_BTUT AS DECIMAL(10, 2)) AS "KPB_BTUT"
FROM CARIHR AS CHR
WHERE CHR.BLCRKODU = C.BLKODU
ORDER BY CHR.TARIHI
FOR XML PATH('CARIHR'), TYPE)
FROM CARI AS C
WHERE C.CARIKODU = 'CR00001'
FOR XML PATH ('CARI')
Python Code
import pyodbc
import uuid
import codecs
import query
import core
conn = pyodbc.connect(core.connection_string, commit=True)
cursor = conn.cursor()
cursor.execute(query.ctr)
row = cursor.fetchval()
id = uuid.uuid4()
xml_file = "./temp/"+str(id)+".xml"
xml = codecs.open(xml_file, "w", "utf-8")
xml.write(row)
xml.close()
I've tried to use pymssql and it didn't change anything.
cursor.fetchvall(), cursor.fetchone() is gives me same result.
cursor.fetchall() gives me full data. But its gives as a list. When its gives as a list i need to convert to string. Before converting to string i need to select first element in the list. So i came with then idea like this below. But result didn't change at all. Its still gives only first 2037 character.
conn = pyodbc.connect(connect_string, commit=True)
cursor = conn.cursor()
cursor.execute(query.ctr)
row = cursor.fetchall()
data = ','.join(row[0])
id = uuid.uuid4()
xml_file = "./temp/"+str(id)+".xml"
xml = codecs.open(xml_file, "w", "utf-8")
xml.write(data)
xml.close()
For XML queries are split to multiple lines by SQL Server automatically if they're long enough. Some clients like Management Studio "merge" these to single row but it's not actually one row.
So you need to concatenate your string yourself:
#code in pseudo-python
xmlString = ""
rows = cursor.fetchall()
for row in rows:
xmlString = xmlString + row[0]
I'm currently trying to query a deltadna database. Their Direct SQL Access guide states that any PostgreSQL ODBC compliant tools should be able to connect without issue. Using the guide, I set up an ODBC data source in windows
I have tried adding Set nocount on, changed various formats for the connection string, changed the table name to be (account).(system).(tablename), all to no avail. The simple query works in Excel and I have cross referenced with how Excel formats everything as well, so it is all the more strange that I get the no query problem.
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor().execute(query1)
row = query1_cursor.fetchone()
print(row)
Result is ProgrammingError: No results. Previous SQL was not a query.
Try it like this:
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor()
query1_cursor.execute(query1)
row = query1_cursor.fetchone()
print(row)
You can't do the cursor declaration and execution in the same row. Since then your query1_cursor variable will point to a cursor object which hasn't executed any query.
I have the following python code, it reads through a text file line by line and takes characters x to y of each line as the variable "Contract".
import os
import pyodbc
cnxn = pyodbc.connect(r'DRIVER={SQL Server};CENSORED;Trusted_Connection=yes;')
cursor = cnxn.cursor()
claimsfile = open('claims.txt','r')
for line in claimsfile:
#ldata = claimsfile.readline()
contract = line[18:26]
print(contract)
cursor.execute("USE calms SELECT XREF_PLAN_CODE FROM calms_schema.APP_QUOTE WHERE APPLICATION_ID = "+str(contract))
print(cursor.fetchall())
When including the line cursor.fetchall(), the following error is returned:
Programming Error: Previous SQL was not a query.
The query runs in SSMS and replace str(contract) with the actual value of the variable results will be returned as expected.
Based on the data, the query will return one value as a result formatted as NVARCHAR(4).
Most other examples have variables declared prior to the loop and the proposed solution is to set NO COUNT on, this does not apply to my problem so I am slightly lost.
P.S. I have also put the query in its own standalone file without the loop to iterate through the file in case this was causing the problem without success.
In your SQL query, you are actually making two commands: USE and SELECT and the cursor is not set up with multiple statements. Plus, with database connections, you should be selecting the database schema in the connection string (i.e., DATABASE argument), so TSQL's USE is not needed.
Consider the following adjustment with parameterization where APPLICATION_ID is assumed to be integer type. Add credentials as needed:
constr = 'DRIVER={SQL Server};SERVER=CENSORED;Trusted_Connection=yes;' \
'DATABASE=calms;UID=username;PWD=password'
cnxn = pyodbc.connect(constr)
cur = cnxn.cursor()
with open('claims.txt','r') as f:
for line in f:
contract = line[18:26]
print(contract)
# EXECUTE QUERY
cur.execute("SELECT XREF_PLAN_CODE FROM APP_QUOTE WHERE APPLICATION_ID = ?",
[int(contract)])
# FETCH ROWS ITERATIVELY
for row in cur.fetchall():
print(row)
cur.close()
cnxn.close()
I am trying to load a big list of sql queries into a table in Vertica using PYODBC. Here's my code:
tablename = DVXTEMP.my_table_name
sql = my_sql_query.strip().strip(';')
samplesize = 1000
createstring = 'CREATE TABLE %s AS %s \n limit %s;' %(tablename, sql, samplesize)
cursor.execute(createstring)
when I print createstring and run it in Toad, it works fine. when I try to execute it in pyodbc, it gives me the following error:
'Syntax error at or near "DVXTEMP" at character 1\n (4856) (SQLExecDirectW)'
We are using Vertica Analytic Database v7.1.2-6
Any ideas what might be causing this?
Thanks
1) did you import pyodbc?
2) did you define "cursor" from "pyodbc.connect"?
import pyodbc
DB = '[string for dbfile]'
DRV = '[string of which driver you are going to use]'
con = pyodbc.connect('DRIVER={};DBQ={}'.format(DRV,DB))
cursor = con.cursor()
##build SQL code and execute as you have done
Try SQL commands after you can connect without an error.
3) I use pyodbc for mdb files (MS Access) and some of my queries will not run unless I put single quotes outside double quotes on table/field names.
mytbl_1 = "mytbl"
SQL = 'SELECT * FROM ' + mytbl_1
print SQL
print result -> SELECT * FROM mytbl
(this fails)
mytbl_2 = '"mytbl"' #single quotes outside of double quote
SQL = 'SELECT * FROM ' + mytbl_2
print SQL
print result -> SELECT * FROM "mytbl"
(this string gets passed w/o error works for me with MDB files)