Develop Reference

create SQL table from existing table using pyodbc/turbodc in python

I would like to create a SQL table from an existing table. I'm using the turbodbc module (which is very similar to pyodbc).
# connect to database
conn = turbodbc.connect(connection_string="my_connection_string")
cursor = conn.cursor()
# execute SQL code
cursor.execute((" create table Test_Puts as"
" select * from OptionValue"
" where call_put = 'P'"))
However, I get the error message:
ODBC error
state: 42000
native error code: 156
message: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near the keyword 'select'.

Try to use this syntax:
select * into Test_Puts from OptionValue where call_put = 'P'
So, instead of this:
" create table Test_Puts as"
" select * from OptionValue"
" where call_put = 'P'"
use this:
" select * into Test_Puts"
" from OptionValue"
" where call_put = 'P'"

DatabaseError: Execution failed on sql object': The first argument to execute must be a string or unicode query

server = "UAT101"
db = "mydb"
con = pyodbc.connect('DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + db +';Uid=uat101_read;Pwd=*****')
max_week = pd.read_sql("select max(FWEEK) from mytable", con).astype('int')
max_week = max_week.iloc[0].astype('str')
min_week = pd.read_sql("select min(FWEEK) from SOE_DC_ML_8K.dbo.RAWDATA_DC_CONSOLIDATED where FWEEK in (select distinct top 2 FWEEK from mytable order by FWEEK desc)", con).astype('int')
min_week = min_week.iloc[0].astype('str')
query = "Select [FWEEK],[DPS_NUM] from mytable where (fweek = '"+max_week+"' or (fweek = '"+min_week+"' and NEW_DPS_FLG = '1'))"
df = pd.read_sql(query, con)
DatabaseError: Execution failed on sql ' Select
[FWEEK],[DPS_NUM],[COMMODITYDESC],[SYST... Name: 0, dtype: object':
The first argument to execute must be a string or unicode query.
i am getting error while executing df = pd.read_sql(query, con) statement

Python cannot interprete special characters in path to HANA table (SQL)

I want to read a table stored in HANA directly from python. For that I use the following code:
from hdbcli import dbapi
import pandas as pd
conn = dbapi.connect(
address="address",
port=XYZ,
user="user",
password="password"
)
print (conn.isconnected())
# Fetch table data
stmnt = "select * from '_SYS_NAME'.'part1.part2.part3.part4.part5.part6/table_name'"
cursor = conn.cursor()
cursor.execute(stmnt)
result = cursor.fetchall()
print('Create the dataframe')
The problem is in the line stmnt: I tried different ways of puting the path name so that python can read it as a string but none is working. I know the problem is not relying on the technique, because if the path is simple and not containing the special characters then the code works.
I tried all the following combinations (among others):
stmnt = "select * from '_SYS_NAME'.'part1.part2.part3.part4.part5.part6/table_name'"
stmnt = """select * from '_SYS_NAME'.'part1.part2.part3.part4.part5.part6/table_name'"""
stmnt = "select * from \'_SYS_NAME\'\.\'part1.part2.part3.part4.part5.part6/table_name\'
stmnt = """select * from \'_SYS_NAME\'\.\'part1.part2.part3.part4.part5.part6/table_name\'"""
The error I get is always the following:
hdbcli.dbapi.Error: (257, 'sql syntax error: incorrect syntax near "_SYS_NAME": line 1 col 1 (at pos 1)')
And the original path as I get it from SQL is:
'_SYS_NAME'.'part1.part2.part3.part4.part5.part6/table_name'
Any ideas what I am missing?

You should reverse your quotes:
stmnt = 'select * from "_SYS_BIC"."rwev.dev.bw.project.si.churn/SI_CV_CHU_7_DATA_MODEL"'

How to parameterize datestamp in pyODBC query?

I'm using pyodbc together with QODBC to construct an ODBC query.
I'm having trouble inserting datestamp parameters. Here you can see the escalation starting from the literal version (1) to string-format version (2) to error-state versions. (Note DateFrom & DateTo):
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = {d'2018-02-12'}, DateTo = {d'2018-02-18'}, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = %s, DateTo = %s, SummarizeColumnsBy='TotalOnly', ReturnRows='All'" % (q_startdate, q_enddate)
Subsequent attempts with the insertion syntax ?, cursor.execute(sql, (q_startdate), (q_enddate)) and the variables:
q_startdate = ("{d'%s'}" % dates[0])
q_enddate = ("{d'%s'}" % dates[1])
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = ?, DateTo = ?, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
>>> ('HY004', '[HY004] [Microsoft][ODBC Driver Manager] SQL data type out of range (0) (SQLBindParameter)')
q_startdate = (dates[0])
q_enddate = (dates[1])
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = {d'?'}, DateTo = {d'?'}, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
>>> ('42000', "[42000] [QODBC] [sql syntax error] Expected lexical element not found: = {d'?'} (11015) (SQLPrepare)")
Reading the pyodbc Wiki page on inserting data, I don't read about any speed bumps with insertion strings. This must have something to do with how pyodbc processes (escapes) the datestamp.
How do you parameterize datestamp--Especially with the qodbc flavor of datestamp.

It is almost never necessary to use ODBC escape sequences like {d'2018-02-12'} in a pyodbc parameterized query. If the parameter value is a true Python date object
q_startdate = date(2018, 2, 12)
then pyodbc will inform the ODBC driver that the parameter value is a SQL_TYPE_DATE as shown in the ODBC trace log
[ODBC][2984][1532535987.825823][SQLBindParameter.c][217]
Entry:
Statement = 0x1f1a6b0
Param Number = 1
Param Type = 1
C Type = 91 SQL_C_TYPE_DATE
SQL Type = 91 SQL_TYPE_DATE
Col Def = 10
Scale = 0
Rgb Value = 0x1f3ac78
Value Max = 0
StrLen Or Ind = 0x1f3ac58
and we can just use a bare parameter placeholder in our SQL command text
... parameters DateFrom = ?, ...

Passing a folder location as an SQL parameter in python causes an error

I am fairly new to python and the only SQL I know is from this project so forgive the lack of technical knowledge:
def importFolder(self):
user = getuser()
filename = askopenfilename(title = "Choose an image from the folder to import", initialdir='C:/Users/%s' % user)
for i in range (0,len(filename) - 1):
if filename[-i] == "/":
folderLocation = filename[:len(filename) - i]
break
cnxn = pyodbc.connect('DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\Users\Public\dbsDetectorBookingSystem.accdb')
cursor = cnxn.cursor()
cursor.execute("SELECT * FROM tblRuns")
cursor.execute("insert into tblRuns(RunID,RunFilePath,TotalAlphaCount,TotalBetaCount,TotalGammaCount) values (%s,%s,0,0,0)" %(str(self.runsCount + 1), folderLocation))
cnxn.commit()
self.runsCount = cursor.rowcount
rowString = str(self.runsCount) + " " + folderLocation + " " + str(0) + " " + str(0) + " " + str(0) + " " + str(0)
self.runsTreeView.insert("","end", text = "", values = (rowString))
That is one routine from my current program meant to create a new record which is mostly empty apart from an index and a file location. This location needs to be saved as a string however when it is passed as a paramenter to the SQL string the following error occurs:
cursor.execute("insert into tblRuns(RunID,RunFilePath,TotalAlphaCount,TotalBetaCount,TotalGammaCount) values (%s,%s,0,0,0)" %(str(self.runsCount + 1), folderLocation))
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error (missing operator) in query expression 'C:/Users/Jacob/Documents/USB backup'. (-3100) (SQLExecDirectW)") I assume this is because the SQL recognises a file path and wantsto user it. Does anybody know how to fix this?

You're not using the db-api correctly. Instead of using string formatting to pass your query params - which is error-prone (as you just noticed) AND a security issue, you want to pass them as arguments to cursor.execute(), ie:
sql = "insert into tblRuns(RunID, RunFilePath, TotalAlphaCount, TotalBetaCount, TotalGammaCount) values (%s, %s, 0, 0, 0)"
cursor.execute(sql, (self.runsCount + 1, folderLocation))
Note that we DONT use string formatting here (no "%" between sql and the params)
NB : note that the placeholder for parameterized queries depends on your db connector. python-MySQLdb uses % but your one may use a ? or anything else.
wrt/ your exact problem: since you didn't put quotes around your placeholders, the sql query you send looks something like:
"insert into tblRuns(
RunID, RunFilePath,
TotalAlphaCount, TotalBetaCount, TotalGammaCount
)
values (1,/path/to/folder,0,0,0)"
Which cannot work, obviously (it needs quotes around /path/to/folder to be valid SQL).
By passing query parameters the right way, your db connector will take care of all the quoting and escaping.