I have this code snippet with a stored procedure Read_records_from_to
cleaned_data = from_to_form.cleaned_data
with connections["mssql_database"].cursor() as cursor:
cursor.execute("Read_records_from_to '2021-12-01 07:55:39.000', '2021-12-14 07:55:39.000'")
result = cursor.fetchall()
class FromToForm(Form):
start_date = DateField(widget=AdminDateWidget())
start_time = TimeField(widget=AdminTimeWidget())
end_date = DateField(widget=AdminDateWidget())
end_time = TimeField(widget=AdminTimeWidget())
The stored procedure takes to parameters from_datetime and to_datetime. I'd like to assign it values taken from FromtoForm. How can I do this?
I tried
start = datetime.combine(from_to_form.cleaned_data['start_date'], from_to_form.cleaned_data['start_time']).utcnow().isoformat()
end = datetime.combine(from_to_form.cleaned_data['end_date'], from_to_form.cleaned_data['end_time']).utcnow().isoformat()
context['start'] = start
with connections["mssql_database"].cursor() as cursor:
cursor.execute("EXEC Readrecords_from_to #dt_from='start' ,#dt_to='end'")
result = cursor.fetchall()
according to this answer. But it ended with error
Django Version: 2.2.4
Exception Type: DataError
Exception Value:
('22007', '[22007] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')
The error was in the execute statement. This is the right code.
start = datetime.combine(from_to_form.cleaned_data['start_date'], from_to_form.cleaned_data['start_time']).isoformat()
end = datetime.combine(from_to_form.cleaned_data['end_date'], from_to_form.cleaned_data['end_time']).isoformat()
with connections["mssql_database"].cursor() as cursor:
cursor.execute("EXEC Read_records_from_to #dt_od='%s', #dt_do='%s'" % (start, end))
result = cursor.fetchall()
context['result'] = result
Related
SELECT_NO_PROCESSED_QRCODES = "SELECT id, visit_date, added_in_monitoring FROM locations_qrcode " \
"WHERE DATE_FORMAT(visit_date, '%Y%m%d') = %(value)s"
This SQL works perfectly outside a python (def) function.
Is this because of MySQL's DATE_FORMAT that stops working because it is inside a Python function?
My visit_date field is a datetime field, I need to use strftime within SQL to convert to ('% Y-% m-%d), how can I do this?
like: strftime(visit_date,'% Y-% m-% d')
1370 (42000): execute command denied to user 'erico'#'%' for routine 'visit_date.srftime'
user - password and database is ok
def process_qrcodes():
now = datetime.now().strftime('%Y-%m-%d')
while True:
try:
cur = conn.cursor()
#QRCODES = "SELECT id, added_in_monitoring, string FROM locations_qrcode WHERE added_in_monitoring='0'"
now = datetime.now().strftime("%Y-%m-%d")
NON_PROCESSED_QRCODES = "SELECT id, added_in_monitoring, string FROM locations_qrcode " \
"WHERE visit_date.srftime('%%Y-%%m-%%d') =%s" % (now,)
cur.execute(NON_PROCESSED_QRCODES)
results = cur.fetchall()
for result in results:
idqrcode = result[0]
string = result[2]
process_status_qrcode(idqrcode, string)
print("OI")
except Exception as e:
print(e)
time.sleep(10)
When running a query to a mysql database using MySqlHook, cursor.execute(query) returns int 1
My code is
import logging
from airflow.hooks.mysql_hook import MySqlHook
query = "SELECT col1, col2 FROM myschema.mytable LIMIT 1"
mysql = MySqlHook(mysql_conn_id=conn_id)
conn = mysql.get_conn()
cursor = conn.cursor()
result_cursor = cursor.execute(query)
logging.info(result_cursor) # this prints out "INFO - 1" in the log
df = pd.DataFrame(result_cursor.fetchall(), columns=result_cursor.keys()) # this triggers error "ERROR - 'int' object has no attribute 'fetchall'"
I would have expected result_cursor to return a "fetchable" result, since the query is working fine.
Cursor.execute() return value is not defined by the db-api spec, but for most implementations it returns the number of rows affected by the query.
To retrieve data, you have to either iterate over the cursor or call .fetchall().
It seems I cannot save cursor.execute(query) into variable result_cursor.
To make the code work, I simply needed to define the data for the data-frame as cursor.fetchall()
cursor.execute(query)
df = pd.DataFrame(list(cursor.fetchall()), column=[col[0] for col in cursor.description])
Error when using timedelta from datetime.now() in SQL Server where clause
python 3.6
yesterday = datetime.now() - timedelta(days=1)
sql = "SELECT submit_dt, api_job_name, job_status, xml_record_count, x_successful_number, x_failed_number, " \
f"job_run_time, mf_job_name FROM JOB_LOG where submit_dt > {yesterday}"
try:
db = Database()
db.cursor.execute(sql)
rows = db.cursor.fetchall()
SQL ODBC Error: Incorrect syntax near '22' --- which is the time part of the datetime.
I've tried wrapping it in '' but then get convert from string error.
Consider parameterizing your query without any need of string conversion of datetime or string interpolation including F-strings.
yesterday = datetime.now() - timedelta(days=1)
sql = """SELECT submit_dt, api_job_name, job_status, xml_record_count,
x_successful_number, x_failed_number,
job_run_time, mf_job_name
FROM JOB_LOG
WHERE submit_dt > ?"""
try:
db = Database()
db.cursor.execute(sql, yesterday)
rows = db.cursor.fetchall()
The error was due to including the microseconds in the compare value. I was able to use:
yesterday_sql = yesterday.strftime("%Y-%m-$d %H:%M:%S")
I'm using pyodbc together with QODBC to construct an ODBC query.
I'm having trouble inserting datestamp parameters. Here you can see the escalation starting from the literal version (1) to string-format version (2) to error-state versions. (Note DateFrom & DateTo):
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = {d'2018-02-12'}, DateTo = {d'2018-02-18'}, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = %s, DateTo = %s, SummarizeColumnsBy='TotalOnly', ReturnRows='All'" % (q_startdate, q_enddate)
Subsequent attempts with the insertion syntax ?, cursor.execute(sql, (q_startdate), (q_enddate)) and the variables:
q_startdate = ("{d'%s'}" % dates[0])
q_enddate = ("{d'%s'}" % dates[1])
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = ?, DateTo = ?, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
>>> ('HY004', '[HY004] [Microsoft][ODBC Driver Manager] SQL data type out of range (0) (SQLBindParameter)')
q_startdate = (dates[0])
q_enddate = (dates[1])
sql = "sp_report ProfitAndLossStandard show Amount_Title, Text, Label, Amount parameters DateFrom = {d'?'}, DateTo = {d'?'}, SummarizeColumnsBy='TotalOnly', ReturnRows='All'"
>>> ('42000', "[42000] [QODBC] [sql syntax error] Expected lexical element not found: = {d'?'} (11015) (SQLPrepare)")
Reading the pyodbc Wiki page on inserting data, I don't read about any speed bumps with insertion strings. This must have something to do with how pyodbc processes (escapes) the datestamp.
How do you parameterize datestamp--Especially with the qodbc flavor of datestamp.
It is almost never necessary to use ODBC escape sequences like {d'2018-02-12'} in a pyodbc parameterized query. If the parameter value is a true Python date object
q_startdate = date(2018, 2, 12)
then pyodbc will inform the ODBC driver that the parameter value is a SQL_TYPE_DATE as shown in the ODBC trace log
[ODBC][2984][1532535987.825823][SQLBindParameter.c][217]
Entry:
Statement = 0x1f1a6b0
Param Number = 1
Param Type = 1
C Type = 91 SQL_C_TYPE_DATE
SQL Type = 91 SQL_TYPE_DATE
Col Def = 10
Scale = 0
Rgb Value = 0x1f3ac78
Value Max = 0
StrLen Or Ind = 0x1f3ac58
and we can just use a bare parameter placeholder in our SQL command text
... parameters DateFrom = ?, ...
I am using Python 3.6, pyodbc, and connect to SQL Server.
I am trying make connection to a database, then creating a query with parameters.
Here is the code:
import sys
import pyodbc
# connection parameters
nHost = 'host'
nBase = 'base'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
print("connection successfull")
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
# if run WITHOUT parameters THEN everything is OK
ask = input ('Go WITHOUT parameters y/n ?')
if ask == 'y':
# SQL without parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN '2017-03-01T00:00:00.000' AND '2017-03-01T01:00:00.000'
''')
# SQL without parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
# if run WITH parameters THEN ERROR
ask = input ('Go WITH parameters y/n ?')
if ask == 'y':
# parameters start
STARTDATE = "'2017-03-01T00:00:00.000'"
ENDDATE = "'2017-03-01T01:00:00.000'"
# parameters end
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN :STARTDATE AND :ENDDATE
''', {"STARTDATE": STARTDATE, "ENDDATE": ENDDATE})
# SQL with parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
When I run the program without parameters in SQL, it works.
When I try running it with parameters, an error occurred.
Parameters in an SQL statement via ODBC are positional, and marked by a ?. Thus:
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN ? AND ?
''', STARTDATE, ENDDATE)
# SQL with parameters stop
Plus, it's better to avoid passing dates as strings. Let pyodbc take care of that using Python's datetime:
from datetime import datetime
...
STARTDATE = datetime(year=2017, month=3, day=1)
ENDDATE = datetime(year=2017, month=3, day=1, hour=0, minute=0, second=1)
then just pass the parameters as above. If you prefer string parsing, see this answer.
If you're trying to use pd.to_sql() like me I fixed the problem by passing a parameter called chunksize.
df.to_sql("tableName", engine ,if_exists='append', chunksize=50)
hope this helps
i tryied and have a lot of different errors: 42000, 22007, 07002 and others
The work version is bellow:
import sys
import pyodbc
import datetime
# connection parameters
nHost = 'host'
nBase = 'DBname'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
STARTDATE = '11/2/2017'
ENDDATE = '12/2/2017'
params = (STARTDATE, ENDDATE)
# SQL with parameters start
sql = ('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN CAST(? as datetime) AND CAST(? as datetime)
''')
# SQL with parameters stop
# print result to console start
query = cursor.execute(sql, params)
row = query.fetchone()
while row:
print (row)
row = query.fetchone()
# print result to console stop
say = input ('everething is ok, you can close console')
I fixed this issue with code if you are using values through csv.
for i, row in read_csv_data.iterrows():
cursor.execute('INSERT INTO ' + self.schema + '.' + self.table + '(first_name, last_name, email, ssn, mobile) VALUES (?,?,?,?,?)', tuple(row))
I had a similar issue. Saw that downgrading the version of PyODBC to 4.0.6 and SQLAlchemy to 1.2.9 fixed the error,using Python 3.6