Error when using timedelta from datetime.now() in SQL Server where clause
python 3.6
yesterday = datetime.now() - timedelta(days=1)
sql = "SELECT submit_dt, api_job_name, job_status, xml_record_count, x_successful_number, x_failed_number, " \
f"job_run_time, mf_job_name FROM JOB_LOG where submit_dt > {yesterday}"
try:
db = Database()
db.cursor.execute(sql)
rows = db.cursor.fetchall()
SQL ODBC Error: Incorrect syntax near '22' --- which is the time part of the datetime.
I've tried wrapping it in '' but then get convert from string error.
Consider parameterizing your query without any need of string conversion of datetime or string interpolation including F-strings.
yesterday = datetime.now() - timedelta(days=1)
sql = """SELECT submit_dt, api_job_name, job_status, xml_record_count,
x_successful_number, x_failed_number,
job_run_time, mf_job_name
FROM JOB_LOG
WHERE submit_dt > ?"""
try:
db = Database()
db.cursor.execute(sql, yesterday)
rows = db.cursor.fetchall()
The error was due to including the microseconds in the compare value. I was able to use:
yesterday_sql = yesterday.strftime("%Y-%m-$d %H:%M:%S")
Related
I have this code snippet with a stored procedure Read_records_from_to
cleaned_data = from_to_form.cleaned_data
with connections["mssql_database"].cursor() as cursor:
cursor.execute("Read_records_from_to '2021-12-01 07:55:39.000', '2021-12-14 07:55:39.000'")
result = cursor.fetchall()
class FromToForm(Form):
start_date = DateField(widget=AdminDateWidget())
start_time = TimeField(widget=AdminTimeWidget())
end_date = DateField(widget=AdminDateWidget())
end_time = TimeField(widget=AdminTimeWidget())
The stored procedure takes to parameters from_datetime and to_datetime. I'd like to assign it values taken from FromtoForm. How can I do this?
I tried
start = datetime.combine(from_to_form.cleaned_data['start_date'], from_to_form.cleaned_data['start_time']).utcnow().isoformat()
end = datetime.combine(from_to_form.cleaned_data['end_date'], from_to_form.cleaned_data['end_time']).utcnow().isoformat()
context['start'] = start
with connections["mssql_database"].cursor() as cursor:
cursor.execute("EXEC Readrecords_from_to #dt_from='start' ,#dt_to='end'")
result = cursor.fetchall()
according to this answer. But it ended with error
Django Version: 2.2.4
Exception Type: DataError
Exception Value:
('22007', '[22007] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')
The error was in the execute statement. This is the right code.
start = datetime.combine(from_to_form.cleaned_data['start_date'], from_to_form.cleaned_data['start_time']).isoformat()
end = datetime.combine(from_to_form.cleaned_data['end_date'], from_to_form.cleaned_data['end_time']).isoformat()
with connections["mssql_database"].cursor() as cursor:
cursor.execute("EXEC Read_records_from_to #dt_od='%s', #dt_do='%s'" % (start, end))
result = cursor.fetchall()
context['result'] = result
I'm trying to execute multiple queries in SQL Server using the pymssql library.
This is my code:
cur = conn.cursor()
cur.execute("DECLARE #begin_time datetime, #end_time datetime, #from_lsn binary(10), #to_lsn binary(10);
SET #begin_time = DATEADD(day, -1, GETDATE()) ;
SET #end_time = GETDATE();
SET #from_lsn = sys.fn_cdc_map_time_to_lsn('smallest greater than or equal', #begin_time);
SET #to_lsn = sys.fn_cdc_map_time_to_lsn('largest less than or equal', #end_time);
SELECT * FROM cdc.fn_cdc_get_net_changes_dbo_users(#from_lsn, #to_lsn, 'all');")
output = cur.fetchall()
print(output)
conn.close()
The code is running fine and fetching the result, however when I'm calculating the date using Python library and passing it to the code, I'm getting an error.
Sample code
from datetime import datetime, timedelta
end_date = datetime.now()
start_date = datetime.now() + timedelta(hours=-1)
cur = conn.cursor()
query = f"""DECLARE #begin_time datetime, #end_time datetime, #from_lsn binary(10), #to_lsn binary(10);
SET #begin_time = {start_date};
SET #end_time = {end_date};
SET #from_lsn = sys.fn_cdc_map_time_to_lsn('smallest greater than or equal', #begin_time);
SET #to_lsn = sys.fn_cdc_map_time_to_lsn('largest less than or equal', #end_time);
SELECT * FROM cdc.fn_cdc_get_all_changes_dbo_users (#from_lsn, #to_lsn, 'all');"""
print(query)
cur.execute(query)
output = cur.fetchall()
print(output)
conn.close()
Error:
ProgrammingError: (102, Incorrect syntax near '11'.
DB-Lib error message 20018, severity 15:
General SQL Server error: Check messages from the SQL Server
I'm not sure what I'm doing wrong here. Would really appreciate if someone can help me and explain the issue.
Consider actual SQL parameterization of the time variables and not string interpolation or concatenation with F-strings which generally is not safe or efficient for passing values from application layer to backend database. The library, pymssql, supports parameters. Python's datetime.datetime should translate to MSSQL's DATETIME.
# PREPARED STATEMENTS WITH %s PLACEHOLDERS
query = """DECLARE #begin_time datetime, #end_time datetime, #from_lsn binary(10), #to_lsn binary(10);
SET #begin_time = %s;
SET #end_time = %s;
SET #from_lsn = sys.fn_cdc_map_time_to_lsn('smallest greater than or equal', #begin_time);
SET #to_lsn = sys.fn_cdc_map_time_to_lsn('largest less than or equal', #end_time);
SELECT * FROM cdc.fn_cdc_get_all_changes_dbo_users (#from_lsn, #to_lsn, 'all');
"""
print(query)
# EXECUTE QUERY WITH BINDED PARAMS
cur.execute(query, [start_date, end_date])
In fact, you can shorten the query since parameters do not need declaration:
# PREPARED STATEMENTS WITH %s PLACEHOLDERS
query = """DECLARE #from_lsn binary(10), #to_lsn binary(10);
SET #from_lsn = sys.fn_cdc_map_time_to_lsn('smallest greater than or equal', %s);
SET #to_lsn = sys.fn_cdc_map_time_to_lsn('largest less than or equal', %s);
SELECT * FROM cdc.fn_cdc_get_all_changes_dbo_users (#from_lsn, #to_lsn, 'all');
"""
By the way, please note pymmsql is no longer a maintained library. Consider pyodbc for most secure, updated DB-API for Python-SQL Server connections. But note the parameter placeholder for pyodbc is qmarks ? and not %s (unlike most Python DB-APIs).
This way works:
import pandas
import pyodbc
import datetime as dt
server = 'myserver'
db = 'mydb'
myparams = ['2017-02-01','2017-02-28', None] # None substitutes NULL in sql
connection_string = pyodbc.connect('DRIVER={SQL Server};server='+server+';DATABASE='+ db+';Trusted_Connection=yes;')
df = pandas.read_sql_query('EXEC PythonTest_Align_RSrptAccountCurrentMunich #EffectiveDateFrom=?,#EffectiveDateTo=?,#ProducerLocationID=?', connection_string, params = myparams)
But this way does not work:
import datetime as dt
today = dt.date.today()
prev_monday = today - dt.timedelta(days=9)
prev_sunday = today - dt.timedelta(days=3)
myparams = [prev_monday,prev_sunday, None] # None substitutes NULL in sql
connection_string = pyodbc.connect('DRIVER={SQL Server};server='+server+';DATABASE='+ db+';Trusted_Connection=yes;')
df = pandas.read_sql_query('EXEC PythonTest_Align_RSrptAccountCurrentMunich #EffectiveDateFrom=?,#EffectiveDateTo=?,#ProducerLocationID=?', connection_string, params = myparams)
Error message:
cur.execute(*args)
pandas.io.sql.DatabaseError: Execution failed on sql 'EXEC PythonTest_Align_RSrptAccountCurrentMunich #EffectiveDateFrom=?,#EffectiveDateTo=?,#ProducerLocationID=?': ('HYC00', '[HYC00] [Microsoft][ODBC SQL Server Driver]Optional feature not implemented (0) (SQLBindParameter)')
Is there specific way for passing date parameters in python?
my guess here is that SQL can't handle pandas datetime values,
try passing the datetime values into strigns using strftime
import datetime as dt
today = dt.date.today()
prev_monday = (today - dt.timedelta(days=9)).strftime('%Y-%m-%d')
prev_sunday = (today - dt.timedelta(days=3)).strftime('%Y-%m-%d')
myparams = [prev_monday,prev_sunday, None]
print(myparams)
['2019-10-14', '2019-10-20', None]
I've a script that makes a query to my database on MySQL. And my doubt is if I can pass any parameter to that query through Python.
For example, on the following script I want to calculate the date_filter using Python and then apply that filter on the query.
now = dt.datetime.now()
date_filter = now - timedelta(days=3)
dq_connection = mysql.connector.connect(user='user', password='pass', host='localhost', database='db')
engine = create_engine('localhost/db')
cursor = connection.cursor(buffered=True)
query = ('''
SELECT *
FROM myTable
WHERE date >= ''' + date_filter + '''
''')
I try it on that way but I got the following error:
builtins.TypeError: can only concatenate str (not "datetime.datetime") to str
It is possible to apply the filter like that?
Thanks!
Yes, you can do it. To avoid sql injections, the best way is not using the python formatting facilities, but the sql parameters & placeholders (see that you don´t need the single quotes ' as the placeholder does the job and converts the type of the variable):
now = dt.datetime.now()
date_filter = now - timedelta(days=3)
dq_connection = mysql.connector.connect(user='user', password='pass', host='localhost', database='db')
engine = create_engine('localhost/db')
cursor = db_connection.cursor(buffered=True)
query = "SELECT * FROM myTable WHERE date >=%s"
cursor.execute(query,(date_filter,))
Also, you had a mistake in your cursor, it should be db_connection.cursor. The last comma after date_filter is ok because you need to send a tuple.
In case you need more than one paremeter, you can place more than one placeholder:
query = "SELECT * FROM myTable WHERE date >=%s and date<=%s"
cursor.execute(query,(date_filter,other_date))
You can just do something like:
WHERE date >= ''' + str(date_filter) + '''
to represent the date as string as not a datetime object.
You can try with this:
date_f = str(date_filter)
query = ('''
SELECT *
FROM myTable
WHERE date >= "{}"
'''.format(date_f))
I am using Python 3.6, pyodbc, and connect to SQL Server.
I am trying make connection to a database, then creating a query with parameters.
Here is the code:
import sys
import pyodbc
# connection parameters
nHost = 'host'
nBase = 'base'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
print("connection successfull")
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
# if run WITHOUT parameters THEN everything is OK
ask = input ('Go WITHOUT parameters y/n ?')
if ask == 'y':
# SQL without parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN '2017-03-01T00:00:00.000' AND '2017-03-01T01:00:00.000'
''')
# SQL without parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
# if run WITH parameters THEN ERROR
ask = input ('Go WITH parameters y/n ?')
if ask == 'y':
# parameters start
STARTDATE = "'2017-03-01T00:00:00.000'"
ENDDATE = "'2017-03-01T01:00:00.000'"
# parameters end
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN :STARTDATE AND :ENDDATE
''', {"STARTDATE": STARTDATE, "ENDDATE": ENDDATE})
# SQL with parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
When I run the program without parameters in SQL, it works.
When I try running it with parameters, an error occurred.
Parameters in an SQL statement via ODBC are positional, and marked by a ?. Thus:
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN ? AND ?
''', STARTDATE, ENDDATE)
# SQL with parameters stop
Plus, it's better to avoid passing dates as strings. Let pyodbc take care of that using Python's datetime:
from datetime import datetime
...
STARTDATE = datetime(year=2017, month=3, day=1)
ENDDATE = datetime(year=2017, month=3, day=1, hour=0, minute=0, second=1)
then just pass the parameters as above. If you prefer string parsing, see this answer.
If you're trying to use pd.to_sql() like me I fixed the problem by passing a parameter called chunksize.
df.to_sql("tableName", engine ,if_exists='append', chunksize=50)
hope this helps
i tryied and have a lot of different errors: 42000, 22007, 07002 and others
The work version is bellow:
import sys
import pyodbc
import datetime
# connection parameters
nHost = 'host'
nBase = 'DBname'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
STARTDATE = '11/2/2017'
ENDDATE = '12/2/2017'
params = (STARTDATE, ENDDATE)
# SQL with parameters start
sql = ('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN CAST(? as datetime) AND CAST(? as datetime)
''')
# SQL with parameters stop
# print result to console start
query = cursor.execute(sql, params)
row = query.fetchone()
while row:
print (row)
row = query.fetchone()
# print result to console stop
say = input ('everething is ok, you can close console')
I fixed this issue with code if you are using values through csv.
for i, row in read_csv_data.iterrows():
cursor.execute('INSERT INTO ' + self.schema + '.' + self.table + '(first_name, last_name, email, ssn, mobile) VALUES (?,?,?,?,?)', tuple(row))
I had a similar issue. Saw that downgrading the version of PyODBC to 4.0.6 and SQLAlchemy to 1.2.9 fixed the error,using Python 3.6