I am trying to get all records that are within the current week(monday to sunday) and then I was planning to insert them back into the database but with the date of lesson being increased by 7. However I get the following error:
sqlite3.OperationalError: near ">=": syntax error
I may be wrong but I think that this is due to how python stores dates is there a way around this, if not I can always get all records in table into array and filter that array in python. The code for the sql is underneath:
with sqlite3.connect("GuitarLessons.db") as db:
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
sql = "select *"\
"from tblBookings"\
"where DateOfLesson >= ?"\
"and DateOfLesson <= ?"
cursor.execute(sql,(startweekd,endweekd))
BookingList = cursor.fetchall()
print(BookingList)
The rest of my code is just calculating the start and end date for that week.
import datetime
from datetime import date, timedelta
import sqlite3
tdate = datetime.datetime.today()
tday = datetime.datetime.today().weekday()
tdadd = 7 - (tday+1)
endweekd = date.today() + timedelta(days=tdadd)
startweekd = endweekd - timedelta(days=7)
endweekd = endweekd.strftime("%d/%m/%y")
startweekd = startweekd.strftime("%d/%m/%y")
print(startweekd)
print(endweekd)
SQLite (nowhere, not just in Python) does not support dates.
So you have to convert the dates, on query, but also on storage, to some format it will understand. There are two options:
Number of seconds since some epoch, e.g. unix time, possibly fractional.
Strings.
To make comparison of strings work, the dates must be stored in the ISO 8601 format (or at least that order). ISO 8601 timestamp has format specification "%FT%T" (or on systems that don't understand %F or %T "%Y-%m-%dT%H:%M:%S"). Or just dates as "%F"/"%Y-%m-%d". You can use different separators, but the only thing that will gain you is some confusion. Also SQLite has some built-in functions to work with date in ISO 8601 format.
I believe you can define the conversion somewhere so it will then be used automatically when binding query parameters, but I don't remember where. Manually is guaranteed to work.
sqllite requires date to be in YYYY:MM:DD format. You probably should use strftime with the following parameters:
endweekd = endweekd.strftime("%Y:%m:%d")
startweekd = startweekd.strftime("%Y:%m:%d")
Related
I am building a small program in Python and SQlite where a client can type a date in the format YYYY-MM-DD and get back all the plays that take place in that specific date. All the plays are store in Play.db which saves the date as: YYYY-MM-DD HH:MM
I do not want the user to type the exact date with hour and minutes. He just has to type the year-month-day and all the plays in that specific date should appear.
I realise that I need the keyword 'LIKE' and pass my variable between '%'variable'%' in order to get a similar (not exact) match.
However I am not sure how to do so by using the syntax of my SQLite query:
user_input = input("Please, type the date of the play in format: 'YYYY-MM-DD':")
search_date = """SELECT * FROM play WHERE play_date LIKE ? """
cursor.execute(search_date, [(user_input)])
results = cursor.fetchall()
The issue is that 'results' is not populated because the SQL query is looking for exact match.
THEREFORE I tried with %user_input% or %?% but, obsiously, the variable '%user_input%' or %?% is not defined.
I understand how logically is should be done:
how to search similar strings in SQLite
But I am not sure how the "LIKE '%'variable'%'" can be obtained using my code syntax.
import datetime
user_input = datetime.datetime(input("Please, type the date of the play in format: 'YYYY-MM-DD':"), '%Y-%m-%d')
# 2011-11-11
search_date = f'SELECT * FROM play WHERE play_date LIKE {user_input}'
print(search_date)
# SELECT * FROM play WHERE play_date LIKE 2011-11-11
Is this what you're looking for?
Edited to prevent sql injections
Use the date function in your query to just compare the dates of your stored datetime values and user input.
SELECT * FROM play WHERE date(play_date) = date(?)
You need a date function. Also you can use slicing for the string. Like:
theDate = input() #example: 2018-12-24
Year = theDate[:3]
Month = theDate[5:6]
Day = theDate[8:9]
Good day,
Well it may not be exactly what you want but I would share you some pieces of code I wrote when I wanted to build something like this but this time using Flask and SQLAlchemy.
First is to convert the data to a date/datetime to ensure the data is valid
converted_date = datetime.datetime.strptime(user_input, "%Y-%m-%d").date()
Converted to regular SQL query should be
SELECT * FROM play WHERE play_date LIKE {"%" + converted_date}
Please note that the "%" is very important, and putting it at the beginning means you want to search for any date that starts with the inputted date.
I am trying to build a SQL query that will filter based on system date (Query for all sales done in the last 7 days):
import datetime
import pandas as pd
import psycopg2
con = p.connect(db_details)
cur = con.cursor()
df = pd.read_sql("""select store_name,count(*) from sales
where created_at between datetime.datetime.now() - (datetime.today() - timedelta(7))""",con=con)
I get an error
psycopg2.NotSupportedError: cross-database references are not implemented: datetime.datetime.now
You are mixing Python syntax into your SQL query. SQL is parsed and executed by the database, not by Python, and the database knows nothing about datetime.datetime.now() or datetime.date() or timedelta()! The specific error you see is caused by your Python code being interpreted as SQL instead and as SQL, datetime.datetime.now references the now column of the datetime table in the datetime database, which is a cross-database reference, and psycopg2 doesn't support queries that involve multiple databases.
Instead, use SQL parameters to pass in values from Python to the database. Use placeholders in the SQL to show the database driver where the values should go:
params = {
# all rows after this timestamp, 7 days ago relative to 'now'
'earliest': datetime.datetime.now() - datetime.timedelta(days=7),
# if you must have a date *only* (no time component), use
# 'earliest': datetime.date.today() - datetime.timedelta(days=7),
}
df = pd.read_sql("""
select store_name,count(*) from sales
where created_at >= %(latest)s""", params=params, con=con)
This uses placeholders as defined by the psycopg2 parameters documentation, where %(latest)s refers to the latest key in the params dictionary. datetime.datetime() instances are directly supported by the driver.
Note that I also fixed your 7 days ago expression, and replaced your BETWEEN syntax with >=; without a second date you are not querying for values between two dates, so use >= to limit the column to dates at or after the given date.
datetime.datetime.now() is not a proper SQL syntax, and thus cannot be executed by read_sql(). I suggest either using the correct SQL syntax that computes current time, or creating variables for each datetime.datetime.now() and datetime.today() - timedelta(7) and replacing them in your string.
edit: Do not follow the second suggestion. See comments below by Martijn Pieters.
Maybe you should remove that Python code inside your SQL, compute your dates in python and then use the strftime function to convert them to strings.
Then you'll be able to use them in your SQL query.
Actually, you do not necessarily need any params or computations in Python. Just use the corresponding SQL statement which should look like this:
select store_name,count(*)
from sales
where created_at >= now()::date - 7
group by store_name
Edit: I also added a group by which I think is missing.
I'm trying to understand the raw manner in which PostgreSQL saves timestamp data types. I get 2 different results depending on the client I use:
1. psql
# SELECT date_incorporated FROM client;
date_incorporated
------------------------
2017-06-14 19:42:15-04
2. records python module
rows = db.query('SELECT date_incorporated FROM client')
print(rows[0])
# {"date_incorporated": "2017-06-14T19:42:15-04:00"}
Since the psql interface and records module are both supposed to be giving me back the raw data, I can't understand why both are giving me back different formats of the timestamp they have stored.
The two differences I see so far are the T's in the middle between the date and time in the records version, and also the differing ways in which it shows the time zone at the end of the string
Is one of them altering it? Which one is showing the real data?
https://www.postgresql.org/docs/current/static/datatype-datetime.html
All timezone-aware dates and times are stored internally in UTC. They
are converted to local time in the zone specified by the TimeZone
configuration parameter before being displayed to the client.
https://www.postgresql.org/docs/current/static/datatype-datetime.html#DATATYPE-DATETIME-OUTPUT
The output format of the date/time types can be set to one of the four
styles ISO 8601, SQL (Ingres), traditional POSTGRES (Unix date
format), or German. The default is the ISO format.
EG:
t=# select now();
now
-------------------------------
2017-11-29 09:07:31.716276+00
(1 row)
t=# set datestyle to SQL;
SET
t=# select now();
now
--------------------------------
11/29/2017 09:07:52.341416 UTC
(1 row)
so the time is saved not the way it is returned. at least not neseserely. You can control up to some level how it it returned to your client. psql does not process time. but python does. not records I believe but python itself
https://en.wikipedia.org/wiki/ISO_8601
T is the time designator that precedes the time components of the
representation.
And that T is definetely not added by postgres itself (unless you deliberately format the date with to_char)
Issue:
I stored some dates as text in my Postgres table, and I want to convert it over to actual dates, again in Postgres.
Im not sure if there is a better way to do this or what im doing wrong. I have pulled a bunch of data into a PostgreSQL database in just text format. As a result I need to go back through and clean it up. I am running into issues with the data. I need to convert it into a format that PostgreSQL can use. I went about pulling it back into python and trying to convert and kick it back. Is this the best way to do this? Also I am having issue with datetime.strptime.. I believe i've got the directive correct but no go. :/
import psycopg2
from datetime import datetime
# connect to the PostgreSQL database
conn = psycopg2.connect(
"dbname='postgres' user='postgres' host=10.0.75.1 password='mysecretpassword'")
# create a new cursor
cur = conn.cursor()
cur.execute("""SELECT "Hash","Date" FROM nas """)
# commit the changes to the database
myDate = cur.fetchall()
for rows in myDate:
target = rows[1]
datetime.strptime(target, '%B %d, %Y, %H:%M:%S %p %Z')
Here is a Postgres query which can convert your strings into actual timestamps:
select
ts_col,
to_timestamp(ts_col, 'Month DD, YYYY HH:MI:SS PM')::timestamp with time zone
from your_table;
For a full solution, you might take the following steps:
create a new timestamp column ts_col_new in your table
update that column using the logic from the above query
then delete the old column containing text
The update might look something like this:
update your_table
set ts_col_new = to_timestamp(ts_col, 'Month DD, YYYY HH:MI:SS PM')::timestamp with time zone;
I need help to create query to compare between date with time and date without time. I am using python with access db (pypyodbc).
In the database I have a column that contains date/time (includes time), and in python I have a datetime object (without time).
I want to write a sql query that compares just the dates of the two.
For Example:
cur.execute("SELECT * FROM MDSSDB WHERE [ValidStartTime] = #2016-05-17#")
The ValidStartTime includes time so it doesn't work. I want just the date from the ValidStartTime.
Consider using MS Access' DateValue function that extracts only the date component (TimeValue being the time component counterpart).
Also, consider passing your date value as parameter to better integrate with your Python environment with no need to concatenate into Access' # form. Below passes a parameter as tuple of one item:
from datetime import datetime
...
cur.execute("SELECT * FROM MDSSDB WHERE DateValue([ValidStartTime]) = ?", (datetime(2016, 5, 17),))