I'm trying to understand how I can use the local server time to quickly filter results on google appengine. It seems to me that there should be a simple way of doing this using DATETIME(time.localtime()).
For example (where 'timestamp' is of type db.DateTimeProperty)...
q = db.GqlQuery("SELECT * FROM LiveData WHERE timestamp > DATETIME(:1)", time.localtime())
Is there a GqlQuery and/or python construct that lets me do this with one method call? It seems as though I need to create strings for DATETIME() parameters.
You do not have to create strings when querying DateTimeProperty types. Try this:
import datetime
q = db.GqlQuery("SELECT * FROM LiveData WHERE timestamp > :1", datetime.datetime.now())
Related
I am trying to build a SQL query that will filter based on system date (Query for all sales done in the last 7 days):
import datetime
import pandas as pd
import psycopg2
con = p.connect(db_details)
cur = con.cursor()
df = pd.read_sql("""select store_name,count(*) from sales
where created_at between datetime.datetime.now() - (datetime.today() - timedelta(7))""",con=con)
I get an error
psycopg2.NotSupportedError: cross-database references are not implemented: datetime.datetime.now
You are mixing Python syntax into your SQL query. SQL is parsed and executed by the database, not by Python, and the database knows nothing about datetime.datetime.now() or datetime.date() or timedelta()! The specific error you see is caused by your Python code being interpreted as SQL instead and as SQL, datetime.datetime.now references the now column of the datetime table in the datetime database, which is a cross-database reference, and psycopg2 doesn't support queries that involve multiple databases.
Instead, use SQL parameters to pass in values from Python to the database. Use placeholders in the SQL to show the database driver where the values should go:
params = {
# all rows after this timestamp, 7 days ago relative to 'now'
'earliest': datetime.datetime.now() - datetime.timedelta(days=7),
# if you must have a date *only* (no time component), use
# 'earliest': datetime.date.today() - datetime.timedelta(days=7),
}
df = pd.read_sql("""
select store_name,count(*) from sales
where created_at >= %(latest)s""", params=params, con=con)
This uses placeholders as defined by the psycopg2 parameters documentation, where %(latest)s refers to the latest key in the params dictionary. datetime.datetime() instances are directly supported by the driver.
Note that I also fixed your 7 days ago expression, and replaced your BETWEEN syntax with >=; without a second date you are not querying for values between two dates, so use >= to limit the column to dates at or after the given date.
datetime.datetime.now() is not a proper SQL syntax, and thus cannot be executed by read_sql(). I suggest either using the correct SQL syntax that computes current time, or creating variables for each datetime.datetime.now() and datetime.today() - timedelta(7) and replacing them in your string.
edit: Do not follow the second suggestion. See comments below by Martijn Pieters.
Maybe you should remove that Python code inside your SQL, compute your dates in python and then use the strftime function to convert them to strings.
Then you'll be able to use them in your SQL query.
Actually, you do not necessarily need any params or computations in Python. Just use the corresponding SQL statement which should look like this:
select store_name,count(*)
from sales
where created_at >= now()::date - 7
group by store_name
Edit: I also added a group by which I think is missing.
I need help to create query to compare between date with time and date without time. I am using python with access db (pypyodbc).
In the database I have a column that contains date/time (includes time), and in python I have a datetime object (without time).
I want to write a sql query that compares just the dates of the two.
For Example:
cur.execute("SELECT * FROM MDSSDB WHERE [ValidStartTime] = #2016-05-17#")
The ValidStartTime includes time so it doesn't work. I want just the date from the ValidStartTime.
Consider using MS Access' DateValue function that extracts only the date component (TimeValue being the time component counterpart).
Also, consider passing your date value as parameter to better integrate with your Python environment with no need to concatenate into Access' # form. Below passes a parameter as tuple of one item:
from datetime import datetime
...
cur.execute("SELECT * FROM MDSSDB WHERE DateValue([ValidStartTime]) = ?", (datetime(2016, 5, 17),))
I am trying to get all records that are within the current week(monday to sunday) and then I was planning to insert them back into the database but with the date of lesson being increased by 7. However I get the following error:
sqlite3.OperationalError: near ">=": syntax error
I may be wrong but I think that this is due to how python stores dates is there a way around this, if not I can always get all records in table into array and filter that array in python. The code for the sql is underneath:
with sqlite3.connect("GuitarLessons.db") as db:
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
sql = "select *"\
"from tblBookings"\
"where DateOfLesson >= ?"\
"and DateOfLesson <= ?"
cursor.execute(sql,(startweekd,endweekd))
BookingList = cursor.fetchall()
print(BookingList)
The rest of my code is just calculating the start and end date for that week.
import datetime
from datetime import date, timedelta
import sqlite3
tdate = datetime.datetime.today()
tday = datetime.datetime.today().weekday()
tdadd = 7 - (tday+1)
endweekd = date.today() + timedelta(days=tdadd)
startweekd = endweekd - timedelta(days=7)
endweekd = endweekd.strftime("%d/%m/%y")
startweekd = startweekd.strftime("%d/%m/%y")
print(startweekd)
print(endweekd)
SQLite (nowhere, not just in Python) does not support dates.
So you have to convert the dates, on query, but also on storage, to some format it will understand. There are two options:
Number of seconds since some epoch, e.g. unix time, possibly fractional.
Strings.
To make comparison of strings work, the dates must be stored in the ISO 8601 format (or at least that order). ISO 8601 timestamp has format specification "%FT%T" (or on systems that don't understand %F or %T "%Y-%m-%dT%H:%M:%S"). Or just dates as "%F"/"%Y-%m-%d". You can use different separators, but the only thing that will gain you is some confusion. Also SQLite has some built-in functions to work with date in ISO 8601 format.
I believe you can define the conversion somewhere so it will then be used automatically when binding query parameters, but I don't remember where. Manually is guaranteed to work.
sqllite requires date to be in YYYY:MM:DD format. You probably should use strftime with the following parameters:
endweekd = endweekd.strftime("%Y:%m:%d")
startweekd = startweekd.strftime("%Y:%m:%d")
In Django, I am trying to filter my query only to objects that were created before a certain hour in the day. I have a datetime field called 'created_at' that stored the datetime from which that object was created.
What I would like to do is:
query = query.filter(created_at__hour__lte=10)
Which I would expect to get all the objects that were created before 10am. However, when I try that I get a:
FieldError: Join on field 'created_at' not permitted. Did you misspell 'hour' for the lookup type?
I could loop through each day and get that day's objects, but that seems highly inefficient. Is there a way I can do this in a single query? If not, what is the fastest way to run this sort of filter?
__hour on a DateTimeField is a lookup type, so you can't mix it with another lookup type like __lte. You could construct a filter with Q objects, EG:
before_ten = Q(created_at__hour=0)
for hour in range(1, 11):
before_ten = before_ten | Q(created_at__hour=hour)
query = query.filter(before_ten)
If you can change your data model, it might be more convenient to save a creation time TimeField as well as your existing created_at.
In Django 1.9+, you can chain hour lookups like created_at__hour__lte, so the query from the question will work.
query = query.filter(created_at__hour__lte=10)
import datetime
start_time = datetime.datetime.now().replace(hour=00, minute=00)
certain_hour = 10
end_time = start_time.replace(hour=certain_hour)
query = query.filter(created_at__range=(start_time, end_time)
I have a NDB datetime property stored on Google App Engine. I'm trying to query for all records since yesterday by converting the datetime to a date, then filtering the query for anything greater or equal to yesterdays date. However, I am getting the following error:
follower_trans = fol.query(
datetime.datetime.date(fol.followed_date) >= self.yesterday).fetch()
TypeError: descriptor 'date' requires a 'datetime.datetime' object but received a 'DateTimeProperty'
If i use just datetime.date() it sends an error requiring an integer as an argument.
You can't do a conversion as part of the query, so you'll have to create 'yesterday' as a datetime and filter using that.
fol.followed_date is the model's property, not an entity's value. You cannot convert a property to a date, as it has no value. You need to fix the query constructor:
follower_trans = fol.query(fol.followed_date >= self.yesterday).fetch()
(assuming you've created self.yesterday as a datetime object.)
Also, it is proper to use a capital letter for any Class names. Should have been:
class Fol(ndb.Model):
followed_date = ndb.DateTimeProperty(auto_now_add = True)
and
Fol.query(Fol.followed_date >= self.yesterday).fetch()