I want to take some action based on comparing two dates. Date 1 is stored in a python variable. Date 2 is retrieved from the database in the select statement. For example I want to retrieve some records from the database where the associated date in the record (in form of the timestamp) is later than the date defined by the python variable. Preferably, I would like the comparison to be in readable date format rather than in timestamps.
I am a beginner with python.
----edit -----
Sorry for being ambiguous. Here's what I am trying to do:
import MySQLdb as mdb
from datetime import datetime
from datetime import date
import time
conn = mdb.connect('localhost','root','root','my_db')
cur = conn.cursor()
right_now = date.today()// python date
this is the part which I want to figure out
The database has a table which has timestamp. I want to compare that timestamp with this date and then retrieve records based on that comparison. For example I want to retrieve all records for which timestamp is above this date
cur.execute("SELECT created from node WHERE timestamp > right_now")
results = cur.fetchall()
for row in results:
print row
first of all, I guess Date 1 (python variable) is a datetime object. http://docs.python.org/2/library/datetime.html
As far as I have used it, MySQLdb gives you results in a (python) datetime object if the sql type was datetime.
So actually you have nothing to do, you can use python datetime comparison methods with date 1 and date 2.
I am a little bit confused about "comparison to be in readable date format rather than in timestamps". I mean the timestamps is readable enough, right?
If Date 1 is timestamps data, then you just simply do comparison. If not, then convert it to timestamps or convert the date in database to date type, both way works.
If you are asking how to write the code to do the comparison, you would use either '_mysql' or sqlalchemy to help you. The detailed syntax can be found at any where.
Anyway, the question itself is not clear enough, so the answer is blur, too.
Related
I can't seem to get reliable results from the query against a sqlite database using a datetime string as a comparison as so:
select *
from table_1
where mydate >= '1/1/2009' and mydate <= '5/5/2009'
how should I handle datetime comparisons to sqlite?
update:
field mydate is a DateTime datatype
To solve this problem, I store dates as YYYYMMDD. Thus,
where mydate >= '20090101' and mydate <= '20050505'
It just plain WORKS all the time. You may only need to write a parser to handle how users might enter their dates so you can convert them to YYYYMMDD.
SQLite doesn't have dedicated datetime types, but does have a few datetime functions. Follow the string representation formats (actually only formats 1-10) understood by those functions (storing the value as a string) and then you can use them, plus lexicographical comparison on the strings will match datetime comparison (as long as you don't try to compare dates to times or datetimes to times, which doesn't make a whole lot of sense anyway).
Depending on which language you use, you can even get automatic conversion. (Which doesn't apply to comparisons in SQL statements like the example, but will make your life easier.)
I had the same issue recently, and I solved it like this:
SELECT * FROM table WHERE
strftime('%s', date) BETWEEN strftime('%s', start_date) AND strftime('%s', end_date)
The following is working fine for me using SQLite:
SELECT *
FROM ingresosgastos
WHERE fecharegistro BETWEEN "2010-01-01" AND "2013-01-01"
Following worked for me.
SELECT *
FROM table_log
WHERE DATE(start_time) <= '2017-01-09' AND DATE(start_time) >= '2016-12-21'
Sqlite can not compare on dates. we need to convert into seconds and cast it as integer.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
I have a situation where I want data from up to two days ago and up until the end of today.
I arrived at the following.
WHERE dateTimeRecorded between date('now', 'start of day','-2 days')
and date('now', 'start of day', '+1 day')
Ok, technically I also pull in midnight on tomorrow like the original poster, if there was any data, but my data is all historical.
The key thing to remember, the initial poster excluded all data after 2009-11-15 00:00:00. So, any data that was recorded at midnight on the 15th was included but any data after midnight on the 15th was not.
If their query was,
select *
from table_1
where mydate between Datetime('2009-11-13 00:00:00')
and Datetime('2009-11-15 23:59:59')
Use of the between clause for clarity.
It would have been slightly better. It still does not take into account leap seconds in which an hour can actually have more than 60 seconds, but good enough for discussions here :)
I had to store the time with the time-zone information in it, and was able to get queries working with the following format:
"SELECT * FROM events WHERE datetime(date_added) BETWEEN
datetime('2015-03-06 20:11:00 -04:00') AND datetime('2015-03-06 20:13:00 -04:00')"
The time is stored in the database as regular TEXT in the following format:
2015-03-06 20:12:15 -04:00
Right now i am developing using System.Data.SQlite NuGet package (version 1.0.109.2). Which using SQLite version 3.24.0.
And this works for me.
SELECT * FROM tables WHERE datetime
BETWEEN '2018-10-01 00:00:00' AND '2018-10-10 23:59:59';
I don't need to use the datetime() function. Perhaps they already updated the SQL query on that SQLite version.
Below are the methods to compare the dates but before that we need to identify the format of date stored in DB
I have dates stored in MM/DD/YYYY HH:MM format so it has to be compared in that format
Below query compares the convert the date into MM/DD/YYY format and get data from last five days till today. BETWEEN operator will help and you can simply specify start date AND end date.
select * from myTable where myColumn BETWEEN strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day') AND strftime('%m/%d/%Y %H:%M',datetime('now','localtime'));
Below query will use greater than operator (>).
select * from myTable where myColumn > strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day');
All the computation I have done is using current time, you can change the format and date as per your need.
Hope this will help you
Summved
You could also write up your own user functions to handle dates in the format you choose. SQLite has a fairly simple method for writing your own user functions. For example, I wrote a few to add time durations together.
My query I did as follows:
SELECT COUNT(carSold)
FROM cars_sales_tbl
WHERE date
BETWEEN '2015-04-01' AND '2015-04-30'
AND carType = "Hybrid"
I got the hint by #ifredy's answer. The all I did is, I wanted this query to be run in iOS, using Objective-C. And it works!
Hope someone who does iOS Development, will get use out of this answer too!
Here is a working example in C# in three ways:
string tableName = "TestTable";
var startDate = DateTime.Today.ToString("yyyy-MM-dd 00:00:00"); \\From today midnight
var endDate = date.AddDays(1).ToString("yyyy-MM-dd HH:mm:ss"); \\ Whole day
string way1 /*long way*/ = $"SELECT * FROM {tableName} WHERE strftime(\'%s\', DateTime)
BETWEEN strftime('%s', \'{startDate}\') AND strftime('%s', \'{endDate}\')";
string way2= $"SELECT * FROM {tableName} WHERE DateTime BETWEEN \'{startDate}\' AND \'{endDate}\'";
string way3= $"SELECT * FROM {tableName} WHERE DateTime >= \'{startDate}\' AND DateTime <=\'{endDate}\'";
select *
from table_1
where date(mydate) >= '1/1/2009' and date(mydate) <= '5/5/2009'
This work for me
I have some discrepancy when subtracting dates in Postgresql and SQLAlchemy. For instance, I have the following in Postgresql:
SELECT trunc(EXTRACT(EPOCH FROM ('2019-07-05 15:20:10.111497-07:00'::timestamp - '2019-07-04 11:45:17.293328-07:00'::timestamp)))
--99292
and the following query in SQLAlchemy:
date_diff = session.query(func.trunc((func.extract('epoch',
func.date('2019-07-05 15:20:10.111497-07:00'))-
func.extract('epoch',
func.date('2019-07-04 11:45:17.293328-07:00'))))).all()
print(date_diff)
#[(86400.0,)]
We can see that the most exact difference is coming from Postgresql query. How can I get the same result using SQLAlchemy? I have not been able to spot what is the cause of this difference. If you know please let me know.
Thanks a lot.
Have never used SQLAlchemy before but it looks like you are trying to truncate to a date instead of a timestamp or datetime
Don't worry, this is an easy mistake to make. DateTime libraries can be confusing with their definitions (a date is a literally a Date so YYYY-MM-DD whereas a timestamp includes both the date and time to some denomination)
This is why you have a difference of 86,400 (one day) because it is comparing the dates of the two objects (2019-07-05 - 2019-07-04)
Try using the func.time.as_utc() or something similar to get a timestamp
You want to be comparing the WHOLE timestamp
EDIT: Sorry, didn't see your comment until after posting.
I have a DB with time entries formatted as follow:
2018-11-05T08:58:00Z
I'm trying to generate SQL queries to compare "now()" with the time in the DB to determine which row(s) to return.
I'm battling to "convert" my local time (now()) to an equivalent time format so that I can use < or > operations against the DB values.
Additionally, I am not sure if the problem has two parts. The example fo the time above is not in a "Datetime" field in MySQL but stored simply as TEXT, leaving me to suspect that I would need to "convert" the DB entries into another format first?
The following code, using the datetime module, works for me (tested in Python 3.6):
import datetime
value = "2018-11-05T08:58:00Z"
dt = datetime.datetime.strptime(value, "%Y-%m-%dT%H:%M:%SZ")
# Result is: datetime.datetime(2018, 11, 5, 8, 58)
This will convert your string values to datetime instances, which you can then compare to now(). The values that get created should be naive (meaning they have no associated timezone information).
However, if you are sure that now() for you is not UTC (aka Zulu time), you may need to do a conversion. This could be possible if, for example, you are using Django's timezone.now() and your configured timezone is something other than UTC. In this case, I might convert the result of now() to UTC, so you only have to convert one value. The pytz module can easily handle this kind of thing.
Check this :
import time
time = time.strftime('%Y-%m-%dT%H:%M:%SZ')
print(time)
I am using SQLite with python. I have a database with two fields (timestamp, reading).
the timestamp is an ISO8601 string formatted like this "YYYY-MM-DD HH:MM:SS.SSSSSS".
When I run this SQL query:
SELECT timestamp, value FROM 'readings' WHERE timestamp > datetime('2017-08-30 14:19:28.684314')
I get all the appropriate readings where the timestamp is since the date provided but I also get the reading from the datetime I pass in (in the example: '2017-08-30 14:19:28.684314').
My question is why is the greater than comparison operator pretending it's a greater than or equal to operator?
SQLite does not have a separate data type for timestamps.
datetime() returns just a string in SQLite's default format:
> select datetime('2017-08-30 14:19:28.684314');
2017-08-30 14:19:28
This does not include milliseconds. So the comparison ends up between a string with milliseconds against a string without milliseconds; the first one is larger because (after the first 19 characters are equal) it has more characters.
Calling datetime() on both values removes the milliseconds from both values.
It might be a better idea to call datetime() on neither value and to compare them directly.
I solve the problem. I will detail it here in case it is helpful to someone else.
It was with my query. SQLite does not have a direct type for date's or datetime's.
My old query:
SELECT timestamp, value FROM 'readings' WHERE timestamp > datetime('2017-08-30 14:19:28.684314')
was implicitly relying on SQL to figure out that the timestamp field was a datetime. SQLite stores them as TEXT fields internally.
When I modified my query to the following:
SELECT timestamp, value FROM 'readings' WHERE datetime(timestamp) > datetime('2017-08-30 14:19:28.684314')
I started to get the results that I was expecting.
I need to insert rows into PG one of the fields is date and time with time stamp, this is the time of incident, so I can not use --> current_timestamp function of Postgres at the time of insertion, so how can I then insert the time and date which I collected before into pg row in the same format as it would have been created by current_timestamp at that point in time.
If you use psycopg2 (and possibly some other client library), you can simply pass a Python datetime object as a parameter to a SQL-query:
from datetime import datetime, timezone
dt = datetime.now(timezone.utc)
cur.execute('INSERT INTO mytable (mycol) VALUES (%s)', (dt,))
(This assumes that the timestamp with time zone type is used on the database side.)
More Python types that can be adapted into SQL (and returned as Python objects when a query is executed) are listed here.
A timestamp does not have "a format".
The recommended way to deal with timestamps is to use a PreparedStatement where you just pass a placeholder in the SQL and pass a "real" object through the API of your programming language. As I don't know Python, I don't know if it supports PreparedStatements and how the syntax for that would be.
If you want to put a timestamp literal into your generated SQL, you will need to follow some formatting rules when specifying the value (a literal does have a format).
Ivan's method will work, although I'm not 100% sure if it depends on the configuration of the PostgreSQL server.
A configuration (and language) independent solution to specify a timestamp literal is the ANSI SQL standard:
INSERT INTO some_table
(ts_column)
VALUES
(TIMESTAMP '2011-05-16 15:36:38');
Yes, that's the keyword TIMESTAMP followed by a timestamp formatted in ISO style (the TIMESTAMP keyword defines that format)
The other solution would be to use the to_timestamp() function where you can specify the format of the input literal.
INSERT INTO some_table
(ts_column)
VALUES
(to_timestamp('16-05-2011 15:36:38', 'dd-mm-yyyy hh24:mi:ss'));
Just use 'now'
http://www.postgresql.org/docs/8.0/static/datatype-datetime.html
Date and time input is accepted in almost any reasonable format,
including ISO 8601, SQL-compatible, traditional POSTGRES, and others.
For some formats, ordering of month, day, and year in date input is
ambiguous and there is support for specifying the expected ordering of
these fields.
In other words: just write anything and it will work.
Or check this table with all the unambiguous formats.
Sure, just pass a string value for that timestamp column in the format: '2011-05-16 15:36:38' (you can also append a timezone there, like 'PST'). PostgreSQL will automatically convert the string to a timestamp. See http://www.postgresql.org/docs/9.0/static/datatype-datetime.html#DATATYPE-DATETIME-INPUT
from datetime import datetime as dt
then use this in your code:
cur.execute('INSERT INTO my_table (dt_col) VALUES (%s)', (dt.now(),))
Just use
now()
or
CURRENT_TIMESTAMP
I prefer the latter as I like not having additional parenthesis but thats just personal preference.