I need to insert rows into PG one of the fields is date and time with time stamp, this is the time of incident, so I can not use --> current_timestamp function of Postgres at the time of insertion, so how can I then insert the time and date which I collected before into pg row in the same format as it would have been created by current_timestamp at that point in time.
If you use psycopg2 (and possibly some other client library), you can simply pass a Python datetime object as a parameter to a SQL-query:
from datetime import datetime, timezone
dt = datetime.now(timezone.utc)
cur.execute('INSERT INTO mytable (mycol) VALUES (%s)', (dt,))
(This assumes that the timestamp with time zone type is used on the database side.)
More Python types that can be adapted into SQL (and returned as Python objects when a query is executed) are listed here.
A timestamp does not have "a format".
The recommended way to deal with timestamps is to use a PreparedStatement where you just pass a placeholder in the SQL and pass a "real" object through the API of your programming language. As I don't know Python, I don't know if it supports PreparedStatements and how the syntax for that would be.
If you want to put a timestamp literal into your generated SQL, you will need to follow some formatting rules when specifying the value (a literal does have a format).
Ivan's method will work, although I'm not 100% sure if it depends on the configuration of the PostgreSQL server.
A configuration (and language) independent solution to specify a timestamp literal is the ANSI SQL standard:
INSERT INTO some_table
(ts_column)
VALUES
(TIMESTAMP '2011-05-16 15:36:38');
Yes, that's the keyword TIMESTAMP followed by a timestamp formatted in ISO style (the TIMESTAMP keyword defines that format)
The other solution would be to use the to_timestamp() function where you can specify the format of the input literal.
INSERT INTO some_table
(ts_column)
VALUES
(to_timestamp('16-05-2011 15:36:38', 'dd-mm-yyyy hh24:mi:ss'));
Just use 'now'
http://www.postgresql.org/docs/8.0/static/datatype-datetime.html
Date and time input is accepted in almost any reasonable format,
including ISO 8601, SQL-compatible, traditional POSTGRES, and others.
For some formats, ordering of month, day, and year in date input is
ambiguous and there is support for specifying the expected ordering of
these fields.
In other words: just write anything and it will work.
Or check this table with all the unambiguous formats.
Sure, just pass a string value for that timestamp column in the format: '2011-05-16 15:36:38' (you can also append a timezone there, like 'PST'). PostgreSQL will automatically convert the string to a timestamp. See http://www.postgresql.org/docs/9.0/static/datatype-datetime.html#DATATYPE-DATETIME-INPUT
from datetime import datetime as dt
then use this in your code:
cur.execute('INSERT INTO my_table (dt_col) VALUES (%s)', (dt.now(),))
Just use
now()
or
CURRENT_TIMESTAMP
I prefer the latter as I like not having additional parenthesis but thats just personal preference.
Related
I can't seem to get reliable results from the query against a sqlite database using a datetime string as a comparison as so:
select *
from table_1
where mydate >= '1/1/2009' and mydate <= '5/5/2009'
how should I handle datetime comparisons to sqlite?
update:
field mydate is a DateTime datatype
To solve this problem, I store dates as YYYYMMDD. Thus,
where mydate >= '20090101' and mydate <= '20050505'
It just plain WORKS all the time. You may only need to write a parser to handle how users might enter their dates so you can convert them to YYYYMMDD.
SQLite doesn't have dedicated datetime types, but does have a few datetime functions. Follow the string representation formats (actually only formats 1-10) understood by those functions (storing the value as a string) and then you can use them, plus lexicographical comparison on the strings will match datetime comparison (as long as you don't try to compare dates to times or datetimes to times, which doesn't make a whole lot of sense anyway).
Depending on which language you use, you can even get automatic conversion. (Which doesn't apply to comparisons in SQL statements like the example, but will make your life easier.)
I had the same issue recently, and I solved it like this:
SELECT * FROM table WHERE
strftime('%s', date) BETWEEN strftime('%s', start_date) AND strftime('%s', end_date)
The following is working fine for me using SQLite:
SELECT *
FROM ingresosgastos
WHERE fecharegistro BETWEEN "2010-01-01" AND "2013-01-01"
Following worked for me.
SELECT *
FROM table_log
WHERE DATE(start_time) <= '2017-01-09' AND DATE(start_time) >= '2016-12-21'
Sqlite can not compare on dates. we need to convert into seconds and cast it as integer.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
I have a situation where I want data from up to two days ago and up until the end of today.
I arrived at the following.
WHERE dateTimeRecorded between date('now', 'start of day','-2 days')
and date('now', 'start of day', '+1 day')
Ok, technically I also pull in midnight on tomorrow like the original poster, if there was any data, but my data is all historical.
The key thing to remember, the initial poster excluded all data after 2009-11-15 00:00:00. So, any data that was recorded at midnight on the 15th was included but any data after midnight on the 15th was not.
If their query was,
select *
from table_1
where mydate between Datetime('2009-11-13 00:00:00')
and Datetime('2009-11-15 23:59:59')
Use of the between clause for clarity.
It would have been slightly better. It still does not take into account leap seconds in which an hour can actually have more than 60 seconds, but good enough for discussions here :)
I had to store the time with the time-zone information in it, and was able to get queries working with the following format:
"SELECT * FROM events WHERE datetime(date_added) BETWEEN
datetime('2015-03-06 20:11:00 -04:00') AND datetime('2015-03-06 20:13:00 -04:00')"
The time is stored in the database as regular TEXT in the following format:
2015-03-06 20:12:15 -04:00
Right now i am developing using System.Data.SQlite NuGet package (version 1.0.109.2). Which using SQLite version 3.24.0.
And this works for me.
SELECT * FROM tables WHERE datetime
BETWEEN '2018-10-01 00:00:00' AND '2018-10-10 23:59:59';
I don't need to use the datetime() function. Perhaps they already updated the SQL query on that SQLite version.
Below are the methods to compare the dates but before that we need to identify the format of date stored in DB
I have dates stored in MM/DD/YYYY HH:MM format so it has to be compared in that format
Below query compares the convert the date into MM/DD/YYY format and get data from last five days till today. BETWEEN operator will help and you can simply specify start date AND end date.
select * from myTable where myColumn BETWEEN strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day') AND strftime('%m/%d/%Y %H:%M',datetime('now','localtime'));
Below query will use greater than operator (>).
select * from myTable where myColumn > strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day');
All the computation I have done is using current time, you can change the format and date as per your need.
Hope this will help you
Summved
You could also write up your own user functions to handle dates in the format you choose. SQLite has a fairly simple method for writing your own user functions. For example, I wrote a few to add time durations together.
My query I did as follows:
SELECT COUNT(carSold)
FROM cars_sales_tbl
WHERE date
BETWEEN '2015-04-01' AND '2015-04-30'
AND carType = "Hybrid"
I got the hint by #ifredy's answer. The all I did is, I wanted this query to be run in iOS, using Objective-C. And it works!
Hope someone who does iOS Development, will get use out of this answer too!
Here is a working example in C# in three ways:
string tableName = "TestTable";
var startDate = DateTime.Today.ToString("yyyy-MM-dd 00:00:00"); \\From today midnight
var endDate = date.AddDays(1).ToString("yyyy-MM-dd HH:mm:ss"); \\ Whole day
string way1 /*long way*/ = $"SELECT * FROM {tableName} WHERE strftime(\'%s\', DateTime)
BETWEEN strftime('%s', \'{startDate}\') AND strftime('%s', \'{endDate}\')";
string way2= $"SELECT * FROM {tableName} WHERE DateTime BETWEEN \'{startDate}\' AND \'{endDate}\'";
string way3= $"SELECT * FROM {tableName} WHERE DateTime >= \'{startDate}\' AND DateTime <=\'{endDate}\'";
select *
from table_1
where date(mydate) >= '1/1/2009' and date(mydate) <= '5/5/2009'
This work for me
I am using SQLite with python. I have a database with two fields (timestamp, reading).
the timestamp is an ISO8601 string formatted like this "YYYY-MM-DD HH:MM:SS.SSSSSS".
When I run this SQL query:
SELECT timestamp, value FROM 'readings' WHERE timestamp > datetime('2017-08-30 14:19:28.684314')
I get all the appropriate readings where the timestamp is since the date provided but I also get the reading from the datetime I pass in (in the example: '2017-08-30 14:19:28.684314').
My question is why is the greater than comparison operator pretending it's a greater than or equal to operator?
SQLite does not have a separate data type for timestamps.
datetime() returns just a string in SQLite's default format:
> select datetime('2017-08-30 14:19:28.684314');
2017-08-30 14:19:28
This does not include milliseconds. So the comparison ends up between a string with milliseconds against a string without milliseconds; the first one is larger because (after the first 19 characters are equal) it has more characters.
Calling datetime() on both values removes the milliseconds from both values.
It might be a better idea to call datetime() on neither value and to compare them directly.
I solve the problem. I will detail it here in case it is helpful to someone else.
It was with my query. SQLite does not have a direct type for date's or datetime's.
My old query:
SELECT timestamp, value FROM 'readings' WHERE timestamp > datetime('2017-08-30 14:19:28.684314')
was implicitly relying on SQL to figure out that the timestamp field was a datetime. SQLite stores them as TEXT fields internally.
When I modified my query to the following:
SELECT timestamp, value FROM 'readings' WHERE datetime(timestamp) > datetime('2017-08-30 14:19:28.684314')
I started to get the results that I was expecting.
I want to take some action based on comparing two dates. Date 1 is stored in a python variable. Date 2 is retrieved from the database in the select statement. For example I want to retrieve some records from the database where the associated date in the record (in form of the timestamp) is later than the date defined by the python variable. Preferably, I would like the comparison to be in readable date format rather than in timestamps.
I am a beginner with python.
----edit -----
Sorry for being ambiguous. Here's what I am trying to do:
import MySQLdb as mdb
from datetime import datetime
from datetime import date
import time
conn = mdb.connect('localhost','root','root','my_db')
cur = conn.cursor()
right_now = date.today()// python date
this is the part which I want to figure out
The database has a table which has timestamp. I want to compare that timestamp with this date and then retrieve records based on that comparison. For example I want to retrieve all records for which timestamp is above this date
cur.execute("SELECT created from node WHERE timestamp > right_now")
results = cur.fetchall()
for row in results:
print row
first of all, I guess Date 1 (python variable) is a datetime object. http://docs.python.org/2/library/datetime.html
As far as I have used it, MySQLdb gives you results in a (python) datetime object if the sql type was datetime.
So actually you have nothing to do, you can use python datetime comparison methods with date 1 and date 2.
I am a little bit confused about "comparison to be in readable date format rather than in timestamps". I mean the timestamps is readable enough, right?
If Date 1 is timestamps data, then you just simply do comparison. If not, then convert it to timestamps or convert the date in database to date type, both way works.
If you are asking how to write the code to do the comparison, you would use either '_mysql' or sqlalchemy to help you. The detailed syntax can be found at any where.
Anyway, the question itself is not clear enough, so the answer is blur, too.
I have a SQLite DB which has few columns of type datetime.
These column store time (just the time and not the date) in HH:mm:ss format.
When I try to access this field it is returned as null.
In models.py this field have been mapped to equivalent models.DateTimeField.
What is the correct way of accessing such fields?
Is it required that models.DateTimeField HAS TO BE stored in YYYY-MM-DD HH:mm:ss format insqlite?
I am not suprised that DateTimeField expects both a date and a time.
Try TimeField instead.
(Please note that SQLite does not have a native date/time type; it's just a string.)
I am having design problems with date storage/retrieval using Python and SQLite.
I understand that a SQLite date column stores dates as text in ISO format
(ie. '2010-05-25'). So when I display a British date (eg. on a web-page) I
convert the date using
datetime.datetime.strptime(mydate,'%Y-%m-%d').strftime('%d/%m/%Y')
However, when it comes to writing-back data to the table, SQLite is very
forgiving and is quite happy to store '25/06/2003' in a date field, but this
is not ideal because
I could be left with a mixture of date formats in the same
column,
SQLite's date functions only work with ISO format.
Therefore I need to convert the date string back to ISO format before
committing, but then I would need a generic function which checks data about to
be written in all date fields and converts to ISO if necessary. That sounds a
bit tedious to me, but maybe it is inevitable.
Are there simpler solutions? Would it be easier to change the date field to a
10-character field and store 'dd/mm/yyyy' throughout the table? This way no
conversion is required when reading or writing from the table, and I could use
datetime() functions if I needed to perform any date-arithmetic.
How have other developers overcome this problem? Any help would be appreciated.
For the record, I am using SQLite3 with Python 3.1.
If you set detect_types=sqlite3.PARSE_DECLTYPES in sqlite3.connect,
then the connection will try to convert sqlite data types to Python data types
when you draw data out of the database.
This is a very good thing since its much nicer to work with datetime objects than
random date-like strings which you then have to parse with
datetime.datetime.strptime or dateutil.parser.parse.
Unfortunately, using detect_types does not stop sqlite from accepting
strings as DATE data, but you will get an error when you try to
draw the data out of the database (if it was inserted in some format other than YYYY-MM-DD)
because the connection will fail to convert it to a datetime.date object:
conn=sqlite3.connect(':memory:',detect_types=sqlite3.PARSE_DECLTYPES)
cur=conn.cursor()
cur.execute('CREATE TABLE foo(bar DATE)')
# Unfortunately, this is still accepted by sqlite
cur.execute("INSERT INTO foo(bar) VALUES (?)",('25/06/2003',))
# But you won't be able to draw the data out later because parsing will fail
try:
cur.execute("SELECT * FROM foo")
except ValueError as err:
print(err)
# invalid literal for int() with base 10: '25/06/2003'
conn.rollback()
But at least the error will alert you to the fact that you've inserted
a string for a DATE when you really should be inserting datetime.date objects:
cur.execute("INSERT INTO foo(bar) VALUES (?)",(datetime.date(2003,6,25),))
cur.execute("SELECT ALL * FROM foo")
data=cur.fetchall()
data=zip(*data)[0]
print(data)
# (datetime.date(2003, 6, 25),)
You may also insert strings as DATE data as long as you use the YYYY-MM-DD format. Notice that although you inserted a string, it comes back out as a datetime.date object:
cur.execute("INSERT INTO foo(bar) VALUES (?)",('2003-06-25',))
cur.execute("SELECT ALL * FROM foo")
data=cur.fetchall()
data=zip(*data)[0]
print(data)
# (datetime.date(2003, 6, 25), datetime.date(2003, 6, 25))
So if you are disciplined about inserting only datetime.date objects into the DATE field, then you'll have no problems later when drawing the data out.
If your users are input-ing date data in various formats, check out dateutil.parser.parse. It may be able to help you convert those various strings into datetime.datetime objects.
Note that SQLite itself does not have a native date/time type. As #unutbu answered, you can make the pysqlite/sqlite3 module try to guess (and note that it really is a guess) which columns/values are dates/times. SQL expressions will easily confuse it.
SQLite does have a variety of date time functions and can work with various strings, numbers in both unixepoch and julian format, and can do transformations. See the documentation:
http://www.sqlite.org/lang_datefunc.html
You may find it more convenient to get SQLite to do the date/time work you need instead of importing the values into Python and using Python libraries to do it. Note that you can put constraints in the SQL table definition for example requiring that string value be present, be a certain length etc.