Related
I have some datetime strings containing miliseconds from midnight.
How do I parse these to yyyy-MM-dd HH:mm:ss, for instance 2020-04-07 08:30:35 using Python?
2020-04-01#21120000
2020-04-06#60300000
2020-04-07#55620000
2020-04-08#56820000
2020-04-09#57240000
2020-04-10#56580000
2020-04-14#27720000
2020-04-15#20880000
2020-04-16#21300000
2020-04-17#21000000
2020-04-20#21060000
2020-04-21#20940000
Some manual splitting and parsing, adding the millisecond part to the midnight point of the date using timedelta:
from datetime import datetime, timedelta
ts = '2020-04-01#21120000'
d, ms = ts.split('#')
dt = datetime.strptime(d, '%Y-%m-%d') + timedelta(milliseconds=int(ms))
# datetime.datetime(2020, 4, 1, 5, 52)
I am parsing a 3rd party website HTML with dates and times which are always in UK time format, however they don't have any timezone info in the source. Converting the string to an object is easy enough using datetime.strptime(), but how do I add timezone info?
Ultimately, I need to convert these strings to a datetime object in UTC format. The code will always run on a PC which is timezone aware, i.e. datetime.now() will return UK time.
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
Is there a nicer way to do this?
offset = datetime.now() - datetime.utcnow()
dt -= offset
Use pytz
import datetime
import pytz
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
timezone = pytz.timezone("Etc/Greenwich")
d_aware = timezone.localize(dt)
d_aware.tzinfo
> <DstTzInfo 'Etc/Greenwich' PST-1 day, 16:00:00 STD>
d_aware
datetime.datetime(2017, 12, 7, 13, 30, tzinfo=<StaticTzInfo 'Etc/Greenwich'>)
There are some good libraries that make working with dates so much easier. I like dateparser, parsedatetime, and arrow;
import dateparser as dp
dt = dp.parse('07-12-2017 13:30 PST')
print (dt)
dt = dp.parse("Yesterday at 3:00am EST")
print(dt)
2017-07-12 13:30:00-08:00
2017-12-06 17:07:07.557109-05:00
I am getting a random datetime between two datetimes with following code
start_date = datetime.datetime(2013, 1, 1, tzinfo=pytz.UTC).toordinal()
end_date = datetime.datetime.now(tz=pytz.utc).toordinal()
return datetime.date.fromordinal(random.randint(start_date, end_date))
The problem is that it is not timezone aware.
I have already tried making it timezone aware by using tzinfo=pytz.UTC as seen in the code above but it doesn't work. I guess datetime.date.fromordinal() makes it a naive datetime format.
If you use datetime.datetime instead of datetime.date in your code then .replace(tzinfo=pytz.utc) works on the result.
To support arbitrary steps (not only 1 day):
#!/usr/bin/env python3
import random
from datetime import datetime, timedelta, timezone
step = timedelta(days=1)
start = datetime(2013, 1, 1, tzinfo=timezone.utc)
end = datetime.now(timezone.utc)
random_date = start + random.randrange((end - start) // step + 1) * step
Note: if start uses a timezone with a non-fixed utc offset then call tz.normalize() at the end.
I am dealing with dates in Python and I need to convert them to UTC timestamps to be used
inside Javascript. The following code does not work:
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(time.mktime(d.timetuple()))
datetime.datetime(2010, 12, 31, 23, 0)
Converting the date object first to datetime also does not help. I tried the example at this link from, but:
from pytz import utc, timezone
from datetime import datetime
from time import mktime
input_date = datetime(year=2011, month=1, day=15)
and now either:
mktime(utc.localize(input_date).utctimetuple())
or
mktime(timezone('US/Eastern').localize(input_date).utctimetuple())
does work.
So general question: how can I get a date converted to seconds since epoch according to UTC?
If d = date(2011, 1, 1) is in UTC:
>>> from datetime import datetime, date
>>> import calendar
>>> timestamp1 = calendar.timegm(d.timetuple())
>>> datetime.utcfromtimestamp(timestamp1)
datetime.datetime(2011, 1, 1, 0, 0)
If d is in local timezone:
>>> import time
>>> timestamp2 = time.mktime(d.timetuple()) # DO NOT USE IT WITH UTC DATE
>>> datetime.fromtimestamp(timestamp2)
datetime.datetime(2011, 1, 1, 0, 0)
timestamp1 and timestamp2 may differ if midnight in the local timezone is not the same time instance as midnight in UTC.
mktime() may return a wrong result if d corresponds to an ambiguous local time (e.g., during DST transition) or if d is a past(future) date when the utc offset might have been different and the C mktime() has no access to the tz database on the given platform. You could use pytz module (e.g., via tzlocal.get_localzone()) to get access to the tz database on all platforms. Also, utcfromtimestamp() may fail and mktime() may return non-POSIX timestamp if "right" timezone is used.
To convert datetime.date object that represents date in UTC without calendar.timegm():
DAY = 24*60*60 # POSIX day in seconds (exact value)
timestamp = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * DAY
timestamp = (utc_date - date(1970, 1, 1)).days * DAY
How can I get a date converted to seconds since epoch according to UTC?
To convert datetime.datetime (not datetime.date) object that already represents time in UTC to the corresponding POSIX timestamp (a float).
Python 3.3+
datetime.timestamp():
from datetime import timezone
timestamp = dt.replace(tzinfo=timezone.utc).timestamp()
Note: It is necessary to supply timezone.utc explicitly otherwise .timestamp() assume that your naive datetime object is in local timezone.
Python 3 (< 3.3)
From the docs for datetime.utcfromtimestamp():
There is no method to obtain the timestamp from a datetime instance,
but POSIX timestamp corresponding to a datetime instance dt can be
easily calculated as follows. For a naive dt:
timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
And for an aware dt:
timestamp = (dt - datetime(1970,1,1, tzinfo=timezone.utc)) / timedelta(seconds=1)
Interesting read: Epoch time vs. time of day on the difference between What time is it? and How many seconds have elapsed?
See also: datetime needs an "epoch" method
Python 2
To adapt the above code for Python 2:
timestamp = (dt - datetime(1970, 1, 1)).total_seconds()
where timedelta.total_seconds() is equivalent to (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6 computed with true division enabled.
Example
from __future__ import division
from datetime import datetime, timedelta
def totimestamp(dt, epoch=datetime(1970,1,1)):
td = dt - epoch
# return td.total_seconds()
return (td.microseconds + (td.seconds + td.days * 86400) * 10**6) / 10**6
now = datetime.utcnow()
print now
print totimestamp(now)
Beware of floating-point issues.
Output
2012-01-08 15:34:10.022403
1326036850.02
How to convert an aware datetime object to POSIX timestamp
assert dt.tzinfo is not None and dt.utcoffset() is not None
timestamp = dt.timestamp() # Python 3.3+
On Python 3:
from datetime import datetime, timedelta, timezone
epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
timestamp = (dt - epoch) / timedelta(seconds=1)
integer_timestamp = (dt - epoch) // timedelta(seconds=1)
On Python 2:
# utc time = local time - utc offset
utc_naive = dt.replace(tzinfo=None) - dt.utcoffset()
timestamp = (utc_naive - datetime(1970, 1, 1)).total_seconds()
For unix systems only:
>>> import datetime
>>> d = datetime.date(2011, 1, 1)
>>> d.strftime("%s") # <-- THIS IS THE CODE YOU WANT
'1293832800'
Note 1: dizzyf observed that this applies localized timezones. Don't use in production.
Note 2: Jakub Narębski noted that this ignores timezone information even for offset-aware datetime (tested for Python 2.7).
Assumption 1: You're attempting to convert a date to a timestamp, however since a date covers a 24 hour period, there isn't a single timestamp that represents that date. I'll assume that you want to represent the timestamp of that date at midnight (00:00:00.000).
Assumption 2: The date you present is not associated with a particular time zone, however you want to determine the offset from a particular time zone (UTC). Without knowing the time zone the date is in, it isn't possible to calculate a timestamp for a specific time zone. I'll assume that you want to treat the date as if it is in the local system time zone.
First, you can convert the date instance into a tuple representing the various time components using the timetuple() member:
dtt = d.timetuple() # time.struct_time(tm_year=2011, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=-1)
You can then convert that into a timestamp using time.mktime:
ts = time.mktime(dtt) # 1293868800.0
You can verify this method by testing it with the epoch time itself (1970-01-01), in which case the function should return the timezone offset for the local time zone on that date:
d = datetime.date(1970,1,1)
dtt = d.timetuple() # time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)
ts = time.mktime(dtt) # 28800.0
28800.0 is 8 hours, which would be correct for the Pacific time zone (where I'm at).
I defined my own two functions
utc_time2datetime(utc_time, tz=None)
datetime2utc_time(datetime)
here:
import time
import datetime
from pytz import timezone
import calendar
import pytz
def utc_time2datetime(utc_time, tz=None):
# convert utc time to utc datetime
utc_datetime = datetime.datetime.fromtimestamp(utc_time)
# add time zone to utc datetime
if tz is None:
tz_datetime = utc_datetime.astimezone(timezone('utc'))
else:
tz_datetime = utc_datetime.astimezone(tz)
return tz_datetime
def datetime2utc_time(datetime):
# add utc time zone if no time zone is set
if datetime.tzinfo is None:
datetime = datetime.replace(tzinfo=timezone('utc'))
# convert to utc time zone from whatever time zone the datetime is set to
utc_datetime = datetime.astimezone(timezone('utc')).replace(tzinfo=None)
# create a time tuple from datetime
utc_timetuple = utc_datetime.timetuple()
# create a time element from the tuple an add microseconds
utc_time = calendar.timegm(utc_timetuple) + datetime.microsecond / 1E6
return utc_time
follow the python2.7 document, you have to use calendar.timegm() instead of time.mktime()
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(calendar.timegm(d.timetuple()))
datetime.datetime(2011, 1, 1, 0, 0)
A complete time-string contains:
date
time
utcoffset [+HHMM or -HHMM]
For example:
1970-01-01 06:00:00 +0500 == 1970-01-01 01:00:00 +0000 == UNIX timestamp:3600
$ python3
>>> from datetime import datetime
>>> from calendar import timegm
>>> tm = '1970-01-01 06:00:00 +0500'
>>> fmt = '%Y-%m-%d %H:%M:%S %z'
>>> timegm(datetime.strptime(tm, fmt).utctimetuple())
3600
Note:
UNIX timestamp is a floating point number expressed in seconds since the epoch, in UTC.
Edit:
$ python3
>>> from datetime import datetime, timezone, timedelta
>>> from calendar import timegm
>>> dt = datetime(1970, 1, 1, 6, 0)
>>> tz = timezone(timedelta(hours=5))
>>> timegm(dt.replace(tzinfo=tz).utctimetuple())
3600
Using the arrow package:
>>> import arrow
>>> arrow.get(2010, 12, 31).timestamp
1293753600
>>> time.gmtime(1293753600)
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31,
tm_hour=0, tm_min=0, tm_sec=0,
tm_wday=4, tm_yday=365, tm_isdst=0)
the question is a little confused. timestamps are not UTC - they're a Unix thing. the date might be UTC? assuming it is, and if you're using Python 3.2+, simple-date makes this trivial:
>>> SimpleDate(date(2011,1,1), tz='utc').timestamp
1293840000.0
if you actually have the year, month and day you don't need to create the date:
>>> SimpleDate(2011,1,1, tz='utc').timestamp
1293840000.0
and if the date is in some other timezone (this matters because we're assuming midnight without an associated time):
>>> SimpleDate(date(2011,1,1), tz='America/New_York').timestamp
1293858000.0
[the idea behind simple-date is to collect all python's date and time stuff in one consistent class, so you can do any conversion. so, for example, it will also go the other way:
>>> SimpleDate(1293858000, tz='utc').date
datetime.date(2011, 1, 1)
]
Considering you have a datetime object called d,
use the following to get the timestamp in UTC:
d.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
And for the opposite direction, use following :
d = datetime.strptime("2008-09-03T20:56:35.450686Z", "%Y-%m-%dT%H:%M:%S.%fZ")
This works for me, pass through a function.
from datetime import timezone, datetime, timedelta
import datetime
def utc_converter(dt):
dt = datetime.datetime.now(timezone.utc)
utc_time = dt.replace(tzinfo=timezone.utc)
utc_timestamp = utc_time.timestamp()
return utc_timestamp
# create start and end timestamps
_now = datetime.datetime.now()
str_start = str(utc_converter(_now))
_end = _now + timedelta(seconds=10)
str_end = str(utc_converter(_end))
i'm impressed of the deep discussion.
my 2 cents:
from datetime import datetime
import time
the timestamp in utc is:
timestamp = \
(datetime.utcnow() - datetime(1970,1,1)).total_seconds()
or,
timestamp = time.time()
if now results from datetime.now(), in the same DST
utcoffset = (datetime.now() - datetime.utcnow()).total_seconds()
timestamp = \
(now - datetime(1970,1,1)).total_seconds() - utcoffset
I am dealing with dates in Python and I need to convert them to UTC timestamps to be used
inside Javascript. The following code does not work:
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(time.mktime(d.timetuple()))
datetime.datetime(2010, 12, 31, 23, 0)
Converting the date object first to datetime also does not help. I tried the example at this link from, but:
from pytz import utc, timezone
from datetime import datetime
from time import mktime
input_date = datetime(year=2011, month=1, day=15)
and now either:
mktime(utc.localize(input_date).utctimetuple())
or
mktime(timezone('US/Eastern').localize(input_date).utctimetuple())
does work.
So general question: how can I get a date converted to seconds since epoch according to UTC?
If d = date(2011, 1, 1) is in UTC:
>>> from datetime import datetime, date
>>> import calendar
>>> timestamp1 = calendar.timegm(d.timetuple())
>>> datetime.utcfromtimestamp(timestamp1)
datetime.datetime(2011, 1, 1, 0, 0)
If d is in local timezone:
>>> import time
>>> timestamp2 = time.mktime(d.timetuple()) # DO NOT USE IT WITH UTC DATE
>>> datetime.fromtimestamp(timestamp2)
datetime.datetime(2011, 1, 1, 0, 0)
timestamp1 and timestamp2 may differ if midnight in the local timezone is not the same time instance as midnight in UTC.
mktime() may return a wrong result if d corresponds to an ambiguous local time (e.g., during DST transition) or if d is a past(future) date when the utc offset might have been different and the C mktime() has no access to the tz database on the given platform. You could use pytz module (e.g., via tzlocal.get_localzone()) to get access to the tz database on all platforms. Also, utcfromtimestamp() may fail and mktime() may return non-POSIX timestamp if "right" timezone is used.
To convert datetime.date object that represents date in UTC without calendar.timegm():
DAY = 24*60*60 # POSIX day in seconds (exact value)
timestamp = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * DAY
timestamp = (utc_date - date(1970, 1, 1)).days * DAY
How can I get a date converted to seconds since epoch according to UTC?
To convert datetime.datetime (not datetime.date) object that already represents time in UTC to the corresponding POSIX timestamp (a float).
Python 3.3+
datetime.timestamp():
from datetime import timezone
timestamp = dt.replace(tzinfo=timezone.utc).timestamp()
Note: It is necessary to supply timezone.utc explicitly otherwise .timestamp() assume that your naive datetime object is in local timezone.
Python 3 (< 3.3)
From the docs for datetime.utcfromtimestamp():
There is no method to obtain the timestamp from a datetime instance,
but POSIX timestamp corresponding to a datetime instance dt can be
easily calculated as follows. For a naive dt:
timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
And for an aware dt:
timestamp = (dt - datetime(1970,1,1, tzinfo=timezone.utc)) / timedelta(seconds=1)
Interesting read: Epoch time vs. time of day on the difference between What time is it? and How many seconds have elapsed?
See also: datetime needs an "epoch" method
Python 2
To adapt the above code for Python 2:
timestamp = (dt - datetime(1970, 1, 1)).total_seconds()
where timedelta.total_seconds() is equivalent to (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6 computed with true division enabled.
Example
from __future__ import division
from datetime import datetime, timedelta
def totimestamp(dt, epoch=datetime(1970,1,1)):
td = dt - epoch
# return td.total_seconds()
return (td.microseconds + (td.seconds + td.days * 86400) * 10**6) / 10**6
now = datetime.utcnow()
print now
print totimestamp(now)
Beware of floating-point issues.
Output
2012-01-08 15:34:10.022403
1326036850.02
How to convert an aware datetime object to POSIX timestamp
assert dt.tzinfo is not None and dt.utcoffset() is not None
timestamp = dt.timestamp() # Python 3.3+
On Python 3:
from datetime import datetime, timedelta, timezone
epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
timestamp = (dt - epoch) / timedelta(seconds=1)
integer_timestamp = (dt - epoch) // timedelta(seconds=1)
On Python 2:
# utc time = local time - utc offset
utc_naive = dt.replace(tzinfo=None) - dt.utcoffset()
timestamp = (utc_naive - datetime(1970, 1, 1)).total_seconds()
For unix systems only:
>>> import datetime
>>> d = datetime.date(2011, 1, 1)
>>> d.strftime("%s") # <-- THIS IS THE CODE YOU WANT
'1293832800'
Note 1: dizzyf observed that this applies localized timezones. Don't use in production.
Note 2: Jakub Narębski noted that this ignores timezone information even for offset-aware datetime (tested for Python 2.7).
Assumption 1: You're attempting to convert a date to a timestamp, however since a date covers a 24 hour period, there isn't a single timestamp that represents that date. I'll assume that you want to represent the timestamp of that date at midnight (00:00:00.000).
Assumption 2: The date you present is not associated with a particular time zone, however you want to determine the offset from a particular time zone (UTC). Without knowing the time zone the date is in, it isn't possible to calculate a timestamp for a specific time zone. I'll assume that you want to treat the date as if it is in the local system time zone.
First, you can convert the date instance into a tuple representing the various time components using the timetuple() member:
dtt = d.timetuple() # time.struct_time(tm_year=2011, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=-1)
You can then convert that into a timestamp using time.mktime:
ts = time.mktime(dtt) # 1293868800.0
You can verify this method by testing it with the epoch time itself (1970-01-01), in which case the function should return the timezone offset for the local time zone on that date:
d = datetime.date(1970,1,1)
dtt = d.timetuple() # time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)
ts = time.mktime(dtt) # 28800.0
28800.0 is 8 hours, which would be correct for the Pacific time zone (where I'm at).
I defined my own two functions
utc_time2datetime(utc_time, tz=None)
datetime2utc_time(datetime)
here:
import time
import datetime
from pytz import timezone
import calendar
import pytz
def utc_time2datetime(utc_time, tz=None):
# convert utc time to utc datetime
utc_datetime = datetime.datetime.fromtimestamp(utc_time)
# add time zone to utc datetime
if tz is None:
tz_datetime = utc_datetime.astimezone(timezone('utc'))
else:
tz_datetime = utc_datetime.astimezone(tz)
return tz_datetime
def datetime2utc_time(datetime):
# add utc time zone if no time zone is set
if datetime.tzinfo is None:
datetime = datetime.replace(tzinfo=timezone('utc'))
# convert to utc time zone from whatever time zone the datetime is set to
utc_datetime = datetime.astimezone(timezone('utc')).replace(tzinfo=None)
# create a time tuple from datetime
utc_timetuple = utc_datetime.timetuple()
# create a time element from the tuple an add microseconds
utc_time = calendar.timegm(utc_timetuple) + datetime.microsecond / 1E6
return utc_time
follow the python2.7 document, you have to use calendar.timegm() instead of time.mktime()
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(calendar.timegm(d.timetuple()))
datetime.datetime(2011, 1, 1, 0, 0)
A complete time-string contains:
date
time
utcoffset [+HHMM or -HHMM]
For example:
1970-01-01 06:00:00 +0500 == 1970-01-01 01:00:00 +0000 == UNIX timestamp:3600
$ python3
>>> from datetime import datetime
>>> from calendar import timegm
>>> tm = '1970-01-01 06:00:00 +0500'
>>> fmt = '%Y-%m-%d %H:%M:%S %z'
>>> timegm(datetime.strptime(tm, fmt).utctimetuple())
3600
Note:
UNIX timestamp is a floating point number expressed in seconds since the epoch, in UTC.
Edit:
$ python3
>>> from datetime import datetime, timezone, timedelta
>>> from calendar import timegm
>>> dt = datetime(1970, 1, 1, 6, 0)
>>> tz = timezone(timedelta(hours=5))
>>> timegm(dt.replace(tzinfo=tz).utctimetuple())
3600
Using the arrow package:
>>> import arrow
>>> arrow.get(2010, 12, 31).timestamp
1293753600
>>> time.gmtime(1293753600)
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31,
tm_hour=0, tm_min=0, tm_sec=0,
tm_wday=4, tm_yday=365, tm_isdst=0)
the question is a little confused. timestamps are not UTC - they're a Unix thing. the date might be UTC? assuming it is, and if you're using Python 3.2+, simple-date makes this trivial:
>>> SimpleDate(date(2011,1,1), tz='utc').timestamp
1293840000.0
if you actually have the year, month and day you don't need to create the date:
>>> SimpleDate(2011,1,1, tz='utc').timestamp
1293840000.0
and if the date is in some other timezone (this matters because we're assuming midnight without an associated time):
>>> SimpleDate(date(2011,1,1), tz='America/New_York').timestamp
1293858000.0
[the idea behind simple-date is to collect all python's date and time stuff in one consistent class, so you can do any conversion. so, for example, it will also go the other way:
>>> SimpleDate(1293858000, tz='utc').date
datetime.date(2011, 1, 1)
]
Considering you have a datetime object called d,
use the following to get the timestamp in UTC:
d.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
And for the opposite direction, use following :
d = datetime.strptime("2008-09-03T20:56:35.450686Z", "%Y-%m-%dT%H:%M:%S.%fZ")
This works for me, pass through a function.
from datetime import timezone, datetime, timedelta
import datetime
def utc_converter(dt):
dt = datetime.datetime.now(timezone.utc)
utc_time = dt.replace(tzinfo=timezone.utc)
utc_timestamp = utc_time.timestamp()
return utc_timestamp
# create start and end timestamps
_now = datetime.datetime.now()
str_start = str(utc_converter(_now))
_end = _now + timedelta(seconds=10)
str_end = str(utc_converter(_end))
i'm impressed of the deep discussion.
my 2 cents:
from datetime import datetime
import time
the timestamp in utc is:
timestamp = \
(datetime.utcnow() - datetime(1970,1,1)).total_seconds()
or,
timestamp = time.time()
if now results from datetime.now(), in the same DST
utcoffset = (datetime.now() - datetime.utcnow()).total_seconds()
timestamp = \
(now - datetime(1970,1,1)).total_seconds() - utcoffset