Develop Reference

how to covert date to milisecond in python

I have a date in this format 04-07-2018
the following Javascript produces
const date1 = new Date("04-07-2018").getTime();
date1; // 1523084400000
how do I parse the date and convert them to milliseconds in python

We can perform this in three steps:
first we convert the string to a datetime object;
next we calculate the difference between that datetime object and the first of January, 1970;
we calculate this difference in seconds, and then multiply it with 1000.
For example:
from datetime import datetime
def total_millis(text):
dt = datetime.strptime(text, '%m-%d-%Y')
return 1000 * (dt - datetime(1970, 1, 1)).total_seconds()
This implementation however is timezone independent, whereas the JavaScript version depends on the timezone.

The JavaScript getTime() method returns the number of milliseconds since January 1, 1970 00:00:00.
You can use datetime but you should compute it with a deltatime object since according to this Python doesn't guarantee any particular epoch time
import datetime
delta = datetime.datetime(2018, 4, 7) - datetime.datetime(1970, 1, 1)
milliseconds = delta.total_seconds()*1000 # 1523059200000.0

Why is converting this datetime to Epoch off by 30 minutes?

I have the following string representing a UTC timestamp: 2017-12-03T20:38:00.971261Z
I would like to convert it into Posix timestamp (IE: seconds since the epoch)
Using this online converter (https://www.epochconverter.com/) I know the answer is 1512333480
But when I do the following code, the result is off by 1800 seconds -- 30 minutes:
>>> temp_time1 = datetime.datetime.strptime('2017-12-03T20:38:00.971261Z', '%Y-%m-%dT%H:%M:%S.%fZ')
>>> ctime = int(datetime.datetime(temp_time1.year,
temp_time1.month,
temp_time1.day,
temp_time1.hour,
temp_time1.minute,
temp_time1.second,
temp_time1.microsecond,
pytz.timezone('Europe/London')).strftime('%s'))
>>> print ctime
1512351480
Anyone know what I'm missing here??

You created a new timestamp and put it in the Europe/London timezone. That is not the same thing as UTC. The Europe/London timezone from the PyTZ database includes historical offsets, and those affect how datetime.datetime() interprets the timezone.
Just use the datetime.timestamp() method on the datetime object you already parsed from the string:
>>> import datetime
>>> temp_time1 = datetime.datetime.strptime('2017-12-03T20:38:00.971261Z', '%Y-%m-%dT%H:%M:%S.%fZ')
>>> temp_time1.timestamp()
1512333480.971261
Your original temp_time1 datetime object is timezone agnostic, so the timestamp() object already assumes no timezone conversion has to take place.
If you must apply the Europe/London timezone first for whatever reason, then at least use the timezone.localize() method to get the right offset applied:
>>> import pytz
>>> pytz.timezone('Europe/London').localize(temp_time1)
datetime.datetime(2017, 12, 3, 20, 38, 0, 971261, tzinfo=<DstTzInfo 'Europe/London' GMT0:00:00 STD>)
>>> pytz.timezone('Europe/London').localize(temp_time1).timestamp()
1512333480.971261
See How to make an unaware datetime timezone aware in python
For Python 2 and Python 3.0, 3.1 or 3.2, where datetime.timestamp() is not available, subtract the epoch date:
>>> (temp_time1 - datetime.datetime(1970, 1, 1)).total_seconds()
1512333480.971261
Add in the UTC timezone when dealing with timezone-aware datetime instances:
>>> (pytz.timezone('Europe/London').localize(temp_time1) - datetime.datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()
1512333480.971261
Combined into a function:
def datetime_to_timestamp(dt, epoch=datetime.datetime(1970, 1, 1)):
if dt.tzinfo is not None:
epoch = pytz.utc.localize(epoch)
return (dt - epoch).total_seconds()

Convert datetime format to unix timestamp in Python [duplicate]

I have dt = datetime(2013,9,1,11), and I would like to get a Unix timestamp of this datetime object.
When I do (dt - datetime(1970,1,1)).total_seconds() I got the timestamp 1378033200.
When converting it back using datetime.fromtimestamp I got datetime.datetime(2013, 9, 1, 6, 0).
The hour doesn't match. What did I miss here?

solution is
import time
import datetime
d = datetime.date(2015,1,5)
unixtime = time.mktime(d.timetuple())

If you want to convert a python datetime to seconds since epoch you should do it explicitly:
>>> import datetime
>>> datetime.datetime(2012, 04, 01, 0, 0).strftime('%s')
'1333234800'
>>> (datetime.datetime(2012, 04, 01, 0, 0) - datetime.datetime(1970, 1, 1)).total_seconds()
1333238400.0
In Python 3.3+ you can use timestamp() instead:
>>> import datetime
>>> datetime.datetime(2012, 4, 1, 0, 0).timestamp()
1333234800.0

What you missed here is timezones.
Presumably you've five hours off UTC, so 2013-09-01T11:00:00 local and 2013-09-01T06:00:00Z are the same time.
You need to read the top of the datetime docs, which explain about timezones and "naive" and "aware" objects.
If your original naive datetime was UTC, the way to recover it is to use utcfromtimestamp instead of fromtimestamp.
On the other hand, if your original naive datetime was local, you shouldn't have subtracted a UTC timestamp from it in the first place; use datetime.fromtimestamp(0) instead.
Or, if you had an aware datetime object, you need to either use a local (aware) epoch on both sides, or explicitly convert to and from UTC.
If you have, or can upgrade to, Python 3.3 or later, you can avoid all of these problems by just using the timestamp method instead of trying to figure out how to do it yourself. And even if you don't, you may want to consider borrowing its source code.
(And if you can wait for Python 3.4, it looks like PEP 341 is likely to make it into the final release, which means all of the stuff J.F. Sebastian and I were talking about in the comments should be doable with just the stdlib, and working the same way on both Unix and Windows.)

Rather than this expression to create a POSIX timestamp from dt,
(dt - datetime(1970,1,1)).total_seconds()
Use this:
int(dt.strftime("%s"))
I get the right answer in your example using the second method.
EDIT: Some followup... After some comments (see below), I was curious about the lack of support or documentation for %s in strftime. Here's what I found:
In the Python source for datetime and time, the string STRFTIME_FORMAT_CODES tells us:
"Other codes may be available on your platform.
See documentation for the C library strftime function."
So now if we man strftime (on BSD systems such as Mac OS X), you'll find support for %s:
"%s is replaced by the number of seconds since the Epoch, UTC (see mktime(3))."
Anyways, that's why %s works on the systems it does. But there are better solutions to OP's problem (that take timezones into account). See #abarnert's accepted answer here.

For working with UTC timezones:
time_stamp = calendar.timegm(dt.timetuple())
datetime.utcfromtimestamp(time_stamp)

You've missed the time zone info (already answered, agreed)
arrow package allows to avoid this torture with datetimes; It is already written, tested, pypi-published, cross-python (2.6 — 3.xx).
All you need: pip install arrow (or add to dependencies)
Solution for your case
dt = datetime(2013,9,1,11)
arrow.get(dt).timestamp
# >>> 1378033200
bc = arrow.get(1378033200).datetime
print(bc)
# >>> datetime.datetime(2013, 9, 1, 11, 0, tzinfo=tzutc())
print(bc.isoformat())
# >>> '2013-09-01T11:00:00+00:00'

If your datetime object represents UTC time, don't use time.mktime, as it assumes the tuple is in your local timezone. Instead, use calendar.timegm:
>>> import datetime, calendar
>>> d = datetime.datetime(1970, 1, 1, 0, 1, 0)
>>> calendar.timegm(d.timetuple())
60

def dt2ts(dt, utc=False):
if utc:
return calendar.timegm(dt.timetuple())
if dt.tzinfo is None:
return int(time.mktime(dt.timetuple()))
utc_dt = dt.astimezone(tz.tzutc()).timetuple()
return calendar.timegm(utc_dt)
If you want UTC timestamp :time.mktime just for local dt .Use calendar.timegm is safe but dt must the utc zone so change the zone to utc. If dt in UTC just use calendar.timegm.

def datetime_to_epoch(d1):
"""
January 1st, 1970 at 00:00:00 UTC is referred to as the Unix epoch
:param d1: input date
:return: seconds since unix epoch
"""
if not d1.tzinfo:
raise ValueError("date is missing timezone information")
d2 = datetime(1970, 1, 1, tzinfo=timezone.utc)
time_delta = d1 - d2
ts = int(time_delta.total_seconds())
return ts
def epoch_to_datetime_string(timestamp, tz_name="UTC", **kwargs):
"""
method to convert unix timestamp to date time string
:param ts: 10 digit unix timestamp in seconds
:param tz_name: timezone name
:param kwargs: formatter=<formatter-string>
:return: date time string in timezone
"""
naive_date = datetime.fromtimestamp(timestamp)
aware_date = naive_date.astimezone(pytz.timezone(tz_name))
formatter = kwargs.pop("formatter", "%d %b %Y %H:%M:%S")
return aware_date.strftime(formatter)

Well, when converting TO unix timestamp, python is basically assuming UTC, but while converting back it will give you a date converted to your local timezone.
See this question/answer;
Get timezone used by datetime.datetime.fromtimestamp()

This class will cover your needs, you can pass the variable into ConvertUnixToDatetime & call which function you want it to operate based off.
from datetime import datetime
import time
class ConvertUnixToDatetime:
def __init__(self, date):
self.date = date
# Convert unix to date object
def convert_unix(self):
unix = self.date
# Check if unix is a string or int & proceeds with correct conversion
if type(unix).__name__ == 'str':
unix = int(unix[0:10])
else:
unix = int(str(unix)[0:10])
date = datetime.utcfromtimestamp(unix).strftime('%Y-%m-%d %H:%M:%S')
return date
# Convert date to unix object
def convert_date(self):
date = self.date
# Check if datetime object or raise ValueError
if type(date).__name__ == 'datetime':
unixtime = int(time.mktime(date.timetuple()))
else:
raise ValueError('You are trying to pass a None Datetime object')
return type(unixtime).__name__, unixtime
if __name__ == '__main__':
# Test Date
date_test = ConvertUnixToDatetime(datetime.today())
date_test = date_test.convert_date()
print(date_test)
# Test Unix
unix_test = ConvertUnixToDatetime(date_test[1])
print(unix_test.convert_unix())

import time
from datetime import datetime
time.mktime(datetime.now().timetuple())

bson.json_util datetime encode and decode best practice

I'm trying to encode & decode python datetime object using pymongo's bson utils.
What's best practice here?
>>> from bson import json_util
>>> import datetime
>>> utcnow = datetime.datetime.utcnow()
>>> x = json_util.dumps({'now': utcnow})
>>> json_util.loads(x)['now'] == utcnow
False
>>> json_util.loads(x)['now'] - utcnow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't subtract offset-naive and offset-aware datetimes
>>> json_util.loads(x)['now'].replace(tzinfo=None) - utcnow
datetime.timedelta(-1, 86399, 999088)
>>> datetime.datetime.utcfromtimestamp(1424297808578 / 1000) == json_util.loads(x)['now'].replace(tzinfo=None)
True
^ Is this really the best way?
Or write your own encode/decode and use the json lib?

It seems bson rounds to milliseconds:
>>> from datetime import datetime
>>> import bson # $ pip install bson
>>> d = datetime.utcnow(); d, abs(d - bson.loads(bson.dumps({'utcnow': d}))['utcnow'].replace(tzinfo=None))
(datetime.datetime(2015, 2, 18, 23, 54, 47, 733092), datetime.timedelta(0, 0, 92))
It is a documented behavior:
UTC datetime - The int64 is UTC milliseconds since the Unix epoch.
If you need microseconds: you could store an integer number of microseconds since the Unix epoch instead:
from datetime import datetime
td = utc_dt - datetime(1970, 1, 1)
micros = td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6
To convert microseconds since the Unix epoch back into a naive datetime object that represents UTC time:
from datetime import datetime, timedelta
utc_dt = datetime(1970, 1, 1) + timedelta(microseconds=micros)
int64 ("\x12") is more than enough to represent the Unix time with the microsecond resolution (it exceeds datetime range anyway).
Note: POSIX timestamp "forgets" leap seconds e.g.:
import time
tt = time.strptime("2015-07-01 01:59:60", "%Y-%m-%d %H:%M:%S")
ts_leap = time.mktime(tt)
tt = time.strptime("2015-07-01 02:00:00", "%Y-%m-%d %H:%M:%S")
ts_after = time.mktime(tt)
assert ts_leap == ts_after # assuming "right" timezone is not used
If you care about microseconds; you should find out what your system does around leap seconds.
Time (hardware clocks, software timers) on an ordinary computer are not very accurate therefore the millisecond resolution should be enough in many cases e.g., if you use ntp to synchronize time between machine then NTP v3 is accurate to 1-2ms in a LAN and 10s of ms in WAN nets.
Though, sometimes, you want to preserve digits in the input even if they are not accurate.

In Python, how do you convert seconds since epoch to a `datetime` object?

The time module can be initialized using seconds since epoch:
>>> import time
>>> t1=time.gmtime(1284286794)
>>> t1
time.struct_time(tm_year=2010, tm_mon=9, tm_mday=12, tm_hour=10, tm_min=19,
tm_sec=54, tm_wday=6, tm_yday=255, tm_isdst=0)
Is there an elegant way to initialize a datetime.datetime object in the same way?

datetime.datetime.fromtimestamp will do, if you know the time zone, you could produce the same output as with time.gmtime
>>> datetime.datetime.fromtimestamp(1284286794)
datetime.datetime(2010, 9, 12, 11, 19, 54)
or
>>> datetime.datetime.utcfromtimestamp(1284286794)
datetime.datetime(2010, 9, 12, 10, 19, 54)

Seconds since epoch to datetime to strftime:
>>> ts_epoch = 1362301382
>>> ts = datetime.datetime.fromtimestamp(ts_epoch).strftime('%Y-%m-%d %H:%M:%S')
>>> ts
'2013-03-03 01:03:02'

From the docs, the recommended way of getting a timezone aware datetime object from seconds since epoch is:
Python 3:
from datetime import datetime, timezone
datetime.fromtimestamp(timestamp, timezone.utc)
Python 2, using pytz:
from datetime import datetime
import pytz
datetime.fromtimestamp(timestamp, pytz.utc)

Note that datetime.datetime.fromtimestamp(timestamp) and .utcfromtimestamp(timestamp) fail on windows for dates before Jan. 1, 1970 while negative unix timestamps seem to work on unix-based platforms. The docs say this:
"This may raise ValueError, if the timestamp is out of the range of
values supported by the platform C gmtime() function. It’s common for
this to be restricted to years in 1970 through 2038"
See also Issue1646728

For those that want it ISO 8601 compliant, since the other solutions do not have the T separator nor the time offset (except Meistro's answer):
from datetime import datetime, timezone
result = datetime.fromtimestamp(1463288494, timezone.utc).isoformat('T', 'microseconds')
print(result) # 2016-05-15T05:01:34.000000+00:00
Note, I use fromtimestamp because if I used utcfromtimestamp I would need to chain on .astimezone(...) anyway to get the offset.
If you don't want to go all the way to microseconds you can choose a different unit with the
isoformat() method.