Parsing datetime string containing milliseconds to datetime - python

I have some datetime strings containing miliseconds from midnight.
How do I parse these to yyyy-MM-dd HH:mm:ss, for instance 2020-04-07 08:30:35 using Python?
2020-04-01#21120000
2020-04-06#60300000
2020-04-07#55620000
2020-04-08#56820000
2020-04-09#57240000
2020-04-10#56580000
2020-04-14#27720000
2020-04-15#20880000
2020-04-16#21300000
2020-04-17#21000000
2020-04-20#21060000
2020-04-21#20940000

Some manual splitting and parsing, adding the millisecond part to the midnight point of the date using timedelta:
from datetime import datetime, timedelta
ts = '2020-04-01#21120000'
d, ms = ts.split('#')
dt = datetime.strptime(d, '%Y-%m-%d') + timedelta(milliseconds=int(ms))
# datetime.datetime(2020, 4, 1, 5, 52)

Related

Python - Add time zone to timestamp

I use chrome timestamp and convert it do readable date but time isn't right
timestamp_formated = str(datetime.datetime(1601, 1, 1) + datetime.timedelta(microseconds=last_visit_time))
Seems to be timezone need to be added
example for last_visit_time : 13292010189305268
Assuming that Chrome timestamps denote microseconds since 1601 UTC, you'll want to make your datetime aware:
from datetime import datetime, timezone, timedelta
epoch = datetime(1601, 1, 1, tzinfo=timezone.utc)
timestamp = epoch + timedelta(microseconds=last_visit_time)
print(timestamp)
If you want to format it for a non-UTC timezone, add a conversion step:
local_timestamp = timestamp.astimezone(the_timezone)
if you'd like to localize(use timezone specific dates and time) you can use pytz for that
t = datetime(
2013, 5, 11, hour=11, minute=0,
tzinfo=pytz.timezone('Europe/Warsaw')
)

How to convert a date string zone oriented and hour string into datetime

I have two strings one: date='2021-12-30T23:00Z' where Z means UTC timezone and 23:00 means hour. I also have an hour string hour='3'. What I want is to convert date to datetime object and add this hour string to date as a delta. In result I would get a datetime object with hour: '2021-12-31T02:00Z' I tried function datetime.datetime.fromisoformat() with no luck.
Use strftime with their format.
from datetime import datetime, timedelta
date='2021-12-30T23:00Z'
date = datetime.strptime(date, '%Y-%m-%dT%H:%MZ')
new_date = date + timedelta(hours=3)
new_date = new_date.strftime('%Y-%m-%dT%H:%MZ')
print(new_date)
Output:
2021-12-31T02:00Z
You could do something like this:
from datetime import datetime
from datetime import timedelta
date = "2021-12-30T23:00Z"
hour = "3"
d = datetime.strptime(date, "%Y-%m-%dT%H:%M%z") + timedelta(hours=int(hour))
print(d)
output:
2021-12-31 02:00:00+00:00

Python: converting date string to UTC

I am parsing a 3rd party website HTML with dates and times which are always in UK time format, however they don't have any timezone info in the source. Converting the string to an object is easy enough using datetime.strptime(), but how do I add timezone info?
Ultimately, I need to convert these strings to a datetime object in UTC format. The code will always run on a PC which is timezone aware, i.e. datetime.now() will return UK time.
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
Is there a nicer way to do this?
offset = datetime.now() - datetime.utcnow()
dt -= offset
Use pytz
import datetime
import pytz
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
timezone = pytz.timezone("Etc/Greenwich")
d_aware = timezone.localize(dt)
d_aware.tzinfo
> <DstTzInfo 'Etc/Greenwich' PST-1 day, 16:00:00 STD>
d_aware
datetime.datetime(2017, 12, 7, 13, 30, tzinfo=<StaticTzInfo 'Etc/Greenwich'>)
There are some good libraries that make working with dates so much easier. I like dateparser, parsedatetime, and arrow;
import dateparser as dp
dt = dp.parse('07-12-2017 13:30 PST')
print (dt)
dt = dp.parse("Yesterday at 3:00am EST")
print(dt)
2017-07-12 13:30:00-08:00
2017-12-06 17:07:07.557109-05:00

Converting datetime.date to UTC timestamp in Python

I am dealing with dates in Python and I need to convert them to UTC timestamps to be used
inside Javascript. The following code does not work:
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(time.mktime(d.timetuple()))
datetime.datetime(2010, 12, 31, 23, 0)
Converting the date object first to datetime also does not help. I tried the example at this link from, but:
from pytz import utc, timezone
from datetime import datetime
from time import mktime
input_date = datetime(year=2011, month=1, day=15)
and now either:
mktime(utc.localize(input_date).utctimetuple())
or
mktime(timezone('US/Eastern').localize(input_date).utctimetuple())
does work.
So general question: how can I get a date converted to seconds since epoch according to UTC?
If d = date(2011, 1, 1) is in UTC:
>>> from datetime import datetime, date
>>> import calendar
>>> timestamp1 = calendar.timegm(d.timetuple())
>>> datetime.utcfromtimestamp(timestamp1)
datetime.datetime(2011, 1, 1, 0, 0)
If d is in local timezone:
>>> import time
>>> timestamp2 = time.mktime(d.timetuple()) # DO NOT USE IT WITH UTC DATE
>>> datetime.fromtimestamp(timestamp2)
datetime.datetime(2011, 1, 1, 0, 0)
timestamp1 and timestamp2 may differ if midnight in the local timezone is not the same time instance as midnight in UTC.
mktime() may return a wrong result if d corresponds to an ambiguous local time (e.g., during DST transition) or if d is a past(future) date when the utc offset might have been different and the C mktime() has no access to the tz database on the given platform. You could use pytz module (e.g., via tzlocal.get_localzone()) to get access to the tz database on all platforms. Also, utcfromtimestamp() may fail and mktime() may return non-POSIX timestamp if "right" timezone is used.
To convert datetime.date object that represents date in UTC without calendar.timegm():
DAY = 24*60*60 # POSIX day in seconds (exact value)
timestamp = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * DAY
timestamp = (utc_date - date(1970, 1, 1)).days * DAY
How can I get a date converted to seconds since epoch according to UTC?
To convert datetime.datetime (not datetime.date) object that already represents time in UTC to the corresponding POSIX timestamp (a float).
Python 3.3+
datetime.timestamp():
from datetime import timezone
timestamp = dt.replace(tzinfo=timezone.utc).timestamp()
Note: It is necessary to supply timezone.utc explicitly otherwise .timestamp() assume that your naive datetime object is in local timezone.
Python 3 (< 3.3)
From the docs for datetime.utcfromtimestamp():
There is no method to obtain the timestamp from a datetime instance,
but POSIX timestamp corresponding to a datetime instance dt can be
easily calculated as follows. For a naive dt:
timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
And for an aware dt:
timestamp = (dt - datetime(1970,1,1, tzinfo=timezone.utc)) / timedelta(seconds=1)
Interesting read: Epoch time vs. time of day on the difference between What time is it? and How many seconds have elapsed?
See also: datetime needs an "epoch" method
Python 2
To adapt the above code for Python 2:
timestamp = (dt - datetime(1970, 1, 1)).total_seconds()
where timedelta.total_seconds() is equivalent to (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6 computed with true division enabled.
Example
from __future__ import division
from datetime import datetime, timedelta
def totimestamp(dt, epoch=datetime(1970,1,1)):
td = dt - epoch
# return td.total_seconds()
return (td.microseconds + (td.seconds + td.days * 86400) * 10**6) / 10**6
now = datetime.utcnow()
print now
print totimestamp(now)
Beware of floating-point issues.
Output
2012-01-08 15:34:10.022403
1326036850.02
How to convert an aware datetime object to POSIX timestamp
assert dt.tzinfo is not None and dt.utcoffset() is not None
timestamp = dt.timestamp() # Python 3.3+
On Python 3:
from datetime import datetime, timedelta, timezone
epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
timestamp = (dt - epoch) / timedelta(seconds=1)
integer_timestamp = (dt - epoch) // timedelta(seconds=1)
On Python 2:
# utc time = local time - utc offset
utc_naive = dt.replace(tzinfo=None) - dt.utcoffset()
timestamp = (utc_naive - datetime(1970, 1, 1)).total_seconds()
For unix systems only:
>>> import datetime
>>> d = datetime.date(2011, 1, 1)
>>> d.strftime("%s") # <-- THIS IS THE CODE YOU WANT
'1293832800'
Note 1: dizzyf observed that this applies localized timezones. Don't use in production.
Note 2: Jakub Narębski noted that this ignores timezone information even for offset-aware datetime (tested for Python 2.7).
Assumption 1: You're attempting to convert a date to a timestamp, however since a date covers a 24 hour period, there isn't a single timestamp that represents that date. I'll assume that you want to represent the timestamp of that date at midnight (00:00:00.000).
Assumption 2: The date you present is not associated with a particular time zone, however you want to determine the offset from a particular time zone (UTC). Without knowing the time zone the date is in, it isn't possible to calculate a timestamp for a specific time zone. I'll assume that you want to treat the date as if it is in the local system time zone.
First, you can convert the date instance into a tuple representing the various time components using the timetuple() member:
dtt = d.timetuple() # time.struct_time(tm_year=2011, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=-1)
You can then convert that into a timestamp using time.mktime:
ts = time.mktime(dtt) # 1293868800.0
You can verify this method by testing it with the epoch time itself (1970-01-01), in which case the function should return the timezone offset for the local time zone on that date:
d = datetime.date(1970,1,1)
dtt = d.timetuple() # time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)
ts = time.mktime(dtt) # 28800.0
28800.0 is 8 hours, which would be correct for the Pacific time zone (where I'm at).
I defined my own two functions
utc_time2datetime(utc_time, tz=None)
datetime2utc_time(datetime)
here:
import time
import datetime
from pytz import timezone
import calendar
import pytz
def utc_time2datetime(utc_time, tz=None):
# convert utc time to utc datetime
utc_datetime = datetime.datetime.fromtimestamp(utc_time)
# add time zone to utc datetime
if tz is None:
tz_datetime = utc_datetime.astimezone(timezone('utc'))
else:
tz_datetime = utc_datetime.astimezone(tz)
return tz_datetime
def datetime2utc_time(datetime):
# add utc time zone if no time zone is set
if datetime.tzinfo is None:
datetime = datetime.replace(tzinfo=timezone('utc'))
# convert to utc time zone from whatever time zone the datetime is set to
utc_datetime = datetime.astimezone(timezone('utc')).replace(tzinfo=None)
# create a time tuple from datetime
utc_timetuple = utc_datetime.timetuple()
# create a time element from the tuple an add microseconds
utc_time = calendar.timegm(utc_timetuple) + datetime.microsecond / 1E6
return utc_time
follow the python2.7 document, you have to use calendar.timegm() instead of time.mktime()
>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(calendar.timegm(d.timetuple()))
datetime.datetime(2011, 1, 1, 0, 0)
A complete time-string contains:
date
time
utcoffset [+HHMM or -HHMM]
For example:
1970-01-01 06:00:00 +0500 == 1970-01-01 01:00:00 +0000 == UNIX timestamp:3600
$ python3
>>> from datetime import datetime
>>> from calendar import timegm
>>> tm = '1970-01-01 06:00:00 +0500'
>>> fmt = '%Y-%m-%d %H:%M:%S %z'
>>> timegm(datetime.strptime(tm, fmt).utctimetuple())
3600
Note:
UNIX timestamp is a floating point number expressed in seconds since the epoch, in UTC.
Edit:
$ python3
>>> from datetime import datetime, timezone, timedelta
>>> from calendar import timegm
>>> dt = datetime(1970, 1, 1, 6, 0)
>>> tz = timezone(timedelta(hours=5))
>>> timegm(dt.replace(tzinfo=tz).utctimetuple())
3600
Using the arrow package:
>>> import arrow
>>> arrow.get(2010, 12, 31).timestamp
1293753600
>>> time.gmtime(1293753600)
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31,
tm_hour=0, tm_min=0, tm_sec=0,
tm_wday=4, tm_yday=365, tm_isdst=0)
the question is a little confused. timestamps are not UTC - they're a Unix thing. the date might be UTC? assuming it is, and if you're using Python 3.2+, simple-date makes this trivial:
>>> SimpleDate(date(2011,1,1), tz='utc').timestamp
1293840000.0
if you actually have the year, month and day you don't need to create the date:
>>> SimpleDate(2011,1,1, tz='utc').timestamp
1293840000.0
and if the date is in some other timezone (this matters because we're assuming midnight without an associated time):
>>> SimpleDate(date(2011,1,1), tz='America/New_York').timestamp
1293858000.0
[the idea behind simple-date is to collect all python's date and time stuff in one consistent class, so you can do any conversion. so, for example, it will also go the other way:
>>> SimpleDate(1293858000, tz='utc').date
datetime.date(2011, 1, 1)
]
Considering you have a datetime object called d,
use the following to get the timestamp in UTC:
d.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
And for the opposite direction, use following :
d = datetime.strptime("2008-09-03T20:56:35.450686Z", "%Y-%m-%dT%H:%M:%S.%fZ")
This works for me, pass through a function.
from datetime import timezone, datetime, timedelta
import datetime
def utc_converter(dt):
dt = datetime.datetime.now(timezone.utc)
utc_time = dt.replace(tzinfo=timezone.utc)
utc_timestamp = utc_time.timestamp()
return utc_timestamp
# create start and end timestamps
_now = datetime.datetime.now()
str_start = str(utc_converter(_now))
_end = _now + timedelta(seconds=10)
str_end = str(utc_converter(_end))
i'm impressed of the deep discussion.
my 2 cents:
from datetime import datetime
import time
the timestamp in utc is:
timestamp = \
(datetime.utcnow() - datetime(1970,1,1)).total_seconds()
or,
timestamp = time.time()
if now results from datetime.now(), in the same DST
utcoffset = (datetime.now() - datetime.utcnow()).total_seconds()
timestamp = \
(now - datetime(1970,1,1)).total_seconds() - utcoffset

How to convert a timedelta object into a datetime object

What is the proper way to convert a timedelta object into a datetime object?
I immediately think of something like datetime(0)+deltaObj, but that's not very nice... Isn't there a toDateTime() function or something of the sort?
It doesn't make sense to convert a timedelta into a datetime, but it does make sense to pick an initial or starting datetime and add or subtract a timedelta from that.
>>> import datetime
>>> today = datetime.datetime.today()
>>> today
datetime.datetime(2010, 3, 9, 18, 25, 19, 474362)
>>> today + datetime.timedelta(days=1)
datetime.datetime(2010, 3, 10, 18, 25, 19, 474362)
Since a datetime represents a time within a single day, your timedelta should be less than 24 hours (86400 seconds), even though timedeltas are not subject to this constraint.
import datetime
seconds = 86399
td = datetime.timedelta(seconds=seconds)
print(td)
dt = datetime.datetime.strptime(str(td), "%H:%M:%S")
print(dt)
23:59:59
1900-01-01 23:59:59
If you don't want a default date and know the date of your timedelta:
date = "05/15/2020"
dt2 = datetime.datetime.strptime("{} {}".format(date, td), "%m/%d/%Y %H:%M:%S")
print(dt2)
2020-05-15 23:59:59
I found that I could take the .total_seconds() and use that to create a new time object (or datetime object if needed).
import time
import datetime
start_dt_obj = datetime.datetime.fromtimestamp(start_timestamp)
stop_dt_obj = datetime.datetime.fromtimestamp(stop_timestamp)
delta = stop_dt_obj - start_dt_obj
delta_as_time_obj = time.gmtime(delta.total_seconds())
This allows you to do something like:
print('The duration was {0}'.format(
time.strftime('%H:%M', delta_as_time_obj)
)
Improving #sadpanduar answer with example on converting one column in pandas.DataFrame:
from datetime import timedelta
import time
def seconds_to_datetime(seconds, format='%Y-%m-%d %H:%M:%S'):
td = timedelta(seconds=seconds)
time_obj = time.gmtime(td.total_seconds())
return time.strftime(format, time_obj)
df = pd.read_csv(CSV_PATH)
df['TIMESTAMP_COLUMN'] = df['TIMESTAMP_COLUMN'].apply(seconds_to_datetime)
import datetime`enter code here
lastDownloadedDate = datetime.date(2022,8,4)
print('lastDownloadedDate: ', lastDownloadedDate)
fdate = lastDownloadedDate + datetime.timedelta(days=1)
fdate = datetime.datetime.strptime(str(fdate), "%Y-%m-%d")
fdate = datetime.date(fdate.year, fdate.month, fdate.day)
print('fdate: ', dt3)`

Categories

Resources