As part of a logging system, I would like to parse a string timestamp coming from a Cisco device, which has the following format:
# show clock
16:26:19.990 GMT+1 Wed Sep 11 2013
The parsing result should be a UTC datetime instance which will be stored in a SQLite database, thus the need for a timezone conversion.
Using just datetime.strptime is not enough, because the %Z directive only recognises local timezones (i.e. those related to the current $LANG or $LC_* environment). Therefore, I need to make use of the pytz package.
Because the format is always the same, I can do something like the following:
import pytz
from datetime import datetime
s = '16:26:19.990 CEST Wed Sep 11 2013'
tm, tz, dt = s.split(" ", 2)
naive = datetime.strptime("%s %s" % (tm, dt), "%H:%M:%S.%f %a %b %d %Y")
aware = naive.replace(timezone=pytz.timezone(tz))
universal = aware.astimezone(pytz.UTC)
This, however, does not work without some modifications. The value of tz must be corrected to a name that is recognized by pytz. In the example, pytz.timezone('CEST') raises an UnknownTimezoneError because the real timezone is CET. The problem is that the daylight savings correction is not applied then:
>>> from datetime import datetime
>>> from pytz import UTC, timezone
>>> a = datetime.strptime('16:18:57.925 Wed Sep 11 2013', '%H:%M:%S.%f %a %b %d %Y')
>>> b = a.replace(tzinfo=timezone('CET'))
>>> a
datetime.datetime(2013, 9, 11, 16, 18, 57, 925000)
>>> b
datetime.datetime(2013, 9, 11, 16, 18, 57, 925000, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)
>>> b.astimezone(UTC)
datetime.datetime(2013, 9, 11, 15, 18, 57, 925000, tzinfo=<UTC>)
Using normalize does not seem to help:
>>> timezone('CET').normalize(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/etanol/virtualenvs/plexus/local/lib/python2.7/site-packages/pytz/tzinfo.py", line 235, in normalize
raise ValueError('Naive time - no tzinfo set')
ValueError: Naive time - no tzinfo set
>>> timezone('CET').normalize(b)
datetime.datetime(2013, 9, 11, 17, 18, 57, 925000, tzinfo=<DstTzInfo 'CET' CEST+2:00:00 DST>)
I don't really know what am I missing, but the wanted result is:
datetime.datetime(2013, 9, 11, 14, 18, 57, 925000, tzinfo=<UTC>)
Thanks in advance.
Using timezone.localize:
>>> from datetime import datetime
>>> from pytz import UTC, timezone
>>>
>>> CET = timezone('CET')
>>>
>>> a = datetime.strptime('16:18:57.925 Wed Sep 11 2013', '%H:%M:%S.%f %a %b %d %Y')
>>> print CET.localize(a).astimezone(UTC)
2013-09-11 14:18:57.925000+00:00
Related
There is a time string 2019-05-15T16:08:37+08:00 I got from front-end, and I want to convert it to a normal datetime object by using datetime
How can I do it correctly?
I tried
start_time_utc_s = "2019-05-15T16:08:37+08:00"
r = datetime.datetime.strptime(start_time_utc_s, "%Y-%m-%dT%H:%M:%S+08:00")
# print(r)
# 2019-05-15 16:08:37
but I don't think it's right...
It's not right because you're simply discarding the timezone information (in this case, the UTC offset (+8 hours)). For more details, check [Python 3.Docs]: datetime - strftime() and strptime() Behavior (%z directive).
>>> import datetime
>>>
>>> start_time = "2019-05-15T16:08:37+08:00"
>>>
>>> dt0 = datetime.datetime.strptime(start_time, "%Y-%m-%dT%H:%M:%S+08:00")
>>> dt0
datetime.datetime(2019, 5, 15, 16, 8, 37)
>>> dt0.tzinfo
>>>
>>> dt_format_string = "%Y-%m-%dT%H:%M:%S%z" # This format string also takes timezone info (UTC offset) into account
>>> dt1 = datetime.datetime.strptime(start_time, dt_format_string)
>>> dt1
datetime.datetime(2019, 5, 15, 16, 8, 37, tzinfo=datetime.timezone(datetime.timedelta(seconds=28800)))
>>> dt1.tzinfo
datetime.timezone(datetime.timedelta(seconds=28800))
>>>
>>> dt2 = datetime.datetime.strptime("2019-05-15T16:08:37+08:45", dt_format_string) # Changed the offset to +08:45
>>> dt2
datetime.datetime(2019, 5, 15, 16, 8, 37, tzinfo=datetime.timezone(datetime.timedelta(seconds=31500)))
The time offset in question corresponds to a bunch of timezones: AWST, CST, HKT, IRKT, MYT, PHT, SGT, WITA, ... (check [TimeAndDate]: Time Zone Abbreviations – Worldwide List for the complete list).
#EDIT0:
Everything above is on Python 3.7(.3) (and it seems that the doc and functionality are NOT in sync). On Python 3.6(.8):
>>> datetime.datetime.strptime("2019-05-15T16:08:37+08:45", "%Y-%m-%dT%H:%M:%S%z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\install\x64\python\python\03.06.08\Lib\_strptime.py", line 565, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "c:\install\x64\python\python\03.06.08\Lib\_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '2019-05-15T16:08:37+08:45' does not match format '%Y-%m-%dT%H:%M:%S%z'
For that, as shown in:
[SO]: ISO to datetime object: 'z' is a bad directive [duplicate]
[SO]: Python strptime() and timezones?
[SO]: Converting string with UTC offset to a datetime object [duplicate]
one way would be to (install and) use dateutil:
>>> import sys
>>> sys.version
'3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]'
>>>
>>> from dateutil import parser
>>> parser.parse("2019-05-15T16:08:37+08:00")
datetime.datetime(2019, 5, 15, 16, 8, 37, tzinfo=tzoffset(None, 28800))
Or you could strip out the (last) colon between hours and minutes (for UTC offset). Note that the below code relies on UTC offset being specified in the fixed format from the question:
>>> start_time = "2019-05-15T16:08:37+08:00"
>>>
>>> dt_format_string = "%Y-%m-%dT%H:%M:%S%z"
>>>
>>> datetime.datetime.strptime(start_time[:-3] + start_time[-2:], dt_format_string)
datetime.datetime(2019, 5, 15, 16, 8, 37, tzinfo=datetime.timezone(datetime.timedelta(0, 28800)))
But both these seem to be just workarounds.
I want convert GMT time to EST time and get a timestamp. I tried the following but don't know how to set time zone.
time = "Tue, 12 Jun 2012 14:03:10 GMT"
timestamp2 = time.mktime(time.strptime(time, '%a, %d %b %Y %H:%M:%S GMT'))
Time zones aren't built into standard Python - you need to use another library. pytz is a good choice.
>>> gmt = pytz.timezone('GMT')
>>> eastern = pytz.timezone('US/Eastern')
>>> time = "Tue, 12 Jun 2012 14:03:10 GMT"
>>> date = datetime.datetime.strptime(time, '%a, %d %b %Y %H:%M:%S GMT')
>>> date
datetime.datetime(2012, 6, 12, 14, 3, 10)
>>> dategmt = gmt.localize(date)
>>> dategmt
datetime.datetime(2012, 6, 12, 14, 3, 10, tzinfo=<StaticTzInfo 'GMT'>)
>>> dateeastern = dategmt.astimezone(eastern)
>>> dateeastern
datetime.datetime(2012, 6, 12, 10, 3, 10, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
Using pytz
from datetime import datetime
from pytz import timezone
fmt = "%Y-%m-%d %H:%M:%S %Z%z"
now_time = datetime.now(timezone('US/Eastern'))
print now_time.strftime(fmt)
With Python 3.9, time zone handling is built into the standard lib with the zoneinfo module (for older Python versions, use zoneinfo via backports.zoneinfo).
Ex:
from zoneinfo import ZoneInfo
from datetime import datetime, timezone
time = "Tue, 12 Jun 2012 14:03:10 GMT"
# parse to datetime, using %Z for the time zone abbreviation
dtobj = datetime.strptime(time, '%a, %d %b %Y %H:%M:%S %Z')
# note that "GMT" (=UTC) is ignored:
# datetime.datetime(2012, 6, 12, 14, 3, 10)
# ...so let's correct that:
dtobj = dtobj.replace(tzinfo=timezone.utc)
# datetime.datetime(2012, 6, 12, 14, 3, 10, tzinfo=datetime.timezone.utc)
# convert to US/Eastern (EST or EDT, depending on time of the year)
dtobj = dtobj.astimezone(ZoneInfo('US/Eastern'))
# datetime.datetime(2012, 6, 12, 10, 3, 10, tzinfo=zoneinfo.ZoneInfo(key='US/Eastern'))
print(dtobj)
# 2012-06-12 10:03:10-04:00
And there's also dateutil, which follows the same semantics:
import dateutil
# no strptime needed...
# correctly localizes to GMT (=UTC) directly
dtobj = dateutil.parser.parse(time)
dtobj = dtobj.astimezone(dateutil.tz.gettz('US/Eastern'))
# datetime.datetime(2012, 6, 12, 10, 3, 10, tzinfo=tzfile('US/Eastern'))
print(dtobj)
# 2012-06-12 10:03:10-04:00
If it is your local timezone and the timezone rules are the same for the given time as they are now then you could use stdlib-only solution (except for some edge cases):
#!/usr/bin/env python
from email.utils import parsedate_tz, mktime_tz
from datetime import datetime
timestamp = mktime_tz(parsedate_tz("Tue, 12 Jun 2012 14:03:10 GMT"))
print(datetime.fromtimestamp(timestamp))
# -> 2012-06-12 10:03:10
Otherwise you need data from a historical timezone database to get the correct utc offset. pytz module provides access to the tz database:
#!/usr/bin/env python
from email.utils import parsedate_tz, mktime_tz
from datetime import datetime
import pytz # $ pip install pytz
timestamp = mktime_tz(parsedate_tz("Tue, 12 Jun 2012 14:03:10 GMT"))
eastern_dt = datetime.fromtimestamp(timestamp, pytz.timezone('America/New_York'))
print(eastern_dt.strftime('%a, %d %b %Y %H:%M:%S %z (%Z)'))
# -> Tue, 12 Jun 2012 10:03:10 -0400 (EDT)
Note: POSIX timestamp is the same around the world i.e., your local timezone doesn't matter if you want to find the timestamp (unless your timezone is of "right" kind). Here's how to convert a utc time to the timestamp.
I have a system (developed in Python) that accepts datetime as string in VARIOUS formats and i have to parse them..Currently datetime string formats are :
Fri Sep 25 18:09:49 -0500 2009
2008-06-29T00:42:18.000Z
2011-07-16T21:46:39Z
1294989360
Now i want a generic parser that can convert any of these datetime formats in appropriate datetime object...
Otherwise, i have to go with parsing them individually. So please also provide method for parsing them individually (if there is no generic parser)..!!
As #TimPietzcker suggested, the dateutil package is the way to go, it handles the first 3 formats correctly and automatically:
>>> from dateutil.parser import parse
>>> parse("Fri Sep 25 18:09:49 -0500 2009")
datetime.datetime(2009, 9, 25, 18, 9, 49, tzinfo=tzoffset(None, -18000))
>>> parse("2008-06-29T00:42:18.000Z")
datetime.datetime(2008, 6, 29, 0, 42, 18, tzinfo=tzutc())
>>> parse("2011-07-16T21:46:39Z")
datetime.datetime(2011, 7, 16, 21, 46, 39, tzinfo=tzutc())
The unixtime format it seems to hiccough on, but luckily the standard datetime.datetime is up for the task:
>>> from datetime import datetime
>>> datetime.utcfromtimestamp(float("1294989360"))
datetime.datetime(2011, 1, 14, 7, 16)
It is rather easy to make a function out of this that handles all 4 formats:
from dateutil.parser import parse
from datetime import datetime
def parse_time(s):
try:
ret = parse(s)
except ValueError:
ret = datetime.utcfromtimestamp(s)
return ret
You should look into the dateutil package.
This question already has answers here:
Python time to age
(6 answers)
Closed 8 years ago.
Following on from my previous question, Python time to age, I have now come across a problem regarding the timezone, and it turns out that it's not always going to be "+0200". So when strptime tries to parse it as such, it throws up an exception.
I thought about just chopping off the +0200 with [:-6] or whatever, but is there a real way to do this with strptime?
I am using Python 2.5.2 if it matters.
>>> from datetime import datetime
>>> fmt = "%a, %d %b %Y %H:%M:%S +0200"
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200", fmt)
datetime.datetime(2008, 7, 22, 8, 17, 41)
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0300", fmt)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/_strptime.py", line 330, in strptime
(data_string, format))
ValueError: time data did not match format: data=Tue, 22 Jul 2008 08:17:41 +0300 fmt=%a, %d %b %Y %H:%M:%S +0200
is there a real way to do this with strptime?
No, but since your format appears to be an RFC822-family date, you can read it much more easily using the email library instead:
>>> import email.utils
>>> email.utils.parsedate_tz('Tue, 22 Jul 2008 08:17:41 +0200')
(2008, 7, 22, 8, 17, 41, 0, 1, 0, 7200)
(7200 = timezone offset from UTC in seconds)
New in version 2.6.
For a naive object, the %z and %Z
format codes are replaced by empty
strings.
It looks like this is implemented only in >= 2.6, and I think you have to manually parse it.
I can't see another solution than to remove the time zone data:
from datetime import timedelta,datetime
try:
offset = int("Tue, 22 Jul 2008 08:17:41 +0300"[-5:])
except:
print "Error"
delta = timedelta(hours = offset / 100)
fmt = "%a, %d %b %Y %H:%M:%S"
time = datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200"[:-6], fmt)
time -= delta
You can use the dateutil library which is very useful:
from datetime import datetime
from dateutil.parser import parse
dt = parse("Tue, 22 Jul 2008 08:17:41 +0200")
## datetime.datetime(2008, 7, 22, 8, 17, 41, tzinfo=tzoffset(None, 7200)) <- dt
print dt
2008-07-22 08:17:41+02:00
As far as I know, strptime() doesn't recognize numeric time zone codes. If you know that the string is always going to end with a time zone specification of that form (+ or - followed by 4 digits), just chopping it off and parsing it manually seems like a perfectly reasonable thing to do.
It seems that %Z corresponds to time zone names, not offsets.
For example, given:
>>> format = '%a, %d %b %Y %H:%M:%S %Z'
I can parse:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 GMT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
Although it seems that it doesn't do anything with the time zone, merely observing that it exists and is valid:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 NZDT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
I suppose if you wished, you could locate a mapping of offsets to names, convert your input, and then parse it. It might be simpler to just truncate your input, though.
This question already has answers here:
Python time to age
(6 answers)
Closed 8 years ago.
Following on from my previous question, Python time to age, I have now come across a problem regarding the timezone, and it turns out that it's not always going to be "+0200". So when strptime tries to parse it as such, it throws up an exception.
I thought about just chopping off the +0200 with [:-6] or whatever, but is there a real way to do this with strptime?
I am using Python 2.5.2 if it matters.
>>> from datetime import datetime
>>> fmt = "%a, %d %b %Y %H:%M:%S +0200"
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200", fmt)
datetime.datetime(2008, 7, 22, 8, 17, 41)
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0300", fmt)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/_strptime.py", line 330, in strptime
(data_string, format))
ValueError: time data did not match format: data=Tue, 22 Jul 2008 08:17:41 +0300 fmt=%a, %d %b %Y %H:%M:%S +0200
is there a real way to do this with strptime?
No, but since your format appears to be an RFC822-family date, you can read it much more easily using the email library instead:
>>> import email.utils
>>> email.utils.parsedate_tz('Tue, 22 Jul 2008 08:17:41 +0200')
(2008, 7, 22, 8, 17, 41, 0, 1, 0, 7200)
(7200 = timezone offset from UTC in seconds)
New in version 2.6.
For a naive object, the %z and %Z
format codes are replaced by empty
strings.
It looks like this is implemented only in >= 2.6, and I think you have to manually parse it.
I can't see another solution than to remove the time zone data:
from datetime import timedelta,datetime
try:
offset = int("Tue, 22 Jul 2008 08:17:41 +0300"[-5:])
except:
print "Error"
delta = timedelta(hours = offset / 100)
fmt = "%a, %d %b %Y %H:%M:%S"
time = datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200"[:-6], fmt)
time -= delta
You can use the dateutil library which is very useful:
from datetime import datetime
from dateutil.parser import parse
dt = parse("Tue, 22 Jul 2008 08:17:41 +0200")
## datetime.datetime(2008, 7, 22, 8, 17, 41, tzinfo=tzoffset(None, 7200)) <- dt
print dt
2008-07-22 08:17:41+02:00
As far as I know, strptime() doesn't recognize numeric time zone codes. If you know that the string is always going to end with a time zone specification of that form (+ or - followed by 4 digits), just chopping it off and parsing it manually seems like a perfectly reasonable thing to do.
It seems that %Z corresponds to time zone names, not offsets.
For example, given:
>>> format = '%a, %d %b %Y %H:%M:%S %Z'
I can parse:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 GMT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
Although it seems that it doesn't do anything with the time zone, merely observing that it exists and is valid:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 NZDT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
I suppose if you wished, you could locate a mapping of offsets to names, convert your input, and then parse it. It might be simpler to just truncate your input, though.