I am trying to retrieve date from an email. At first it's easy:
message = email.parser.Parser().parse(file)
date = message['Date']
print date
and I receive:
'Mon, 16 Nov 2009 13:32:02 +0100'
But I need a nice datetime object, so I use:
datetime.strptime('Mon, 16 Nov 2009 13:32:02 +0100', '%a, %d %b %Y %H:%M:%S %Z')
which raises ValueError, since %Z isn't format for +0100. But I can't find proper format for timezone in the documentation, there is only this %Z for zone. Can someone help me on that?
email.utils has a parsedate() function for the RFC 2822 format, which as far as I know is not deprecated.
>>> import email.utils
>>> import time
>>> import datetime
>>> email.utils.parsedate('Mon, 16 Nov 2009 13:32:02 +0100')
(2009, 11, 16, 13, 32, 2, 0, 1, -1)
>>> time.mktime((2009, 11, 16, 13, 32, 2, 0, 1, -1))
1258378322.0
>>> datetime.datetime.fromtimestamp(1258378322.0)
datetime.datetime(2009, 11, 16, 13, 32, 2)
Please note, however, that the parsedate method does not take into account the time zone and time.mktime always expects a local time tuple.
>>> (time.mktime(email.utils.parsedate('Mon, 16 Nov 2009 13:32:02 +0900')) ==
... time.mktime(email.utils.parsedate('Mon, 16 Nov 2009 13:32:02 +0100'))
True
So you'll still need to parse out the time zone and take into account the local time difference, too:
>>> REMOTE_TIME_ZONE_OFFSET = +9 * 60 * 60
>>> (time.mktime(email.utils.parsedate('Mon, 16 Nov 2009 13:32:02 +0900')) +
... time.timezone - REMOTE_TIME_ZONE_OFFSET)
1258410122.0
Use email.utils.parsedate_tz(date):
msg=email.message_from_file(open(file_name))
date=None
date_str=msg.get('date')
if date_str:
date_tuple=email.utils.parsedate_tz(date_str)
if date_tuple:
date=datetime.datetime.fromtimestamp(email.utils.mktime_tz(date_tuple))
if date:
... # valid date found
For python 3.3+ you can use parsedate_to_datetime function:
>>> from email.utils import parsedate_to_datetime
>>> parsedate_to_datetime('Mon, 16 Nov 2009 13:32:02 +0100')
...
datetime.datetime(2009, 11, 16, 13, 32, 2, tzinfo=datetime.timezone(datetime.timedelta(0, 3600)))
Official documentation:
The inverse of format_datetime(). Performs the same function as
parsedate(), but on success returns a datetime. If the input date has
a timezone of -0000, the datetime will be a naive datetime, and if the
date is conforming to the RFCs it will represent a time in UTC but
with no indication of the actual source timezone of the message the
date comes from. If the input date has any other valid timezone
offset, the datetime will be an aware datetime with the corresponding
a timezone tzinfo. New in version 3.3.
In Python 3.3+, email message can parse the headers for you:
import email
import email.policy
headers = email.message_from_file(file, policy=email.policy.default)
print(headers.get('date').datetime)
# -> 2009-11-16 13:32:02+01:00
Since Python 3.2+, it works if you replace %Z with %z:
>>> from datetime import datetime
>>> datetime.strptime("Mon, 16 Nov 2009 13:32:02 +0100",
... "%a, %d %b %Y %H:%M:%S %z")
datetime.datetime(2009, 11, 16, 13, 32, 2,
tzinfo=datetime.timezone(datetime.timedelta(0, 3600)))
Or using email package (Python 3.3+):
>>> from email.utils import parsedate_to_datetime
>>> parsedate_to_datetime("Mon, 16 Nov 2009 13:32:02 +0100")
datetime.datetime(2009, 11, 16, 13, 32, 2,
tzinfo=datetime.timezone(datetime.timedelta(0, 3600)))
if UTC offset is specified as -0000 then it returns a naive datetime object that represents time in UTC otherwise it returns an aware datetime object with the corresponding tzinfo set.
To parse rfc 5322 date-time string on earlier Python versions (2.6+):
from calendar import timegm
from datetime import datetime, timedelta, tzinfo
from email.utils import parsedate_tz
ZERO = timedelta(0)
time_string = 'Mon, 16 Nov 2009 13:32:02 +0100'
tt = parsedate_tz(time_string)
#NOTE: mktime_tz is broken on Python < 2.7.4,
# see https://bugs.python.org/issue21267
timestamp = timegm(tt) - tt[9] # local time - utc offset == utc time
naive_utc_dt = datetime(1970, 1, 1) + timedelta(seconds=timestamp)
aware_utc_dt = naive_utc_dt.replace(tzinfo=FixedOffset(ZERO, 'UTC'))
aware_dt = aware_utc_dt.astimezone(FixedOffset(timedelta(seconds=tt[9])))
print(aware_utc_dt)
print(aware_dt)
# -> 2009-11-16 12:32:02+00:00
# -> 2009-11-16 13:32:02+01:00
where FixedOffset is based on tzinfo subclass from the datetime documentation:
class FixedOffset(tzinfo):
"""Fixed UTC offset: `time = utc_time + utc_offset`."""
def __init__(self, offset, name=None):
self.__offset = offset
if name is None:
seconds = abs(offset).seconds
assert abs(offset).days == 0
hours, seconds = divmod(seconds, 3600)
if offset < ZERO:
hours = -hours
minutes, seconds = divmod(seconds, 60)
assert seconds == 0
#NOTE: the last part is to remind about deprecated POSIX
# GMT+h timezones that have the opposite sign in the
# name; the corresponding numeric value is not used e.g.,
# no minutes
self.__name = '<%+03d%02d>GMT%+d' % (hours, minutes, -hours)
else:
self.__name = name
def utcoffset(self, dt=None):
return self.__offset
def tzname(self, dt=None):
return self.__name
def dst(self, dt=None):
return ZERO
def __repr__(self):
return 'FixedOffset(%r, %r)' % (self.utcoffset(), self.tzname())
Have you tried
rfc822.parsedate_tz(date) # ?
More on RFC822, http://docs.python.org/library/rfc822.html
It's deprecated (parsedate_tz is now in email.utils.parsedate_tz), though.
But maybe these answers help:
How to parse dates with -0400 timezone string in python?
python time to age part 2, timezones
# Parses Nginx' format of "01/Jan/1999:13:59:59 +0400"
# Unfortunately, strptime doesn't support %z for the UTC offset (despite what
# the docs actually say), hence the need # for this function.
def parseDate(dateStr):
date = datetime.datetime.strptime(dateStr[:-6], "%d/%b/%Y:%H:%M:%S")
offsetDir = dateStr[-5]
offsetHours = int(dateStr[-4:-2])
offsetMins = int(dateStr[-2:])
if offsetDir == "-":
offsetHours = -offsetHours
offsetMins = -offsetMins
return date + datetime.timedelta(hours=offsetHours, minutes=offsetMins)
For those who want to get the correct local time, here is what I did:
from datetime import datetime
from email.utils import parsedate_to_datetime
mail_time_str = 'Mon, 16 Nov 2009 13:32:02 +0100'
local_time_str = datetime.fromtimestamp(parsedate_to_datetime(mail_time_str).timestamp()).strftime('%Y-%m-%d %H:%M:%S')
print(local_time_str)
ValueError: 'z' is a bad directive in format...
(note: I have to stick to python 2.7 in my case)
I have had a similar problem parsing commit dates from the output of git log --date=iso8601 which actually isn't the ISO8601 format (hence the addition of --date=iso8601-strict in a later version).
Since I am using django I can leverage the utilities there.
https://github.com/django/django/blob/master/django/utils/dateparse.py
>>> from django.utils.dateparse import parse_datetime
>>> parse_datetime('2013-07-23T15:10:59.342107+01:00')
datetime.datetime(2013, 7, 23, 15, 10, 59, 342107, tzinfo=+0100)
Instead of strptime you could use your own regular expression.
Related
I have a CSV dumpfile from a Blackberry IPD backup, created using IPDDump.
The date/time strings in here look something like this
(where EST is an Australian time-zone):
Tue Jun 22 07:46:22 EST 2010
I need to be able to parse this date in Python. At first, I tried to use the strptime() function from datettime.
>>> datetime.datetime.strptime('Tue Jun 22 12:10:20 2010 EST', '%a %b %d %H:%M:%S %Y %Z')
However, for some reason, the datetime object that comes back doesn't seem to have any tzinfo associated with it.
I did read on this page that apparently datetime.strptime silently discards tzinfo, however, I checked the documentation, and I can't find anything to that effect documented here.
Is there any way to get strptime() to play nicely with timezones?
I recommend using python-dateutil. Its parser has been able to parse every date format I've thrown at it so far.
>>> from dateutil import parser
>>> parser.parse("Tue Jun 22 07:46:22 EST 2010")
datetime.datetime(2010, 6, 22, 7, 46, 22, tzinfo=tzlocal())
>>> parser.parse("Fri, 11 Nov 2011 03:18:09 -0400")
datetime.datetime(2011, 11, 11, 3, 18, 9, tzinfo=tzoffset(None, -14400))
>>> parser.parse("Sun")
datetime.datetime(2011, 12, 18, 0, 0)
>>> parser.parse("10-11-08")
datetime.datetime(2008, 10, 11, 0, 0)
and so on. No dealing with strptime() format nonsense... just throw a date at it and it Does The Right Thing.
The datetime module documentation says:
Return a datetime corresponding to date_string, parsed according to format. This is equivalent to datetime(*(time.strptime(date_string, format)[0:6])).
See that [0:6]? That gets you (year, month, day, hour, minute, second). Nothing else. No mention of timezones.
Interestingly, [Win XP SP2, Python 2.6, 2.7] passing your example to time.strptime doesn't work but if you strip off the " %Z" and the " EST" it does work. Also using "UTC" or "GMT" instead of "EST" works. "PST" and "MEZ" don't work. Puzzling.
It's worth noting this has been updated as of version 3.2 and the same documentation now also states the following:
When the %z directive is provided to the strptime() method, an aware datetime object will be produced. The tzinfo of the result will be set to a timezone instance.
Note that this doesn't work with %Z, so the case is important. See the following example:
In [1]: from datetime import datetime
In [2]: start_time = datetime.strptime('2018-04-18-17-04-30-AEST','%Y-%m-%d-%H-%M-%S-%Z')
In [3]: print("TZ NAME: {tz}".format(tz=start_time.tzname()))
TZ NAME: None
In [4]: start_time = datetime.strptime('2018-04-18-17-04-30-+1000','%Y-%m-%d-%H-%M-%S-%z')
In [5]: print("TZ NAME: {tz}".format(tz=start_time.tzname()))
TZ NAME: UTC+10:00
Since strptime returns a datetime object which has tzinfo attribute, We can simply replace it with desired timezone.
>>> import datetime
>>> date_time_str = '2018-06-29 08:15:27.243860'
>>> date_time_obj = datetime.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S.%f').replace(tzinfo=datetime.timezone.utc)
>>> date_time_obj.tzname()
'UTC'
Your time string is similar to the time format in rfc 2822 (date format in email, http headers). You could parse it using only stdlib:
>>> from email.utils import parsedate_tz
>>> parsedate_tz('Tue Jun 22 07:46:22 EST 2010')
(2010, 6, 22, 7, 46, 22, 0, 1, -1, -18000)
See solutions that yield timezone-aware datetime objects for various Python versions: parsing date with timezone from an email.
In this format, EST is semantically equivalent to -0500. Though, in general, a timezone abbreviation is not enough, to identify a timezone uniquely.
Ran into this exact problem.
What I ended up doing:
# starting with date string
sdt = "20190901"
std_format = '%Y%m%d'
# create naive datetime object
from datetime import datetime
dt = datetime.strptime(sdt, sdt_format)
# extract the relevant date time items
dt_formatters = ['%Y','%m','%d']
dt_vals = tuple(map(lambda formatter: int(datetime.strftime(dt,formatter)), dt_formatters))
# set timezone
import pendulum
tz = pendulum.timezone('utc')
dt_tz = datetime(*dt_vals,tzinfo=tz)
I want to transfer date string into python datetime format. The string is below:
Mon, 26 Dec 2011 20:42:08 +0200
Sat, 24 Dec 2011 16:28:59 +0200
Is there any faster way to transfer the string into python datetime format without using pytz API?
>>> import datetime
>>> s = "Mon, 26 Dec 2011 20:42:08 +0200"
>>> t = datetime.datetime.strptime(s, "%a, %d %b %Y %H:%M:%S %z")
>>> t
datetime.datetime(2011, 12, 26, 20, 42, 8, tzinfo=datetime.timezone(datetime.timedelta(0, 7200)))
See the docs for a complete list of datetime placeholders.
Here's a variant that works on Python 2.5+, PyPy, Jython:
from datetime import datetime, timedelta
from email.utils import mktime_tz, parsedate_tz
s = "Mon, 26 Dec 2011 20:42:08 +0200"
t = parsedate_tz(s)
if t[9] is not None:
utc_offset = timedelta(seconds=t[9]) # local = utc + offset
print("utc offset: %s" % (utc_offset,))
print(repr(datetime.utcfromtimestamp(mktime_tz(t)))) # naive datetime in UTC
Output
utc offset: 2:00:00
datetime.datetime(2011, 12, 26, 18, 42, 8)
Try this?
Assuming input of 2016-01-02 and you want to standardize all dates.
def NormalizaData(input):
input = str(input)
num = input.split('-')
year = num[0]
newnum = num[1]
num = newnum.split('-')
month = num[0]
day = '01'
return (year + '-' + month + '-' + day)
I am trying to convert a string timestamp into a proper datetime object. The problem I am having is that there is a timezone offset and everything I am doing doesn't seem to work.
Ultimately I want to convert the string timestamp into a datetime object in my machines timezone.
# string timestamp
date = "Fri, 16 Jul 2010 07:08:23 -0700"
The dateutil package is handy for parsing date/times:
In [10]: date = u"Fri, 16 Jul 2010 07:08:23 -0700"
In [11]: from dateutil.parser import parse
In [12]: parse(date)
Out[12]: datetime.datetime(2010, 7, 16, 7, 8, 23, tzinfo=tzoffset(None, -25200))
Finally, to convert into your local timezone,
In [13]: parse(date).astimezone(YOUR_LOCAL_TIMEZONE)
It looks like datetime.datetime.strptime(d, '%a, %d %b %Y %H:%M:%S %z') should work, but according to this bug report there are issues with the %z processing. So you'll probably have to handle the timezone on your own:
import datetime
d = u"Fri, 16 Jul 2010 07:08:23 -0700"
d, tz_info = d[:-5], d[-5:]
neg, hours, minutes = tz_info[0], int(tz_info[1:3]), int(tz_info[3:])
if neg == '-':
hours, minutes = hours * -1, minutes * -1
d = datetime.datetime.strptime(d, '%a, %d %b %Y %H:%M:%S ')
print d
print d + datetime.timedelta(hours = hours, minutes = minutes)
Here's a stdlib solution:
>>> from datetime import datetime
>>> from email.utils import mktime_tz, parsedate_tz
>>> datetime.fromtimestamp(mktime_tz(parsedate_tz(u"Fri, 16 Jul 2010 07:08:23 -0700")))
datetime.datetime(2010, 7, 16, 16, 8, 23) # your local time may be different
See also, Python: parsing date with timezone from an email.
Note: fromtimestamp() may fail if the local timezone had different UTC offset in the past (2010) and if it does not use a historical timezone database on the given platform. To fix it, you could use tzlocal.get_localzone(), to get a pytz tzinfo object representing your local timezone. pytz provides access to the tz database in a portable manner:
>>> timestamp = mktime_tz(parsedate_tz(u"Fri, 16 Jul 2010 07:08:23 -0700"))
>>> import tzlocal # $ pip install tzlocal
>>> str(datetime.fromtimestamp(timestamp, tzlocal.get_localzone()))
'2010-07-16 16:08:23+02:00'
I have a CSV dumpfile from a Blackberry IPD backup, created using IPDDump.
The date/time strings in here look something like this
(where EST is an Australian time-zone):
Tue Jun 22 07:46:22 EST 2010
I need to be able to parse this date in Python. At first, I tried to use the strptime() function from datettime.
>>> datetime.datetime.strptime('Tue Jun 22 12:10:20 2010 EST', '%a %b %d %H:%M:%S %Y %Z')
However, for some reason, the datetime object that comes back doesn't seem to have any tzinfo associated with it.
I did read on this page that apparently datetime.strptime silently discards tzinfo, however, I checked the documentation, and I can't find anything to that effect documented here.
Is there any way to get strptime() to play nicely with timezones?
I recommend using python-dateutil. Its parser has been able to parse every date format I've thrown at it so far.
>>> from dateutil import parser
>>> parser.parse("Tue Jun 22 07:46:22 EST 2010")
datetime.datetime(2010, 6, 22, 7, 46, 22, tzinfo=tzlocal())
>>> parser.parse("Fri, 11 Nov 2011 03:18:09 -0400")
datetime.datetime(2011, 11, 11, 3, 18, 9, tzinfo=tzoffset(None, -14400))
>>> parser.parse("Sun")
datetime.datetime(2011, 12, 18, 0, 0)
>>> parser.parse("10-11-08")
datetime.datetime(2008, 10, 11, 0, 0)
and so on. No dealing with strptime() format nonsense... just throw a date at it and it Does The Right Thing.
The datetime module documentation says:
Return a datetime corresponding to date_string, parsed according to format. This is equivalent to datetime(*(time.strptime(date_string, format)[0:6])).
See that [0:6]? That gets you (year, month, day, hour, minute, second). Nothing else. No mention of timezones.
Interestingly, [Win XP SP2, Python 2.6, 2.7] passing your example to time.strptime doesn't work but if you strip off the " %Z" and the " EST" it does work. Also using "UTC" or "GMT" instead of "EST" works. "PST" and "MEZ" don't work. Puzzling.
It's worth noting this has been updated as of version 3.2 and the same documentation now also states the following:
When the %z directive is provided to the strptime() method, an aware datetime object will be produced. The tzinfo of the result will be set to a timezone instance.
Note that this doesn't work with %Z, so the case is important. See the following example:
In [1]: from datetime import datetime
In [2]: start_time = datetime.strptime('2018-04-18-17-04-30-AEST','%Y-%m-%d-%H-%M-%S-%Z')
In [3]: print("TZ NAME: {tz}".format(tz=start_time.tzname()))
TZ NAME: None
In [4]: start_time = datetime.strptime('2018-04-18-17-04-30-+1000','%Y-%m-%d-%H-%M-%S-%z')
In [5]: print("TZ NAME: {tz}".format(tz=start_time.tzname()))
TZ NAME: UTC+10:00
Since strptime returns a datetime object which has tzinfo attribute, We can simply replace it with desired timezone.
>>> import datetime
>>> date_time_str = '2018-06-29 08:15:27.243860'
>>> date_time_obj = datetime.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S.%f').replace(tzinfo=datetime.timezone.utc)
>>> date_time_obj.tzname()
'UTC'
Your time string is similar to the time format in rfc 2822 (date format in email, http headers). You could parse it using only stdlib:
>>> from email.utils import parsedate_tz
>>> parsedate_tz('Tue Jun 22 07:46:22 EST 2010')
(2010, 6, 22, 7, 46, 22, 0, 1, -1, -18000)
See solutions that yield timezone-aware datetime objects for various Python versions: parsing date with timezone from an email.
In this format, EST is semantically equivalent to -0500. Though, in general, a timezone abbreviation is not enough, to identify a timezone uniquely.
Ran into this exact problem.
What I ended up doing:
# starting with date string
sdt = "20190901"
std_format = '%Y%m%d'
# create naive datetime object
from datetime import datetime
dt = datetime.strptime(sdt, sdt_format)
# extract the relevant date time items
dt_formatters = ['%Y','%m','%d']
dt_vals = tuple(map(lambda formatter: int(datetime.strftime(dt,formatter)), dt_formatters))
# set timezone
import pendulum
tz = pendulum.timezone('utc')
dt_tz = datetime(*dt_vals,tzinfo=tz)
This question already has answers here:
Python time to age
(6 answers)
Closed 8 years ago.
Following on from my previous question, Python time to age, I have now come across a problem regarding the timezone, and it turns out that it's not always going to be "+0200". So when strptime tries to parse it as such, it throws up an exception.
I thought about just chopping off the +0200 with [:-6] or whatever, but is there a real way to do this with strptime?
I am using Python 2.5.2 if it matters.
>>> from datetime import datetime
>>> fmt = "%a, %d %b %Y %H:%M:%S +0200"
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200", fmt)
datetime.datetime(2008, 7, 22, 8, 17, 41)
>>> datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0300", fmt)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/_strptime.py", line 330, in strptime
(data_string, format))
ValueError: time data did not match format: data=Tue, 22 Jul 2008 08:17:41 +0300 fmt=%a, %d %b %Y %H:%M:%S +0200
is there a real way to do this with strptime?
No, but since your format appears to be an RFC822-family date, you can read it much more easily using the email library instead:
>>> import email.utils
>>> email.utils.parsedate_tz('Tue, 22 Jul 2008 08:17:41 +0200')
(2008, 7, 22, 8, 17, 41, 0, 1, 0, 7200)
(7200 = timezone offset from UTC in seconds)
New in version 2.6.
For a naive object, the %z and %Z
format codes are replaced by empty
strings.
It looks like this is implemented only in >= 2.6, and I think you have to manually parse it.
I can't see another solution than to remove the time zone data:
from datetime import timedelta,datetime
try:
offset = int("Tue, 22 Jul 2008 08:17:41 +0300"[-5:])
except:
print "Error"
delta = timedelta(hours = offset / 100)
fmt = "%a, %d %b %Y %H:%M:%S"
time = datetime.strptime("Tue, 22 Jul 2008 08:17:41 +0200"[:-6], fmt)
time -= delta
You can use the dateutil library which is very useful:
from datetime import datetime
from dateutil.parser import parse
dt = parse("Tue, 22 Jul 2008 08:17:41 +0200")
## datetime.datetime(2008, 7, 22, 8, 17, 41, tzinfo=tzoffset(None, 7200)) <- dt
print dt
2008-07-22 08:17:41+02:00
As far as I know, strptime() doesn't recognize numeric time zone codes. If you know that the string is always going to end with a time zone specification of that form (+ or - followed by 4 digits), just chopping it off and parsing it manually seems like a perfectly reasonable thing to do.
It seems that %Z corresponds to time zone names, not offsets.
For example, given:
>>> format = '%a, %d %b %Y %H:%M:%S %Z'
I can parse:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 GMT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
Although it seems that it doesn't do anything with the time zone, merely observing that it exists and is valid:
>>> datetime.datetime.strptime('Tue, 22 Jul 2008 08:17:41 NZDT', format)
datetime.datetime(2008, 7, 22, 8, 17, 41)
I suppose if you wished, you could locate a mapping of offsets to names, convert your input, and then parse it. It might be simpler to just truncate your input, though.