I need to parse many different dates in many different formats. I am having trouble with the following and wondered if anyopne could explain why;
The following works on a linux system:
from datetime import datetime
datetime.strptime('Tue 23 Aug 2011 09:00:07 PM BST','%a %d %b %Y %H:%M:%S %p %Z')
But running under windows it raises
ValueError: time data does not match format
However, if I try GMT not BST on windows, it works fine;
from datetime import datetime
datetime.strptime('Tue 23 Aug 2011 09:00:07 PM GMT','%a %d %b %Y %H:%M:%S %p %Z')
Is there a reason python does not understand the BST timezone under windows, but it works fine under Linux?
thanks,
Matt.
In my opinion, parsing a three-letter time zone code like this is not a good practice (unless of course you have no choice). For example, "EST" is commonly used in the USA for UTC-4/5 and is also commonly used in Australia. So any support for "EST" must therefore be dependent on locale. It would not surprise me if "BST" was similarly ambiguous.
I highly recommend using the pytz module in which British civil time is given the string identifier Europe/London and UTC is called Etc/UTC. The pytz API will give consistent results regardless of the locale of the user or system running the application.
If you are working on a UI that must be tied to locale, or parsing inputs with formats you cannot change, then consider using a dictionary of abbreviations to pytz timezone objects. For example: {'BST': 'Europe/London'}. Then your application can work with UTC dates and times uniformly, which will greatly reduce the possibility of errors.
Related
I have the following error which happens only in CI:
ValueError: time data '09.30.2019 17:50 EDT' does not match format '%m.%d.%Y %H:%M %Z'
Here's my test:
def test_extract_time_from_page(pjm_html):
expected_time = datetime.strptime("09.30.2019 17:50 EDT", "%m.%d.%Y %H:%M %Z")
res = demand.extract_time_from_page(pjm_html)
assert res == expected_time
It passes locally. I'm not sure how this could be different running in a CI environment
Edit: I can reproduce this by changing my machine timezone to something other than EDT. Can you not use a timezone different than your current timezone with datetime.strptime?
This is a known issue with the %Z directive. The current documentation is confusing and there is a pending request to have the documentation revised. The pending documentation change explains the issue you are experiencing:
Note that strptime only accepts certain values for %Z: UTC
and GMT, and what is defined in time.tzname for your own
locales. It will return a ValueError for any invalid strings. For
example, someone living in Japan will have UTC, GMT and
JST as valid values, but probably not EST.
Might be a problem with pytz lib. Compare pytz libs on your CI worker and locally. Here you can find an explanation which points to wikipedia list of timezones which says that EDT is a deprecated timezone. You can try updating your pytz lib on CI worker.
I'm currently trying to writing a script to automate a function at work, but I'm not intimately familiar with Python. I'm trying to take a XML dump and compare a specific entry's date to see if the time has passed or not.
The date is in a particular format, given:
<3-letter Month> <DD> <HH:MM:SS> <YYYY> <3-letter Timezone>
For example:
May 14 20:11:20 2014 GMT
I've parsed out a string in that raw form, and need to somehow compare it with the current time to find out if the time has passed or not. That said, I'm having a bit of trouble figuring out how I should go about either formatting my text, or choosing the right mask/time format in Python.
I've been messing around with different variations of the same basic format:
if(trimmed < time.strftime("%x") ):
Trimmed is the clean date/time string. Time is derived from import time.
Is there a simple way to fix this or will I have to dig into converting the format etc.? I know the above attempt is simplistic, but I'm still very new to Python. Thanks for your time and patience!
You should use combination of gmtime (for GMT time),mktime and datetime.
from time import gmtime,mktime
from datetime import datetime
s = "May 14 20:11:20 2014 GMT"
f = "%b %d %H:%M:%S %Y GMT"
dt = datetime.strptime(s, f)
gmt = datetime.fromtimestamp(mktime(gmtime()))
if dt<gmt:
print(dt)
else:
print(gmt)
Using '%z' pattern of datetime.strptime()
I have a string text that represent a date and I'm perfectly able to parse it and transform it into a clean datetime object:
date = "[24/Aug/2014:17:57:26"
dt = datetime.strptime(date, "[%d/%b/%Y:%H:%M:%S")
Except that I can't catch the entire date string with the timezone using the %z pattern as specified here
date_tz = 24/Aug/2014:17:57:26 +0200
dt = datetime.strptime(date, "[%d/%b/%Y:%H:%M:%S %z]")
>>> ValueError: 'z' is a bad directive in format '[%d/%b/%Y:%H:%M:%S %z]'
Because as this bug report says
strftime() is implemented per platform
I precise that there is no such a problem with the naive tzinfo directive '%Z'
Workaround : Casting tzinfo string into tzinfo object
I can perfectly make the following workaround by transforming the GST time format string into a tzinfo object [as suggested here][4] using dateutil module
and then insert tzinfo into datetime object
Question: Make %z available for my plateform?
But as I will obviously need %z pattern for further project I would like to find a solution to avoid this workaround and using external module for this simple task.
Can you suggest me some reading on it? I supposed that newer version of python (I'm on 2.7) can handle it but I'd rather not changing my version now for this little but crucial detail.
[EDIT]
Well, seeing comments make me reformulated my question how to parse Email time zone indicator using strptime() without being aware of locale time?
strptime() is implemented in pure Python. Unlike strftime(); it [which directives are supported] doesn't depend on platform. %z is supported since Python 3.2:
>>> from datetime import datetime
>>> datetime.strptime('24/Aug/2014:17:57:26 +0200', '%d/%b/%Y:%H:%M:%S %z')
datetime.datetime(2014, 8, 24, 17, 57, 26, tzinfo=datetime.timezone(datetime.timedelta(0, 7200)))
how to parse Email time zone indicator using strptime() without being aware of locale time?
There is no concrete timezone implementation in Python 2.7. You could easily implement the UTC offset parsing, see How to parse dates with -0400 timezone string in python?
In continue to #j-f-sebastians 's answer, here is a fix for python 2.7
Instead of using:
datetime.strptime(t,'%Y-%m-%dT%H:%M %z')
use the timedelta to account for the timezone, like this:
from datetime import datetime,timedelta
def dt_parse(t):
ret = datetime.strptime(t[0:16],'%Y-%m-%dT%H:%M')
if t[17]=='+':
ret-=timedelta(hours=int(t[18:20]),minutes=int(t[20:]))
elif t[17]=='-':
ret+=timedelta(hours=int(t[18:20]),minutes=int(t[20:]))
return ret
print(dt_parse('2017-01-12T14:12 -0530'))
The Answer of Uri is great, saved my life, but when you have
USE_TZ = True you need to be careful with the time, for avoid the warning "RuntimeWarning: DateTimeField" is better if you add the utc to the return.
import pytz
from datetime import datetime, timedelta
def dt_parse(t):
ret = datetime.strptime(t[0:19],'%Y-%m-%dT%H:%M:%S')
if t[23]=='+':
ret-=timedelta(hours=int(t[24:26]), minutes=int(t[27:]))
elif t[23]=='-':
ret+=timedelta(hours=int(t[24:26]), minutes=int(t[27:]))
return ret.replace(tzinfo=pytz.UTC)
I read time stamps from text file. These time stamps are in UTC-4. I need to convert them to US/Eastern.
import datetime
datetime_utc4 = datetime.datetime.strptime("12/31/2012 16:15", "%m/%d/%Y %H:%M")
How do I convert it to US/Eastern? One-line answer would be best.
Note: my original question stated EST to EDT. But it does not change the essence of the question, which is how to go from one time zone to another. Upon some reading (following comments) I gather that python (pytz in particular) does not treat EST and EDT as separate time zones, rather as two flavors of US/Eastern. But this is an implementation detail. It is common to refer to EST and EDT as two different time zones, see e.g. here.
Based on your update and comments, I now understand that you have data that is fixed at UTC-4 and you want to correct this so that it is valid in US Eastern Time, including both EST/EDT where appropriate. Here is how you do that with pytz.
from datetime import datetime
import pytz
dt = datetime.strptime("12/31/2012 16:15", "%m/%d/%Y %H:%M") \
.replace(tzinfo = pytz.FixedOffset(-240)) \
.astimezone(pytz.timezone('America/New_York'))
Note that I used the America/New_York time zone id. This is the most correct form of identifier. You could instead use US/Eastern and it would work just fine, but be aware that this is an alias, and it is just there for backwards compatibility.
I am trying to calculate difference(in seconds) between two date/times formatted as following:
2010-05-11 17:07:33 UTC
2010-05-11 17:07:33 EDT
time1 = '2010-05-11 17:07:33 UTC'
time2 = '2010-05-11 17:07:33 EDT'
delta = time.mktime(time.strptime(time1,"%Y-%m-%d %H:%M:%S %Z"))-\
time.mktime(time.strptime(time2, "%Y-%m-%d %H:%M:%S %Z"))
The problem I got is EDT is not recognized, the specific error is
ValueError: time data '2010-05-11 17:07:33 EDT' does not match format '%Y-%m-%d %H:%M:%S %Z'
Check out the pytz world timezone definitions library.
This library allows accurate and cross platform timezone calculations using Python 2.3 or higher. It also solves the issue of ambiguous times at the end of daylight savings, which you can read more about in the Python Library Reference (datetime.tzinfo).
It takes advantage of the tz database, which should include EDT, and allow you to perform the calculations you need to (and probably more reliably & accurately than your current implementation).
In addition to pytz, check out python-dateutil. The relativedelta functionality is outstanding.
Here's a sample of using them together:
from datetime import datetime
from dateutil.relativedelta import *
import pytz
if __name__ == '__main__':
date_one = datetime.now(pytz.timezone('US/Eastern'))
date_two = datetime.now(pytz.timezone('US/Mountain'))
rdelta = relativedelta(date_one, date_two)
print(rdelta)
From docs for strptime
Support for the %Z directive is based
on the values contained in tzname and
whether daylight is true. Because of
this, it is platform-specific except
for recognizing UTC and GMT which are
always known (and are considered to be
non-daylight savings timezones).