I've been struggling for way too long on dates/timezones in Python and was thinking someone could give me a hand here.
Basically I want to do a conversion in UTC and taking into account DST changes.
I've created the following tzinfo class from one of the Python tutorials (not 100% accurate I know but it doesn't need to):
from datetime import tzinfo, timedelta, datetime
ZERO = timedelta(0)
HOUR = timedelta(hours=1)
def first_sunday_on_or_after(dt):
days_to_go = 6 - dt.weekday()
if days_to_go:
dt += timedelta(days_to_go)
return dt
DSTSTART_2007 = datetime(1, 3, 8, 2)
DSTEND_2007 = datetime(1, 11, 1, 1)
DSTSTART_1987_2006 = datetime(1, 4, 1, 2)
DSTEND_1987_2006 = datetime(1, 10, 25, 1)
DSTSTART_1967_1986 = datetime(1, 4, 24, 2)
DSTEND_1967_1986 = DSTEND_1987_2006
class USTimeZone(tzinfo):
def __init__(self, hours, reprname, stdname, dstname):
self.stdoffset = timedelta(hours=hours)
self.reprname = reprname
self.stdname = stdname
self.dstname = dstname
def __repr__(self):
return self.reprname
def tzname(self, dt):
if self.dst(dt):
return self.dstname
else:
return self.stdname
def utcoffset(self, dt):
return self.stdoffset + self.dst(dt)
def dst(self, dt):
if dt is None or dt.tzinfo is None:
# An exception may be sensible here, in one or both cases.
# It depends on how you want to treat them. The default
# fromutc() implementation (called by the default astimezone()
# implementation) passes a datetime with dt.tzinfo is self.
return ZERO
assert dt.tzinfo is self
# Find start and end times for US DST. For years before 1967, return
# ZERO for no DST.
if 2006 < dt.year:
dststart, dstend = DSTSTART_2007, DSTEND_2007
elif 1986 < dt.year < 2007:
dststart, dstend = DSTSTART_1987_2006, DSTEND_1987_2006
elif 1966 < dt.year < 1987:
dststart, dstend = DSTSTART_1967_1986, DSTEND_1967_1986
else:
return ZERO
start = first_sunday_on_or_after(dststart.replace(year=dt.year))
end = first_sunday_on_or_after(dstend.replace(year=dt.year))
# Can't compare naive to aware objects, so strip the timezone from
# dt first.
if start <= dt.replace(tzinfo=None) < end:
return HOUR
else:
return ZERO
On the other side I have an arbitrary date object in EST, and I want to know the number of hours they differ by taking into account DST.
I've tried something like this:
>>> Eastern = ustimezone.USTimeZone(-5, "Eastern", "EST", "EDT")
>>> x = datetime.date.today() # I actually get an arbitrary date but this is for the example
>>> x_dt = datetime.datetime.combine(x, datetime.time())
>>> x_dt_tz = x_dt.astimezone(Eastern)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: astimezone() cannot be applied to a naive datetime
I've seen several posts who tell to use localize from the pytz module, but unfortunately I am not able to use additional modules, so impossible to use pyzt
Does anyone know how I can get this naive datetime into a timezoned object without using pytz?
For what it's worth, the answer #skyl provided is more-or-less equivalent to what pytz does.
Here is the relevant pytz source. It just calls replace on the datetime object with the tzinfo kwarg:
def localize(self, dt, is_dst=False):
'''Convert naive time to local time'''
if dt.tzinfo is not None:
raise ValueError('Not naive datetime (tzinfo is already set)')
return dt.replace(tzinfo=self)
Use x_dt.replace(tzinfo=Eastern) (found from this Google Groups thread).
x_dt.replace(tzinfo=Eastern).utcoffset() returns datetime.timedelta(-1, 72000) which corresponds to -4 hours! (from Question's comment)
Related
Somewhat related to this post: dateutil parser for month/year format: return beginning of month
Given a date string of the form 'Sep-2020', dateutil.parser.parse correctly identifies the month and the year but adds the day as well. If a default is provided, it takes the day from it. Else, it will just use today's day. Is there anyway to tell if the parser used any of the default terms?
For example, how can I tell from the three options below that the input date string in the first case did not include day and that the default value was used?
>>> from datetime import datetime
>>> from dateutil import parser
>>> d = datetime(1978, 1, 1, 0, 0)
>>> parser.parse('Sep-2020', default=d)
datetime.datetime(2020, 9, 1, 0, 0)
>>> parser.parse('1-Sep-2020', default=d)
datetime.datetime(2020, 9, 1, 0, 0)
>>> parser.parse('Sep-1-2020', default=d)
datetime.datetime(2020, 9, 1, 0, 0)
``
I did something a little mad to solve this. It's mad since it's not guaranteed to work with future versions of dateutil (since it's relying on some dateutil internals).
Currently I'm using: python-dateutil 2.8.1.
I wrote my own class and passed it as default to the parser:
from datetime import datetime
class SentinelDateTime:
def __init__(self, year=0, month=0, day=0, default=None):
self._year = year
self._month = month
self._day = day
if default is None:
default = datetime.now().replace(
hour=0, minute=0,
second=0, microsecond=0
)
self.year = default.year
self.month = default.month
self.day = default.day
self.default = default
#property
def has_year(self):
return self._year != 0
#property
def has_month(self):
return self._month != 0
#property
def has_day(self):
return self._day != 0
def todatetime(self):
res = {
attr: value
for attr, value in [
("year", self._year),
("month", self._month),
("day", self._day),
] if value
}
return self.default.replace(**res)
def replace(self, **result):
return SentinelDateTime(**result, default=self.default)
def __repr__(self):
return "%s(%d, %d, %d)" % (
self.__class__.__qualname__,
self._year,
self._month,
self._day
)
The dateutils method now returns this SentinelDateTime class:
>>> from dateutil import parser
>>> from datetime import datetime
>>> from snippet1 import SentinelDateTime
>>>
>>> sentinel = SentinelDateTime()
>>> s = parser.parse('Sep-2020', default=sentinel)
>>> s
SentinelDateTime(2020, 9, 0)
>>> s.has_day
False
>>> s.todatetime()
datetime.datetime(2020, 9, 9, 0, 0)
>>> d = datetime(1978, 1, 1)
>>> sentinel = SentinelDateTime(default=d)
>>> s = parser.parse('Sep-2020', default=sentinel)
>>> s
SentinelDateTime(2020, 9, 0)
>>> s.has_day
False
>>> s.todatetime()
datetime.datetime(2020, 9, 1, 0, 0)
I wrote this answer into a little package: https://github.com/foxyblue/sentinel-datetime
I have found a solution that's a little less complicated:
from datetime import datetime
from dataclasses import dataclass
from dateutil import parser
#dataclass
class Result:
dt: datetime
data: dict
class subparser(parser.parser):
def _build_naive(self, res, default):
naive = super()._build_naive(res, default)
return Result(dt=naive, data=res)
In an example:
>>> PARSER = subparser()
>>> info = PARSER.parse("2020")
>>> info.data.year)
2020
>>> info.data.month
None
>>> info.dt
2020-01-10 00:00:00
I am searching for a way to parse a rather unusual timestamp string to a Python datetime object. The problem here is, that this string includes the corresponding quarter, which seems not to be supported by the datetime.strptime function. The format of the string is as follows: YYYY/qq/mm/dd/HH/MM e.g 1970/Q1/01/01/00/00. I am searching for a function, which is allows me to parse string in such a format, including a validity check, if the quarter is correct for the date.
Question: Datetime String with Quarter to Python datetime
This implements a OOP solution which extends Python datetime with a directive: %Q.
Possible values: Q1|Q2|Q3|Q4, for example:
data_string = '1970/Q1/01/01/00/00'
# '%Y/%Q/%m/%d/%H/%M'
Note: This depends on the module _strptime class TimeRE and may fail if the internal implementation changes!
from datetime import datetime
class Qdatetime(datetime):
re_compile = None
#classmethod
def _strptime(cls):
import _strptime
_class = _strptime.TimeRE
if not 'strptime_compile' in _class.__dict__:
setattr(_class, 'strptime_compile', getattr(_class, 'compile'))
setattr(_class, 'compile', cls.compile)
def compile(self, format):
import _strptime
self = _strptime._TimeRE_cache
# Add directive %Q
if not 'Q' in self:
self.update({'Q': r"(?P<Q>Q[1-4])"})
Qdatetime.re_compile = self.strptime_compile(format)
return Qdatetime.re_compile
def validate(self, quarter):
# 1970, 1, 1 is the lowest date used in timestamp
month = [1, 4, 7, 10][quarter - 1]
day = [31, 30, 30, 31][quarter - 1]
q_start = datetime(self.year, month, 1).timestamp()
q_end = datetime(self.year, month + 2, day).timestamp()
dtt = self.timestamp()
return dtt >= q_start and dtt<= q_end
#property
def quarter(self): return self._quarter
#quarter.setter
def quarter(self, data):
found_dict = Qdatetime.re_compile.match(data).groupdict()
self._quarter = int(found_dict['Q'][1])
#property
def datetime(self):
return datetime(self.year, self.month, self.day,
hour=self.hour, minute=self.minute, second=self.second)
def __str__(self):
return 'Q{} {}'.format(self.quarter, super().__str__())
#classmethod
def strptime(cls, data_string, _format):
cls._strptime()
dt = super().strptime(data_string, _format)
dt.quarter = data_string
if not dt.validate(dt.quarter):
raise ValueError("time data '{}' does not match quarter 'Q{}'"\
.format(data_string, dt.quarter))
return dt
Usage:
for data_string in ['1970/Q1/01/01/00/00',
'1970/Q3/12/31/00/00',
'1970/Q2/05/05/00/00',
'1970/Q3/07/01/00/00',
'1970/Q4/12/31/00/00',
]:
try:
d = Qdatetime.strptime(data_string, '%Y/%Q/%m/%d/%H/%M')
except ValueError as e:
print(e)
else:
print(d, d.datetime)
Output:
Q1 1970-01-01 00:00:00 1970-01-01 00:00:00
time data '1970/Q3/12/31/00/00' does not match quarter 'Q3'
Q2 1970-05-05 00:00:00 1970-05-05 00:00:00
Q3 1970-07-01 00:00:00 1970-07-01 00:00:00
Q4 1970-12-31 00:00:00 1970-12-31 00:00:00
Tested with Python: 3.6 - verified with Python 3.8 source
I want to find out if a entry has been updated in the last 6 months.
This is what I have tried:
def is_old(self):
"""
Is older than 6 months (since last update)
"""
time_threshold = datetime.date.today() - datetime.timedelta(6*365/12)
if self.last_update < time_threshold:
return False
return True
but i get the error:
if self.last_update < time_threshold:
TypeError: can't compare datetime.datetime to datetime.date
You need the days keyword
>>> import datetime
>>> datetime.date.today() - datetime.timedelta(days=30)
datetime.date(2014, 5, 26)
>>> datetime.date.today() - datetime.timedelta(days=180)
datetime.date(2013, 12, 27)
>>> datetime.date.today() - datetime.timedelta(days=6*365/12)
datetime.date(2013, 12, 25)
Also, coming to your actual error: TypeError: can't compare datetime.datetime to datetime.date
You can just do
def is_old(self):
time_threshold = datetime.date.today() - datetime.timedelta(days=6*365/12)
#The following code can be simplified, i shall let you figure that out yourself.
if self.last_update and self.last_update.date() < time_threshold:
return False
return True
Your database field last_update is datetime field and you are comparing it against date hence the error, Instead of datetime.date.today() use datetime.datetime.now(). Better use django.utils.timezone which will respect the TIME_ZONE in settings:
from django.utils import timezone
def is_old(self):
"""
Is older than 6 months (since last update)
"""
time_threshold = timezone.now() - datetime.timedelta(6*365/12)
return bool(self.last_update > time_threshold)
You can use the external module dateutil:
from dateutil.relativedelta import relativedelta
def is_old(last_update):
time_threshold = date.today() - relativedelta(months=6)
return last_update < time_threshold
That's assuming the type of last_update is a date object
I'm playing with a function in Python 3 that queries small blocks of XML from the eBird API, parsing them with minidom. The function locates and compares dates from two requested blocks of XML, returning the most recent. The code below does its job, but I wanted to ask if there was a simpler way of doing this (the for loops seem unnecessary since each bit of XML will only ever have one date, and comparing pieces of the returned string bit by bit seems clunky). Is there a faster way to produce the same result?
from xml.dom import minidom
import requests
def report(owl):
#GETS THE MOST RECENT OBSERVATION FROM BOTH USA AND CANADA
usa_xml = requests.get('http://ebird.org/ws1.1/data/obs/region_spp/recent?rtype=country&r=US&sci=surnia%20ulula&back=30&maxResults=1&includeProvisional=true')
canada_xml = requests.get('http://ebird.org/ws1.1/data/obs/region_spp/recent?rtype=country&r=CA&sci=surnia%20ulula&back=30&maxResults=1&includeProvisional=true')
usa_parsed = minidom.parseString(usa_xml.text)
canada_parsed = minidom.parseString(canada_xml.text)
#COMPARES THE RESULTS AND RETURNS THE MOST RECENT
usa_raw_date = usa_parsed.getElementsByTagName('obs-dt')
canada_raw_date = canada_parsed.getElementsByTagName('obs-dt')
for date in usa_raw_date:
usa_date = str(date.childNodes[0].nodeValue)
for date in canada_raw_date:
canada_date = str(date.childNodes[0].nodeValue)
if int(usa_date[0:4]) > int(canada_date[0:4]):
most_recent = usa_date
elif int(usa_date[5:7]) > int(canada_date[5:7]):
most_recent = usa_date
elif int(usa_date[8:10]) > int(canada_date[8:10]):
most_recent = usa_date
elif int(usa_date[11:13]) > int(canada_date[11:13]):
most_recent = usa_date
elif int(usa_date[14:16]) > int(canada_date[14:16]):
most_recent = usa_date
else:
most_recent = canada_date
return most_recent
Use the datetime.datetime.strftime() to parse the dates into datetime.datetime() objects, then us max() to return the greater value (most recent):
usa_date = datetime.datetime.strptime(
usa_raw_date[-1].childNodes[0].nodeValue, '%Y-%m-%d %H:%M')
canada_date = datetime.datetime.strptime(
canada_raw_date[-1].childNodes[0].nodeValue, '%Y-%m-%d %H:%M')
return max(usa_date, canada_date)
Running this now against the URLs you provided, that results in:
>>> usa_date = datetime.datetime.strptime(
... usa_raw_date[-1].childNodes[0].nodeValue, '%Y-%m-%d %H:%M')
>>> canada_date = datetime.datetime.strptime(
... canada_raw_date[-1].childNodes[0].nodeValue, '%Y-%m-%d %H:%M')
>>> usa_date, canada_date
(datetime.datetime(2014, 5, 5, 11, 0), datetime.datetime(2014, 5, 11, 18, 0))
>>> max(usa_date, canada_date)
datetime.datetime(2014, 5, 11, 18, 0)
This returns a datetime.datetime() object; if returning a string is important to you, you can always still return:
return max(usa_date, canada_date).strftime('%Y-%m-%d %H:%M')
e.g. format the datetime object to a string again.
How to increment the day of a datetime?
for i in range(1, 35)
date = datetime.datetime(2003, 8, i)
print(date)
But I need pass through months and years correctly? Any ideas?
date = datetime.datetime(2003,8,1,12,4,5)
for i in range(5):
date += datetime.timedelta(days=1)
print(date)
Incrementing dates can be accomplished using timedelta objects:
import datetime
datetime.datetime.now() + datetime.timedelta(days=1)
Look up timedelta objects in the Python docs: http://docs.python.org/library/datetime.html
All of the current answers are wrong in some cases as they do not consider that timezones change their offset relative to UTC. So in some cases adding 24h is different from adding a calendar day.
Proposed solution
The following solution works for Samoa and keeps the local time constant.
def add_day(today):
"""
Add a day to the current day.
This takes care of historic offset changes and DST.
Parameters
----------
today : timezone-aware datetime object
Returns
-------
tomorrow : timezone-aware datetime object
"""
today_utc = today.astimezone(datetime.timezone.utc)
tz = today.tzinfo
tomorrow_utc = today_utc + datetime.timedelta(days=1)
tomorrow_utc_tz = tomorrow_utc.astimezone(tz)
tomorrow_utc_tz = tomorrow_utc_tz.replace(hour=today.hour,
minute=today.minute,
second=today.second)
return tomorrow_utc_tz
Tested Code
# core modules
import datetime
# 3rd party modules
import pytz
# add_day methods
def add_day(today):
"""
Add a day to the current day.
This takes care of historic offset changes and DST.
Parameters
----------
today : timezone-aware datetime object
Returns
-------
tomorrow : timezone-aware datetime object
"""
today_utc = today.astimezone(datetime.timezone.utc)
tz = today.tzinfo
tomorrow_utc = today_utc + datetime.timedelta(days=1)
tomorrow_utc_tz = tomorrow_utc.astimezone(tz)
tomorrow_utc_tz = tomorrow_utc_tz.replace(hour=today.hour,
minute=today.minute,
second=today.second)
return tomorrow_utc_tz
def add_day_datetime_timedelta_conversion(today):
# Correct for Samoa, but dst shift
today_utc = today.astimezone(datetime.timezone.utc)
tz = today.tzinfo
tomorrow_utc = today_utc + datetime.timedelta(days=1)
tomorrow_utc_tz = tomorrow_utc.astimezone(tz)
return tomorrow_utc_tz
def add_day_dateutil_relativedelta(today):
# WRONG!
from dateutil.relativedelta import relativedelta
return today + relativedelta(days=1)
def add_day_datetime_timedelta(today):
# WRONG!
return today + datetime.timedelta(days=1)
# Test cases
def test_samoa(add_day):
"""
Test if add_day properly increases the calendar day for Samoa.
Due to economic considerations, Samoa went from 2011-12-30 10:00-11:00
to 2011-12-30 10:00+13:00. Hence the country skipped 2011-12-30 in its
local time.
See https://stackoverflow.com/q/52084423/562769
A common wrong result here is 2011-12-30T23:59:00-10:00. This date never
happened in Samoa.
"""
tz = pytz.timezone('Pacific/Apia')
today_utc = datetime.datetime(2011, 12, 30, 9, 59,
tzinfo=datetime.timezone.utc)
today_tz = today_utc.astimezone(tz) # 2011-12-29T23:59:00-10:00
tomorrow = add_day(today_tz)
return tomorrow.isoformat() == '2011-12-31T23:59:00+14:00'
def test_dst(add_day):
"""Test if add_day properly increases the calendar day if DST happens."""
tz = pytz.timezone('Europe/Berlin')
today_utc = datetime.datetime(2018, 3, 25, 0, 59,
tzinfo=datetime.timezone.utc)
today_tz = today_utc.astimezone(tz) # 2018-03-25T01:59:00+01:00
tomorrow = add_day(today_tz)
return tomorrow.isoformat() == '2018-03-26T01:59:00+02:00'
to_test = [(add_day_dateutil_relativedelta, 'relativedelta'),
(add_day_datetime_timedelta, 'timedelta'),
(add_day_datetime_timedelta_conversion, 'timedelta+conversion'),
(add_day, 'timedelta+conversion+dst')]
print('{:<25}: {:>5} {:>5}'.format('Method', 'Samoa', 'DST'))
for method, name in to_test:
print('{:<25}: {:>5} {:>5}'
.format(name,
test_samoa(method),
test_dst(method)))
Test results
Method : Samoa DST
relativedelta : 0 0
timedelta : 0 0
timedelta+conversion : 1 0
timedelta+conversion+dst : 1 1
Here is another method to add days on date using dateutil's relativedelta.
from datetime import datetime
from dateutil.relativedelta import relativedelta
print 'Today: ',datetime.now().strftime('%d/%m/%Y %H:%M:%S')
date_after_month = datetime.now()+ relativedelta(day=1)
print 'After a Days:', date_after_month.strftime('%d/%m/%Y %H:%M:%S')
Output:
Today: 25/06/2015 20:41:44
After a Days: 01/06/2015 20:41:44
Most Simplest solution
from datetime import timedelta, datetime
date = datetime(2003,8,1,12,4,5)
for i in range(5):
date += timedelta(days=1)
print(date)
This was a straightforward solution for me:
from datetime import timedelta, datetime
today = datetime.today().strftime("%Y-%m-%d")
tomorrow = datetime.today() + timedelta(1)
You can also import timedelta so the code is cleaner.
from datetime import datetime, timedelta
date = datetime.now() + timedelta(seconds=[delta_value])
Then convert to date to string
date = date.strftime('%Y-%m-%d %H:%M:%S')
Python one liner is
date = (datetime.now() + timedelta(seconds=[delta_value])).strftime('%Y-%m-%d %H:%M:%S')
A short solution without libraries at all. :)
d = "8/16/18"
day_value = d[(d.find('/')+1):d.find('/18')]
tomorrow = f"{d[0:d.find('/')]}/{int(day_value)+1}{d[d.find('/18'):len(d)]}".format()
print(tomorrow)
# 8/17/18
Make sure that "string d" is actually in the form of %m/%d/%Y so that you won't have problems transitioning from one month to the next.