Develop Reference

datetime.combine with timezone is different from datetime.now with timezone

In the below code:
from datetime import datetime
import pytz
EDT = pytz.timezone('US/Eastern')
d1 = datetime.now(tz=EDT)
d2 = datetime.combine(d1.date(), d1.time(), tzinfo=EDT)
Why are d1 and d2 showing different timezone information?
>> d1
datetime.datetime(2021, 4, 8, 7, 0, 44, 316514, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
>> d2
datetime.datetime(2021, 4, 8, 7, 0, 44, 316514, tzinfo=<DstTzInfo 'US/Eastern' LMT-1 day, 19:04:00 STD>)
How do I get the same datetime as datetime.now when using datetime.combine?

datetime.now effectively converts (localizes) your datetime with the pytz timezone object - from the docs:
If tz is not None, it must be an instance of a tzinfo subclass, and
the current date and time are converted to tz’s time zone.
datetime.combine does not do that. It is as if you would write something like datetime(2020,1,1, tzinfo=pytz.timezone('US/Eastern')) - effectively not adjusting the time zone to the provided date/time. See also e.g. pytz localize vs datetime replace and pytz: The Fastest Footgun in the West for more background info.
The correct way to get d2 with pytz would be
d2 = EDT.localize(datetime.combine(d1.date(), d1.time()))
No such issues if using timezone objects from dateutil or zoneinfo (Python 3.9+):
from datetime import datetime
from zoneinfo import ZoneInfo
EDT = ZoneInfo('US/Eastern')
d1 = datetime.now(tz=EDT)
d2 = datetime.combine(d1.date(), d1.time(), tzinfo=EDT)
# d1
# Out[75]: datetime.datetime(2021, 4, 8, 7, 57, 18, 309209, tzinfo=zoneinfo.ZoneInfo(key='US/Eastern'))
# d2
# Out[76]: datetime.datetime(2021, 4, 8, 7, 57, 18, 309209, tzinfo=zoneinfo.ZoneInfo(key='US/Eastern'))
# d1 == d2
# Out[77]: True

datetime.now(tz) and datetime(year, month, day, tzinfo=tz) don't have same UTC offset

Why are the UTC offsets in the 2 datetimes created in the code below different? I'm using pytz==2019.1
from datetime import datetime
import pytz
EASTERN = pytz.timezone('US/Eastern')
dt1 = datetime.now(EASTERN)
dt2 = datetime(2020, 8, 11, 15, 30, tzinfo=EASTERN)
print(f'dt1: {dt1}')
print(f'dt2: {dt2}')
results:
dt1: 2020-08-11 18:35:47.984745-04:00
dt2: 2020-08-11 15:30:00-04:56
The first one shows an UTC offset of -04:00 which is correct (for this time of year), but the 2nd one is giving an UTC offset of -04:56.
How can I declare a datetime with the 2nd method and get the UTC offset from the first method.

The issue with the shown code is that datetime.now(tz) can handle a pytz.timezone as tz argument while the default constructor datetime(y,m,d,...) cannot. In this case, you have to use the localize method of the timezone,
from datetime import datetime
import pytz
EASTERN = pytz.timezone('US/Eastern')
dt1 = datetime.now(EASTERN)
dt2 = EASTERN.localize(datetime(2020, 8, 11, 15, 30))
print(f'dt1: {dt1}')
print(f'dt2: {dt2}')
# prints
# dt1: 2020-08-12 10:07:09.738763-04:00
# dt2: 2020-08-11 15:30:00-04:00
dateutil avoids this issue, more info can be found here. That would make the code work "as it is":
from dateutil.tz import gettz
EASTERN = gettz('US/Eastern')
dt1 = datetime.now(EASTERN)
dt2 = datetime(2020, 8, 11, 15, 30, tzinfo=EASTERN)
print(f'dt1: {dt1}')
print(f'dt2: {dt2}')
# prints e.g.
# dt1: 2020-08-12 10:20:07.796811-04:00
# dt2: 2020-08-11 15:30:00-04:00
In addition pytz is likely to be deprecated with Python 3.9 as you'll have zoneinfo as part of the standard library for these kinds of jobs.

with regards to dt1:
datetime.now() will return the tzinfo from utc. You can see this if you do the following:
>>> repr(EASTERN.fromutc(datetime.now()))
"datetime.datetime(2020, 8, 11, 12, 33, 28, 849873, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)"
the tz.fromutc() is what datetime.now() does behind the scenes.
with regards to dt2:
if you check the repr of EASTERN you will see the following:
>>> repr(EASTERN)
"<DstTzInfo 'US/Eastern' LMT-1 day, 19:04:00 STD>"
>>> repr(dt2)
"datetime.datetime(2020, 8, 11, 15, 30, tzinfo=<DstTzInfo 'US/Eastern' LMT-1 day, 19:04:00 STD>)"
I'm not very comfortable with timezone differences so I don't want to steer you in the wrong direction, but from what I can tell both of these two options yield the same result:
dt2 = datetime(2020, 8, 11, 15, 30, tzinfo=EASTERN)
dt2_eastern_corrected = EASTERN.fromutc(dt2)
dt3 = datetime(2020, 8, 11, 15, 30)
dt3_eastern_corrected = EASTERN.fromutc(dt3
>>> repr(dt2_eastern_corrected)
"datetime.datetime(2020, 8, 11, 11, 30, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)"
>>> repr(dt3_eastern_corrected)
"datetime.datetime(2020, 8, 11, 11, 30, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)"

Deal with Birtish summer time

I have a date string 18 May 14:30 which corresponds to the British summertime (WEST or UTC+1). I would like to convert it to central Euopean (summer)time.
Here is my code
# from datetime import datetime
# from pytz import timezone
d = '18 May 14:30'
# Attempt 1
dd=datetime.strptime(d, '%d %b %H:%M').replace(year=datetime.now().year, tzinfo=timezone('WET'))
dd.astimezone(timezone('CET'))
# datetime.datetime(2019, 5, 18, 16, 30, tzinfo=<DstTzInfo 'CET' CEST+2:00:00 DST>)
# It should be 15:30, not 16:30
# Attempt 2
dd=datetime.strptime(d, '%d %b %H:%M').replace(year=datetime.now().year, tzinfo=timezone('WET'))
# Same result as above
# Attempt 3
dd=datetime.strptime(d, '%d %b %H:%M').replace(year=datetime.now().year, tzinfo=timezone('Etc/GMT-1'))
dd.astimezone(timezone('CET'))
# datetime.datetime(2019, 5, 18, 15, 30, tzinfo=<DstTzInfo 'CET' CEST+2:00:00 DST>)
# This works
So my problem in the third attempt I had to manually specify GMT-1 whereas CET automatically transforms to CEST. I hoped this would work identically for WET (to WEST).
Besides, what also confuses me is the fact that according to Wiki British summertime should be UTC +1 but I had to set GMT-1 (as GMT+1 returns 18:30).

In case it interests someone, I did manage to find a workaround
dd=datetime.strptime(d, '%d %b %H:%M').replace(year=datetime.now().year)
timezone('WET').localize(dd).astimezone(timezone('CET'))
# datetime.datetime(2019, 5, 18, 15, 30, tzinfo=<DstTzInfo 'CET' CEST+2:00:00 DST>)
# Correct result without having to specify the shift
# Let's check with another date (non-summertime)
d = '18 Jan 14:30'
dd=datetime.strptime(d, '%d %b %H:%M').replace(year=datetime.now().year)
timezone('WET').localize(dd).astimezone(timezone('CET'))
# datetime.datetime(2019, 1, 18, 15, 30, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)
# Yay!
It's not very elegant but at least it does the job.

Saving a local datetime offset the time by 4 minutes

I'm trying to modify a datetime based on a timezone on save and on load the following way:
An input datetime, along with a input timezone are sent to the server and the server should update the datetime to reflect the timezone. So when it saves in the database (PostregSQL), the UTC time is saved (after the offset caused by the timezone, of course).
To reflect this here's a simpler example that fails in the same way:
Some imports:
>>> import datetime
>>> import pytz
>>> from apps.myapp.models import Project
Creating the two inputs:
>>> input_date = timezone.now()
>>> input_date
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<UTC>)
>>> current_tz = pytz.timezone('America/New_York')
>>> current_tz
<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>
As you can see, the timezone is not 5h (24 - 19 = 5), but 4h56. At this stage I'm thinking that's OK, it may be related to the Daylight Saving Time.
Now I'm replacing the timezone on the input date:
>>> input_date = input_date.replace(tzinfo=current_tz)
>>> input_date
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
As expected, the time hasn't changed, but the timezone has, which is fine.
I'll assign this value to a project (the launch_date is a DateTimeField without any specific option):
>>> project = Project.objects.get(pk=1)
>>> project.launch_date
datetime.datetime(2017, 1, 14, 8, 53, 57, 241718, tzinfo=<UTC>)
>>> project.launch_date = input_date
>>> project.launch_date
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
Now I'll save this into (and refresh from) the database, leaving Django/PostgreSQL do the maths:
>>> project.save()
>>> project.refresh_from_db()
>>> project.launch_date
datetime.datetime(2017, 2, 7, 21, 3, 14, 377429, tzinfo=<UTC>)
As expected the date is now 4h56 ahead of the previous date. I'm trying now to get back the local time:
>>> project.launch_date.astimezone(current_tz)
datetime.datetime(2017, 2, 7, 16, 3, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' EST-1 day, 19:00:00 STD>)
>>> input_date
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
This time, the offset is perfectly 5h. And I'm missing 4 minutes.
3 questions here:
Where is this 4 minutes coming from?
Why is astimezone not using the 4 minutes as well?
How can a datetime be converted to UTC, saved, loaded and converted back to local?

pytz time zones are a little weird, as you can see by multiple questions on StackOverflow. They often won't show the correct offset or timezone name unless they are allowed to adjust themselves to the datetime they are being paired with. Here is what the documentation has to say:
This library only supports two ways of building a localized time. The first is to use the localize() method provided by the pytz library. This is used to localize a naive datetime (datetime with no timezone information):
The second way of building a localized time is by converting an existing localized time using the standard astimezone() method:
Unfortunately using the tzinfo argument of the standard datetime constructors “does not work” with pytz for many timezones.
It does not say so explicitly, but using replace has the same problem as using the datetime constructor.
To accomplish what your code was doing without the 4-minute discrepancy, you can use localize():
>>> input_date
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<UTC>)
>>> current_tz.localize(input_date.replace(tzinfo=None))
datetime.datetime(2017, 2, 7, 16, 7, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' EST-1 day, 19:00:00 STD>)
I suspect that's a bug though, and you really want to do a timezone conversion from UTC:
>>> input_date.astimezone(current_tz)
datetime.datetime(2017, 2, 7, 11, 7, 14, 377429, tzinfo=<DstTzInfo 'America/New_York' EST-1 day, 19:00:00 STD>)

pytz and Etc/GMT-5

I'm having trouble understanding the conversion between the "Etc/GMT-5" timezone and UTC in pytz.
>>> dt = datetime(2009, 9, 9, 10, 0) # September 9 2009, 10:00
>>> gmt_5 = pytz.timezone("Etc/GMT-5")
>>> gmt_5.localize(dt)
datetime.datetime(2009, 9, 9, 10, 0, tzinfo=<StaticTzInfo 'Etc/GMT-5'>)
Everything is fine so far, but then I try to convert that to UTC:
>>> gmt_5.localize(dt).astimezone(pytz.utc)
datetime.datetime(2009, 9, 9, 5, 0, tzinfo=<UTC>)
So to me it seems that when converting from 10:00 in GMT-5 to UTC I get 05:00? I would expect pytz to give me 15:00 instead.
What am I missing?
EDIT: I have confirmed that timezone conversion for the US/Eastern timezone works just as I'd expect:
>>> eastern = pytz.timezone("US/Eastern")
>>> eastern.localize(dt)
datetime.datetime(2009, 9, 9, 10, 0, tzinfo=...) # Too long
>>> pytz.utc.normalize(eastern.localize(dt).astimezone(pytz.utc))
datetime.datetime(2009, 9, 9, 14, 0, tzinfo=<UTC>)
EDIT 2: I have confirmed that when I use Etc/GMT+5 I get 15:00, which is what I'd expect to get from Etc/GMT-5. Is this a pytz bug?

This is apparently a POSIX thing. From Wikipedia:
In order to conform with the POSIX style, those zones beginning with "Etc/GMT" have their sign reversed from what most people expect. In this style, zones west of GMT have a positive sign and those east have a negative sign.

This bug report explains this behavior. Apparently they know that it is all inverted, but that's because anything else would break compatibility.