Develop Reference

Odd timezone behaviour in python (pytz)

I'm encountering a strange issue with timezones in python that I've boiled down to a few lines of code:
from datetime import time, datetime
import pytz
tz = pytz.timezone('Canada/Pacific')
d = datetime.now(tz=tz)
t = time(tzinfo=tz)
When inspecting the tzinfo objects in d and t, the one in t gives a timezone with a utc-offset of 8:12, while the one in d gives a utc-offset of just 8 hours, which is the correct offset. The tz instance also gives a utc-offset of 8:12. What's with the extra 12 minutes?
I realize that datetime.now() is dependent on when you run the code, so I will say that I ran it a few minutes before posting this on StackOverflow and saw the same error. Can someone more knowledgable on timezones help me figure out what the problem is?

pytz is indeed strange. The standard python way since year 2018 is
import datetime
import zoneinfo
ca_pa = zoneinfo.ZoneInfo("Canada/Pacific")
datetime.datetime.now(ca_pa)
Do not use pytz anymore.

#Ruli the answer is just to read the documentation.
tz = pytz.timezone('Canada/Pacific')
dt = tz.normalize(datetime.now())
t = tz.normalize(time())
is the correct way of creating local times.

Timezone Timestamp Weirdness

I am finding a very strange behavior for different timezones in Python. The following code:
import datetime as dtm
import pytz
def test_tz():
america_ny_tz: dtm.tzinfo = pytz.timezone("America/New_York")
est_tz: dtm.tzinfo = pytz.timezone("EST5EDT")
today = dtm.date.today()
ny_dtm = dtm.datetime(
year=today.year, month=today.month, day=today.day, tzinfo=america_ny_tz
)
est_dtm = dtm.datetime(
year=today.year, month=today.month, day=today.day, tzinfo=est_tz
)
print(f"New York: {ny_dtm.timestamp()}, EST: {est_dtm.timestamp()}")
if __name__ == "__main__":
test_tz()
outputs:
New York: 1609995360.0, EST: 1609995600.0
As you may notice the difference is about 4 minutes where one would have to assume the time should be the same.
Am I accessing time zone information incorrectly or is my understanding that the Time Zone information should be the same?
Running on Linux, Ubuntu 20.04 but the behavior on 18.04 was the same.
P.S. I haven't tried other languages or different OSs to see if the behavior will be the same.

You need to localize correctly - with pytz's timezone objects, setting tzinfo directly when creating the datetime object is not the right way. It's in the docs:
This library only supports two ways of building a localized time. The first is to use the localize() method provided by the pytz library.
and
The second way of building a localized time is by converting an existing localized time using the standard astimezone() method
On the other hand, if you happen to use Python 3.9+, you can use the zoneinfo module from the standard lib to make you life easier - see the docs / using-zoneinfo or example usage here.

This seems to be because one timezone uses Local Mean Time and the other is an offset of UTC (Universal Coordinated Time).
>>> pytz.timezone("America/New_York")
<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>
>>> pytz.timezone("EST5EDT")
<DstTzInfo 'EST5EDT' EST-1 day, 19:00:00 STD>
>>> pytz.timezone("LMT")
From https://www.timeanddate.com/time/local-mean-time.html
Local Mean Time was officially used as civil time in many countries
during the 19th century. Each city had a different local time defined
by its longitude, the difference amounting to 4 minutes per degree
longtitude. This equals a distance of 50 miles or 81 kilometers on New
York's latitude.

Python convert timestamps with specific timezone to datetime in UTC

I'm trying to convert a timestamp with a specific timezone(Europe/Paris) to a datetime format in UTC.
From my laptop it works with the solution below but when I'm executing my code in a remote server(AWS- Lambda function in Ireland), I've a shift of 1 hour because the local timezone of the server is different from mine.
How can I have a code who can work on my laptop and at the same time in a remote server(dynamically handle local timezone)?
import pytz
import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
utc = pytz.timezone('UTC')
now_in_utc = datetime.datetime.utcnow().replace(tzinfo=utc).astimezone(pytz.UTC)
fr = pytz.timezone('Europe/Paris')
new_date = datetime.datetime.fromtimestamp(timestamp_received)
return fr.localize(new_date, is_dst=None).astimezone(pytz.UTC)
Thanks

I am not sure what timestamp_received is, but I think what you want is utcfromtimestamp()
import pytz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_naive_utc = datetime.utcfromtimestamp(timestamp_received)
return dt_naive_utc.replace(tzinfo=pytz.utc)
For completeness, here is another way to accomplish the same thing by referencing python-dateutil's tzlocal time zone:
from dateutil import tz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_local = datetime.fromtimestamp(timestamp_received, tz.tzlocal())
if tz.datetime_ambiguous(dt_local):
raise AmbiguousTimeError
if tz.datetime_imaginary(dt_local):
raise ImaginaryTimeError
return dt_local.astimezone(tz.tzutc())
class AmbiguousTimeError(ValueError):
pass
class ImaginaryTimeError(ValueError):
pass
(I added in the AmbiguousTimeError and ImaginaryTimeError conditions to mimic the pytz interface.) Note that I'm including this just in case you have a similar problem that needs to make reference to the local time zone for some reason - if you have something that will give you the right answer in UTC, it's best to use that and then use astimezone to get it into whatever local zone you want it in.
How it works
Since you expressed that you were still a bit confused about how this works in the comments, I thought I would clarify why this works. There are two functions that convert timestamps to datetime.datetime objects, datetime.datetime.fromtimestamp(timestamp, tz=None) and datetime.datetime.utcfromtimestamp(timestamp):
utcfromtimestamp(timestamp) will give you a naive datetime that represents the time in UTC. You can then do dt.replace(tzinfo=pytz.utc) (or any other utc implementation - datetime.timezone.utc, dateutil.tz.tzutc(), etc) to get an aware datetime and convert it to whatever time zone you want.
fromtimestamp(timestamp, tz=None), when tz is not None, will give you an aware datetime equivalent to utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(tz). If tz is None, instead of converting too the specified time zone, it converts to your local time (equivalent to dateutil.tz.tzlocal()), and then returns a naive datetime.
Starting in Python 3.6, you can use datetime.datetime.astimezone(tz=None) on naive datetimes, and the time zone will be assumed to be system local time. So if you're developing a Python >= 3.6 application or library, you can use datetime.fromtimestamp(timestamp).astimezone(whatever_timezone) or datetime.utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(whatever_timezone) as equivalents.

Using python to determine if a timestamp is under daylight savings time

This does not work:
t = os.path.getmtime(filename)
dTime = datetime.datetime.fromtimestamp(t)
justTime = dTime.timetuple()
if justTime.tm_isdst == 0 :
tDelta = datetime.timedelta(hours=0)
else:
tDelta = datetime.timedelta(hours=1)
What happens instead is that the conditional always equals 1, despite some of the timestamps being within daytime savings time.
I am trying to make a python call match the behavior of a c-based call.

To find out whether a given timestamp corresponds to daylight saving time (DST) in the local time zone:
import os
import time
timestamp = os.path.getmtime(filename)
isdst = time.localtime(timestamp).tm_isdst > 0
It may fail. To workaround the issues, you could get a portable access to the tz database using pytz module:
from datetime import datetime
import tzlocal # $ pip install tzlocal
local_timezone = tzlocal.get_localzone() # get pytz tzinfo
isdst = bool(datetime.fromtimestamp(timestamp, local_timezone).dst())
Related: Use Python to find out if a timezone currently in daylight savings time.

Why do you assume that your filesystem writes down timestamps with a timezone?
On a server, you stick to UTC which does not have DST. On a desktop, you should look up the latest moment of the DST change (on or off), and correct the time accordingly. I don't know if pytz has this information, but this information is definitely available on the web in a machine-readable form.
Note that for some moments during the transition from DST some values of local time occur twice, and it's impossible to tell if a particular timestamp (e.g. 2:30 am) occurred before or after the switch (within an hour).

How do you convert a datetime/timestamp from one timezone to another timezone?

Specifically, given the timezone of my server (system time perspective) and a timezone input, how do I calculate the system time as if it were in that new timezone (regardless of daylight savings, etc)?
import datetime
current_time = datetime.datetime.now() #system time
server_timezone = "US/Eastern"
new_timezone = "US/Pacific"
current_time_in_new_timezone = ???

If you know your origin timezone and the new timezone that you want to convert it to, it turns out to be very straightforward:
Make two pytz.timezone objects, one for the current timezone and one for the new timezone e.g. pytz.timezone("US/Pacific"). You can find a list of all official timezones in pytz library: import pytz; pytz.all_timezones
Localize the datetime/timestamp of interest to the current timezone e.g.
current_timezone = pytz.timezone("US/Eastern")
localized_timestamp = current_timezone.localize(timestamp)
Convert to new timezone using .astimezone() on the newly localized datetime/timestamp from step 2 with the desired timezone's pytz object as input e.g. localized_timestamp.astimezone(new_timezone).
Done!
As a full example:
import datetime
import pytz
# a timestamp I'd like to convert
my_timestamp = datetime.datetime.now()
# create both timezone objects
old_timezone = pytz.timezone("US/Eastern")
new_timezone = pytz.timezone("US/Pacific")
# two-step process
localized_timestamp = old_timezone.localize(my_timestamp)
new_timezone_timestamp = localized_timestamp.astimezone(new_timezone)
# or alternatively, as an one-liner
new_timezone_timestamp = old_timezone.localize(my_timestamp).astimezone(new_timezone)
Bonus: but if all you need is the current time in a specific timezone, you can conveniently pass that timezone directly into datetime.now() to get the current times directly:
datetime.datetime.now(new_timezone)
When it comes to needing timezones conversions generally, I would strongly advise that one should store all timestamps in your database in UTC, which has no daylight savings time (DST) transition. And as a good practice, one should always choose to enable time zone support (even if your users are all in a single time zone!). This will help you avoid the DST transition problem that plagues so much software today.
Beyond DST, time in software can be generally quite tricky. To get a sense of just how difficult it is to deal with time in software in general, here is a potentially enlightening resource: http://yourcalendricalfallacyis.com
Even a seemingly simple operation as converting a datetime/timestamp into a date can become non-obvious. As this helpful documentation points out:
A datetime represents a point in time. It’s absolute: it doesn’t depend on anything. On the contrary, a date is a calendaring concept. It’s a period of time whose bounds depend on the time zone in which the date is considered. As you can see, these two concepts are fundamentally different.
Understanding this difference is a key step towards avoiding time-based bugs. Good luck.

With Python 3.9, the standard lib has all you need: zoneinfo. pytz is not needed anymore (deprecated; -> pytz deprecation shim).
Ex:
from datetime import datetime
from zoneinfo import ZoneInfo
server_timezone = "US/Eastern"
new_timezone = "US/Pacific"
current_time = datetime.now(ZoneInfo(server_timezone))
# current_time_in_new_timezone = ???
current_time_in_new_timezone = current_time.astimezone(ZoneInfo(new_timezone))
That gives you for example
print(current_time.isoformat(timespec='seconds'))
# 2021-10-04T02:42:54-04:00
print(repr(current_time))
# datetime.datetime(2021, 10, 4, 2, 42, 54, 40600, tzinfo=zoneinfo.ZoneInfo(key='US/Eastern'))
print(current_time_in_new_timezone.isoformat(timespec='seconds'))
# 2021-10-03T23:42:54-07:00
print(repr(current_time_in_new_timezone))
# datetime.datetime(2021, 10, 3, 23, 42, 54, 40600, tzinfo=zoneinfo.ZoneInfo(key='US/Pacific'))

How do you convert datetime/timestamp from one timezone to another timezone?
There are two steps:
Create an aware datetime objects from the system time and timezone e.g., to get the current system time in the given timezone:
#!/usr/bin/env python
from datetime import datetime
import pytz
server_timezone = pytz.timezone("US/Eastern")
server_time = datetime.now(server_timezone) # you could pass *tz* directly
Note: datetime.now(server_timezone) works even during ambiguous times e.g., during DST transitions while server_timezone.localize(datetime.now()) may fail (50% chance).
If you are sure that your input time exists in the server's timezone and it is unique then you could pass is_dst=None to assert that:
server_time = server_timezone.localize(naive_time, is_dst=None)
It raises an exception for invalid times.
If it is acceptable to ignore upto a day error (though typically an error due to DST is around an hour) then you could drop is_dst parameter:
server_time = server_timezone.normalize(server_timezone.localize(naive_time))
.normalize() is called to adjust non-existing times (local time in the gap, during "spring forward" transitions). If the time zone rules haven't changed; your server shouldn't generate non-existing times. See "Can I just always set is_dst=True?"
Convert an aware datetime object to the target timezone tz:
tz = pytz.timezone("US/Pacific")
server_time_in_new_timezone = server_time.astimezone(tz)

In case you want to convert a timestamp from one timezone to another, you can use this code:
from datetime import datetime
from zoneinfo import ZoneInfo
from_timezone = ZoneInfo('Europe/Moscow') # UTC-3
to_timezone = ZoneInfo('Asia/Tbilisi') # UTC-4
dt = datetime.fromtimestamp(timestamp, to_timezone)
result_timestamp = int(dt.replace(tzinfo=from_timezone).timestamp())
For example, if you take timestamp = 529635600 (1986-14-10 04:00:00 in Moscow) and run this code, you will get result_timestamp = 529639200 (1986-14-10 05:00:00 in Tbilisi).