I'm trying to convert a timestamp with a specific timezone(Europe/Paris) to a datetime format in UTC.
From my laptop it works with the solution below but when I'm executing my code in a remote server(AWS- Lambda function in Ireland), I've a shift of 1 hour because the local timezone of the server is different from mine.
How can I have a code who can work on my laptop and at the same time in a remote server(dynamically handle local timezone)?
import pytz
import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
utc = pytz.timezone('UTC')
now_in_utc = datetime.datetime.utcnow().replace(tzinfo=utc).astimezone(pytz.UTC)
fr = pytz.timezone('Europe/Paris')
new_date = datetime.datetime.fromtimestamp(timestamp_received)
return fr.localize(new_date, is_dst=None).astimezone(pytz.UTC)
Thanks
I am not sure what timestamp_received is, but I think what you want is utcfromtimestamp()
import pytz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_naive_utc = datetime.utcfromtimestamp(timestamp_received)
return dt_naive_utc.replace(tzinfo=pytz.utc)
For completeness, here is another way to accomplish the same thing by referencing python-dateutil's tzlocal time zone:
from dateutil import tz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_local = datetime.fromtimestamp(timestamp_received, tz.tzlocal())
if tz.datetime_ambiguous(dt_local):
raise AmbiguousTimeError
if tz.datetime_imaginary(dt_local):
raise ImaginaryTimeError
return dt_local.astimezone(tz.tzutc())
class AmbiguousTimeError(ValueError):
pass
class ImaginaryTimeError(ValueError):
pass
(I added in the AmbiguousTimeError and ImaginaryTimeError conditions to mimic the pytz interface.) Note that I'm including this just in case you have a similar problem that needs to make reference to the local time zone for some reason - if you have something that will give you the right answer in UTC, it's best to use that and then use astimezone to get it into whatever local zone you want it in.
How it works
Since you expressed that you were still a bit confused about how this works in the comments, I thought I would clarify why this works. There are two functions that convert timestamps to datetime.datetime objects, datetime.datetime.fromtimestamp(timestamp, tz=None) and datetime.datetime.utcfromtimestamp(timestamp):
utcfromtimestamp(timestamp) will give you a naive datetime that represents the time in UTC. You can then do dt.replace(tzinfo=pytz.utc) (or any other utc implementation - datetime.timezone.utc, dateutil.tz.tzutc(), etc) to get an aware datetime and convert it to whatever time zone you want.
fromtimestamp(timestamp, tz=None), when tz is not None, will give you an aware datetime equivalent to utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(tz). If tz is None, instead of converting too the specified time zone, it converts to your local time (equivalent to dateutil.tz.tzlocal()), and then returns a naive datetime.
Starting in Python 3.6, you can use datetime.datetime.astimezone(tz=None) on naive datetimes, and the time zone will be assumed to be system local time. So if you're developing a Python >= 3.6 application or library, you can use datetime.fromtimestamp(timestamp).astimezone(whatever_timezone) or datetime.utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(whatever_timezone) as equivalents.
Related
I am finding a very strange behavior for different timezones in Python. The following code:
import datetime as dtm
import pytz
def test_tz():
america_ny_tz: dtm.tzinfo = pytz.timezone("America/New_York")
est_tz: dtm.tzinfo = pytz.timezone("EST5EDT")
today = dtm.date.today()
ny_dtm = dtm.datetime(
year=today.year, month=today.month, day=today.day, tzinfo=america_ny_tz
)
est_dtm = dtm.datetime(
year=today.year, month=today.month, day=today.day, tzinfo=est_tz
)
print(f"New York: {ny_dtm.timestamp()}, EST: {est_dtm.timestamp()}")
if __name__ == "__main__":
test_tz()
outputs:
New York: 1609995360.0, EST: 1609995600.0
As you may notice the difference is about 4 minutes where one would have to assume the time should be the same.
Am I accessing time zone information incorrectly or is my understanding that the Time Zone information should be the same?
Running on Linux, Ubuntu 20.04 but the behavior on 18.04 was the same.
P.S. I haven't tried other languages or different OSs to see if the behavior will be the same.
You need to localize correctly - with pytz's timezone objects, setting tzinfo directly when creating the datetime object is not the right way. It's in the docs:
This library only supports two ways of building a localized time. The first is to use the localize() method provided by the pytz library.
and
The second way of building a localized time is by converting an existing localized time using the standard astimezone() method
On the other hand, if you happen to use Python 3.9+, you can use the zoneinfo module from the standard lib to make you life easier - see the docs / using-zoneinfo or example usage here.
This seems to be because one timezone uses Local Mean Time and the other is an offset of UTC (Universal Coordinated Time).
>>> pytz.timezone("America/New_York")
<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>
>>> pytz.timezone("EST5EDT")
<DstTzInfo 'EST5EDT' EST-1 day, 19:00:00 STD>
>>> pytz.timezone("LMT")
From https://www.timeanddate.com/time/local-mean-time.html
Local Mean Time was officially used as civil time in many countries
during the 19th century. Each city had a different local time defined
by its longitude, the difference amounting to 4 minutes per degree
longtitude. This equals a distance of 50 miles or 81 kilometers on New
York's latitude.
I have a naïve datetime in hand and I want to promote it to include the system timezone. I see lots of examples where the timezone is known a priori and the datetime is localized against it. How do I obtain the local timezone if I don't know it in advance?
local_tz = ???
dt = local_tz.localize(naive_time)
You can get the appropriate local_tz by doing:
import time
import pytz
local_tz = pytz.timezone(time.tzname[0])
EDIT: It appears that time.tzname[0] isn't always a name that corresponds to those found in pytz.all_timezones
The simplest solution is to use the tzlocal package, which uses a number of tricks to find the local timezone from the system, and can also localize for you in place of pytz:
import tzlocal
local_tz = tzlocal.get_localzone()
local_tz.localize(naive_time)
Here is as close as I've managed to get:
local_tz = dateutil.tz.tzlocal()
timestamp = timestamp.replace(tzinfo=local_tz)
Unfortunately local_tz doesn't have a localize() method so I resorted to using replace().
This does not work:
t = os.path.getmtime(filename)
dTime = datetime.datetime.fromtimestamp(t)
justTime = dTime.timetuple()
if justTime.tm_isdst == 0 :
tDelta = datetime.timedelta(hours=0)
else:
tDelta = datetime.timedelta(hours=1)
What happens instead is that the conditional always equals 1, despite some of the timestamps being within daytime savings time.
I am trying to make a python call match the behavior of a c-based call.
To find out whether a given timestamp corresponds to daylight saving time (DST) in the local time zone:
import os
import time
timestamp = os.path.getmtime(filename)
isdst = time.localtime(timestamp).tm_isdst > 0
It may fail. To workaround the issues, you could get a portable access to the tz database using pytz module:
from datetime import datetime
import tzlocal # $ pip install tzlocal
local_timezone = tzlocal.get_localzone() # get pytz tzinfo
isdst = bool(datetime.fromtimestamp(timestamp, local_timezone).dst())
Related: Use Python to find out if a timezone currently in daylight savings time.
Why do you assume that your filesystem writes down timestamps with a timezone?
On a server, you stick to UTC which does not have DST. On a desktop, you should look up the latest moment of the DST change (on or off), and correct the time accordingly. I don't know if pytz has this information, but this information is definitely available on the web in a machine-readable form.
Note that for some moments during the transition from DST some values of local time occur twice, and it's impossible to tell if a particular timestamp (e.g. 2:30 am) occurred before or after the switch (within an hour).
Specifically, given the timezone of my server (system time perspective) and a timezone input, how do I calculate the system time as if it were in that new timezone (regardless of daylight savings, etc)?
import datetime
current_time = datetime.datetime.now() #system time
server_timezone = "US/Eastern"
new_timezone = "US/Pacific"
current_time_in_new_timezone = ???
If you know your origin timezone and the new timezone that you want to convert it to, it turns out to be very straightforward:
Make two pytz.timezone objects, one for the current timezone and one for the new timezone e.g. pytz.timezone("US/Pacific"). You can find a list of all official timezones in pytz library: import pytz; pytz.all_timezones
Localize the datetime/timestamp of interest to the current timezone e.g.
current_timezone = pytz.timezone("US/Eastern")
localized_timestamp = current_timezone.localize(timestamp)
Convert to new timezone using .astimezone() on the newly localized datetime/timestamp from step 2 with the desired timezone's pytz object as input e.g. localized_timestamp.astimezone(new_timezone).
Done!
As a full example:
import datetime
import pytz
# a timestamp I'd like to convert
my_timestamp = datetime.datetime.now()
# create both timezone objects
old_timezone = pytz.timezone("US/Eastern")
new_timezone = pytz.timezone("US/Pacific")
# two-step process
localized_timestamp = old_timezone.localize(my_timestamp)
new_timezone_timestamp = localized_timestamp.astimezone(new_timezone)
# or alternatively, as an one-liner
new_timezone_timestamp = old_timezone.localize(my_timestamp).astimezone(new_timezone)
Bonus: but if all you need is the current time in a specific timezone, you can conveniently pass that timezone directly into datetime.now() to get the current times directly:
datetime.datetime.now(new_timezone)
When it comes to needing timezones conversions generally, I would strongly advise that one should store all timestamps in your database in UTC, which has no daylight savings time (DST) transition. And as a good practice, one should always choose to enable time zone support (even if your users are all in a single time zone!). This will help you avoid the DST transition problem that plagues so much software today.
Beyond DST, time in software can be generally quite tricky. To get a sense of just how difficult it is to deal with time in software in general, here is a potentially enlightening resource: http://yourcalendricalfallacyis.com
Even a seemingly simple operation as converting a datetime/timestamp into a date can become non-obvious. As this helpful documentation points out:
A datetime represents a point in time. It’s absolute: it doesn’t depend on anything. On the contrary, a date is a calendaring concept. It’s a period of time whose bounds depend on the time zone in which the date is considered. As you can see, these two concepts are fundamentally different.
Understanding this difference is a key step towards avoiding time-based bugs. Good luck.
With Python 3.9, the standard lib has all you need: zoneinfo. pytz is not needed anymore (deprecated; -> pytz deprecation shim).
Ex:
from datetime import datetime
from zoneinfo import ZoneInfo
server_timezone = "US/Eastern"
new_timezone = "US/Pacific"
current_time = datetime.now(ZoneInfo(server_timezone))
# current_time_in_new_timezone = ???
current_time_in_new_timezone = current_time.astimezone(ZoneInfo(new_timezone))
That gives you for example
print(current_time.isoformat(timespec='seconds'))
# 2021-10-04T02:42:54-04:00
print(repr(current_time))
# datetime.datetime(2021, 10, 4, 2, 42, 54, 40600, tzinfo=zoneinfo.ZoneInfo(key='US/Eastern'))
print(current_time_in_new_timezone.isoformat(timespec='seconds'))
# 2021-10-03T23:42:54-07:00
print(repr(current_time_in_new_timezone))
# datetime.datetime(2021, 10, 3, 23, 42, 54, 40600, tzinfo=zoneinfo.ZoneInfo(key='US/Pacific'))
How do you convert datetime/timestamp from one timezone to another timezone?
There are two steps:
Create an aware datetime objects from the system time and timezone e.g., to get the current system time in the given timezone:
#!/usr/bin/env python
from datetime import datetime
import pytz
server_timezone = pytz.timezone("US/Eastern")
server_time = datetime.now(server_timezone) # you could pass *tz* directly
Note: datetime.now(server_timezone) works even during ambiguous times e.g., during DST transitions while server_timezone.localize(datetime.now()) may fail (50% chance).
If you are sure that your input time exists in the server's timezone and it is unique then you could pass is_dst=None to assert that:
server_time = server_timezone.localize(naive_time, is_dst=None)
It raises an exception for invalid times.
If it is acceptable to ignore upto a day error (though typically an error due to DST is around an hour) then you could drop is_dst parameter:
server_time = server_timezone.normalize(server_timezone.localize(naive_time))
.normalize() is called to adjust non-existing times (local time in the gap, during "spring forward" transitions). If the time zone rules haven't changed; your server shouldn't generate non-existing times. See "Can I just always set is_dst=True?"
Convert an aware datetime object to the target timezone tz:
tz = pytz.timezone("US/Pacific")
server_time_in_new_timezone = server_time.astimezone(tz)
In case you want to convert a timestamp from one timezone to another, you can use this code:
from datetime import datetime
from zoneinfo import ZoneInfo
from_timezone = ZoneInfo('Europe/Moscow') # UTC-3
to_timezone = ZoneInfo('Asia/Tbilisi') # UTC-4
dt = datetime.fromtimestamp(timestamp, to_timezone)
result_timestamp = int(dt.replace(tzinfo=from_timezone).timestamp())
For example, if you take timestamp = 529635600 (1986-14-10 04:00:00 in Moscow) and run this code, you will get result_timestamp = 529639200 (1986-14-10 05:00:00 in Tbilisi).
I have working code. My question is why does this work?
#"04/13/05 2:30pm EDT" <- original date string
from datetime import datetime, timedelta
from pytz import timezone
import pytz
import time
T= time.struct_time((2005,4,13,14,30,0,0,0,1))
t = time.mktime(T)
print(t)
this prints
1113417000,
which according to an online converter, http://www.epochconverter.com/, prints
Your time zone: 4/13/2005 2:30:00 PM GMT-4
which is correct. My question is, how does it know the time was in EDT which is GMT-4? The last of the 9-tuple is "dst flag", but there are numerous timezones where DST is used. So how does it detect the correct timezone?
It asks your operating system for that information, by way of the stdlib C time functions.
To quote the time module documentation:
Most of the functions defined in this module call platform C library functions with the same name. It may sometimes be helpful to consult the platform documentation, because the semantics of these functions varies among platforms.
You can pass in a -1 for the DST flag to have the library set it for you, see time.struct_time. On my Mac OS X 10.7.5 setup, configured for the Europe/Oslo timezone, that gives me:
>>> import time
>>> time.mktime((2005,4,13,14,30,0,0,0,-1))
1113395400.0
or, as 'translated' by epochconverter:
Your time zone: 4/13/2005 2:30:00 PM GMT+2
According to the documentation the function time.mktime(t) takes an argument (struct_time) expressed in local time, not UTC. Local time is taken from your operating system. You can see your local DST timezone with, for example, time.tzname