Python able to handle timestamps timezone-insensitively except in GMT? - python

I'm trying to write a function that accepts a UTC timestamp (seconds since epoch) and emits a timezone-aware datetime for that timestamp with tz=utc. But I've encountered a strange issue.
Steps to reproduce:
(a) Set your system timezone to anything not-GMT. (In this case, Los Angeles. Also tested with various Russian, Chinese, South American and Icelandic timezones both before and after GMT+0000).
>>> import datetime
>>> from dateutil import tz
>>> epoch_seconds = 0
>>> dt = datetime.datetime.fromtimestamp(epoch_seconds)
>>> print(dt)
1969-12-31 16:00:00
>>> dt = dt.replace(tzinfo=tz.tzlocal())
>>> print(dt)
1969-12-31 16:00:00-08:00
>>> dt = dt.astimezone(tz=tz.tzutc())
>>> print(dt)
1970-01-01 00:00:00+00:00
>>> assert '{:%Y-%m-%d %H:%M:%S %z}'.format(dt) == '1970-01-01 00:00:00 +0000'
>>>
(b) Set your timezone to London/Dublin/something equivalent to GMT.
>>> import datetime
>>> from dateutil import tz
>>> epoch_seconds = 0
>>> dt = datetime.datetime.fromtimestamp(epoch_seconds)
>>> print(dt)
1970-01-01 01:00:00
>>> dt = dt.replace(tzinfo=tz.tzlocal())
>>> print(dt)
1970-01-01 01:00:00+00:00
>>> dt = dt.astimezone(tz=tz.tzutc())
>>> print(dt)
1970-01-01 01:00:00+00:00
>>> assert '{:%Y-%m-%d %H:%M:%S %z}'.format(dt) == '1970-01-01 00:00:00 +0000'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
>>>
Can you help me understand why this seemingly works in any non-GMT timezone but not in GMT?
The intent of the code is:
Create a datetime using 0 seconds since epoch: dt = datetime.datetime.fromtimestamp(0)
Tell that datetime what timezone it is currently in: dt = dt.replace(tzinfo=tz.tzlocal())
"Move" it from the local timezone to UTC: dt = dt.astimezone(tz=tz.tzutc())
I think the problem lies in dt = datetime.datetime.fromtimestamp(0) because even at that point calling print(dt) shows 01:00 as the time instead of 00:00.
Update It gives 1am because the UK was actually in "British Standard Time" (GMT+0100) on 1st January 1970. Maybe dateutil's tz doesn't know that? It's just an abstract representation of the current rules? Maybe?


I ended up solving this using the pytz library. I don't think it can be solved using datetime because datetime's timezone conversion doesn't know that the British used GMT+0100 for the entire of 1970 and not just for summer time.
With system timezone set to Europe/London:
>>> import datetime
>>> from pytz import timezone
>>> epoch_seconds = 0
>>> dt = datetime.datetime.utcfromtimestamp(epoch_seconds)
>>> print(dt)
1970-01-01 00:00:00
>>> dt = timezone('UTC').localize(dt)
>>> print(dt)
1970-01-01 00:00:00+00:00
>>> assert '{:%Y-%m-%d %H:%M:%S %z}'.format(dt) == '1970-01-01 00:00:00 +0000'
With system timezone set to America/Los_Angeles
>>> import datetime
>>> from pytz import timezone
>>> epoch_seconds = 0
>>> dt = datetime.datetime.utcfromtimestamp(epoch_seconds)
>>> print(dt)
1970-01-01 00:00:00
>>> dt = timezone('UTC').localize(dt)
>>> print(dt)
1970-01-01 00:00:00+00:00
>>> assert '{:%Y-%m-%d %H:%M:%S %z}'.format(dt) == '1970-01-01 00:00:00 +0000'
>>>
Local time never has to come into it this way - you can just generate a naïve timestamp, tell it to be UTC with pytz's localize.


I'm trying to write a function that accepts a UTC timestamp (seconds since epoch) and emits a timezone-aware datetime for that timestamp with tz=utc
#!/usr/bin/env python3
from datetime import datetime, timezone
aware_dt = datetime.fromtimestamp(epoch_seconds, timezone.utc)
I don't think it can be solved using datetime because datetime's timezone conversion doesn't know that the British used GMT+0100 for the entire of 1970 and not just for summer time.
If python has access to the tz database on your system then it does know the rules:
$ TZ=Europe/London ptpython
>>> from datetime import datetime, timezone
>>> epoch_seconds = 0
>>> aware_dt = datetime.fromtimestamp(epoch_seconds, timezone.utc)
>>> aware_dt
datetime.datetime(1970, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
>>> aware_dt.astimezone() # local time (1am)
datetime.datetime(1970, 1, 1, 1, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 3600), 'BST'))
>>> _.astimezone(timezone.utc) # back to utc
datetime.datetime(1970, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
To provide a portable access to the tz database, you could use pytz timezone e.g., via tzlocal module:
>>> import tzlocal # $ pip install tzlocal
>>> datetime.fromtimestamp(epoch_seconds, tzlocal.get_localzone()) # local time (1am)
datetime.datetime(1970, 1, 1, 1, 0, tzinfo=<DstTzInfo 'Europe/London' BST+1:00:00 STD>)
btw, dateutil provides access to the tz database too. If it doesn't work: it is either the wrong usage or a bug. Try to update to the latest python-dateutil version, to see whether the issue disappears.

Related

How to convert a datetime ('2019-06-05T10:37:29.353+0100') to UTC timestamp using Python3?

I want to convert datetime, i.e. 2019-06-05T10:37:29.353+0100, to UTC timestamp in Python3.
I understand that +0100 represents the timezone. Why do +0100, +0200, and +0300 all convert to the same timestamp?
How can I convert a datetime containing a timezone to a UTC timestamp?
>>> d=datetime.datetime.strptime('2019-06-05T10:37:29.353+0100', '%Y-%m-%dT%H:%M:%S.%f%z')
>>> unixtime = time.mktime(d.timetuple())
>>> unixtime
1559723849.0
>>> d=datetime.datetime.strptime('2019-06-05T10:37:29.353+0200', '%Y-%m-%dT%H:%M:%S.%f%z')
>>> unixtime = time.mktime(d.timetuple())
>>> unixtime
1559723849.0
>>> d=datetime.datetime.strptime('2019-06-05T10:37:29.353+0300', '%Y-%m-%dT%H:%M:%S.%f%z')
>>> unixtime = time.mktime(d.timetuple())
>>> unixtime
1559723849.0

here's some more explanations (see comments) how to convert back and forth between timestamps as strings with UTC offset and POSIX timestamps.
from datetime import datetime, timezone
s = '2019-06-05T10:37:29.353+0100'
# to datetime object
dt = datetime.strptime('2019-06-05T10:37:29.353+0100', '%Y-%m-%dT%H:%M:%S.%f%z')
# note that the object has tzinfo set to a specific timedelta:
print(repr(dt))
>>> datetime.datetime(2019, 6, 5, 10, 37, 29, 353000, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600)))
# you could store this info
dt_UTCoffset = dt.utcoffset() # datetime.timedelta(seconds=3600)
# to get POSIX seconds since the epoch:
ts = dt.timestamp()
# and back to datetime:
dt_from_ts = datetime.fromtimestamp(ts, tz=timezone.utc)
# note that this is a UTC timestamp; the UTC offset is zero:
print(dt_from_ts.isoformat())
>>> 2019-06-05T09:37:29.353000+00:00
# instead of UTC, you could also set a UTC offset:
dt_from_ts = datetime.fromtimestamp(ts, tz=timezone(dt_UTCoffset))
print(dt_from_ts.isoformat())
>>> 2019-06-05T10:37:29.353000+01:00
...And a note on a pitfall when working with datetime in Python: if you convert from timestamp to datetime and don't set the tz property, local time is returned (same applies the other way 'round!):
print(datetime.fromtimestamp(ts)) # I'm on CEST at the moment, so UTC+2
>>> 2019-06-05 11:37:29.353000

Getting the UTC hour from a UTC+2 datetime object

My Time string looks like this:
03/16/16 15:50 UTC+02:00
so I parse it like so
from dateutil import parser
my_date = parser.parse(date_string)
Since this is UTC+2 time, how do I convert this dateobject to UTC?

Using datetime.datetime.astimezone with pytz.UTC (datetime.timezone.utc if you use Python 3.x), you can get the datetime with UTC timezone:
>>> import pytz
>>> from dateutil import parser
>>>
>>> date_string = '03/16/16 15:50 UTC+02:00'
>>> my_date = parser.parse(date_string)
>>> my_date.astimezone(pytz.UTC)
datetime.datetime(2016, 3, 16, 17, 50, tzinfo=<UTC>)

What is wrong with my conversion from local time to UTC

According to timeanddate.com, currently Chicago is 5 hours behind UTC. However, my Python app thinks differently:
import datetime
import pytz
local_tz = pytz.timezone('America/Chicago')
local_time = datetime.datetime(2015, 8, 6, 0, 0, tzinfo=local_tz)
utc_time = local_time.astimezone(pytz.utc)
print(local_time)
print(utc_time)
2015-08-06 00:00:00-05:51
2015-08-06 05:51:00+00:00
I am getting the same results with both 'America/Chicago' and 'US/Central'. Why is the offset -05:51 instead of -05:00?

pytz timezone objects need to be initialized with a specific time before they're used, and creating a datetime with a tzinfo= parameter does not allow for that. You have to use the localize method of the pytz object to add the timezone to the datetime.
>>> local_tz = pytz.timezone('America/Chicago')
>>> local_time = local_tz.localize(datetime.datetime(2015, 8, 6, 0, 0))
>>> print local_time
2015-08-06 00:00:00-05:00
>>> utc_time = local_time.astimezone(pytz.utc)
>>> print utc_time
2015-08-06 05:00:00+00:00

Python Timezone conversion

How do I convert a time to another timezone in Python?

I have found that the best approach is to convert the "moment" of interest to a utc-timezone-aware datetime object (in python, the timezone component is not required for datetime objects).
Then you can use astimezone to convert to the timezone of interest (reference).
from datetime import datetime
import pytz
utcmoment_naive = datetime.utcnow()
utcmoment = utcmoment_naive.replace(tzinfo=pytz.utc)
# print "utcmoment_naive: {0}".format(utcmoment_naive) # python 2
print("utcmoment_naive: {0}".format(utcmoment_naive))
print("utcmoment: {0}".format(utcmoment))
localFormat = "%Y-%m-%d %H:%M:%S"
timezones = ['America/Los_Angeles', 'Europe/Madrid', 'America/Puerto_Rico']
for tz in timezones:
localDatetime = utcmoment.astimezone(pytz.timezone(tz))
print(localDatetime.strftime(localFormat))
# utcmoment_naive: 2017-05-11 17:43:30.802644
# utcmoment: 2017-05-11 17:43:30.802644+00:00
# 2017-05-11 10:43:30
# 2017-05-11 19:43:30
# 2017-05-11 13:43:30
So, with the moment of interest in the local timezone (a time that exists), you convert it to utc like this (reference).
localmoment_naive = datetime.strptime('2013-09-06 14:05:10', localFormat)
localtimezone = pytz.timezone('Australia/Adelaide')
try:
localmoment = localtimezone.localize(localmoment_naive, is_dst=None)
print("Time exists")
utcmoment = localmoment.astimezone(pytz.utc)
except pytz.exceptions.NonExistentTimeError as e:
print("NonExistentTimeError")

Using pytz
from datetime import datetime
from pytz import timezone
fmt = "%Y-%m-%d %H:%M:%S %Z%z"
timezonelist = ['UTC','US/Pacific','Europe/Berlin']
for zone in timezonelist:
now_time = datetime.now(timezone(zone))
print now_time.strftime(fmt)

import datetime
import pytz
def convert_datetime_timezone(dt, tz1, tz2):
tz1 = pytz.timezone(tz1)
tz2 = pytz.timezone(tz2)
dt = datetime.datetime.strptime(dt,"%Y-%m-%d %H:%M:%S")
dt = tz1.localize(dt)
dt = dt.astimezone(tz2)
dt = dt.strftime("%Y-%m-%d %H:%M:%S")
return dt
-
dt: date time string
tz1: initial time zone
tz2: target time zone
-
> convert_datetime_timezone("2017-05-13 14:56:32", "Europe/Berlin", "PST8PDT")
'2017-05-13 05:56:32'
> convert_datetime_timezone("2017-05-13 14:56:32", "Europe/Berlin", "UTC")
'2017-05-13 12:56:32'
-
> pytz.all_timezones[0:10]
['Africa/Abidjan',
'Africa/Accra',
'Africa/Addis_Ababa',
'Africa/Algiers',
'Africa/Asmara',
'Africa/Asmera',
'Africa/Bamako',
'Africa/Bangui',
'Africa/Banjul',
'Africa/Bissau']

Python 3.9 adds the zoneinfo module so now only the the standard library is needed!
>>> from zoneinfo import ZoneInfo
>>> from datetime import datetime
>>> d = datetime(2020, 10, 31, 12, tzinfo=ZoneInfo('America/Los_Angeles'))
>>> d.astimezone(ZoneInfo('Europe/Berlin')) # 12:00 in Cali will be 20:00 in Berlin
datetime.datetime(2020, 10, 31, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Berlin'))
Wikipedia list of available time zones
Some functions such as now() and utcnow() return timezone-unaware datetimes, meaning they contain no timezone information. I recommend only requesting timezone-aware values from them using the keyword tz=ZoneInfo('localtime').
If astimezone gets a timezone-unaware input, it will assume it is local time, which can lead to errors:
>>> datetime.utcnow() # UTC -- NOT timezone-aware!!
datetime.datetime(2020, 6, 1, 22, 39, 57, 376479)
>>> datetime.now() # Local time -- NOT timezone-aware!!
datetime.datetime(2020, 6, 2, 0, 39, 57, 376675)
>>> datetime.now(tz=ZoneInfo('localtime')) # timezone-aware
datetime.datetime(2020, 6, 2, 0, 39, 57, 376806, tzinfo=zoneinfo.ZoneInfo(key='localtime'))
>>> datetime.now(tz=ZoneInfo('Europe/Berlin')) # timezone-aware
datetime.datetime(2020, 6, 2, 0, 39, 57, 376937, tzinfo=zoneinfo.ZoneInfo(key='Europe/Berlin'))
>>> datetime.utcnow().astimezone(ZoneInfo('Europe/Berlin')) # WRONG!!
datetime.datetime(2020, 6, 1, 22, 39, 57, 377562, tzinfo=zoneinfo.ZoneInfo(key='Europe/Berlin'))
Windows has no system time zone database, so here an extra package is needed:
pip install tzdata
There is a backport to allow use in Python 3.6 to 3.8:
sudo pip install backports.zoneinfo
Then:
from backports.zoneinfo import ZoneInfo

Time conversion
To convert a time in one timezone to another timezone in Python, you could use datetime.astimezone():
so, below code is to convert the local time to other time zone.
datetime.datetime.today() - return current the local time
datetime.astimezone() - convert the time zone, but we have to pass the time zone.
pytz.timezone('Asia/Kolkata') -passing the time zone to pytz module
Strftime - Convert Datetime to string
# Time conversion from local time
import datetime
import pytz
dt_today = datetime.datetime.today() # Local time
dt_India = dt_today.astimezone(pytz.timezone('Asia/Kolkata'))
dt_London = dt_today.astimezone(pytz.timezone('Europe/London'))
India = (dt_India.strftime('%m/%d/%Y %H:%M'))
London = (dt_London.strftime('%m/%d/%Y %H:%M'))
print("Indian standard time: "+India+" IST")
print("British Summer Time: "+London+" BST")
List all the time zones
import pytz
for tz in pytz.all_timezones:
print(tz)

To convert a time in one timezone to another timezone in Python, you could use datetime.astimezone():
time_in_new_timezone = time_in_old_timezone.astimezone(new_timezone)
Given aware_dt (a datetime object in some timezone), to convert it to other timezones and to print the times in a given time format:
#!/usr/bin/env python3
import pytz # $ pip install pytz
time_format = "%Y-%m-%d %H:%M:%S%z"
tzids = ['Asia/Shanghai', 'Europe/London', 'America/New_York']
for tz in map(pytz.timezone, tzids):
time_in_tz = aware_dt.astimezone(tz)
print(f"{time_in_tz:{time_format}}")
If f"" syntax is unavailable, you could replace it with "".format(**vars())
where you could set aware_dt from the current time in the local timezone:
from datetime import datetime
import tzlocal # $ pip install tzlocal
local_timezone = tzlocal.get_localzone()
aware_dt = datetime.now(local_timezone) # the current time
Or from the input time string in the local timezone:
naive_dt = datetime.strptime(time_string, time_format)
aware_dt = local_timezone.localize(naive_dt, is_dst=None)
where time_string could look like: '2016-11-19 02:21:42'. It corresponds to time_format = '%Y-%m-%d %H:%M:%S'.
is_dst=None forces an exception if the input time string corresponds to a non-existing or ambiguous local time such as during a DST transition. You could also pass is_dst=False, is_dst=True. See links with more details at Python: How do you convert datetime/timestamp from one timezone to another timezone?

For Python timezone conversions, I use the handy table from the PyCon 2012 presentation by Taavi Burns.

Please note: The first part of this answer is or version 1.x of pendulum. See below for a version 2.x answer.
I hope I'm not too late!
The pendulum library excels at this and other date-time calculations.
>>> import pendulum
>>> some_time_zones = ['Europe/Paris', 'Europe/Moscow', 'America/Toronto', 'UTC', 'Canada/Pacific', 'Asia/Macao']
>>> heres_a_time = '1996-03-25 12:03 -0400'
>>> pendulum_time = pendulum.datetime.strptime(heres_a_time, '%Y-%m-%d %H:%M %z')
>>> for tz in some_time_zones:
... tz, pendulum_time.astimezone(tz)
...
('Europe/Paris', <Pendulum [1996-03-25T17:03:00+01:00]>)
('Europe/Moscow', <Pendulum [1996-03-25T19:03:00+03:00]>)
('America/Toronto', <Pendulum [1996-03-25T11:03:00-05:00]>)
('UTC', <Pendulum [1996-03-25T16:03:00+00:00]>)
('Canada/Pacific', <Pendulum [1996-03-25T08:03:00-08:00]>)
('Asia/Macao', <Pendulum [1996-03-26T00:03:00+08:00]>)
Answer lists the names of the time zones that may be used with pendulum. (They're the same as for pytz.)
For version 2:
some_time_zones is a list of the names of the time zones that might be used in a program
heres_a_time is a sample time, complete with a time zone in the form '-0400'
I begin by converting the time to a pendulum time for subsequent processing
now I can show what this time is in each of the time zones in show_time_zones
...
>>> import pendulum
>>> some_time_zones = ['Europe/Paris', 'Europe/Moscow', 'America/Toronto', 'UTC', 'Canada/Pacific', 'Asia/Macao']
>>> heres_a_time = '1996-03-25 12:03 -0400'
>>> pendulum_time = pendulum.from_format('1996-03-25 12:03 -0400', 'YYYY-MM-DD hh:mm ZZ')
>>> for tz in some_time_zones:
... tz, pendulum_time.in_tz(tz)
...
('Europe/Paris', DateTime(1996, 3, 25, 17, 3, 0, tzinfo=Timezone('Europe/Paris')))
('Europe/Moscow', DateTime(1996, 3, 25, 19, 3, 0, tzinfo=Timezone('Europe/Moscow')))
('America/Toronto', DateTime(1996, 3, 25, 11, 3, 0, tzinfo=Timezone('America/Toronto')))
('UTC', DateTime(1996, 3, 25, 16, 3, 0, tzinfo=Timezone('UTC')))
('Canada/Pacific', DateTime(1996, 3, 25, 8, 3, 0, tzinfo=Timezone('Canada/Pacific')))
('Asia/Macao', DateTime(1996, 3, 26, 0, 3, 0, tzinfo=Timezone('Asia/Macao')))

For Python 3.2+ simple-date is a wrapper around pytz that tries to simplify things.
If you have a time then
SimpleDate(time).convert(tz="...")
may do what you want. But timezones are quite complex things, so it can get significantly more complicated - see the the docs.

# Program
import time
import os
os.environ['TZ'] = 'US/Eastern'
time.tzset()
print('US/Eastern in string form:',time.asctime())
os.environ['TZ'] = 'Australia/Melbourne'
time.tzset()
print('Australia/Melbourne in string form:',time.asctime())
os.environ['TZ'] = 'Asia/Kolkata'
time.tzset()
print('Asia/Kolkata in string form:',time.asctime())

How to get the difference in Localtime and GMT time python?

I get the server date and I need to get the difference of this date from GMT
I get
Datetime = "2011-04-27 2:17:45"
I would like to get the result like
Datetime = "2011-04-27 2:17:45 +0500"

Try this:
import datetime, pytz
now = datetime.datetime.now(pytz.timezone('Asia/Kolkata'))
print now.strftime('%Y-%m-%d %H:%M:%S %z')
# prints: '2011-04-27 13:56:09 +0530'

From the example you have given, it looks to me that what you are looking for is datetime.isoformat. The example in the page shows how to convert the datetime values to the ISO format with the time zone information.

To do this, you have to know the timezone (or the UTC offset) of the server date. What you have here is a "naive" date, without timezone info, you can't guess the UTC difference.

I think the datetime module is what you need here:
>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2011, 4, 27, 11, 8, 26, 149000)
>>> datetime.utcnow()
datetime.datetime(2011, 4, 27, 8, 8, 47, 712000)
For a difference between two dates:
>>> dtnow = datetime.now()
>>> dtutc = datetime.utcnow()
>>> dtnow - dtutc
datetime.timedelta(0, 10792, 847000)
Look up the datetime module and the relevant classes in Python's docs.

A very powerful extension of the datetime standard python library is the dateutil one, that allows you to easily:
set the delta of your time zone:
parse dates with various convenient options (in our case we will use the default option, which will allow us to set our time zone)
So 1st set time zone, and default date with this zone:
>>> from datetime import datetime
>>> from dateutil import parser
>>> from dateutil.tz import tzoffset
>>> tz_plus_5 = tzoffset(None, 5 * 60 * 60) # offset is in seconds !
>>> default = datetime.now(tz_plus_5)
Now use this default date in the parsing:
>>> Datetime = "2011-04-27 2:17:45"
>>> my_date = parser.parse(Datetime, default=default)
>>> my_date
datetime.datetime(2011, 4, 27, 2, 17, 45, tzinfo=tzoffset(None, 18000))
>>> my_date.strftime("%Y-%m-%d %H:%M:%S %z")
'2011-04-27 02:17:45 +0500'

For those that simply need to get the offset between local time and UTC, the time module has an attribute time.altzone that specifies the difference between UTC and local time in seconds:
The offset of the local DST timezone, in seconds west of UTC, if one is defined. This is negative if the local DST timezone is east of UTC (as in Western Europe, including the UK). Only use this if daylight is nonzero.
Here's an example of how it works:
>>> datetime.now().isoformat()
'2011-09-01T17:26:46.971000'
>>> datetime.utcnow().isoformat()
'2011-09-01T15:27:32.699000'
>>> time.altzone / (60*60)
-2
Doesn't get much cleaner than that.

Categories

Resources