Develop Reference

Python time zone conversion from UTC to EST

I have the below list in python.
[['File_1','2021-09-09 07:05:10'],['File_2','2021-09-08 08:05:11']]
The above timestamp is a string in UTC timezone. I would like to convert this to EST timezone.
I tried using pytz package using
datetime.strptime(timestamp,'%Y-%m-%d %H:%M:%S').astimezone(pytz.timezone('US/Eastern'))
but it gives me result like
['File_1',datetime.datetime(2021,09,03,4,20,5)].
The format of date time is not as expected.

you need to set UTC first, then convert to the desired tz:
from datetime import datetime, timezone
from zoneinfo import ZoneInfo # Python 3.9+ standard lib
l = [['File_1','2021-09-09 07:05:10'],['File_2','2021-09-08 08:05:11']]
out = [[i[0], datetime.fromisoformat(i[1]) # Python 3.7+
.replace(tzinfo=timezone.utc)
.astimezone(ZoneInfo("America/New_York"))] for i in l]
print(out)
# [['File_1', datetime.datetime(2021, 9, 9, 3, 5, 10, tzinfo=zoneinfo.ZoneInfo(key='America/New_York'))], ['File_2', datetime.datetime(2021, 9, 8, 4, 5, 11, tzinfo=zoneinfo.ZoneInfo(key='America/New_York'))]]
# with pytz:
import pytz
outp = [[i[0], datetime.fromisoformat(i[1]) # Python 3.7+
.replace(tzinfo=timezone.utc)
.astimezone(pytz.timezone("America/New_York"))] for i in l]
print(outp)
# [['File_1', datetime.datetime(2021, 9, 9, 3, 5, 10, tzinfo=<DstTzInfo 'America/New_York' EDT-1 day, 20:00:00 DST>)], ['File_2', datetime.datetime(2021, 9, 8, 4, 5, 11, tzinfo=<DstTzInfo 'America/New_York' EDT-1 day, 20:00:00 DST>)]]
If you want a string instead of datetime object, use strftime or even simpler: .isoformat().

You can use the below snippet to get your output in the same format as your input
from datetime import datetime
import pytz
from pytz import timezone
timestamp = '2021-09-09 07:05:10'
datetime.strptime(timestamp,'%Y-%m-%d %H:%M:%S').astimezone(pytz.timezone('US/Eastern')).strftime("%Y-%m-%d %H:%M:%S")
# Output
'2021-09-09 03:05:10'
If you wish to display the timezone in the format, you can use
datetime.strptime(timestamp,'%Y-%m-%d %H:%M:%S').astimezone(pytz.timezone('US/Eastern')).strftime("%Y-%m-%d %H:%M:%S %Z%z")
# Output
'2021-09-09 03:05:10 EDT-0400'

Here's one way to convert it to EST
1st. Declare the list, and identify the date you want to convert as string
list = [['File_1','2021-09-09 07:05:10'],['File_2','2021-09-08 08:05:11']]
date_to_convert = list[1][1]
2nd. Import the needed libraries
import pytz
from datetime import datetime,timezone
3rd. Convert the string to date time
d = datetime.fromisoformat(date_to_convert)
4th. Declare datetime timezone as utc.
d = date_to_string.replace(tzinfo=timezone.utc)
print(d.isoformat())
# the output for the above: '2021-09-08T08:05:11+00:00'
5th. Set the format and convert to EST
fmt = '%Y-%m-%d %H:%M:%S'
est_date = d.astimezone(pytz.timezone('US/Eastern')).strftime(fmt)
print(est_date)
# the output for the above: '2021-09-08 04:05:11'

USE THIS WAY REMBER UPERCASE AND LOWERCASE ALSO MATTER
import time
timer = time.strftime("%Y-%m-%d -- %H:%M:%S %p")
print(timer)

How to convert UTC to EST with Python and take care of daylight saving automatically?

If I have a bunch of data with date & time in UTC format, how can I convert them to EST.
It can determine when they will be -4(in summer) and -5(in winter) automatically every year?
Thanks

You'll need to use the pytz module (available from PyPI):
import pytz
from datetime import datetime
est = pytz.timezone('US/Eastern')
utc = pytz.utc
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
winter = datetime(2016, 1, 24, 18, 0, 0, tzinfo=utc)
summer = datetime(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(winter.strftime(fmt))
print(summer.strftime(fmt))
print(winter.astimezone(est).strftime(fmt))
print(summer.astimezone(est).strftime(fmt))
which will print:
2016-01-24 18:00:00 UTC+0000
2016-07-24 18:00:00 UTC+0000
2016-01-24 13:00:00 EST-0500
2016-07-24 14:00:00 EDT-0400
The reason why you'll need to use 'US/Eastern' and not 'EST' is exemplified in the last two lines of output.

If you have a pandas series with object datatype, you can first convert it into a DateTime series using pd.to_datetime()
df[col] = pd.to_datetime(your_series, format = '%Y-%m-%d %H:%M:%S', errors ='coerce')
Check if it is timezone aware or not by using series.dt.tz
df[col].dt.tz
If it's not timezone aware, we should make it timezone aware by using series.dt.tz_localize(). Also, do read about the ambiguous and nonexistent parameters of this function
df[col] = your_series[col].dt.tz_localize('UTC')
Now convert this series into the required timezone by series.dt.tz_convert()
df[col] = your_series[col].dt.tz_convert('US/Eastern')
The above method will take care of daylight savings time. If you want to check more timezones you can pip install pytz and
import pytz
pytz.common_timezones

In case you just want the normalized hour offset for your existing timedelta shifting:
from datetime import datetime
import pytz
def curr_est_offset():
tz_est = pytz.timezone('US/Eastern')
offset = tz_est.utcoffset(datetime.utcnow())
offset_seconds = (offset.days * 86400) + offset.seconds
offset_hours = offset_seconds // 3600
return offset_hours # -4 or -5

As mentioned above, you can use pandas.DataFrame.tz_convert() like this:
import pandas as pd
from datetime import datetime
df = pd.read_csv("your_data_file_path.csv", index_col=False, engine='python')
df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.tz_localize('US/Eastern').dt.tz_convert('UTC')
df['Date'] = df['Date'].apply(lambda x: datetime.replace(x, tzinfo=None))
What the last row does is removing the timezone info from the datetime object, so you can operate with the date and time only (don't worry, that doesn't change the timezone again, it just strips it from the timestamp string).

This is thebjorn's answer converted from Python 2 to Python 3 with a few additional comments. Thanks to thebjorn.
For convention, I use these terms:
EST: Eastern Standard Time (winter)
EDT: Eastern Daylight Time
(summer)
EPT: Eastern Prevailing Time (standard or daylight saving
time as appropriate)
Code:
# Convert EPT / UTC
import pytz
import datetime from datetime as dtdt
# Timezones
ept = pytz.timezone('US/Eastern')
utc = pytz.utc
# str format
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
print("\nEPT/UTC examples:")
print("\nWinter (EST) example:")
# Create a UTC time in the winter
winter_utc = dtdt(2016, 1, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", winter_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is standard time.
winter_ept = winter_utc.astimezone(ept)
print(" EPT: ", winter_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
winter_utc2 = winter_ept.astimezone(utc)
print(" UTC: ", winter_utc2.strftime(fmt))
# Let's do that again for a summer datetime.
print("\nSummer (EDT) example:")
summer_utc = dtdt(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", summer_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is daylight saving time.
summer_ept = summer_utc.astimezone(ept)
print(" EPT: ", summer_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
summer_utc2 = summer_ept.astimezone(utc)
print(" UTC: ", summer_utc2.strftime(fmt))
Console:
EPT/UTC examples:
Winter (EST) example:
UTC: 2016-01-24 18:00:00 UTC+0000
EPT: 2016-01-24 13:00:00 EST-0500
UTC: 2016-01-24 18:00:00 UTC+0000
Summer (EDT) example:
UTC: 2016-07-24 18:00:00 UTC+0000
EPT: 2016-07-24 14:00:00 EDT-0400
UTC: 2016-07-24 18:00:00 UTC+0000

Python: converting date string to UTC

I am parsing a 3rd party website HTML with dates and times which are always in UK time format, however they don't have any timezone info in the source. Converting the string to an object is easy enough using datetime.strptime(), but how do I add timezone info?
Ultimately, I need to convert these strings to a datetime object in UTC format. The code will always run on a PC which is timezone aware, i.e. datetime.now() will return UK time.
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
Is there a nicer way to do this?
offset = datetime.now() - datetime.utcnow()
dt -= offset

Use pytz
import datetime
import pytz
temp = '07/12/2017 13:30'
dt = datetime.strptime(temp, '%d/%m/%Y %H:%M')
timezone = pytz.timezone("Etc/Greenwich")
d_aware = timezone.localize(dt)
d_aware.tzinfo
> <DstTzInfo 'Etc/Greenwich' PST-1 day, 16:00:00 STD>
d_aware
datetime.datetime(2017, 12, 7, 13, 30, tzinfo=<StaticTzInfo 'Etc/Greenwich'>)

There are some good libraries that make working with dates so much easier. I like dateparser, parsedatetime, and arrow;
import dateparser as dp
dt = dp.parse('07-12-2017 13:30 PST')
print (dt)
dt = dp.parse("Yesterday at 3:00am EST")
print(dt)
2017-07-12 13:30:00-08:00
2017-12-06 17:07:07.557109-05:00

Datetime module and Pandas to_datetime give different results

I have a string containing a UTC datetime
utc_str = '2017-11-21T23:00+0100'
which in my local time (Europe/Berlin) is:
local_time = '2017-11-22 00:00'
And is the desired value I would like to obtain from utc_string.
I can convert utc_string to local_time just fine using:
import datetime as dt
utc_time = dt.datetime.strptime(date_str, '%Y-%m-%dT%H:%M%z')
local_time = utc_time.replace(tzinfo=pytz.utc).astimezone(pytz.timezone('Europe/Berlin'))
print(local_time.strftime('%Y-%m-%d %H:%M'))
>>> 2017-11-22 00:00
However, when I use Pandas, I get a different result. It doesn't seem to apply the UTC offset:
import pandas as pd
pd_date = pd.to_datetime(date_str, utc=True)
print(pd_date.strftime('%Y-%m-%d %H:%M'))
>>> '2017-11-21 22:00'
And naively if I try to do the same process as with the datetime module,
the results are still off:
pd_date = pd.to_datetime(date_str, utc=True)
pd_date = pd_date.replace(tzinfo=pytz.utc).astimezone(pytz.timezone('Europe/Berlin'))
print(pd_date.strftime('%Y-%m-%d %H:%M'))
>>> '2017-11-21 23:00'
Is there something I am not understanding? Am I using pd.to_datetime or something else wrong? On Python 3.6, Windows 7.

As stated in the comment, I think your code for local_time is wrong
utc_time
datetime.datetime(2017, 11, 21, 23, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 3600))
utc_time.replace(tzinfo=pytz.utc)
'datetime.datetime(2017, 11, 21, 23, 0, tzinfo=<UTC>)'
so this replace removes the '+0100 from the datetime, but keeps the rest the same
utc_time.replace(tzinfo=pytz.utc).astimezone(pytz.timezone('Europe/Berlin'))
"datetime.datetime(2017, 11, 22, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin' CET+1:00:00 STD>)"
This then adds 1 hour to 23:00UTC, so become the next day midnight in Berlin as expected
pd.to_datetime(utc_str, utc=True)
Timestamp('2017-11-21 22:00:00+0000', tz='UTC')
The difference in behaviour is due to the constructor. pd.to_datetime calculates the time and timezone back to 22:00UTC instead of 23:00+0100, so if there you replace the timezone info with UTC, it changes nothing
Local time
Your utc_time object is in the correct timezone, so if you want the local time you can just do utc_time.strftime('%Y-%m-%d %H:%M') in pandas you'll have to do pd.to_datetime(utc_str, utc=True).astimezone(pytz.timezone('Europe/Berlin')).strftime('%Y-%m-%d %H:%M')

Python Datetime : use strftime() with a timezone-aware date

Suppose I have date d like this :
>>> d
datetime(2009, 4, 19, 21, 12, tzinfo=tzoffset(None, -7200))
As you can see, it is "timezone aware", there is an offset of 2 Hour, utctime is
>>> d.utctimetuple()
time.struct_time(tm_year=2009, tm_mon=4, tm_mday=19,
tm_hour=23, tm_min=12, tm_sec=0,
tm_wday=6, tm_yday=109, tm_isdst=0)
So, real UTC date is 19th March 2009 23:12:00, right ?
Now I need to format my date in string, I use
>>> d.strftime('%Y-%m-%d %H:%M:%S.%f')
'2009-04-19 21:12:00.000000'
Which doesn't seems to take this offset into account. How to fix that ?

In addition to what #Slam has already answered:
If you want to output the UTC time without any offset, you can do
from datetime import timezone, datetime, timedelta
d = datetime(2009, 4, 19, 21, 12, tzinfo=timezone(timedelta(hours=-2)))
d.astimezone(timezone.utc).strftime('%Y-%m-%d %H:%M:%S.%f')
See datetime.astimezone in the Python docs.

The reason is python actually formatting your datetime object, not some "UTC at this point of time"
To show timezone in formatting, use %z or %Z.
Look for strf docs for details

This will convert your local time to UTC and print it:
import datetime, pytz
from dateutil.tz.tz import tzoffset
loc = datetime.datetime(2009, 4, 19, 21, 12, tzinfo=tzoffset(None, -7200))
print(loc.astimezone(pytz.utc).strftime('%Y-%m-%d %H:%M:%S.%f') )
(http://pytz.sourceforge.net/)

I couldn't import timezone module (and hadn't much time to know why)
so I set TZ environment variable which override the /etc/localtime information
>>> import os
>>> import datetime
>>> print datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
2019-05-17 11:26
>>> os.environ["TZ"] = "UTC"
>>> print datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
2019-05-17 09:26