Develop Reference

datetime append GMT to the end of the string?

I am migrating php code into Python and came across datetime. My code:
date_raw = datetime.datetime.strptime(data["Campaign_Start_Date"], '%Y-%m-%d')
date_new = date_raw.strftime("%Y-%m-%d"+"T"+"%H:%M:%S GMT")
print(date_new)
# 2020-09-14T00:00:00 GMT
My desired output is: 2020-09-14T00:00:00-04:00 So I need to append GMT to the end of the string, but can't find a way to have a proper format back.

strptime doesn't automagically know about the timezone from a time formatted as '%Y-%m-%d', you will have to include that, e.g.
from datetime import datetime
import pytz
# parse the string
datestring = '2020-05-04'
date = datetime.strptime(datestring, '%Y-%m-%d')
# add a timezone info
tz = pytz.timezone('US/Eastern')
date_est = tz.localize(date)
# datetime.datetime(2020, 5, 4, 0, 0, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
print(date_est.isoformat())
# 2020-05-04T00:00:00-04:00

How to convert UTC to EST with Python and take care of daylight saving automatically?

If I have a bunch of data with date & time in UTC format, how can I convert them to EST.
It can determine when they will be -4(in summer) and -5(in winter) automatically every year?
Thanks

You'll need to use the pytz module (available from PyPI):
import pytz
from datetime import datetime
est = pytz.timezone('US/Eastern')
utc = pytz.utc
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
winter = datetime(2016, 1, 24, 18, 0, 0, tzinfo=utc)
summer = datetime(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(winter.strftime(fmt))
print(summer.strftime(fmt))
print(winter.astimezone(est).strftime(fmt))
print(summer.astimezone(est).strftime(fmt))
which will print:
2016-01-24 18:00:00 UTC+0000
2016-07-24 18:00:00 UTC+0000
2016-01-24 13:00:00 EST-0500
2016-07-24 14:00:00 EDT-0400
The reason why you'll need to use 'US/Eastern' and not 'EST' is exemplified in the last two lines of output.

If you have a pandas series with object datatype, you can first convert it into a DateTime series using pd.to_datetime()
df[col] = pd.to_datetime(your_series, format = '%Y-%m-%d %H:%M:%S', errors ='coerce')
Check if it is timezone aware or not by using series.dt.tz
df[col].dt.tz
If it's not timezone aware, we should make it timezone aware by using series.dt.tz_localize(). Also, do read about the ambiguous and nonexistent parameters of this function
df[col] = your_series[col].dt.tz_localize('UTC')
Now convert this series into the required timezone by series.dt.tz_convert()
df[col] = your_series[col].dt.tz_convert('US/Eastern')
The above method will take care of daylight savings time. If you want to check more timezones you can pip install pytz and
import pytz
pytz.common_timezones

In case you just want the normalized hour offset for your existing timedelta shifting:
from datetime import datetime
import pytz
def curr_est_offset():
tz_est = pytz.timezone('US/Eastern')
offset = tz_est.utcoffset(datetime.utcnow())
offset_seconds = (offset.days * 86400) + offset.seconds
offset_hours = offset_seconds // 3600
return offset_hours # -4 or -5

As mentioned above, you can use pandas.DataFrame.tz_convert() like this:
import pandas as pd
from datetime import datetime
df = pd.read_csv("your_data_file_path.csv", index_col=False, engine='python')
df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.tz_localize('US/Eastern').dt.tz_convert('UTC')
df['Date'] = df['Date'].apply(lambda x: datetime.replace(x, tzinfo=None))
What the last row does is removing the timezone info from the datetime object, so you can operate with the date and time only (don't worry, that doesn't change the timezone again, it just strips it from the timestamp string).

This is thebjorn's answer converted from Python 2 to Python 3 with a few additional comments. Thanks to thebjorn.
For convention, I use these terms:
EST: Eastern Standard Time (winter)
EDT: Eastern Daylight Time
(summer)
EPT: Eastern Prevailing Time (standard or daylight saving
time as appropriate)
Code:
# Convert EPT / UTC
import pytz
import datetime from datetime as dtdt
# Timezones
ept = pytz.timezone('US/Eastern')
utc = pytz.utc
# str format
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
print("\nEPT/UTC examples:")
print("\nWinter (EST) example:")
# Create a UTC time in the winter
winter_utc = dtdt(2016, 1, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", winter_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is standard time.
winter_ept = winter_utc.astimezone(ept)
print(" EPT: ", winter_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
winter_utc2 = winter_ept.astimezone(utc)
print(" UTC: ", winter_utc2.strftime(fmt))
# Let's do that again for a summer datetime.
print("\nSummer (EDT) example:")
summer_utc = dtdt(2016, 7, 24, 18, 0, 0, tzinfo=utc)
print(" UTC: ", summer_utc.strftime(fmt))
# Convert from UTC to eastern prevailing time. Since, the timestamp is in the
# winter, prevailing time is daylight saving time.
summer_ept = summer_utc.astimezone(ept)
print(" EPT: ", summer_ept.strftime(fmt))
# Let's convert back to UTC to show we get back to the original value.
summer_utc2 = summer_ept.astimezone(utc)
print(" UTC: ", summer_utc2.strftime(fmt))
Console:
EPT/UTC examples:
Winter (EST) example:
UTC: 2016-01-24 18:00:00 UTC+0000
EPT: 2016-01-24 13:00:00 EST-0500
UTC: 2016-01-24 18:00:00 UTC+0000
Summer (EDT) example:
UTC: 2016-07-24 18:00:00 UTC+0000
EPT: 2016-07-24 14:00:00 EDT-0400
UTC: 2016-07-24 18:00:00 UTC+0000

Determine start and end time of current day (UTC -> EST -> UTC) ; Python

I am storing all my times in UTC and my system is set to UTC (though I am in EST).
I have dates stored as:
Wed, 20 Feb 2013 03:51:39 +0000
However, I would like to select information based off today for EST, so I am attempting to:
Get current time as UTC and change to EST
datetime.utcnow().replace(tzinfo=tz.tzutc()).astimezone(tz.gettz('America/New_York'))
2013-02-19 23:17:20.560898-05:00
Next I want to get the start time for the EST day (2013-02-19 00:00:00.000000-05:00) and the end time (2013-02-19 23:59:59.99999-05:00)
Once I have those values, I'd like to convert back to UTC, so I have a high and low value I can clamp by that's correct my EST (my timezone).
If this isn't the best way to do this, or I'm missing something (does seem overly complicated to me) please help me see the light!
TIA
Update per answer:
d1 = datetime.utcnow().replace(tzinfo=tz.tzutc()).astimezone(tz.gettz('America/New_York'))
print d1.strftime("%m %d %Y") ; d2 = d1.replace(day=d1.day + 1) ; print d2.strftime("%m %d %Y")
That will give me
02 20 2013
02 21 2013
Which is correct. I now need to generate the full EST time from that and then convert to UTC. This I cannot figure out. Actually, I probably want to convert to UTC epoch timestamp when complete because that will make my database operations pretty easy (<, >, ==, etc).

The first step of getting current time as UTC and converting it to EST seems a bit pointless. Do you use that time for anything?
Other than that it seems rather straighforward. You want to get the start and end of a day EST in UTC, so you create them and convert them to UTC. That's not so complicated. :-)
You might want to look at your matching routines though, so that you can use the start of today as the lower value, and the start of tomorrow as the higher, so you don't have to deal with that 23:59:59.9999 time.
Update:
From my original understanding of your question, this is what you want to do:
First you want to get the current date as it is in UTC (so at 11pm EST the 12st, you want the 22nd, as it is the 22nd in UTC then.
>>> from datetime import datetime
>>> today = datetime.utcnow().date()
>>> today
datetime.date(2013, 2, 21)
Secondly you want 00:00:00 of that day in UTC, as start for a search.
>>> from dateutil import tz
>>> start = datetime(today.year, today.month, today.day, tzinfo=tz.tzutc())
datetime.datetime(2013, 2, 21, 0, 0, tzinfo=tzutc())
Except that you want to know what that time is in New York:
>>> from dateutil import tz
>>> est = tz.gettz('America/New_York')
>>> start = start.astimezone(est)
>>> start
datetime.datetime(2013, 2, 20, 19, 0, tzinfo=tzfile('/usr/share/zoneinfo/America/New_York'))
And you also want tomorrow as the end:
>>> from datetime import timedelta
>>> end = start + timedelta(1)
>>> end
datetime.datetime(2013, 2, 21, 19, 0, tzinfo=tzfile('/usr/share/zoneinfo/America/New_York'))
Summary:
today = datetime.utcnow().date()
start = datetime(today.year, today.month, today.day, tzinfo=tz.tzutc()).astimezone(est)
end = start + timedelta(1)

use datetime pytz will solve your problem.
def get_start_and_end():
tz = pytz.timezone('Asia/Shanghai')
today = datetime.now(tz=tz)
start = today.replace(hour=0, minute=0, second=0, microsecond=0)
end = start + timedelta(1)
return start, end

The question is old but maybe this helps:
import datetime
end_of_today = datetime.datetime.combine(datetime.datetime.today(), datetime.time(23, 59, 59, 999999))

I would definitely give Delorean a look, to solve your problem would follow a few steps.
You first need to parse your string. Excellent use the Delorean parse method.
>>> from delorean import parse
>>> d = parse("Wed, 20 Feb 2013 03:51:39 +0000")
>>> d
Delorean(datetime=2013-02-20 03:51:39+00:00, timezone=UTC)
Once you have the datetime that you parsed in a Delorean object you simply convert to EST
>>> d = d.shift('US/Eastern')
>>> d
Delorean(datetime=2013-02-19 22:51:39-05:00, timezone=US/Eastern)
Albeit pointless. You never use it for anything in your question, but super easy with Delorean.
Then you get the time now in EST
from delorean import Delorean
>>> d1 = Delorean(timezone="US/Eastern")
>>> d1
Delorean(datetime=2013-02-21 00:35:56.405256-05:00, timezone=US/Eastern)
Now for the truncation step.
>>> d.truncate('day')
Delorean(datetime=2013-02-21 00:00:00-05:00, timezone=US/Eastern)
do the simple shift as above to UTC.
Now get the end of day.
d = d.next_day(1) # move to the next day
Then to shift back one second. Something that the library needs I will be updating this. Simply get the datetime from the Delorean example by asking for it with datetime attribute.
d.datetime - timedelta(seconds=1)
datetime.datetime(2013, 2, 21, 23, 59, 59, tzinfo=<DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>)
Goodluck, but this library should simply your dealing with datetime operations :)

This is only a partial answer, because the rest has been covered well. I struggled with this for a while, as some technologies have inclusive searches, and I don't want to include any data from the first microsecond of the next day.
My solution for finding the end of day time quickly and correctly is this:
reference_time.replace(hour=0, minute=0, second=0, microsecond=0) + timedelta(days=1,microseconds=-1)

How about this
import datetime
datetime.datetime.combine(datetime.date.today(), datetime.time(00, 00, 00))
datetime.datetime.combine(datetime.date.today(), datetime.time(23, 59, 59))

Just ran into this, here's the simplest option I found:
from delorean import Delorean
today_d = Delorean()
sod_dt = today_d.start_of_day
eod_dt = today_d.end_of_day
sod_d = Delorean(sod_dt)
eod_d = Delorean(eod_dt)
sod_e = sod_d.epoch
eod_e = eod_d.epoch
to confirm:
In [69]: eod_e - sod_e
Out[69]: 86399.99999904633
close enough for most people

Here's a nice solution if you already use Arrow.
import arrow
now = arrow.now('US/Eastern')
start = now.floor('day')
end = now.ceil('day')
# Use the "datetime" property to access the actual datetime
print(start.datetime)

With python3.8, we can use as follow:
from datetime import datetime, timedelta, time
from pytz import timezone
today = datetime.utcnow()
start_of_day = datetime.combine(today, time.min).astimezone(tz=timezone('America/New_York'))
end_of_day = datetime.combine(today, time.max).astimezone(tz=timezone('America/New_York'))

To get the min and max time of the day, you could use datetime.time that gives you the earliest and latest representalbe time:
time.min --> datetime.time(0, 0)
time.max --> datetime.time(23, 59, 59, 999999)
and then you can combine this with datetime.now() or the specific date you want as below to get the desired result
from datetime import datetime, time
tmp = datetime.combine(datetime.now(), time.max)
print(tmp) --> 2022-09-27 23:59:59.999999
from datetime import datetime, time
tmp = datetime.combine(datetime.now(), time.max)
print(tmp) --> 2022-09-27 00:00:00

Simplest solution for getting the start of a day, though it might produce a phantom microsecond in unlucky floating point imprecision cases:
import datetime as dt
d = dt.datetime.now()
d = d - dt.timedelta(seconds=d.timestamp() % dt.timedelta(days=1).total_seconds())

How to get the difference in Localtime and GMT time python?

I get the server date and I need to get the difference of this date from GMT
I get
Datetime = "2011-04-27 2:17:45"
I would like to get the result like
Datetime = "2011-04-27 2:17:45 +0500"

Try this:
import datetime, pytz
now = datetime.datetime.now(pytz.timezone('Asia/Kolkata'))
print now.strftime('%Y-%m-%d %H:%M:%S %z')
# prints: '2011-04-27 13:56:09 +0530'

From the example you have given, it looks to me that what you are looking for is datetime.isoformat. The example in the page shows how to convert the datetime values to the ISO format with the time zone information.

To do this, you have to know the timezone (or the UTC offset) of the server date. What you have here is a "naive" date, without timezone info, you can't guess the UTC difference.

I think the datetime module is what you need here:
>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2011, 4, 27, 11, 8, 26, 149000)
>>> datetime.utcnow()
datetime.datetime(2011, 4, 27, 8, 8, 47, 712000)
For a difference between two dates:
>>> dtnow = datetime.now()
>>> dtutc = datetime.utcnow()
>>> dtnow - dtutc
datetime.timedelta(0, 10792, 847000)
Look up the datetime module and the relevant classes in Python's docs.

A very powerful extension of the datetime standard python library is the dateutil one, that allows you to easily:
set the delta of your time zone:
parse dates with various convenient options (in our case we will use the default option, which will allow us to set our time zone)
So 1st set time zone, and default date with this zone:
>>> from datetime import datetime
>>> from dateutil import parser
>>> from dateutil.tz import tzoffset
>>> tz_plus_5 = tzoffset(None, 5 * 60 * 60) # offset is in seconds !
>>> default = datetime.now(tz_plus_5)
Now use this default date in the parsing:
>>> Datetime = "2011-04-27 2:17:45"
>>> my_date = parser.parse(Datetime, default=default)
>>> my_date
datetime.datetime(2011, 4, 27, 2, 17, 45, tzinfo=tzoffset(None, 18000))
>>> my_date.strftime("%Y-%m-%d %H:%M:%S %z")
'2011-04-27 02:17:45 +0500'

For those that simply need to get the offset between local time and UTC, the time module has an attribute time.altzone that specifies the difference between UTC and local time in seconds:
The offset of the local DST timezone, in seconds west of UTC, if one is defined. This is negative if the local DST timezone is east of UTC (as in Western Europe, including the UK). Only use this if daylight is nonzero.
Here's an example of how it works:
>>> datetime.now().isoformat()
'2011-09-01T17:26:46.971000'
>>> datetime.utcnow().isoformat()
'2011-09-01T15:27:32.699000'
>>> time.altzone / (60*60)
-2
Doesn't get much cleaner than that.

How to convert a timedelta object into a datetime object

What is the proper way to convert a timedelta object into a datetime object?
I immediately think of something like datetime(0)+deltaObj, but that's not very nice... Isn't there a toDateTime() function or something of the sort?

It doesn't make sense to convert a timedelta into a datetime, but it does make sense to pick an initial or starting datetime and add or subtract a timedelta from that.
>>> import datetime
>>> today = datetime.datetime.today()
>>> today
datetime.datetime(2010, 3, 9, 18, 25, 19, 474362)
>>> today + datetime.timedelta(days=1)
datetime.datetime(2010, 3, 10, 18, 25, 19, 474362)

Since a datetime represents a time within a single day, your timedelta should be less than 24 hours (86400 seconds), even though timedeltas are not subject to this constraint.
import datetime
seconds = 86399
td = datetime.timedelta(seconds=seconds)
print(td)
dt = datetime.datetime.strptime(str(td), "%H:%M:%S")
print(dt)
23:59:59
1900-01-01 23:59:59
If you don't want a default date and know the date of your timedelta:
date = "05/15/2020"
dt2 = datetime.datetime.strptime("{} {}".format(date, td), "%m/%d/%Y %H:%M:%S")
print(dt2)
2020-05-15 23:59:59

I found that I could take the .total_seconds() and use that to create a new time object (or datetime object if needed).
import time
import datetime
start_dt_obj = datetime.datetime.fromtimestamp(start_timestamp)
stop_dt_obj = datetime.datetime.fromtimestamp(stop_timestamp)
delta = stop_dt_obj - start_dt_obj
delta_as_time_obj = time.gmtime(delta.total_seconds())
This allows you to do something like:
print('The duration was {0}'.format(
time.strftime('%H:%M', delta_as_time_obj)
)

Improving #sadpanduar answer with example on converting one column in pandas.DataFrame:
from datetime import timedelta
import time
def seconds_to_datetime(seconds, format='%Y-%m-%d %H:%M:%S'):
td = timedelta(seconds=seconds)
time_obj = time.gmtime(td.total_seconds())
return time.strftime(format, time_obj)
df = pd.read_csv(CSV_PATH)
df['TIMESTAMP_COLUMN'] = df['TIMESTAMP_COLUMN'].apply(seconds_to_datetime)

import datetime`enter code here
lastDownloadedDate = datetime.date(2022,8,4)
print('lastDownloadedDate: ', lastDownloadedDate)
fdate = lastDownloadedDate + datetime.timedelta(days=1)
fdate = datetime.datetime.strptime(str(fdate), "%Y-%m-%d")
fdate = datetime.date(fdate.year, fdate.month, fdate.day)
print('fdate: ', dt3)`