Related
I would like to filter pandas using the time stamp. This works fine for all hours except 0. If I filter for dt.hour = 0, only the date is displayed and not the time. How can I have the time displayed too?
import datetime
df = pd.DataFrame({'datetime': [datetime.datetime(2005, 7, 14, 12, 30),
datetime.datetime(2005, 7, 14, 0, 0),
datetime.datetime(2005, 7, 14, 10, 30),
datetime.datetime(2005, 7, 14, 15, 30)]})
print(df[df['datetime'].dt.hour == 10])
print(df[df['datetime'].dt.hour == 0]
use strftime:
print(df[df['datetime'].dt.hour == 0].datetime.dt.strftime("%Y-%m-%d %H:%M:%S"))
The result is:
1 2005-07-14 00:00:00
Name: datetime, dtype: object
I have an array of unixtime timestamps. How do I convert that using
datetime.utcfromtimestamp().strftime("%Y-%M-%D %H:%M:%S")
? My array is saved under "time". How do I utilize that array in this conversion?
Assuming your times are of the format datetime, you can loop through the list and convert each one.
Here is a quick example:
import datetime
time = []
for i in range(10):
time.append(datetime.datetime.now())
print(time) # output: [datetime.datetime(2020, 7, 8, 10, 7, 4, 314614), datetime.datetime(2020, 7, 8, 10, 7, 4, 314622)....
formattedTime = []
for t in time:
formattedTime.append(t.strftime('%Y-%m-%d %H:%M:%S'))
print(formattedTime) # output: ['2020-07-07/08/20 10:07:04', '2020-07-07/08/20 10:07:04', ....
# the update to my answer:
newTimes = []
for date_time_str in formattedTime:
newTimes.append(datetime.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S'))
print(newTimes) # '%Y-%m-%d %H:%M:%S' [datetime.datetime(2020, 7, 8, 18, 56, 47), datetime.datetime(2020, 7, 8, 18, 56, 47), datetime.datetime(2020, 7, 8, 18, 56, 47),...]
Let me know if you have more questions.
I am also attaching this article for datetime which I found really helpful.
Here is the example in repl
Is there any efficient way to generate a list of N random timeframes which do not intersect each other given the initial lower and upper bounds as well as the time intervals that these time periods should have. For example in the following case I want 10 timestamps between 09:00-17:00:
Initial start time: {datetime} YYYY-MM-DD 09:00:00
Initial end time: {datetime} YYYY-MM-DD 17:00:00
Timestamp intervals (in minutes): [32 24 4 20 40 8 27 18 3 4]
where the first time period 32 minutes long, the next 24 and so on.
The way I am doing it at the moment is by using more or less the following code snippet:
def random_time(start, end, timeframe=None):
sec_diff = int((end - start).total_seconds())
secs_to_add = random.randint(0, sec_diff)
return start + timedelta(seconds=secs_to_add)
def in_datetimes_range(self, x, starts, ends):
return np.any((starts <= x) & (x <= ends))
n = 10
dadate = datetime.now()
year = self.dadate.year
month = self.dadate.month
day = self.dadate.day
start = datetime(year, month, day, 9, 0, 0)
end = datetime(year, month, day, 17, 0, 0)
timeframe = [32 24 4 20 40 8 27 18 3 4]
startTimes = []
endTimes = []
for i in range(0, n):
while True:
startTime = random_time(start, end)
endTime = startTime + timedelta(minutes=int(timeframe[i]))
if startTimes:
startTimesAsNpArray = np.array(startTimes)
endTimesAsNpArray = np.array(endTimes)
#check if new time period falls inside existing timeframes or if existing timeframes fall within new time period
inner_bound = np.logical_or(in_datetimes_range(startTime, startTimesAsNpArray, endTimesAsNpArray), in_datetimes_range(endTime, startTimesAsNpArray, endTimesAsNpArray))
outer_bound = np.logical_or(in_datetimes_range(startTimesAsNpArray, startTime, endTime), in_datetimes_range(endTimesAsNpArray, startTime, endTime))
if not inner_bound and not outer_bound:
startTimes.append(startTime)
endTimes.append(endTime)
break
but this is really inefficient and I was looking for something more reliable if possible.
Here is a way to do it: the idea is that if we remove the total duration of the periods from the time available, generate start times in the period that is left, and then postpone them with the cumulated periods before them, we are sure that the intervals won't overlap.
from datetime import datetime, timedelta
import random
def generate_periods(start, end, durations):
durations = [timedelta(minutes=m) for m in durations]
total_duration = sum(durations, timedelta())
nb_periods = len(durations)
open_duration = (end - start) - total_duration
delays = sorted(timedelta(seconds=s)
for s in random.sample(range(0, int(open_duration.total_seconds())), nb_periods))
periods = []
periods_before = timedelta()
for delay, duration in zip(delays, durations):
periods.append((start + delay + periods_before,
start + delay + periods_before + duration))
periods_before += duration
return periods
Sample run:
durations = [32, 24, 4, 20, 40, 8, 27, 18, 3, 4]
start_time = datetime(2019, 9, 2, 9, 0, 0)
end_time = datetime(2019, 9, 2, 17, 0, 0)
generate_periods(start_time, end_time, durations)
# [(datetime.datetime(2019, 9, 2, 9, 16, 1),
# datetime.datetime(2019, 9, 2, 9, 48, 1)),
# (datetime.datetime(2019, 9, 2, 9, 58, 57),
# datetime.datetime(2019, 9, 2, 10, 22, 57)),
# (datetime.datetime(2019, 9, 2, 10, 56, 41),
# datetime.datetime(2019, 9, 2, 11, 0, 41)),
# (datetime.datetime(2019, 9, 2, 11, 2, 37),
# datetime.datetime(2019, 9, 2, 11, 22, 37)),
# (datetime.datetime(2019, 9, 2, 11, 48, 17),
# datetime.datetime(2019, 9, 2, 12, 28, 17)),
# (datetime.datetime(2019, 9, 2, 13, 4, 28),
# datetime.datetime(2019, 9, 2, 13, 12, 28)),
# (datetime.datetime(2019, 9, 2, 15, 13, 3),
# datetime.datetime(2019, 9, 2, 15, 40, 3)),
# (datetime.datetime(2019, 9, 2, 16, 6, 44),
# datetime.datetime(2019, 9, 2, 16, 24, 44)),
# (datetime.datetime(2019, 9, 2, 16, 37, 42),
# datetime.datetime(2019, 9, 2, 16, 40, 42)),
# (datetime.datetime(2019, 9, 2, 16, 42, 50),
# datetime.datetime(2019, 9, 2, 16, 46, 50))]
Like this?
import pandas as pd
from datetime import datetime
date = datetime.now()
start = datetime(date.year, date.month, date.day, 9, 0, 0)
end = datetime(date.year, date.month, date.day, 17, 0, 0)
interval = 32
periods = (end-start).seconds/60/interval
times = pd.date_range(start.strftime("%m/%d/%Y, %H:%M:%S"), periods=periods, freq=str(interval)+'min')
or like this
# =============================================================================
# or if you want the results as a dataframe
# =============================================================================
def xyz(interval):
date = datetime.now()
start = datetime(date.year, date.month, date.day, 9, 0, 0)
end = datetime(date.year, date.month, date.day, 17, 0, 0)
periods = (end-start).seconds/60/interval
return pd.date_range(start.strftime("%m/%d/%Y, %H:%M:%S"), periods=periods, freq=str(interval)+'min')
timeframes = [32,24,4,20,40,8,27,18,3,4]
df_output=pd.DataFrame(index=timeframes, data=[xyz(x) for x in timeframes])
I am trying to generate time interval array. for example:
time_array = ["2016-09-02T17:30:00Z", "2016-09-02T17:45:00Z",
"2016-09-02T18:00:00Z", "2016-09-02T18:15:00Z",
"2016-09-02T18:30:00Z", "2016-09-02T18:45:00Z"]
It should create the element like above in zulu time till 9 pm everyday.
Should generate the elements for next and day after next as well
Start time from 7:00 am - Ed time 9:00 pm,
if current_time is > start_time then generate 15 min time interval array till 9 pm. and then generate for next day and day + 2.
And Interval should be 7:00, 7:15 like that.. not in 7:12, 8:32
Here's a generic datetime_range for you to use.
Code
from datetime import datetime, timedelta
def datetime_range(start, end, delta):
current = start
while current < end:
yield current
current += delta
dts = [dt.strftime('%Y-%m-%d T%H:%M Z') for dt in
datetime_range(datetime(2016, 9, 1, 7), datetime(2016, 9, 1, 9+12),
timedelta(minutes=15))]
print(dts)
Output
['2016-09-01 T07:00 Z', '2016-09-01 T07:15 Z', '2016-09-01 T07:30 Z', '2016-09-01 T07:45 Z', '2016-09-01 T08:00 Z', '2016-09-01 T08:15 Z', '2016-09-01 T08:30 Z', '2016-09-01 T08:45 Z', '2016-09-01 T09:00 Z', '2016-09-01 T09:15 Z', '2016-09-01 T09:30 Z', '2016-09-01 T09:45 Z' ... ]
Here is a Pandas solution:
import pandas as pd
l = (pd.DataFrame(columns=['NULL'],
index=pd.date_range('2016-09-02T17:30:00Z', '2016-09-04T21:00:00Z',
freq='15T'))
.between_time('07:00','21:00')
.index.strftime('%Y-%m-%dT%H:%M:%SZ')
.tolist()
)
Output:
In [165]: l
Out[165]:
['2016-09-02T17:30:00Z',
'2016-09-02T17:45:00Z',
'2016-09-02T18:00:00Z',
'2016-09-02T18:15:00Z',
'2016-09-02T18:30:00Z',
'2016-09-02T18:45:00Z',
'2016-09-02T19:00:00Z',
'2016-09-02T19:15:00Z',
'2016-09-02T19:30:00Z',
'2016-09-02T19:45:00Z',
'2016-09-02T20:00:00Z',
'2016-09-02T20:15:00Z',
'2016-09-02T20:30:00Z',
'2016-09-02T20:45:00Z',
'2016-09-02T21:00:00Z',
'2016-09-03T07:00:00Z',
'2016-09-03T07:15:00Z',
'2016-09-03T07:30:00Z',
'2016-09-03T07:45:00Z',
'2016-09-03T08:00:00Z',
'2016-09-03T08:15:00Z',
'2016-09-03T08:30:00Z',
'2016-09-03T08:45:00Z',
'2016-09-03T09:00:00Z',
'2016-09-03T09:15:00Z',
'2016-09-03T09:30:00Z',
'2016-09-03T09:45:00Z',
'2016-09-03T10:00:00Z',
'2016-09-03T10:15:00Z',
'2016-09-03T10:30:00Z',
'2016-09-03T10:45:00Z',
'2016-09-03T11:00:00Z',
'2016-09-03T11:15:00Z',
'2016-09-03T11:30:00Z',
'2016-09-03T11:45:00Z',
'2016-09-03T12:00:00Z',
'2016-09-03T12:15:00Z',
'2016-09-03T12:30:00Z',
'2016-09-03T12:45:00Z',
'2016-09-03T13:00:00Z',
'2016-09-03T13:15:00Z',
'2016-09-03T13:30:00Z',
'2016-09-03T13:45:00Z',
'2016-09-03T14:00:00Z',
'2016-09-03T14:15:00Z',
'2016-09-03T14:30:00Z',
'2016-09-03T14:45:00Z',
'2016-09-03T15:00:00Z',
'2016-09-03T15:15:00Z',
'2016-09-03T15:30:00Z',
'2016-09-03T15:45:00Z',
'2016-09-03T16:00:00Z',
'2016-09-03T16:15:00Z',
'2016-09-03T16:30:00Z',
'2016-09-03T16:45:00Z',
'2016-09-03T17:00:00Z',
'2016-09-03T17:15:00Z',
'2016-09-03T17:30:00Z',
'2016-09-03T17:45:00Z',
'2016-09-03T18:00:00Z',
'2016-09-03T18:15:00Z',
'2016-09-03T18:30:00Z',
'2016-09-03T18:45:00Z',
'2016-09-03T19:00:00Z',
'2016-09-03T19:15:00Z',
'2016-09-03T19:30:00Z',
'2016-09-03T19:45:00Z',
'2016-09-03T20:00:00Z',
'2016-09-03T20:15:00Z',
'2016-09-03T20:30:00Z',
'2016-09-03T20:45:00Z',
'2016-09-03T21:00:00Z',
'2016-09-04T07:00:00Z',
'2016-09-04T07:15:00Z',
'2016-09-04T07:30:00Z',
'2016-09-04T07:45:00Z',
'2016-09-04T08:00:00Z',
'2016-09-04T08:15:00Z',
'2016-09-04T08:30:00Z',
'2016-09-04T08:45:00Z',
'2016-09-04T09:00:00Z',
'2016-09-04T09:15:00Z',
'2016-09-04T09:30:00Z',
'2016-09-04T09:45:00Z',
'2016-09-04T10:00:00Z',
'2016-09-04T10:15:00Z',
'2016-09-04T10:30:00Z',
'2016-09-04T10:45:00Z',
'2016-09-04T11:00:00Z',
'2016-09-04T11:15:00Z',
'2016-09-04T11:30:00Z',
'2016-09-04T11:45:00Z',
'2016-09-04T12:00:00Z',
'2016-09-04T12:15:00Z',
'2016-09-04T12:30:00Z',
'2016-09-04T12:45:00Z',
'2016-09-04T13:00:00Z',
'2016-09-04T13:15:00Z',
'2016-09-04T13:30:00Z',
'2016-09-04T13:45:00Z',
'2016-09-04T14:00:00Z',
'2016-09-04T14:15:00Z',
'2016-09-04T14:30:00Z',
'2016-09-04T14:45:00Z',
'2016-09-04T15:00:00Z',
'2016-09-04T15:15:00Z',
'2016-09-04T15:30:00Z',
'2016-09-04T15:45:00Z',
'2016-09-04T16:00:00Z',
'2016-09-04T16:15:00Z',
'2016-09-04T16:30:00Z',
'2016-09-04T16:45:00Z',
'2016-09-04T17:00:00Z',
'2016-09-04T17:15:00Z',
'2016-09-04T17:30:00Z',
'2016-09-04T17:45:00Z',
'2016-09-04T18:00:00Z',
'2016-09-04T18:15:00Z',
'2016-09-04T18:30:00Z',
'2016-09-04T18:45:00Z',
'2016-09-04T19:00:00Z',
'2016-09-04T19:15:00Z',
'2016-09-04T19:30:00Z',
'2016-09-04T19:45:00Z',
'2016-09-04T20:00:00Z',
'2016-09-04T20:15:00Z',
'2016-09-04T20:30:00Z',
'2016-09-04T20:45:00Z',
'2016-09-04T21:00:00Z']
Looking at the data file, you should use the built in python date-time objects. followed by strftime to format your dates.
Broadly you can modify the code below to however many date-times you would like
First create a starting date.
Today= datetime.datetime.today()
Replace 100 with whatever number of time intervals you want.
date_list = [Today + datetime.timedelta(minutes=15*x) for x in range(0, 100)]
Finally, format the list in the way that you would like, using code like that below.
datetext=[x.strftime('%Y-%m-%d T%H:%M Z') for x in date_list]
Here is an example using an arbitrary date time
from datetime import datetime
start = datetime(1900,1,1,0,0,0)
end = datetime(1900,1,2,0,0,0)
Now you need to get the timedelta (the difference between two dates or times.) between the start and end
seconds = (end - start).total_seconds()
Define the 15 minutes interval
from datetime import timedelta
step = timedelta(minutes=15)
Iterate over the range of seconds, with step of time delta of 15 minutes (900 seconds) and sum it to start.
array = []
for i in range(0, int(seconds), int(step.total_seconds())):
array.append(start + timedelta(seconds=i))
print array
[datetime.datetime(1900, 1, 1, 0, 0),
datetime.datetime(1900, 1, 1, 0, 15),
datetime.datetime(1900, 1, 1, 0, 30),
datetime.datetime(1900, 1, 1, 0, 45),
datetime.datetime(1900, 1, 1, 1, 0),
...
At the end you can format the datetime objects to str representation.
array = [i.strftime('%Y-%m-%d %H:%M%:%S') for i in array]
print array
['1900-01-01 00:00:00',
'1900-01-01 00:15:00',
'1900-01-01 00:30:00',
'1900-01-01 00:45:00',
'1900-01-01 01:00:00',
...
You can format datetime object at first iteration. But it may hurt your eyes
array.append((start + timedelta(seconds=i)).strftime('%Y-%m-%d %H:%M%:%S'))
I'll provide a solution that does not handle timezones, since the problem is generating dates and times and you can set the timezone afterwards however you want.
You have a starting date and starting and ending time (for each day), plus an interval (in minutes) for these datetimes. The idea is to create a timedelta object that represent the time interval and repeatedly update the datetime until we reach the ending time, then we advance by one day and reset the time to the initial one and repeat.
A simple implementation could be:
def make_dates(start_date, number_of_days, start_time, end_time, interval, timezone):
if isinstance(start_date, datetime.datetime):
start_date = start_date.date()
start_date = datetime.datetime.combine(start_date, start_time)
cur_date = start_date
num_days_passed = 0
step = datetime.timedelta(seconds=interval*60)
while True:
new_date = cur_date + step
if new_date.time() > end_time:
num_days_passed += 1
if num_days_passed > number_of_days:
break
new_date = start_date + datetime.timedelta(days=num_days_passed)
ret_date, cur_date = cur_date, new_date
yield ret_date
In [31]: generator = make_dates(datetime.datetime.now(), 3, datetime.time(hour=17), datetime.time(hour=19), 15, None)
In [32]: next(generator)
Out[32]: datetime.datetime(2016, 9, 2, 17, 0)
In [33]: next(generator)
Out[33]: datetime.datetime(2016, 9, 2, 17, 15)
In [34]: list(generator)
Out[34]:
[datetime.datetime(2016, 9, 2, 17, 30),
datetime.datetime(2016, 9, 2, 17, 45),
datetime.datetime(2016, 9, 2, 18, 0),
datetime.datetime(2016, 9, 2, 18, 15),
datetime.datetime(2016, 9, 2, 18, 30),
datetime.datetime(2016, 9, 2, 18, 45),
datetime.datetime(2016, 9, 2, 19, 0),
datetime.datetime(2016, 9, 3, 17, 0),
datetime.datetime(2016, 9, 3, 17, 15),
datetime.datetime(2016, 9, 3, 17, 30),
datetime.datetime(2016, 9, 3, 17, 45),
datetime.datetime(2016, 9, 3, 18, 0),
datetime.datetime(2016, 9, 3, 18, 15),
datetime.datetime(2016, 9, 3, 18, 30),
datetime.datetime(2016, 9, 3, 18, 45),
datetime.datetime(2016, 9, 3, 19, 0),
datetime.datetime(2016, 9, 4, 17, 0),
datetime.datetime(2016, 9, 4, 17, 15),
datetime.datetime(2016, 9, 4, 17, 30),
datetime.datetime(2016, 9, 4, 17, 45),
datetime.datetime(2016, 9, 4, 18, 0),
datetime.datetime(2016, 9, 4, 18, 15),
datetime.datetime(2016, 9, 4, 18, 30),
datetime.datetime(2016, 9, 4, 18, 45),
datetime.datetime(2016, 9, 4, 19, 0),
datetime.datetime(2016, 9, 5, 17, 0),
datetime.datetime(2016, 9, 5, 17, 15),
datetime.datetime(2016, 9, 5, 17, 30),
datetime.datetime(2016, 9, 5, 17, 45),
datetime.datetime(2016, 9, 5, 18, 0),
datetime.datetime(2016, 9, 5, 18, 15),
datetime.datetime(2016, 9, 5, 18, 30),
datetime.datetime(2016, 9, 5, 18, 45)]
Once you have the datetimes you can use the strftime method to convert them to strings.
This is the final script I have written based on the answers posted on my question:
from datetime import datetime
from datetime import timedelta
import calendar
current_utc = datetime.utcnow().strftime("%Y-%m-%d-%H-%M-%S")
current_year = int(current_utc.split("-")[0])
current_month = int(current_utc.split("-")[1])
current_date = int(current_utc.split("-")[2])
current_hour = int(current_utc.split("-")[3])
current_min = int(current_utc.split("-")[4])
current_sec = int(current_utc.split("-")[5])
#### To make minutes round to quarter ####
min_range_1 = range(1,16)
min_range_2 = range(16,31)
min_range_3 = range(31,46)
min_range_4 = range(46,60)
if current_min in min_range_1:
current_min = 15
elif current_min in min_range_2:
current_min = 30
elif current_min in min_range_3:
current_min = 45
elif current_min in min_range_4:
current_hour = current_hour + 1
current_min = 0
else:
print("Please check current minute.")
current_sec = 00
date_range_31 = range(1,32)
date_range_30 = range(1,31)
month_days_31 = [1,3,5,7,8,10,12]
month_days_30 = [4,6,9,11]
if current_month in month_days_31:
if current_date == 31:
next_day_month = current_month + 1
next_day_date = 1
else:
next_day_month = current_month
next_day_date = current_date
elif current_month == 2:
if calendar.isleap(current_year):
if current_date == 29:
next_day_month = current_month + 1
next_day_date = 1
else:
next_day_month = current_month
next_day_date = current_date
else:
if current_date == 28:
next_day_month = current_month + 1
next_day_date = 1
else:
next_day_month = current_month
next_day_date = current_date
elif current_month in month_days_30:
if current_date == 30:
next_day_month = current_month + 1
next_day_date = 1
else:
next_day_month = current_month
next_day_date = current_date
else:
print("Please check the current month and date to procedd further.")
if current_hour < 11:
current_hour = 11
current_min = 15
next_day_date = current_date + 1
current_start = datetime(current_year,current_month,current_date,current_hour,current_min,current_sec)
current_end = datetime(current_year,current_month,current_date,21,15,0)
next_day_start = datetime(current_year,next_day_month,next_day_date,11,15,0)
next_day_end = datetime(current_year,next_day_month,next_day_date,21,15,0)
current_seconds = (current_end - current_start).total_seconds()
next_day_seconds = (next_day_end - next_day_start).total_seconds()
step = timedelta(minutes=15)
current_day_array = []
next_day_array = []
for i in range(0, int(current_seconds), int(step.total_seconds())):
current_day_array.append(current_start + timedelta(seconds=i))
for i in range(0, int(next_day_seconds), int(step.total_seconds())):
current_day_array.append(next_day_start + timedelta(seconds=i))
current_day_array = [i.strftime('%Y-%m-%dT%H:%M%:%SZ') for i in current_day_array]
print current_day_array
Which produces the following output:
['2016-09-03T11:15:00Z', '2016-09-03T11:30:00Z', '2016-09-03T11:45:00Z', '2016-09-03T12:00:00Z', '2016-09-03T12:15:00Z', '2016-09-03T12:30:00Z', '2016-09-03T12:45:00Z', '2016-09-03T13:00:00Z', '2016-09-03T13:15:00Z', '2016-09-03T13:30:00Z', '2016-09-03T13:45:00Z', '2016-09-03T14:00:00Z', '2016-09-03T14:15:00Z', '2016-09-03T14:30:00Z', '2016-09-03T14:45:00Z', '2016-09-03T15:00:00Z', '2016-09-03T15:15:00Z', '2016-09-03T15:30:00Z', '2016-09-03T15:45:00Z', '2016-09-03T16:00:00Z', '2016-09-03T16:15:00Z', '2016-09-03T16:30:00Z', '2016-09-03T16:45:00Z', '2016-09-03T17:00:00Z', '2016-09-03T17:15:00Z', '2016-09-03T17:30:00Z', '2016-09-03T17:45:00Z', '2016-09-03T18:00:00Z', '2016-09-03T18:15:00Z', '2016-09-03T18:30:00Z', '2016-09-03T18:45:00Z', '2016-09-03T19:00:00Z', '2016-09-03T19:15:00Z', '2016-09-03T19:30:00Z', '2016-09-03T19:45:00Z', '2016-09-03T20:00:00Z', '2016-09-03T20:15:00Z', '2016-09-03T20:30:00Z', '2016-09-03T20:45:00Z', '2016-09-03T21:00:00Z', '2016-09-04T11:15:00Z', '2016-09-04T11:30:00Z', '2016-09-04T11:45:00Z', '2016-09-04T12:00:00Z', '2016-09-04T12:15:00Z', '2016-09-04T12:30:00Z', '2016-09-04T12:45:00Z', '2016-09-04T13:00:00Z', '2016-09-04T13:15:00Z', '2016-09-04T13:30:00Z', '2016-09-04T13:45:00Z', '2016-09-04T14:00:00Z', '2016-09-04T14:15:00Z', '2016-09-04T14:30:00Z', '2016-09-04T14:45:00Z', '2016-09-04T15:00:00Z', '2016-09-04T15:15:00Z', '2016-09-04T15:30:00Z', '2016-09-04T15:45:00Z', '2016-09-04T16:00:00Z', '2016-09-04T16:15:00Z', '2016-09-04T16:30:00Z', '2016-09-04T16:45:00Z', '2016-09-04T17:00:00Z', '2016-09-04T17:15:00Z', '2016-09-04T17:30:00Z', '2016-09-04T17:45:00Z', '2016-09-04T18:00:00Z', '2016-09-04T18:15:00Z', '2016-09-04T18:30:00Z', '2016-09-04T18:45:00Z', '2016-09-04T19:00:00Z', '2016-09-04T19:15:00Z', '2016-09-04T19:30:00Z', '2016-09-04T19:45:00Z', '2016-09-04T20:00:00Z', '2016-09-04T20:15:00Z', '2016-09-04T20:30:00Z', '2016-09-04T20:45:00Z', '2016-09-04T21:00:00Z']
Let's imagine this datetime
>>> import datetime
>>> dt = datetime.datetime(2012, 10, 25, 17, 32, 16)
I'd like to ceil it to the next quarter of hour, in order to get
datetime.datetime(2012, 10, 25, 17, 45)
I imagine something like
>>> quarter = datetime.timedelta(minutes=15)
>>> import math
>>> ceiled_dt = math.ceil(dt / quarter) * quarter
But of course, this does not work
This one takes microseconds into account!
import math
def ceil_dt(dt):
# how many secs have passed this hour
nsecs = dt.minute*60 + dt.second + dt.microsecond*1e-6
# number of seconds to next quarter hour mark
# Non-analytic (brute force is fun) way:
# delta = next(x for x in xrange(0,3601,900) if x>=nsecs) - nsecs
# analytic way:
delta = math.ceil(nsecs / 900) * 900 - nsecs
#time + number of seconds to quarter hour mark.
return dt + datetime.timedelta(seconds=delta)
t1 = datetime.datetime(2017, 3, 6, 7, 0)
assert ceil_dt(t1) == t1
t2 = datetime.datetime(2017, 3, 6, 7, 1)
assert ceil_dt(t2) == datetime.datetime(2017, 3, 6, 7, 15)
t3 = datetime.datetime(2017, 3, 6, 7, 15)
assert ceil_dt(t3) == t3
t4 = datetime.datetime(2017, 3, 6, 7, 16)
assert ceil_dt(t4) == datetime.datetime(2017, 3, 6, 7, 30)
t5 = datetime.datetime(2017, 3, 6, 7, 30)
assert ceil_dt(t5) == t5
t6 = datetime.datetime(2017, 3, 6, 7, 31)
assert ceil_dt(t6) == datetime.datetime(2017, 3, 6, 7, 45)
t7 = datetime.datetime(2017, 3, 6, 7, 45)
assert ceil_dt(t7) == t7
t8 = datetime.datetime(2017, 3, 6, 7, 46)
assert ceil_dt(t8) == datetime.datetime(2017, 3, 6, 8, 0)
Explanation of delta:
900 seconds is 15 minutes (a quarter of an hour sans leap seconds which I don't think datetime handles...)
nsecs / 900 is the number of quarter hour chunks that have transpired. Taking the ceil of this rounds up the number of quarter hour chunks.
Multiply the number of quarter hour chunks by 900 to figure out how many seconds have transpired in since the start of the hour after "rounding".
#Mark Dickinson suggested the best formula so far:
def ceil_dt(dt, delta):
return dt + (datetime.min - dt) % delta
In Python 3, for an arbitrary time delta (not just 15 minutes):
#!/usr/bin/env python3
import math
from datetime import datetime, timedelta
def ceil_dt(dt, delta):
return datetime.min + math.ceil((dt - datetime.min) / delta) * delta
print(ceil_dt(datetime(2012, 10, 25, 17, 32, 16), timedelta(minutes=15)))
# -> 2012-10-25 17:45:00
To avoid intermediate floats, divmod() could be used:
def ceil_dt(dt, delta):
q, r = divmod(dt - datetime.min, delta)
return (datetime.min + (q + 1)*delta) if r else dt
Example:
>>> ceil_dt(datetime(2012, 10, 25, 17, 32, 16), timedelta(minutes=15))
datetime.datetime(2012, 10, 25, 17, 45)
>>> ceil_dt(datetime.min, datetime.resolution)
datetime.datetime(1, 1, 1, 0, 0)
>>> ceil_dt(datetime.min, 2*datetime.resolution)
datetime.datetime(1, 1, 1, 0, 0)
>>> ceil_dt(datetime.max, datetime.resolution)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
>>> ceil_dt(datetime.max, 2*datetime.resolution)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in ceil_dt
OverflowError: date value out of range
>>> ceil_dt(datetime.min+datetime.resolution, datetime.resolution)
datetime.datetime(1, 1, 1, 0, 0, 0, 1)
>>> ceil_dt(datetime.min+datetime.resolution, 2*datetime.resolution)
datetime.datetime(1, 1, 1, 0, 0, 0, 2)
>>> ceil_dt(datetime.max-datetime.resolution, datetime.resolution)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999998)
>>> ceil_dt(datetime.max-datetime.resolution, 2*datetime.resolution)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999998)
>>> ceil_dt(datetime.max-2*datetime.resolution, datetime.resolution)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999997)
>>> ceil_dt(datetime.max-2*datetime.resolution, 2*datetime.resolution)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999998)
>>> ceil_dt(datetime.max-timedelta(1), datetime.resolution)
datetime.datetime(9999, 12, 30, 23, 59, 59, 999999)
>>> ceil_dt(datetime.max-timedelta(1), 2*datetime.resolution)
datetime.datetime(9999, 12, 31, 0, 0)
>>> ceil_dt(datetime.min, datetime.max-datetime.min)
datetime.datetime(1, 1, 1, 0, 0)
>>> ceil_dt(datetime.max, datetime.max-datetime.min)
datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
def ceil(dt):
if dt.minute % 15 or dt.second:
return dt + datetime.timedelta(minutes = 15 - dt.minute % 15,
seconds = -(dt.second % 60))
else:
return dt
This gives you:
>>> ceil(datetime.datetime(2012,10,25, 17,45))
datetime.datetime(2012, 10, 25, 17, 45)
>>> ceil(datetime.datetime(2012,10,25, 17,45,1))
datetime.datetime(2012, 10, 25, 18, 0)
>>> ceil(datetime.datetime(2012,12,31,23,59,0))
datetime.datetime(2013,1,1,0,0)
You just need to calculate correct minutes and add them in datetime object after setting minutes, seconds to zero
import datetime
def quarter_datetime(dt):
minute = (dt.minute//15+1)*15
return dt.replace(minute=0, second=0)+datetime.timedelta(minutes=minute)
for minute in [12, 22, 35, 52]:
print quarter_datetime(datetime.datetime(2012, 10, 25, 17, minute, 16))
It works for all cases:
2012-10-25 17:15:00
2012-10-25 17:30:00
2012-10-25 17:45:00
2012-10-25 18:00:00
The formula proposed here by #Mark Dickinson worked beautifully, but I needed a solution that also handled timezones and Daylight Savings Time (DST).
Using pytz, I arrived at:
import pytz
from datetime import datetime, timedelta
def datetime_ceiling(dt, delta):
# Preserve original timezone info
original_tz = dt.tzinfo
if original_tz:
# If the original was timezone aware, translate to UTC.
# This is necessary because datetime math does not take
# DST into account, so first we normalize the datetime...
dt = dt.astimezone(pytz.UTC)
# ... and then make it timezone naive
dt = dt.replace(tzinfo=None)
# We only do math on a timezone naive object, which allows
# us to pass naive objects directly to the function
dt = dt + ((datetime.min - dt) % delta)
if original_tz:
# If the original was tz aware, we make the result aware...
dt = pytz.UTC.localize(dt)
# ... then translate it from UTC back its original tz.
# This translation applies appropriate DST status.
dt = dt.astimezone(original_tz)
return dt
A nearly identical floor function can be made by changing one line of code:
def datetime_floor(dt, delta):
...
dt = dt - ((datetime.min - dt) % delta)
...
The following datetime is three minutes before the transition from DST back to Standard Time (STD):
datetime.datetime(2020, 11, 1, 1, 57, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
Assuming the above as dt, we can round down to the nearest five minute increment using our floor function:
>>> datetime_floor(dt, timedelta(minutes=5))
datetime.datetime(2020, 11, 1, 1, 55, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
The timezone and relationship to DST is preserved. (The same would be true for the ceiling function.)
On this date DST will end at 2 am, at which point the time will "roll back" to 1am STD. If we use our ceiling function to round up from 1:57am DST, we should not end up at 2am DST, but rather at 1:00am STD, which is the result we get:
>>> datetime_ceiling(dt, timedelta(minutes=5))
datetime.datetime(2020, 11, 1, 1, 0, tzinfo=<DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>)
Here is my code working with any periods:
def floorDT(dt, secperiod):
tstmp = dt.timestamp()
return datetime.datetime.fromtimestamp(
math.floor(tstmp/secperiod)*secperiod).astimezone().astimezone(datetime.timezone.utc)
def ceilDT(dt, secperiod):
tstmp = dt.timestamp()
return datetime.datetime.fromtimestamp(
math.ceil(tstmp/secperiod)*secperiod).astimezone().astimezone(datetime.timezone.utc)
Note: we must use astimezone().astimezone() trick else it uses local timezone during converting from timestamp
Based on the example by Mark Dickinson's, here is an enhanced datetime round up function that also preserves the timezone information of the original timestamp (both naive and timezone aware):
def round_datetime_up(
ts: datetime.datetime,
delta: datetime.timedelta,
offset: datetime.timedelta = datetime.timedelta(minutes=0)) -> datetime.datetime:
"""Snap to next available timedelta.
Preserve any timezone info on `ts`.
If we are at the the given exact delta, then do not round, only add offset.
:param ts: Timestamp we want to round
:param delta: Our snap grid
:param offset: Add a fixed time offset at the top of rounding
:return: Rounded up datetime
"""
rounded = ts + (datetime.datetime.min.replace(tzinfo=ts.tzinfo) - ts) % delta
return rounded + offset
See full code and tests.