Develop Reference

Creating datetime series in python

I have a recording with the start time (t0) and length of the recording (N). How do I create a time series vector (ts) in python for every 10s increments?
For example:
t0 = 2017-06-12T11:05:10.00
N=1000
So there should be an array of 100 (N/10)values such that:
ts = [2017-06-12T11:05:10.00, 2017-06-12T11:05:20.00,2017-06-12T11:05:30.00 and so on...]

You can use the datetime module of Python.
First you need to convert your string into a date with dateteime.strptime:
t0 = datetime.datetime.strptime("2017-06-12T11:05:10.00", "%Y-%m-%dT%H:%M:%S.00")
where the "%Y-%m-%dT%H:%M:%S.00" part is the description of your string format (see documentation)
Then you can increment a datetime object by adding a timedelta to it. Build a sequence like this:
delta = datetime.timedelta(seconds=10)
ts = [t0 + i*delta for i in range(N)]
You can also recover dates as strings by using datetime.strftime with a similar syntax to strptime.
The whole thing would look like
from datetime import datetime, timedelta
date_format = "%Y-%m-%dT%H:%M:%S.00"
t0 = datetime.strptime("2017-06-12T11:05:10.00", date_format)
delta = timedelta(seconds=10)
ts = [datetime.strftime(t0 + i * delta, date_format) for i in range(100)]

import datetime
import numpy as np
t0=datetime.datetime(2017, 6, 12, 11, 5, 10)
dt=datetime.timedelta(seconds=10)
ts=np.arange(100)*dt+t0
There might be some easier way. I've never tried to find one. But this is how I do it.

How to Split a substract of a date in python

My code is the following:
date = datetime.datetime.now()- datetime.datetime.now()
print date
h, m , s = str(date).split(':')
When I print h the result is:
-1 day, 23
How do I get only the hour (the 23) from the substract using datetime?
Thanks.

If you subtract the current date from a past date, you would get a negative timedelta value.
You can get the seconds with td.seconds and corresponding hour value via just dividing by 3600.
from datetime import datetime
import time
date1 = datetime.now()
time.sleep(3)
date2 = datetime.now()
# timedelta object
td = date2 - date1
print(td.days, td.seconds // 3600, td.seconds)
# 0 0 3

You're not too far off but you should just ask your question as opposed to a question with a "real scenario" later as those are often two very different questions. That way you get an answer to your actual question.
All that said, rather than going through a lot of hoop-jumping with splitting the datetime object, assigning it to a variable which you then later use look for what you need in, it's better to just know what DateTime can do since that can be such a common part of your coding. You would also do well to look at timedelta (which is part of datetime) and if you use pandas, timestamp.
from datetime import datetime
date = datetime.now()
print(date)
print(date.hour)

I can get you the hour of datetime.datetime.now()
You could try indexing a list of a string of datetime.datetime.now():
print(list(str(datetime.datetime.now()))[11] + list(str(datetime.datetime.now()))[12])
Output (in my case when tested):
09
Hope I am of help!

Python convert GTFS time to datetime

It is common for a GTFS time to exceed 23:59:59 due to the timetable cycle. Ie, the last time may be 25:20:00 (01:20:00 the next day), so when you convert the times to datetime, you will get an error when these times are encountered.
Is there a way to convert the GTFS time values into standard datetime format, without splitting the hour out and then converting back to a string in the correct format, to then convert it to a datetime.
t = ['24:22:00', '24:30:00', '25:40:00', '26:27:00']
'0'+str(pd.to_numeric(t[0].split(':')[0])%24)+':'+':'.join(t[0].split(':')[1:])
For the above examples, i would expect to just see
['00:22:00', '00:30:00', '01:40:00', '02:27:00']

from datetime import datetime, timedelta
def gtfs_time_to_datetime(gtfs_date, gtfs_time):
hours, minutes, seconds = tuple(
int(token) for token in gtfs_time.split(":")
)
return (
datetime.strptime(gtfs_date, "%Y%m%d") + timedelta(
hours=hours, minutes=minutes, seconds=seconds
)
)
gives the following result
>>> gtfs_time_to_datetime("20191031", "24:22:00")
datetime.datetime(2019, 11, 1, 0, 22)
>>> gtfs_time_to_datetime("20191031", "24:22:00").time().isoformat()
'00:22:00'
>>> t = ['24:22:00', '24:30:00', '25:40:00', '26:27:00']
>>> [ gtfs_time_to_datetime("20191031", tt).time().isoformat() for tt in t]
['00:22:00', '00:30:00', '01:40:00', '02:27:00']

I didn't find an easy way, so i just wrote a function to do it.
If anyone else wants the solution, here is mine:
from datetime import timedelta
import pandas as pd
def list_to_real_datetime(time_list, date_exists=False):
'''
Convert a list of GTFS times to real datetime list
:param time_list: GTFS times
:param date_exists: Flag indicating if the date exists in the list elements
:return: An adjusted list of time to conform with real date times
'''
# new list of times to be returned
new_time = []
for time in time_list:
plus_day = False
hour = int(time[0:2])
if hour >= 24:
hour -= 24
plus_day = True
# reset the time to a real format
time = '{:02d}'.format(hour)+time[2:]
# Convert the time to a datetime
if not date_exists:
time = pd.to_datetime('1970-01-01 '+time, format='%Y-%m-%d')
if plus_day:
time = time + timedelta(days=1)
new_time.append(time)
return new_time

Create a date range in julian date python

I have to create a range between two dates, interval between dates of some minutes, in Julian date, i create a code, but is taking a lot of time about(15 minutes, for the ex.)
my code is:
from astropy.time import Time
import pandas as pd
timedelta = "600s"
start = "2018-01-01"
end = "2018-06-30"
dateslist = pd.date_range(start,end, freq =timedelta ).tolist()
dates = pd.DataFrame({'col':dateslist})
dates["col2"] =""
for i in range(len(dateslist)):
#print(i," / ", len(dateslist))
dates["col2"][i] = (Time(str(dateslist[i]).replace(" ", "T"), format="fits").jd)
I tried using Time without for, but is getting error
time = str(list(dates['col'])).replace("[Timestamp('","").replace(" Timestamp('","").replace("')","").replace(" ","T").split(",")
time
Time(time, format="fits")
ValueError: Input values did not match the format class fits
Is there some way of doing this quickly?
Thanks for now,

Use DatetimeIndex.to_julian_date:
dates["col2"] = pd.date_range(start,end, freq = timedelta).to_julian_date()

The equivalent way in astropy would be:
from astropy.time import Time
import astropy.units as u
timedelta = 600 * u.s
start = "2018-01-01"
end = "2018-06-30"
dates["col2"] = np.arange(Time(start).jd, Time(end).jd, timedelta.to_value('day'))
An alternate (perhaps more idiomatic way in astropy) is:
start = Time("2018-01-01")
end = Time("2018-06-30")
timedelta = 600 * u.s
dates = start + timedelta * np.arange((end - start) / timedelta)
This gives you a vector Time object, which you could convert to JD via the jd attribute.

how to shift a datetime object by 12 hours in python

Datetime objects hurt my head for some reason. I am writing to figure out how to shift a date time object by 12 hours. I also need to know how to figure out if two date time object's differ by say 1 minute or more.

The datetime library has a timedelta object specifically for this kind of thing:
import datetime
mydatetime = datetime.now() # or whatever value you want
twelvelater = mydatetime + datetime.timedelta(hours=12)
twelveearlier = mydatetime - datetime.timedelta(hours=12)
difference = abs(some_datetime_A - some_datetime_B)
# difference is now a timedelta object
# there are a couple of ways to do this comparision:
if difference > timedelta(minutes=1):
print "Timestamps were more than a minute apart"
# or:
if difference.total_seconds() > 60:
print "Timestamps were more than a minute apart"

You'd use datetime.timedelta for something like this.
from datetime import timedelta
datetime arithmetic works kind of like normal arithmetic: you can add a timedelta object to a datetime object to shift its time:
dt = # some datetime object
dt_plus_12 = dt + timedelta(hours=12)
Also you can subtract two datetime objects to get a timedelta representing the difference between them:
dt2 = # some other datetime object
ONE_MINUTE = timedelta(minutes=1)
if abs(dt2 - dt) > ONE_MINUTE:
# do something