Have two data sets with different time format. One in epcoh time, seconds from 1/1/1970, and the other hours since 1/1/1900. Need to convert the first to the second one. How can I do that in python ?
Thanks
So, for starters, here are the necessary tools:
from datetime import datetime, timezone, timedelta
Then you need to establish the starting time:
atmos_epoch = datetime(1900, 1, 1, 0, 0, tzinfo=timezone.utc)
Then you can work with deltas to that time:
>>> d = datetime.now(timezone.utc) - atmos_epoch
>>> d
datetime.timedelta(days=43893, seconds=28215, microseconds=982930)
>>> d.total_seconds() / 60 / 60
1053439.8377730362
That last value is the difference in hours.
You can add deltas to that epoch timestamp:
>>> atmos_epoch + timedelta(hours=1000000)
datetime.datetime(2014, 1, 29, 16, 0, tzinfo=datetime.timezone.utc)
The other thing you talk about, seconds since 1970, are simply UNIX timestamps, which you can work with easily:
>>> datetime.now().timestamp()
1583395060.91666
>>> datetime.fromtimestamp(1500000000)
datetime.datetime(2017, 7, 14, 4, 40)
Now you have a way to convert from both values to datetime objects and from datetime objects to either value.
Related
I've got an array of seconds since the beginning of the year from a netCDF file and want to turn it into a datetime array. The time array is in units of seconds, every minute, i.e.:
timesec = ncfile['time'][:]
>>>timesec
0
60
120
180
240
300
360
420
...
And there is a basetime variable of the number of second since the begining of the year:
basetime = ncfile['base_time'][:]
>>>basetime
1457568000
basetime being March 10th at 12am.
How do I convert it into an array of format '2016-03-10 00:00:00' and the following minute being '2016-03-10 00:01:00' and so on. I'm sure there are many ways of doing it, but if you could recommend me one that'd be excellent.
Thank You!
Without any further information on the exact input and output format: You could just use a list comprehension to add the basetime to each of the seconds in timesec, then convert those to datetime using fromtimestamp:
>>> import datetime
>>> timesec = [60*i for i in range(3)]
>>> basetime = 1457568000
>>> [datetime.datetime.fromtimestamp(basetime + s) for s in timesec]
[datetime.datetime(2016, 3, 10, 1, 0),
datetime.datetime(2016, 3, 10, 1, 1),
datetime.datetime(2016, 3, 10, 1, 2)]
You can convert those to str, which is pretty much the format you want, or use isoformat or strftime for other formats:
>>> [str(datetime.datetime.fromtimestamp(basetime + s)) for s in timesec]
['2016-03-10 01:00:00', '2016-03-10 01:01:00', '2016-03-10 01:02:00']
I want to subtract 2 times and convert that into a time array.
I consulted this How to calculate the time interval between two time strings. Stating this following code
from datetime import datetime as dt
import time
print("Time Subtraction")
FMT = '%Y-%m-%d %H:%M:%S'
time_tuple = (2018, 1, 13, 13, 51, 18, 2, 317, 0)
time2_tuple = (2018, 1, 15, 13, 50, 18, 2, 317, 0)
s1 = time.strftime(FMT, time_tuple)
s2 = time.strftime(FMT, time2_tuple)
tdelta = dt.strptime(s2, FMT) - dt.strptime(s1, FMT)
print(tdelta)
The result is:
Time Subtraction
1 day, 23:59:00
But I want to get a tuple/print that will look like this
tuple = (0,0,1,23,59,0,2,317,0)
I usually use time not datetime so I am not sure what to do. Any ideas?
tdelta is a datetime.timedelta object, therefore you are printing the string representation of that object. You can get the days, hours, minutes, etc by performing simple arithmetic (since they are stored as fractions-of-days) on the attributes:
def days_hours_minutes(delta):
return delta.days, delta.seconds//3600, (delta.seconds//60)%60
You can add as many of these attributes to the tuple as you'd like.
I have gps unsegmented time like this:
Tgps = 1092121243.0
And I'd like to understand what date and time is that. The begining of GPS time is 6 January 1980. Python function
datetime.utcfromtimestamp
could give seconds from 1 January 1970 year.
I found following:
from datetime import datetime
GPSfromUTC = (datetime(1980,1,6) - datetime(1970,1,1)).total_seconds()
curDate = datetime.utcfromtimestamp(Tgps + GPSfromUTC)
Out[83]: datetime.datetime(2014, 8, 15, 7, 0, 43)
I'm not sure about leapseconds are they included in function datetime or I should calculate them and substract from the result?
May be also exists better solution of this problem?
GPS time started in sync with UTC: 1980-01-06 (UTC) == 1980-01-06 (GPS). Both tick in SI seconds. The difference between GPS time and UTC time increases with each (intercalary) leap second.
To find the correct UTC time, you need to know the number of leap seconds occurred before the given GPS time:
#!/usr/bin/env python
from datetime import datetime, timedelta
# utc = 1980-01-06UTC + (gps - (leap_count(2014) - leap_count(1980)))
utc = datetime(1980, 1, 6) + timedelta(seconds=1092121243.0 - (35 - 19))
print(utc)
Output
2014-08-15 07:00:27 # (UTC)
where leap_count(date) is the number of leap seconds introduced before the given date. From TAI-UTC table (note: the site is the authoritative source on leap seconds. It publishes Bulletin C announcing new leap seconds):
1980..: 19s
2012..: 35s
and therefore:
(leap_count(2014) - leap_count(1980)) == (35 - 19)
If you are on Unix then you could use "right" time zone to get UTC time from TAI time
(and it is easy to get TAI time from GPS time: TAI = GPS + 19 seconds (constant offset)):
#!/usr/bin/env python
import os
import time
os.environ['TZ'] = 'right/UTC' # TAI scale with 1970-01-01 00:00:10 (TAI) epoch
time.tzset() # Unix
from datetime import datetime, timedelta
gps_timestamp = 1092121243.0 # input
gps_epoch_as_gps = datetime(1980, 1, 6)
# by definition
gps_time_as_gps = gps_epoch_as_gps + timedelta(seconds=gps_timestamp)
gps_time_as_tai = gps_time_as_gps + timedelta(seconds=19) # constant offset
tai_epoch_as_tai = datetime(1970, 1, 1, 0, 0, 10)
# by definition
tai_timestamp = (gps_time_as_tai - tai_epoch_as_tai).total_seconds()
print(datetime.utcfromtimestamp(tai_timestamp)) # "right" timezone is in effect!
Output
2014-08-15 07:00:27 # (UTC)
You could avoid changing the timezone if you extract the leap seconds list from the corresponding tzfile(5). It is a combination of the first two methods where the leap count computation from the first method is automated and the autoupdating tzdata (system package for the tz database) from the second method is used:
>>> from datetime import datetime, timedelta
>>> import leapseconds
>>> leapseconds.gps_to_utc(datetime(1980,1,6) + timedelta(seconds=1092121243.0))
datetime.datetime(2014, 8, 15, 7, 0, 27)
where leapseconds.py can extract leap seconds from /usr/share/zoneinfo/right/UTC file (part of tzdata package).
All three methods produce the same result.
You can use the astropy.time package to do this:
GPS time to TAI
from astropy.time import Time
mytime = 1092121243.0
t = Time(mytime, format='gps')
t = Time(t, format='iso') # same as scale='tai'
print(t)
which returns 2014-08-15 07:01:02.000
GPS time to UTC
from astropy.time import Time
sec = 1092121243.0
t_in = Time(sec, format='gps')
t_out = Time(t_in, format='iso', scale='utc')
print(t_out)
which outputs 2014-08-15 07:00:27.000
I use the following function that counts leap seconds:
import bisect
from datetime import datetime, timedelta
_LEAP_DATES = ((1981, 6, 30), (1982, 6, 30), (1983, 6, 30),
(1985, 6, 30), (1987, 12, 31), (1989, 12, 31),
(1990, 12, 31), (1992, 6, 30), (1993, 6, 30),
(1994, 6, 30), (1995, 12, 31), (1997, 6, 30),
(1998, 12, 31), (2005, 12, 31), (2008, 12, 31),
(2012, 6, 30), (2015, 6, 30), (2016, 12, 31))
LEAP_DATES = tuple(datetime(i[0], i[1], i[2], 23, 59, 59) for i in _LEAP_DATES)
def leap(date):
"""
Return the number of leap seconds since 1980-01-01
:param date: datetime instance
:return: leap seconds for the date (int)
"""
# bisect.bisect returns the index `date` would have to be
# inserted to keep `LEAP_DATES` sorted, so is the number of
# values in `LEAP_DATES` that are less than `date`, or the
# number of leap seconds.
return bisect.bisect(LEAP_DATES, date)
Of course, you need to update _LEAP_DATES occasionally, but these updates are quite rare.
In general, GPS time consists of two numbers: GPS week and the number of seconds since the beginning of the current GPS week. Therefore, you can use the following:
def gps2utc(week, secs):
"""
:param week: GPS week number, i.e. 1866
:param secs: number of seconds since the beginning of `week`
:return: datetime instance with UTC time
"""
secs_in_week = 604800
gps_epoch = datetime(1980, 1, 6, 0, 0, 0)
date_before_leaps = gps_epoch + timedelta(seconds=week * secs_in_week + secs)
return date_before_leaps - timedelta(seconds=leap(date_before_leaps))
In your case week = 0, so:
In [1]: gps2utc(0, 1092121243.0)
Out[1]: datetime.datetime(2014, 8, 15, 7, 0, 27)
A linear combination between Ru887321 and Yury Kirienko answers.
from astropy.time import Time
def gps2utc(gpsweek, gpsseconds):
""" GPS time to UTC.
Parameters
----------
gpsweek : int
GPS week number, i.e. 1866.
gpsseconds : int
Number of seconds since the beginning of week.
Returns
-------
datetime
datetime instance with UTC time.
"""
secs_in_week = 604800
secs = gpsweek * secs_in_week + gpsseconds
t_gps = Time(secs, format="gps")
t_utc = Time(t_gps, format="iso", scale="utc")
return t_utc.datetime
In your case week = 0, so:
In [1]: gps2utc(0, 1092121243.0)
Out[1]: datetime.datetime(2014, 8, 15, 7, 0, 27)
I have time defined in the following format: "%A %H:%M:%S" i.e "Monday 06:00:00"
Now, using the above format, I define a time window:
"Monday 06:00:00" - "Monday 18:00:00"
I now want to check if the current time falls within the time window or not. To do that, I am using the datetime utility.
import datetime
# current time in str
current_time = datetime.datetime.strftime(datetime.datetime.now(),'%A %H:%M:%S')
# convert to datetime object
current_time = datetime.datetime.strptime(current_time, '%A %H:%M:%S')
# print current_time produces: 1900-01-01 19:13:53
# and day of week information is lost
However, the moment, I convert the current_time to a datetime object, the day of the week information is lost.
What's the best way to compare time windows that includes day-of-week in Python?
BTW: I want the time window to be repeated in the future. For eg, "Monday 06:00 - Tuesday 05:00" would apply to all weeks in the future.
datetime would know nothing about which monday you're talking about. There have been 1000's since the epoch.
I suggest you look into the dateutil package. It applies "human-ish" semantics:
>>> import dateutil.parser
>>> dateutil.parser.parse('Monday')
datetime.datetime(2014, 2, 3, 0, 0)
>>> dateutil.parser.parse('Monday 18:30:00')
datetime.datetime(2014, 2, 3, 18, 30)
Note how it assumes that Monday means today (it's Monday). And Tuesday and Sunday (below) mean this coming Tuesday, Sunday:
>>> dateutil.parser.parse('Tuesday 18:30:00')
datetime.datetime(2014, 2, 4, 18, 30)
>>> dateutil.parser.parse('Sunday 18:30:00')
datetime.datetime(2014, 2, 9, 18, 30)
To take this further one take a look at the rrule module/class in dateutil. And the built-in calendar module. For example, with rrule I can do:
next_five = rrule.rrule(
rrule.WEEKLY,
count=5,
dtstart=dateutil.parser.parse('Sunday 18:30:00'),
)
This returns an iterator of the next 5 weeks starting at 6:30pm Sunday. The iterator will produce
[datetime.datetime(2014, 2, 5, 18, 30),
datetime.datetime(2014, 2, 12, 18, 30),
datetime.datetime(2014, 2, 19, 18, 30),
datetime.datetime(2014, 2, 26, 18, 30),
datetime.datetime(2014, 3, 5, 18, 30)]
This should get you going on the final solution.
In my experience Datetime objects are YYYY:MM:DD HH:MM:SS (more or less). The day of week can be derived from that object using datetime's weekday() - see http://docs.python.org/2/library/datetime.html#datetime.date.weekday
You can solve your problem using greater/less than "<" and ">" operators for datetime evaluation. Here is a proof for simple comparison http://pastebin.com/miTUW9nF
Your logic would be (paraphrased): "if Xtime > windowstart and Xtime < windowend then Xtime is in window"
Datetime compares down to the second, my example is now and (now-10 days), that should cover your needs.
I have a utc timestamp in the IS8601 format and am trying to convert it to unix time. This is my console session:
In [9]: mydate
Out[9]: '2009-07-17T01:21:00.000Z'
In [10]: parseddate = iso8601.parse_date(mydate)
In [14]: ti = time.mktime(parseddate.timetuple())
In [25]: datetime.datetime.utcfromtimestamp(ti)
Out[25]: datetime.datetime(2009, 7, 17, 7, 21)
In [26]: datetime.datetime.fromtimestamp(ti)
Out[26]: datetime.datetime(2009, 7, 17, 2, 21)
In [27]: ti
Out[27]: 1247815260.0
In [28]: parseddate
Out[28]: datetime.datetime(2009, 7, 17, 1, 21, tzinfo=<iso8601.iso8601.Utc object at 0x01D74C70>)
As you can see, I can't get the correct time back. The hour is ahead by one if i use fromtimestamp(), and it's ahead by six hours if i use utcfromtimestamp()
Any advice?
Thanks!
You can create an struct_time in UTC with datetime.utctimetuple() and then convert this to a unix timestamp with calendar.timegm():
calendar.timegm(parseddate.utctimetuple())
This also takes care of any daylight savings time offset, because utctimetuple() normalizes this.
I am just guessing, but one hour difference can be not because of time zones, but because of daylight savings on/off.
naive_utc_dt = parseddate.replace(tzinfo=None)
timestamp = (naive_utc_dt - datetime(1970, 1, 1)).total_seconds()
# -> 1247793660.0
See more details in another answer to similar question.
And back:
utc_dt = datetime.utcfromtimestamp(timestamp)
# -> datetime.datetime(2009, 7, 17, 1, 21)
import time
import datetime
import calendar
def date_time_to_utc_epoch(dt_utc): #convert from utc date time object (yyyy-mm-dd hh:mm:ss) to UTC epoch
frmt="%Y-%m-%d %H:%M:%S"
dtst=dt_utc.strftime(frmt) #convert datetime object to string
time_struct = time.strptime(dtst, frmt) #convert time (yyyy-mm-dd hh:mm:ss) to time tuple
epoch_utc=calendar.timegm(time_struct) #convert time to to epoch
return epoch_utc
#----test function --------
now_datetime_utc = int(date_time_to_utc_epoch(datetime.datetime.utcnow()))
now_time_utc = int(time.time())
print (now_datetime_utc)
print (now_time_utc)
if now_datetime_utc == now_time_utc :
print ("Passed")
else :
print("Failed")