I'm working with some telemetry that uses timestamps measured in hours since January 1st at midnight of the current year.
So I get value 1 at time 8668.12034
I'd like to convert it to a more useful date format and of course I've been doing so with hardcoded math dividing into days, remainder hours, minutes, etc accounting for leap years... and it works but I'm sure there's a simple way using the datetime library or something right?
I'm thinking timedelta is the way to go since it's giving me a delta since the beginning of the year but does that account for leap years?
Curious how others would approach this issue, thanks for any advice.
# import packages we need
import datetime
From elapsed hours to datetime.datetime object
You can for example do:
hours_elapsed = 1000
your_date = datetime.datetime(2020,1,1,0,0)+datetime.timedelta(hours=hours_elapsed)
(Of course change hours_elapsed to whatever hours elapsed in your case.)
your_date will be: datetime.datetime(2020, 2, 11, 16, 0)
Yes, timedelta does know about leap years.
Further processing
If want to process this further, can do, using getattr():
timeunits = ['year', 'month', 'day', 'hour', 'minute', 'second']
[getattr(your_date,timeunit) for timeunit in timeunits]
Resulting in:
[2020, 2, 11, 16, 0, 0]
Related
I have an error in python, when extracting one day. I'm converting to unix and extracting one day, yet the 11th of march - is always missing, no matter how big the dataset. Could anyone tell me what might be the error ?
from time import localtime, mktime, strptime, strftime
day = str(20180313)
one_day = 86400
for i in range(1,5):
print(day)
previous_day_unix = int( mktime( strptime( day, "%Y%m%d")))-one_day
day = strftime("%Y%m%d", localtime(int(previous_day_unix)))
print(day)
Daylight saving time 2018 began at 2:00 AM on March 11, 2018. Thus this day wasn't 86400 seconds.
As you can see subtracting 86400 seconds is not a good way to compute differences in days. In general, all date/time "math" operations are a little more complicated than simple multiplies and adds due to things like timezones, daylight savings, and leap years.
A better way is to use a library, such as the datetime, which handles all of these things for you:
from datetime import datetime, timedelta
day = str(20180313)
for i in range(1, 5):
print(day)
previous_day = (datetime.strptime(day, "%Y%m%d") - timedelta(days=1))
day = previous_day.strftime("%Y%m%d")
#20180313
#20180312
#20180311
#20180310
I have a vector of time in Gregorian type. I want to convert it in the common format of dates with year, month, day, hour, minute and seconds.
I tried this:
from datetime import datetime
datetime.fromordinal(736364.94)
It returns always something only with the information of year, month and day. Like this:
datetime.datetime(2017, 2, 4, 0, 0)
I was expecting a date with hours, minutes and seconds information
I think the ordinal is for days only, but assuming the decimal is representative of the fraction of the day you can use this.
import datetime
datetime.fromordinal(int(736364.94))+datetime.timedelta(days=736364.94%1)
Note this is not quite the same as this question. That question assumes the time you want is "now", which is not the same as for an arbitrary point in time.
I have a UTC, aware, datetime object, call it point_in_time (e.g. datetime(2017, 3, 12, 16, tzinfo=tz.tzutc())).
I have a timezone, call it location (e.g. 'US/Pacific'), because I care about where it is, but its hours offset from UTC may change throughout the year with daylight savings and whatnot.
I want to
1) get the date of point_in_time if I'm standing in location,
2) get midnight of that date if I'm standing in location.
===
I tried to simply use .astimezone(timezone('US/Pacific')) and then .replace(hours=0, ...) to move to midnight, but as you might notice about my example point_in_time, the midnight for that date is on the other side of a daylight savings switch!
The result was that I got a time representing UTC datetime(2017, 3, 12, 7), instead of a time representing UTC datetime(2017, 3, 12, 8), which is the true midnight.
EDIT:
I'm actually thinking the difference between mine and the linked question is that I'm looking for the most recent midnight in the past. That question's answer seems to be able to give a midnight that could be in the past or future, perhaps?
Your example highlights the perils of doing datetime arithmetic in a local time zone.
You can probably achieve this using pytz's normalize() function, but here's the method that occurs to me:
point_in_time = datetime(2017, 3, 12, 16, tzinfo=pytz.utc)
pacific = pytz.timezone("US/Pacific")
pacific_time = point_in_time.astimezone(pacific)
pacific_midnight_naive = pacific_time.replace(hour=0, tzinfo=None)
pacific_midnight_aware = pacific.localize(pacific_midnight_naive)
pacific_midnight_aware.astimezone(pytz.utc) # datetime(2017, 3, 12, 8)
In other words, you first convert to Pacific time to figure out the right date; then you convert again from midnight on that date to get the correct local time.
Named timezones such as "US/Pacific" are by definition daylight-savings aware. If you wish to use a fixed non-daylight-savings-aware offset from GMT you can use the timezones "Etc/GMT+*", where * is the desired offset. For example for US Pacific Standard Time you would use "Etc/GMT+8":
import pandas as pd
point_in_time = pd.to_datetime('2017-03-12 16:00:00').tz_localize('UTC')
# not what you want
local_time = point_in_time.tz_convert("US/Pacific")
(local_time - pd.Timedelta(hours=local_time.hour)).tz_convert('UTC')
# Timestamp('2017-03-12 07:00:00+0000', tz='UTC')
# what you want
local_time = point_in_time.tz_convert("Etc/GMT+8")
(local_time - pd.Timedelta(hours=local_time.hour)).tz_convert('UTC')
# Timestamp('2017-03-12 08:00:00+0000', tz='UTC')
See the docs at http://pvlib-python.readthedocs.io/en/latest/timetimezones.html for more info.
EDIT Now that I think about it, Midnight PST will always be 8am UTC, so you could simplify this as
if point_in_time.hour >=8:
local_midnight = point_in_time - point_in_time.hour + 8
else:
local_midnight = point_in_time - point_in_time.hour - 16
so I am a beginner with python and have been working with the datetime, time, and timedelta libraries a little bit. I am trying to create a piece of code that gives me the date approximately two months ago(exact_two_months_date) from today (whatever today happens to be). The catch is, I want to find that date approx. two months ago AND begin the actual start_date on the Monday of that week. So in theory, the actual start date will not be exactly two months ago. It will be the week beginning on Monday two months ago from today.
Example pseudocode:
today = '20150425' ## '%Y%m%d' ... Saturday
exact_two_months_date = '20150225' ## EXACTLY two months ago ... Wednesday
start_date = '20150223' ## this is the Monday of that week two months ago
So how do I find the 'start_date' above? If the day exactly two months ago begins on a Saturday or Sunday, then I would just want to go to the next Monday. Hope this is clear and makes sense... Once I find the start date, I would like to increment day by day(only week days) up to 'today'.
Appreciate any feedback, thanks.
Calculating with dates using python-dateutil
If a dependency on a third-party package is an option, then ☞ python-dateutil provides a convenient method to calculate with dates.
Browse the docs for ☞ relativedelta to see the wealth of supported parameters. The more calculations a package needs to do with dates, the more a helper module like dateutil justifies its dependency. For more inspiration on what it has to offer see the ☞ examples page.
Quick run-through:
>>> import datetime
>>> from dateutil.relativedelta import relativedelta
>>> today = datetime.date.today()
>>> two_m_ago = today - relativedelta(months=2)
>>> # print two_m_ago ~> datetime.date(2015, 2, 25)
>>> monday = two_m_ago - datetime.timedelta(days=two_m_ago.weekday())
>>> # print monday ~> datetime.date(2015, 2, 23)
Getting the Monday with weekday()
Once we have the date from two months ago in the variable two_m_ago, we subtract the index of the weekday() from it. This index is 0 for Monday and goes all the way to 6 for Sunday. If two_m_ago already is a Monday, then subtracting by 0 will not cause any changes.
Does something like this work for you?
import datetime
today = datetime.date.today()
delta = datetime.timedelta(days=60) # ~ 2 months
thatDay = today - delta
# subtract weekdays to get monday
thatMonday = thatDay - datetime.timedelta(days=thatDay.weekday())
Honestly, I find working with datetimes to be the hardest thing I ever have to regularly do and I make a lot of mistakes, so I'm going to work through this one and show some of the failures I regularly have with it. Here goes.
Two constraints: 1) Date two months ago, 2) Monday of that week
Date Two Months Ago
Okay, so Python's datetime library has a useful method called replace, which seems like it might help here:
>>> import datetime
>>> now = datetime.date.today()
>>> today
datetime.date(2015, 4, 25)
>>> today.month
4
>>> two_months_ago = today.replace(month=today.month-2)
>>> two_months_ago
datetime.date(2015, 2, 25)
>>> two_months_ago.month
2
But wait: what about negative numbers? That won't work:
>>> older = datetime.date(2015, 01, 01)
>>> older.replace(month=older.month-2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: month must be in 1..12
So there are two solutions:
1) I can build a 1-12 range that cycles forwards or back, or
2) To find two months previous, I can merely replace the day part of my date with the 1st day of the month I'm in and then go back 1 day to the previous month and then replace that day in the previous month with the day I want.
(If you think about it, you'll find that either of these may present bugs if I land on day 31 in a month with fewer days than that, for instance. This is part of what makes datetimes difficult.)
def previous_month(date):
current_day = date.day
first_day = date.replace(day=1)
last_day_prev_month = first_day - datetime.timedelta(days=1)
prev_month_day = last_day_prev_month.replace(day=current_day)
return prev_month_day
>>> today = datetime.date.today()
>>> older = previous_month(today)
>>> older
datetime.date(2015, 3, 25)
Well, let's say we're getting close, though, and we need to include some error-checking to make sure the day we want is a valid date inside the month we land in. Ultimately, the problem is that "two months ago" means a lot more than we think it means when we say it out loud.
Next, we'll take a crack at problem number two: How to get to the Monday of that week?
Well, datetime objects have a weekday method, so this part shouldn't be too hard and here's a nice SO answer on how to do that.
Simple version is: use the difference in weekday integers to figure out how many days to go back and do that using datetime.timedelta(days=days_difference).
Takeaway: Working with datetimes can be tough.
Date manipulation in Python is horribly convoluted. You will save a lot of time by using the arrow package which greatly simplifies these operations.
First install it
pip install arrow
Now your question:
import arrow
# get local current time
now = arrow.now('local')
# move 2 months back
old = now.replace(months=-2)
# what day of the week was that?
dow = old.isoweekday()
# reset old to Monday, for instance at 9:32 in the morning (this is just an example, just to show case)
old = old.replace(days=-dow, hour=9, minute=32, second=0)
print('now is {now}, we went back to {old}'.format(now=now.isoformat(), old=old.isoformat()))
The output:
now is 2015-04-25T20:37:38.174000+02:00, we went back to 2015-02-22T09:32:00.174000+01:00
Note that the various formats, timezones etc. are now transparent and you just need to rely on one package.
I wish to get the total duration of a relativedelta in terms of days.
Expected:
dateutil.timedelta(1 month, 24 days) -> dateutil.timedelta(55 days)
What I tried:
dateutil.timedelta(1 month, 24 days).days -> 24 (WRONG)
Is there a simple way to do this? Thanks!
This one bothered me as well. There isn't a very clean way to get the span of time in a particular unit. This is partly because of the date-range dependency on units.
relativedelta() takes an argument for months. But when you think about how long a month is, the answer is "it depends". With that said, it's technically impossible to convert a relativedelta() directly to days, without knowing which days the delta lands on.
Here is what I ended up doing.
from datetime import datetime, timedelta
from dateutil.relativedelta import relativedelta
rd = relativedelta(years=3, months=7, days=19)
# I use 'now', but you may want to adjust your start and end range to a specific set of dates.
now = datetime.now()
# calculate the date difference from the relativedelta span
then = now - rd
# unlike normal timedelta 'then' is returned as a datetime
# subtracting two dates will give you a timedelta which contains the value you're looking for
diff = now - then
print diff.days
Simple date diff does it actually.
>>> from datetime import datetime
>>> (datetime(2017, 12, 1) - datetime(2018, 1, 1)).days
-31
To get positive number You can swap dates or use abs:
>>> abs((datetime(2017, 12, 1) - datetime(2018, 1, 1)).days)
31
In many situations you have a much restricted relativedelta, in my case, my relativedelta had only relative fields set (years, months, weeks, and days) and no other field. You may be able to get away with the simple method.
This is definitely off by few days, but it may be all you need
(365 * duration.years) + (30 * duration.months) + (duration.days)