Is it possible to translate pandas datetime to weekday/weekend and season? - python

I have a pandas dataframe with one if its columns in datetime format (year-month-day). Is there any way to create a new column that says weekend or weekday, and a column that says what season it is? Even better (but this one seems much more complex so not totally necessary) would also be whether it's a federal holiday (United States) or not.
For example, an instance at date 2019-10-23 is a Wednesday, so I'd want to create a column called day_type that fills 'weekday', and a column called season that fills 'fall'.

It's easy with strftime:
import pandas as pd
df = pd.DataFrame({'string_date': ['2019-10-23', '2019-10-24', '2019-10-23']})
df['date'] = pd.to_datetime(df['string_date'], format='%Y-%m-%d', errors='ignore')
df['day_of_week'] = df['date'].dt.strftime('%A')
Result1 :
string_date date day_of_week
0 2019-10-23 2019-10-23 Wednesday
1 2019-10-24 2019-10-24 Thursday
2 2019-10-23 2019-10-23 Wednesday
For adding the season you can use this formula :
Link
df['season'] = (df['date'].dt.month%12 + 3)//3
Result2:
string_date date day_of_week season
0 2019-10-23 2019-10-23 Wednesday 4
1 2019-10-24 2019-10-24 Thursday 4
2 2019-10-23 2019-10-23 Wednesday 4
Strftime reference:
http://strftime.org/

I wrote a function that takes in a string argument in the form 'yyyy-mm-dd' and returns a tuple of season and day-of-week. Please modify the function accordingly to your needs.
from datetime import date, datetime
def date_week(date_str):
"""
this method returns season and day of week tuple from str
arg in the format 'yyyy-mm-dd'
"""
datetime_obj = datetime.strptime(date_str, '%Y-%m-%d')
weekdays = {0:'monday',
1:'tuesday',
2:'wednesday',
3:'thursday',
4:'friday',
5:'saturday',
6:'sunday'}
day_of_week = weekdays[datetime_obj.weekday()] # returns day of week
Y = 2000 # dummy leap year to allow input X-02-29 (leap day)
seasons = [('winter', (date(Y, 1, 1), date(Y, 3, 20))),
('spring', (date(Y, 3, 21), date(Y, 6, 20))),
('summer', (date(Y, 6, 21), date(Y, 9, 22))),
('autumn', (date(Y, 9, 23), date(Y, 12, 20))),
('winter', (date(Y, 12, 21), date(Y, 12, 31)))]
if isinstance(datetime_obj, datetime):
datetime_obj = datetime_obj.date()
datetime_obj = datetime_obj.replace(year=Y)
season = next(season for season, (start, end) in seasons
if start <= datetime_obj <= end) # returns season
return day_of_week, season
If we call date_week("2019-10-23"), it will return ('wednesday', 'autumn')

Related

How to get date of last friday in each quarter?

I want to find the last friday date in each quater for the given year. For example, the output as follows
last friday in March
last friday in June
last friday in september
last friday in december
How can I find this intelligently based on given year as an input
I'm assuming you don't want the output to be literally
last friday in March
last friday in June
last friday in september
last friday in december
and you just didn't feel like looking up those dates for an example year to include in your question.
Since we know that quarters always end in those months, we can create the last dates of those months. Then, datetime.date.weekday() tells us which day of the week it is. Friday is 4, so we just need to find out how many days we need to go back to achieve this. Then, use datetime.timedelta can subtract that many days, and we should be good to go:
import datetime
def last_fridays_of_quarter(year):
last_days = [datetime.date(year, 3, 31), datetime.date(year, 6, 30), datetime.date(year, 9, 30), datetime.date(year, 12, 31)]
for day in last_days:
days_to_sub = (day.weekday() - 4) % 7
last_friday = day - datetime.timedelta(days=days_to_sub)
print(last_friday)
Testing this for 2022:
>>> last_fridays_of_quarter(2022)
2022-03-25
2022-06-24
2022-09-30
2022-12-30
Here's one way to solve it. You store last days of each quarter in a given year in an array. Then you iterate over said quarter ending dates, and calculate the offset that you would need to subtract in order to get to the previous friday.
from datetime import datetime, timedelta
year = 2022
quarter_ending_dates = [
datetime(year, 3, 31),
datetime(year, 6, 30),
datetime(year, 9, 30),
datetime(year, 12, 31)
]
for ending_date in quarter_ending_dates:
offset = (ending_date.weekday() - 4) % 7
last_friday = ending_date - timedelta(days=offset)
print(last_friday)
try:
import pandas as pd
def fridays(year):
df = pd.date_range(start=str(year), end=str(year+1),
freq='W-FRI').strftime('%m/%d/%Y')\
.to_frame().reset_index(drop = True)
df.columns = ['date']
df.index = pd.to_datetime(df['date']).dt.to_period('Q')
df = df.groupby(df.index).last()
df.index.name = 'quarter'
return df
fridays(2022)
output is:
quarter
date
2022Q1
03/25/2022
2022Q2
06/24/2022
2022Q3
09/30/2022
2022Q4
12/30/2022

How to get the first and last date of the month with Year and Month in python

In the below code , from the month and year, need 1st and last date of the month
for ex month =3 and year= 2022
first_date = 2021-03-01 00:00:00
last_date = 2021-03-31 00:00:00
I tried with calendar but it simply returns (1 ,31)
import calendar
month =3
year= 2022
print( calendar.monthrange(2002, 1))
From monthrange() you will get the first day and the number of days in that month. You can use the datetime method to convert it to date.
month =3
year= 2022
first, last = calendar.monthrange(year, month)
print(first, last)
print(datetime.datetime(year, month, 1))
print(datetime.datetime(year, month, last))
You could write your own function to calculate the last day of the month:
def last_day_of_month(date):
if date.month == 12:
return date.replace(day=31)
return date.replace(month=date.month+1, day=1) - datetime.timedelta(days=1)
So:
>>> last_day_of_month(datetime.date(2021, 3, 19))
datetime.date(2021, 3, 31)
Similarly, we can use a one-liner for the first date:
(dt.replace(day=1) + datetime.timedelta(days=32)).replace(day=1)
So:
dt = datetime.datetime(2021, 3, 19)
print((dt.replace(day=1) + datetime.timedelta(days=32)).replace(day=1))
>>> 2021-03-01 00:00:00
you can do it using datetime module
import datetime
def get_last_day_of_month(day):
next_month = day.replace(day=28) + datetime.timedelta(days=4)
return next_month - datetime.timedelta(days=next_month.day)
the output will be:
for month in range(1, 13):
print(get_last_day_of_month(datetime.date(2020, month, 1)))
2020-01-31
2020-02-29
2020-03-31
2020-04-30
2020-05-31
2020-06-30
2020-07-31
2020-08-31
2020-09-30
2020-10-31
2020-11-30
2020-12-31
for the first day you can just put the day always 1

How can I turn a date into the day of the week (as an integer)?

I have a pandas dataframe which has dates in the format YYYYMMDD stored as a float32. Is it possible to somehow calculate what day of the week each day is? Eg. monday = 1, tuesday =2, etc?
I don't even know where to start.
We can convert the column to str first, then to datetime and use .dt.weekday:
df = pd.DataFrame({'date': [20210101, 20210102]})
df['weekday'] = pd.to_datetime(df['date'].astype(str)).dt.weekday
df
Output:
date weekday
0 20210101 4
1 20210102 5
P.S. Here Monday = 0, ..., Sunday = 6, so if you want 1-7 instead, you can add 1 to column values

Assigning a numerical value to day and time

I have a column in df that contains numerical values that correlate to the customers hour of week for that row. For example 0 = Sunday 12:00am and 24 would be Monday 12:00am and 5 would be Sunday 5:00am.
Value
0
4
10
24
Value Expected Output Column
0 Sunday 12:00am
4 Sunday 4:00am
10 Sunday 10:00am
24 Monday 12:00am
49 Tuesday 1:00am
How can I create the new column if I want all values to assign to a correct corresponding Day and time for the week? Values start at 0 which would represent Sunday 12:00am first value of week and would end at 167 which would be Saturday 11:59pm of that week.
Thanks!
This does what you want:
Value = pd.Series([0, 4, 10, 24, 49])
AnyGivenSunday = pd.to_datetime('Sunday 2020') # yes this works
(AnyGivenSunday + pd.to_timedelta(Value, 'h')).dt.strftime('%A %I:%M%P')
import pandas as pd
from datetime import datetime
import calendar as cal
df = pd.DataFrame(data={'val': [0, 0, 0, 0, 0], 'date': ["Sunday 12:00am", "Sunday 4:00am", "Sunday 10:00am", "Monday 12:00am", "Tuesday 1:00am"]})
def convertDate(dateString):
hour = datetime.strptime(dateString, '%A %I:%M%p').hour # Convert to 24 hour format
day, time = dateString.split(" ") # Get text of day, cannot get datetime.weekday() without full date
if day.lower() == cal.day_name[6].lower(): # cal.day_name index starts at Monday and not Sunday
return hour + ((list(cal.day_name).index(day) - 6) * 24)
else:
return hour + ((list(cal.day_name).index(day) + 1) * 24)
df['val'] = df['date'].apply(convertDate) # Apply function to all columns

Return datetime object of previous month

If only timedelta had a month argument in it's constructor. So what's the simplest way to do this?
EDIT: I wasn't thinking too hard about this as was pointed out below. Really what I wanted was any day in the last month because eventually I'm going to grab the year and month only. So given a datetime object, what's the simplest way to return any datetime object that falls in the previous month?
You can use the third party dateutil module (PyPI entry here).
import datetime
import dateutil.relativedelta
d = datetime.datetime.strptime("2013-03-31", "%Y-%m-%d")
d2 = d - dateutil.relativedelta.relativedelta(months=1)
print d2
output:
2013-02-28 00:00:00
After the original question's edit to "any datetime object in the previous month", you can do it pretty easily by subtracting 1 day from the first of the month.
from datetime import datetime, timedelta
def a_day_in_previous_month(dt):
return dt.replace(day=1) - timedelta(days=1)
Try this:
def monthdelta(date, delta):
m, y = (date.month+delta) % 12, date.year + ((date.month)+delta-1) // 12
if not m: m = 12
d = min(date.day, [31,
29 if y%4==0 and (not y%100==0 or y%400 == 0) else 28,
31,30,31,30,31,31,30,31,30,31][m-1])
return date.replace(day=d,month=m, year=y)
>>> for m in range(-12, 12):
print(monthdelta(datetime.now(), m))
2009-08-06 16:12:27.823000
2009-09-06 16:12:27.855000
2009-10-06 16:12:27.870000
2009-11-06 16:12:27.870000
2009-12-06 16:12:27.870000
2010-01-06 16:12:27.870000
2010-02-06 16:12:27.870000
2010-03-06 16:12:27.886000
2010-04-06 16:12:27.886000
2010-05-06 16:12:27.886000
2010-06-06 16:12:27.886000
2010-07-06 16:12:27.886000
2010-08-06 16:12:27.901000
2010-09-06 16:12:27.901000
2010-10-06 16:12:27.901000
2010-11-06 16:12:27.901000
2010-12-06 16:12:27.901000
2011-01-06 16:12:27.917000
2011-02-06 16:12:27.917000
2011-03-06 16:12:27.917000
2011-04-06 16:12:27.917000
2011-05-06 16:12:27.917000
2011-06-06 16:12:27.933000
2011-07-06 16:12:27.933000
>>> monthdelta(datetime(2010,3,30), -1)
datetime.datetime(2010, 2, 28, 0, 0)
>>> monthdelta(datetime(2008,3,30), -1)
datetime.datetime(2008, 2, 29, 0, 0)
Edit Corrected to handle the day as well.
Edit See also the answer from puzzlement which points out a simpler calculation for d:
d = min(date.day, calendar.monthrange(y, m)[1])
A vectorized, pandas solution is very simple:
df['date'] - pd.DateOffset(months=1)
A variation on Duncan's answer (I don't have sufficient reputation to comment), which uses calendar.monthrange to dramatically simplify the computation of the last day of the month:
import calendar
def monthdelta(date, delta):
m, y = (date.month+delta) % 12, date.year + ((date.month)+delta-1) // 12
if not m: m = 12
d = min(date.day, calendar.monthrange(y, m)[1])
return date.replace(day=d,month=m, year=y)
Info on monthrange from Get Last Day of the Month in Python
I think the simple way is to use DateOffset from Pandas like so:
import pandas as pd
date_1 = pd.to_datetime("2013-03-31", format="%Y-%m-%d") - pd.DateOffset(months=1)
The result will be a Timestamp object
If only timedelta had a month argument
in it's constructor. So what's the
simplest way to do this?
What do you want the result to be when you subtract a month from, say, a date that is March 30? That is the problem with adding or subtracting months: months have different lengths! In some application an exception is appropriate in such cases, in others "the last day of the previous month" is OK to use (but that's truly crazy arithmetic, when subtracting a month then adding a month is not overall a no-operation!), in others yet you'll want to keep in addition to the date some indication about the fact, e.g., "I'm saying Feb 28 but I really would want Feb 30 if it existed", so that adding or subtracting another month to that can set things right again (and the latter obviously requires a custom class holding a data plus s/thing else).
There can be no real solution that is tolerable for all applications, and you have not told us what your specific app's needs are for the semantics of this wretched operation, so there's not much more help that we can provide here.
If all you want is any day in the last month, the simplest thing you can do is subtract the number of days from the current date, which will give you the last day of the previous month.
For instance, starting with any date:
>>> import datetime
>>> today = datetime.date.today()
>>> today
datetime.date(2016, 5, 24)
Subtracting the days of the current date we get:
>>> last_day_previous_month = today - datetime.timedelta(days=today.day)
>>> last_day_previous_month
datetime.date(2016, 4, 30)
This is enough for your simplified need of any day in the last month.
But now that you have it, you can also get any day in the month, including the same day you started with (i.e. more or less the same as subtracting a month):
>>> same_day_last_month = last_day_previous_month.replace(day=today.day)
>>> same_day_last_month
datetime.date(2016, 4, 24)
Of course, you need to be careful with 31st on a 30 day month or the days missing from February (and take care of leap years), but that's also easy to do:
>>> a_date = datetime.date(2016, 3, 31)
>>> last_day_previous_month = a_date - datetime.timedelta(days=a_date.day)
>>> a_date_minus_month = (
... last_day_previous_month.replace(day=a_date.day)
... if a_date.day < last_day_previous_month.day
... else last_day_previous_month
... )
>>> a_date_minus_month
datetime.date(2016, 2, 29)
Returns last day of last month:
>>> import datetime
>>> datetime.datetime.now() - datetime.timedelta(days=datetime.datetime.now().day)
datetime.datetime(2020, 9, 30, 14, 13, 15, 67582)
Returns the same day last month:
>>> x = datetime.datetime.now() - datetime.timedelta(days=datetime.datetime.now().day)
>>> x.replace(day=datetime.datetime.now().day)
datetime.datetime(2020, 9, 7, 14, 22, 14, 362421)
For most use cases, what about
from datetime import date
current_date =date.today()
current_month = current_date.month
last_month = current_month - 1 if current_month != 1 else 12
today_a_month_ago = date(current_date.year, last_month, current_date.day)
That seems the simplest to me.
Note: I've added the second to last line so that it would work if the current month is January as per #Nick's comment
Note 2: In most cases, if the current date is the 31st of a given month the result will be an invalid date as the previous month would not have 31 days (Except for July & August), as noted by #OneCricketeer
I use this for government fiscal years where Q4 starts October 1st. Note I convert the date into quarters and undo it as well.
import pandas as pd
df['Date'] = '1/1/2020'
df['Date'] = pd.to_datetime(df['Date']) #returns 2020-01-01
df['NewDate'] = df.Date - pd.DateOffset(months=3) #returns 2019-10-01 <---- answer
# For fun, change it to FY Quarter '2019Q4'
df['NewDate'] = df['NewDate'].dt.year.astype(str) + 'Q' + df['NewDate'].dt.quarter.astype(str)
# Convert '2019Q4' back to 2019-10-01
df['NewDate'] = pd.to_datetime(df.NewDate)
One liner ?
previous_month_date = (current_date - datetime.timedelta(days=current_date.day+1)).replace(day=current_date.day)
Simplest Way that i have tried Just now
from datetime import datetime
from django.utils import timezone
current = timezone.now()
if current.month == 1:
month = 12
else:
month = current.month - 1
current = datetime(current.year, month, current.day)
Here is some code to do just that. Haven't tried it out myself...
def add_one_month(t):
"""Return a `datetime.date` or `datetime.datetime` (as given) that is
one month earlier.
Note that the resultant day of the month might change if the following
month has fewer days:
>>> add_one_month(datetime.date(2010, 1, 31))
datetime.date(2010, 2, 28)
"""
import datetime
one_day = datetime.timedelta(days=1)
one_month_later = t + one_day
while one_month_later.month == t.month: # advance to start of next month
one_month_later += one_day
target_month = one_month_later.month
while one_month_later.day < t.day: # advance to appropriate day
one_month_later += one_day
if one_month_later.month != target_month: # gone too far
one_month_later -= one_day
break
return one_month_later
def subtract_one_month(t):
"""Return a `datetime.date` or `datetime.datetime` (as given) that is
one month later.
Note that the resultant day of the month might change if the following
month has fewer days:
>>> subtract_one_month(datetime.date(2010, 3, 31))
datetime.date(2010, 2, 28)
"""
import datetime
one_day = datetime.timedelta(days=1)
one_month_earlier = t - one_day
while one_month_earlier.month == t.month or one_month_earlier.day > t.day:
one_month_earlier -= one_day
return one_month_earlier
Given a (year,month) tuple, where month goes from 1-12, try this:
>>> from datetime import datetime
>>> today = datetime.today()
>>> today
datetime.datetime(2010, 8, 6, 10, 15, 21, 310000)
>>> thismonth = today.year, today.month
>>> thismonth
(2010, 8)
>>> lastmonth = lambda (yr,mo): [(y,m+1) for y,m in (divmod((yr*12+mo-2), 12),)][0]
>>> lastmonth(thismonth)
(2010, 7)
>>> lastmonth( (2010,1) )
(2009, 12)
Assumes there are 12 months in every year.
def month_sub(year, month, sub_month):
result_month = 0
result_year = 0
if month > (sub_month % 12):
result_month = month - (sub_month % 12)
result_year = year - (sub_month / 12)
else:
result_month = 12 - (sub_month % 12) + month
result_year = year - (sub_month / 12 + 1)
return (result_year, result_month)
>>> month_sub(2015, 7, 1)
(2015, 6)
>>> month_sub(2015, 7, -1)
(2015, 8)
>>> month_sub(2015, 7, 13)
(2014, 6)
>>> month_sub(2015, 7, -14)
(2016, 9)
I Used the following code to go back n Months from a specific Date:
your_date = datetime.strptime(input_date, "%Y-%m-%d") #to convert date(2016-01-01) to timestamp
start_date=your_date #start from current date
#Calculate Month
for i in range(0,n): #n = number of months you need to go back
start_date=start_date.replace(day=1) #1st day of current month
start_date=start_date-timedelta(days=1) #last day of previous month
#Calculate Day
if(start_date.day>your_date.day):
start_date=start_date.replace(day=your_date.day)
print start_date
For eg:
input date = 28/12/2015
Calculate 6 months previous date.
I) CALCULATE MONTH:
This step will give you the start_date as 30/06/2015.
Note that after the calculate month step you will get the last day of the required month.
II)CALCULATE DAY:
Condition if(start_date.day>your_date.day) checks whether the day from input_date is present in the required month. This handles condition where input date is 31(or 30) and the required month has less than 31(or 30 in case of feb) days. It handles leap year case as well(For Feb). After this step you will get result as 28/06/2015
If this condition is not satisfied, the start_date remains the last date of the previous month. So if you give 31/12/2015 as input date and want 6 months previous date, it will give you 30/06/2015
You can use below given function to get date before/after X month.
from datetime import date
def next_month(given_date, month):
yyyy = int(((given_date.year * 12 + given_date.month) + month)/12)
mm = int(((given_date.year * 12 + given_date.month) + month)%12)
if mm == 0:
yyyy -= 1
mm = 12
return given_date.replace(year=yyyy, month=mm)
if __name__ == "__main__":
today = date.today()
print(today)
for mm in [-12, -1, 0, 1, 2, 12, 20 ]:
next_date = next_month(today, mm)
print(next_date)
I think this answer is quite readable:
def month_delta(dt, delta):
year_delta, month = divmod(dt.month + delta, 12)
if month == 0:
# convert a 0 to december
month = 12
if delta < 0:
# if moving backwards, then it's december of last year
year_delta -= 1
year = dt.year + year_delta
return dt.replace(month=month, year=year)
for delta in range(-20, 21):
print(delta, "->", month_delta(datetime(2011, 1, 1), delta))
-20 -> 2009-05-01 00:00:00
-19 -> 2009-06-01 00:00:00
-18 -> 2009-07-01 00:00:00
-17 -> 2009-08-01 00:00:00
-16 -> 2009-09-01 00:00:00
-15 -> 2009-10-01 00:00:00
-14 -> 2009-11-01 00:00:00
-13 -> 2009-12-01 00:00:00
-12 -> 2010-01-01 00:00:00
-11 -> 2010-02-01 00:00:00
-10 -> 2010-03-01 00:00:00
-9 -> 2010-04-01 00:00:00
-8 -> 2010-05-01 00:00:00
-7 -> 2010-06-01 00:00:00
-6 -> 2010-07-01 00:00:00
-5 -> 2010-08-01 00:00:00
-4 -> 2010-09-01 00:00:00
-3 -> 2010-10-01 00:00:00
-2 -> 2010-11-01 00:00:00
-1 -> 2010-12-01 00:00:00
0 -> 2011-01-01 00:00:00
1 -> 2011-02-01 00:00:00
2 -> 2011-03-01 00:00:00
3 -> 2011-04-01 00:00:00
4 -> 2011-05-01 00:00:00
5 -> 2011-06-01 00:00:00
6 -> 2011-07-01 00:00:00
7 -> 2011-08-01 00:00:00
8 -> 2011-09-01 00:00:00
9 -> 2011-10-01 00:00:00
10 -> 2011-11-01 00:00:00
11 -> 2012-12-01 00:00:00
12 -> 2012-01-01 00:00:00
13 -> 2012-02-01 00:00:00
14 -> 2012-03-01 00:00:00
15 -> 2012-04-01 00:00:00
16 -> 2012-05-01 00:00:00
17 -> 2012-06-01 00:00:00
18 -> 2012-07-01 00:00:00
19 -> 2012-08-01 00:00:00
20 -> 2012-09-01 00:00:00
Some time ago I came across the following algorithm which works very well for incrementing and decrementing months on either a date or datetime.
CAVEAT: This will fail if day is not available in the new month. I use this on date objects where day == 1 always.
Python 3.x:
def increment_month(d, add=1):
return date(d.year+(d.month+add-1)//12, (d.month+add-1) % 12+1, 1)
For Python 2.7 change the //12 to just /12 since integer division is implied.
I recently used this in a defaults file when a script started to get these useful globals:
MONTH_THIS = datetime.date.today()
MONTH_THIS = datetime.date(MONTH_THIS.year, MONTH_THIS.month, 1)
MONTH_1AGO = datetime.date(MONTH_THIS.year+(MONTH_THIS.month-2)//12,
(MONTH_THIS.month-2) % 12+1, 1)
MONTH_2AGO = datetime.date(MONTH_THIS.year+(MONTH_THIS.month-3)//12,
(MONTH_THIS.month-3) % 12+1, 1)
I used the following method to substract "n_months" months to a datetime object:
from datetime import datetime, timedelta
def substract_months(original_date: datetime, n_months:int) -> datetime:
ref_date = original_date
for i in range(0, number_of_months):
ref_date = (ref_date.replace(day=1) - timedelta(days=1)).replace(day=1)
ref_date = ref_date.replace(day=original_date.day)
return ref_date
You can use it as:
print(substract_months(original_date=datetime(2022, 11, 16), number_of_months=2))
which returns:
2022-09-16 00:00:00
import datetime
date_str = '08/01/2018'
format_str = '%d/%m/%Y'
datetime_obj = datetime.datetime.strptime(date_str, format_str)
datetime_obj.replace(month=datetime_obj.month-1)
Simple solution, no need for special libraries.
You could do it in two lines like this:
now = datetime.now()
last_month = datetime(now.year, now.month - 1, now.day)
remember the imports
from datetime import datetime

Categories

Resources