I have a datetime instance declared as follows:
dtDate = datetime.datetime(2016,1,1,0,0)
How do I get the previous month and previous year from dtDate?
e.g. something like:
dtDate.minusOneMonth()
# to return datetime.datetime(2015,12,1,0,0)
You can use:
dtDate = datetime.datetime(2016,1,1,0,0)
print (dtDate - pd.DateOffset(months=1))
2015-12-01 00:00:00
print (dtDate - pd.DateOffset(years=1))
2015-01-01 00:00:00
Add s is important, because if use year only:
print (dtDate - pd.DateOffset(year=1))
0001-01-01 00:00:00
You can use DateOffset:
In [32]:
dtDate = dt.datetime(2016,1,1,0,0)
dtDate - pd.DateOffset(months=1)
Out[32]:
Timestamp('2015-12-01 00:00:00')
To Manipulate an entire pandas series.
Use pd.DateOffset() with .dt.to_period("M")
df['year_month'] = df['timestamp'].dt.to_period("M")
df['prev_year_month'] = (df['timestamp'] - pd.DateOffset(months=1)).dt.to_period("M")
If you want to go forward a month, set months=-1.
Use relativedelta from dateutil:
import datetime
import dateutil.relativedelta
dtDate = datetime.datetime(2016,1,1,0,0)
# get previous month
print ((dtDate+dateutil.relativedelta.relativedelta(months=-1)).month)
# get previous year
print ((dtDate+dateutil.relativedelta.relativedelta(years=-1)).year)
Output:
12
2015
Related
So, Basically, I got this 2 df columns with data content. The initial content is in the dd/mm/YYYY format, and I want to subtract them. But I can't really subtract string, so I converted it to datetime, but when I do such thing for some reason the format changes to YYYY-dd-mm, so when I try to subtract them, I got a wrong result. For example:
Initial Content:
a: 05/09/2022
b: 30/09/2021
result expected: 25 days.
Converted to DateTime:
a: 2022-05-09
b: 2021-09-30 (For some reason this date stills the same)
result: 144 days.
I'm using pandas and datetime to make this project.
So, I wanted to know a way I can subtract this 2 columns with the proper result.
--- Answer
When I used
pd.to_datetime(date, format="%d/%m/%Y")
It worked. Thank you all for your time. This is my first project in pandas. :)
df = pd.DataFrame({'Date1': ['05/09/2021'], 'Date2': ['30/09/2021']})
df = df.apply(lambda x:pd.to_datetime(x,format=r'%d/%m/%Y')).assign(Delta=lambda x: (x.Date2-x.Date1).dt.days)
print(df)
Date1 Date2 Delta
0 2021-09-05 2021-09-30 25
I just answered a similar query here subtracting dates in python
import datetime
from datetime import date
from datetime import datetime
import pandas as pd
date_format_str = '%Y-%m-%d %H:%M:%S.%f'
date_1 = '2016-09-24 17:42:27.839496'
date_2 = '2017-01-18 10:24:08.629327'
start = datetime.strptime(date_1, date_format_str)
end = datetime.strptime(date_2, date_format_str)
diff = end - start
# Get interval between two timstamps as timedelta object
diff_in_hours = diff.total_seconds() / 3600
print(diff_in_hours)
# get the difference between two dates as timedelta object
diff = end.date() - start.date()
print(diff.days)
Pandas
import datetime
from datetime import date
from datetime import datetime
import pandas as pd
date_1 = '2016-09-24 17:42:27.839496'
date_2 = '2017-01-18 10:24:08.629327'
start = pd.to_datetime(date_1, format='%Y-%m-%d %H:%M:%S.%f')
end = pd.to_datetime(date_2, format='%Y-%m-%d %H:%M:%S.%f')
# get the difference between two datetimes as timedelta object
diff = end - start
print(diff.days)
I'm trying to subtract a day from this date 06-30-2019 in order to make it 06-29-2019 but can't figure out any way to achive that.
I've tried with:
import datetime
date = "06-30-2019"
date = datetime.datetime.strptime(date,'%m-%d-%Y').strftime('%m-%d-%Y')
print(date)
It surely gives me back the date I used above.
How can I subtract a day from a date in the above format?
try this
import datetime
date = "06/30/19"
date = datetime.datetime.strptime(date, "%m/%d/%y")
NewDate = date + datetime.timedelta(days=-1)
print(NewDate) # 2019-06-29 00:00:00
Your code:
date = "06-30-2019"
date = datetime.datetime.strptime(date,'%m-%d-%Y').strftime('%m-%d-%Y')
Check type of date variable.
type(date)
Out[]: str
It is in string format. To perform subtraction operation you must convert it into date format first. You can use pd.to_datetime()
# Import packages
import pandas as pd
from datetime import timedelta
# input date
date = "06-30-2019"
# Convert it into pd.to_datetime format
date = pd.to_datetime(date)
print(date)
# Substracting days
number_of_days = 1
new_date = date - timedelta(number_of_days)
print(new_date)
output:
2019-06-29 00:00:00
If you want to get rid of timestamp you can use:
str(new_date.date())
Out[]: '2019-06-29'
use timedelta
import datetime
date = datetime.datetime.strptime("06/30/19" ,"%m/%d/%y")
print( date - datetime.timedelta(days=1))
I'd like to get the break of a variable by year, month and day. Here's what I got:
import datetime
from datetime import date, timedelta
yesterday = date.today() - timedelta(1)
print (yesterday)
year = datetime.date.yesterday.year
month = datetime.date.yesterday.month
day=datetime.date.yesterday.day
print (year)
print (month)
print (day)
I'm getting an error that datetime.date has no attribute. I'm a total noob at python and I'm stuck, any help is appreciated
you were close
import datetime
from datetime import date, timedelta
yesterday = date.today() - timedelta(1)
print (yesterday)
year = yesterday.year
month = yesterday.month
day=yesterday.day
print (year)
print (month)
print (day)
result is
2019-03-10
2019
3
10
You can use strftime method
A simple example:
>>> from datetime import datetime
>>> now = datetime.utcnow()
>>> year_month_day_format = '%Y-%m-%d'
>>> now.strftime(year_month_day_format)
'2020-11-06'
>>> hour_minute_format = '%H:%M'
>>> now.strftime(hour_minute_format)
'22:54'
Hopping, it will help someones
You can also simplify your import statements like so:
from datetime import datetime, timedelta
yesterday = datetime.today() - timedelta(1)
print(yesterday)
year = yesterday.year
month = yesterday.month
day = yesterday.day
print(year)
print(month)
print(day)
You will get the output:
2019-03-10 21:19:36.695577
2019
3
10
For current day
import datetime
current_datetime=datetime.datetime.now()
print("current_year:{}".format(current_datetime.year))
print("current_month:{}".format(current_datetime.month))
print("current_day:{}".format(current_datetime.day))
If you want in this format for example "10-Oct-2018". You can try this code for current day.
from datetime import datetime, timezone
now_utc = datetime.now(timezone.utc)
year = now_utc.strftime("%Y")
month = now_utc.strftime("%b")
day = now_utc.strftime("%d")
result = day+"-"+month+"-"+year
print(result)
I have written a function to convert pandas datetime dates to month-end:
import pandas
import numpy
import datetime
from pandas.tseries.offsets import Day, MonthEnd
def get_month_end(d):
month_end = d - Day() + MonthEnd()
if month_end.month == d.month:
return month_end # 31/March + MonthEnd() returns 30/April
else:
print "Something went wrong while converting dates to EOM: " + d + " was converted to " + month_end
raise
This function seems to be quite slow, and I was wondering if there is any faster alternative? The reason I noticed it's slow is that I am running this on a dataframe column with 50'000 dates, and I can see that the code is much slower since introducing that function (before I was converting dates to end-of-month).
df = pandas.read_csv(inpath, na_values = nas, converters = {open_date: read_as_date})
df[open_date] = df[open_date].apply(get_month_end)
I am not sure if that's relevant, but I am reading the dates in as follows:
def read_as_date(x):
return datetime.datetime.strptime(x, fmt)
Revised, converting to period and then back to timestamp does the trick
In [104]: df = DataFrame(dict(date = [Timestamp('20130101'),Timestamp('20130131'),Timestamp('20130331'),Timestamp('20130330')],value=randn(4))).set_index('date')
In [105]: df
Out[105]:
value
date
2013-01-01 -0.346980
2013-01-31 1.954909
2013-03-31 -0.505037
2013-03-30 2.545073
In [106]: df.index = df.index.to_period('M').to_timestamp('M')
In [107]: df
Out[107]:
value
2013-01-31 -0.346980
2013-01-31 1.954909
2013-03-31 -0.505037
2013-03-31 2.545073
Note that this type of conversion can also be done like this, the above would be slightly faster, though.
In [85]: df.index + pd.offsets.MonthEnd(0)
Out[85]: DatetimeIndex(['2013-01-31', '2013-01-31', '2013-03-31', '2013-03-31'], dtype='datetime64[ns]', name=u'date', freq=None, tz=None)
If the date column is in datetime format and is set to starting day of the month, this will add one month of time to it:
df['date1']=df['date'] + pd.offsets.MonthEnd(0)
import pandas as pd
import numpy as np
import datetime as dt
df0['Calendar day'] = pd.to_datetime(df0['Calendar day'], format='%m/%d/%Y')
df0['Calendar day'] = df0['Calendar day'].apply(pd.datetools.normalize_date)
df0['Month Start Date'] = df0['Calendar day'].dt.to_period('M').apply(lambda r: r.start_time)
This code should work. Calendar Day is a column in which date is given in the format %m/%d/%Y. For example: 12/28/2014 is 28 December, 2014. The output comes out to be 2014-12-01 in class 'pandas.tslib.Timestamp' type.
you can also use numpy to do it faster:
import numpy as np
date_array = np.array(['2013-01-01', '2013-01-15', '2013-01-30']).astype('datetime64[ns]')
month_start_date = date_array.astype('datetime64[M]')
In case the date is not in the index but in another column (works for Pandas 0.25.0):
import pandas as pd
import numpy as np
df = pd.DataFrame(dict(date = [pd.Timestamp('20130101'),
pd.Timestamp('20130201'),
pd.Timestamp('20130301'),
pd.Timestamp('20130401')],
value = np.random.rand(4)))
print(df.to_string())
df.date = df.date.dt.to_period('M').dt.to_timestamp('M')
print(df.to_string())
Output:
date value
0 2013-01-01 0.295791
1 2013-02-01 0.278883
2 2013-03-01 0.708943
3 2013-04-01 0.483467
date value
0 2013-01-31 0.295791
1 2013-02-28 0.278883
2 2013-03-31 0.708943
3 2013-04-30 0.483467
What you are looking for might be:
df.resample('M').last()
The other method as said earlier by #Jeff:
df.index = df.index.to_period('M').to_timestamp('M')
Let's say i have 2 strings 'Jan-2010' and 'Mar-2010' and i want to parse it such that it returns 2 datetime objects: 1-Jan-2010 and 31-Mar-2010 (i.e. the last day).
What would be the best strategy in python? Should i just split the string into tokens or use regular expressions and then use the calendar functions to get say the last day of the month for 'Mar-2010' (getting the first day is trivial, its always 1 in this case unless i wanted the first working day of the month).
Any suggestions? Thanks in advance.
strptime does the string parsing into dates on your behalf:
def firstofmonth(MmmYyyy):
return datetime.datetime.strptime(MmmYyyy, '%b-%Y').date()
much better than messing around with tokenization, regexp, &c!-).
To get the date of the last day of the month, you can indeed use the calendar module:
def lastofmonth(MmmYyyy):
first = firstofmonth(MmmYyyy)
_, lastday = calendar.monthrange(first.year, first.month)
return datetime.date(first.year, first.month, lastday)
You could ALMOST do it neatly with datetime alone, e.g., an ALMOST working approach:
def lastofmonth(MmmYyyy):
first = firstofmonth(MmmYyyy)
return first.replace(month=first.month+1, day=1
) - datetime.timedelta(days=1)
but, alas!, this breaks for December, and the code needed to specialcase December makes the overall approach goofier than calendar affords;-).
I highly recommend using the python timeseries module, which you can download and read about here:
http://pytseries.sourceforge.net/
You should also use the dateutil package for parsing the date string, which you can find here:
http://labix.org/python-dateutil
Then you can do something like this
import datetime
import dateutil.parser
import scikits.timeseries as TS
m1 = TS.Date('M', datetime=dateutil.parser.parse('Jan-2010'))
m2 = TS.Date('M', datetime=dateutil.parser.parse('Mar-2010'))
d1 = m1.asfreq('D', relation='START') # returns a TS.Date object
d2 = m2.asfreq('D', relation='END')
firstDay = d1.datetime
lastDay = d2.datetime
This solution is dependent out outside modules, but they're very powerful and well written.
from datetime import datetime, timedelta
def first_day(some_date):
return some_date.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
def next_month(some_date):
return first_day(first_day(some_date) + timedelta(days=31))
def last_day(some_date):
return next_month(some_date) - timedelta(days=1)
# testing:
months = [('Jan-2010', 'Mar-2010'), # your example
('Apr-2009', 'Apr-2009'), # same month, 30 days
('Jan-2008', 'Dec-2008'), # whole year
('Jan-2007', 'Feb-2007')] # february involved
for date1, date2 in months:
print first_day(datetime.strptime(date1, '%b-%Y')),
print '-',
print last_day(datetime.strptime(date2, '%b-%Y'))
That prints:
2010-01-01 00:00:00 - 2010-03-31 00:00:00
2009-04-01 00:00:00 - 2009-04-30 00:00:00
2008-01-01 00:00:00 - 2008-12-31 00:00:00
2007-01-01 00:00:00 - 2007-02-28 00:00:00
i know it's long time gone, but if someone needs:
from dateutil import rrule
from dateutil import parser
from datetime import datetime
first_day = parser.parse('Jan-2010',default=datetime(1,1,1))
last_day = rrule.rrule(rrule.MONTHLY,count=1,bymonthday=-1, bysetpos=1,dtstart=parser.parse('Mar-2010'))
Riffing on Alex Martelli's:
import datetime
def lastofmonthHelper(MmmYyyy): # Takes a date
return MmmYyyy.replace(year=MmmYyyy.year+(MmmYyyy.month==12), month=MmmYyyy.month%12 + 1, day=1) - datetime.timedelta(days=1)
>>> for month in range(1,13):
... t = datetime.date(2009,month,1)
... print t, lastofmonthHelper(t)
...
2009-01-01 2009-01-31
2009-02-01 2009-02-28
2009-03-01 2009-03-31
2009-04-01 2009-04-30
2009-05-01 2009-05-31
2009-06-01 2009-06-30
2009-07-01 2009-07-31
2009-08-01 2009-08-31
2009-09-01 2009-09-30
2009-10-01 2009-10-31
2009-11-01 2009-11-30
2009-12-01 2009-12-31
You don't have to use the first day of the month, BTW. I would have put this in a comment but we all know how the formatting would have turned out. Feel free to upvote Alex.
If you call with the result of a firstofmonth() call, you get the desired result:
>>> lastofmonthHelper(firstofmonth('Apr-2009'))
datetime.date(2009, 4, 30)