This question already has answers here:
Resample a pandas timeseries by "1st Monday of month" etc
(2 answers)
Closed 3 years ago.
I need to generate an array of dates at specific intervals. for example every 3 sunday of the month:
[2019-02-17, 2019-03-17, 2019-04-21]
Is it possible to do this using standard pandas functions? For example, specifying some particular freq field in the pd.date_range method or using pd.Dateoffset?
You could do something like this:
s = pd.date_range('2019-01-01','2019-12-31', freq='D')
s[(s.dayofweek == 6) & (s.day>=15) & (s.day<=21)]
Output:
DatetimeIndex(['2019-01-20', '2019-02-17', '2019-03-17', '2019-04-21',
'2019-05-19', '2019-06-16', '2019-07-21', '2019-08-18',
'2019-09-15', '2019-10-20', '2019-11-17', '2019-12-15'],
dtype='datetime64[ns]', freq=None)
Related
This question already has answers here:
How to change the datetime format in Pandas
(8 answers)
Closed 12 months ago.
I have a pandas dataframe with dates in the following format:
Dec 11, 2018
Wondering is there an easy way to change the format to 11/12/2018? I know I can go through each month manually but not sure what my next step would be to switch around the month and day and add the /.
Thanks in advance!
Use strftime('%m/%d/%Y'):
s = pd.Series(['Dec 11, 2018'])
pd.to_datetime(s).dt.strftime('%m/%d/%Y')
Output:
0 12/11/2018
dtype: object
This question already has answers here:
How to get year and week number aligned for a date
(3 answers)
Closed 2 years ago.
Is there a better solution than df['weekofyear'] = df['date'].dt.weekofyear?
The problem of this solution is that, sometimes, the days after the last week of the year n but before the first week of the year n+1 are counted as week 1 and and not as week 0.
I am working with pyspark and koalas (no pandas allowed).
Here is an example:
As you can see, the first column is Date, the second one is week, the third is month and last is year.
Not sure if this is what you want...? I suppose you can use case when to replace the undesired values of week of year.
df['weekofyear'] = df['date'].dt.weekofyear
df2 = ks.sql("""
select
date,
case when weekofyear = 1 and month = 12 then 53 else weekofyear end as weekofyear,
month,
year
from {df}""")
This question already has answers here:
Subtract a year from a datetime column in pandas
(4 answers)
Closed 4 years ago.
I have a pandas Time Series (called df) that has one column (with name data) that contains data with a daily frequency over a time period of 5 years. The following code produces some random data:
import pandas as pd
import numpy as np
df_index = pd.date_range('01-01-2012', periods=5 * 365 + 2, freq='D')
df = pd.DataFrame({'data': np.random.rand(len(df_index))}, index=df_index)
I want to perform a simple yearly trend decomposition, where for each day I subtract its value one year ago. Aditionally, I want to attend leap years in the subtraction. Is there any elegant way to do that? My way to do this is to perform differences with 365 and 366 days and assign them to new columns.
df['diff_365'] = df['data'].diff(365)
df['diff_366'] = df['data'].diff(366)
Afterwards, I apply a function to each row thats selects the right value based on whether the same date from last year is 365 or 366 days ago.
def decide(row):
if (row.name - 59).is_leap_year:
return row[1]
else:
return row[0]
df['yearly_diff'] = df[['diff_365', 'diff_366']].apply(decide, axis=1)
Explanation: the function decide takes as argument a row from the DataFrame consisting of the columns diff_365 and diff_366 (along with the DatetimeIndex). The expression row.name returns the date of the row and assuming the time series has daily frequency (freq = 'D'), 59 days are subtracted which is the number of days from 1st January to 28th February. Based on whether the resulting date is a day from a leap year, the value from the diff_366 column is returned, otherwise the value from the diff_365 column.
This took 8 lines and it feels that the subtraction can be performed in one or two lines. I tried to apply a similiar function directly to the data column (via apply and taking the default argument axis=0). But in this case, I cannot take my DatetimeIndex into account. Is there a better to perform the subtraction?
You may not need to worry about dealing with leap years explicitly. When you construct a DatetimeIndex, you can specify start and end parameters. As per the docs:
Of the four parameters start, end, periods, and freq, exactly three
must be specified.
Here's an example of how you can restructure your logic:
df_index = pd.date_range(start='01-01-2012', end='12-31-2016', freq='D')
df = pd.DataFrame({'data': np.random.rand(len(df_index))}, index=df_index)
df['yearly_diff'] = df['data'] - (df_index - pd.DateOffset(years=1)).map(df['data'].get)
Explanation
We construct a DatetimeIndex object by supplying start, end and freq arguments.
Subtract 1 year from your index by subtracting pd.DateOffset(years=1).
Use pd.Series.map to map these 1yr behind dates to data.
Subtract the resulting series from the original data series.
This question already has answers here:
How to calculate number of days between two given dates
(15 answers)
Closed 7 years ago.
How can I subtract on datefield from another and get result in integer?
Like 11.05.2015-10.05.2015 will return 1
I tried
entry.start_devation= each.start_on_fact - timedelta(days=data.calendar_start)
Take the difference between two dates, which is a timedelta, and ask for the days attribute of that timedelta.
This question already has answers here:
How do I get the day of week given a date?
(30 answers)
Closed 7 years ago.
Please suggest me on the following.
How to find whether a particular day is weekday or weekend in Python?
You can use the .weekday() method of a datetime.date object
import datetime
weekno = datetime.datetime.today().weekday()
if weekno < 5:
print "Weekday"
else: # 5 Sat, 6 Sun
print "Weekend"
Use the date.weekday() method. Digits 0-6 represent the consecutive days of the week, starting from Monday.