In pandas I have a variable defined as
start_date = date (2020,7,1)
How do I update it to be the next day? I have a dataframe and I am filtering on individual days but I want to iterate through a full time range. I suppose I could have a for loop like so
for x < 10:
start_date = date (2020,7,x)
x +=1
But is there another way? I couldn't find any other stack exchange questions for python dates.
Assuming date is the regular python one you can add a day as follows:
from datetime import date, timedelta
start_date = date(2020, 7, 1)
next_date = start_date + timedelta(days=1)
Related
What I wanted to do is get 1 year of data.
By calculate latest date from the column date, as my end date. Then use the end date - 1 year to get the start date. After that, I can filter the data in between those start and end date.
I did manage to get the end date, but can't find how I can get the start date.
Below is the code that I have used so far. -1 year is what needs to be solved.
and if you know how to filter in pyspark is also welcome.
from pyspark.sql.functions import min, max
import datetime
import pyspark.sql.function as F
from pyspark.sql.functions import date_format, col
#convert string to date type
df = df.withColumn('risk_date', F.to_date(F.col('chosen_risk_prof_date'), 'dd.MM.yyyy'))
#filter only 1 year of data from big data set.
#calculate the start date and end date. lastest_date = end end.
latest_date = df.select((max("risk_date"))).show()
start_date = latest_date - *1 year*
new_df = df.date > start_date & df.date < end_date
Then after this get all the data between start date and end date
you can use relativedelta as below
from datetime import datetime
from dateutil.relativedelta import relativedelta
print(datetime.now() - relativedelta(years=1))
I'm not a Python developer but have to fix an existing code.
In this code, a method (extract) is called providing an interval of dates:
extract(start_date, end_date)
The parameters can have for exemple the values:
start_date : 2020-10-01
end_date : 2022-01-03
The problem
The issue with this call is that the extract method only support a 1 year max interval of dates. If greater, the interval must be split, for exemple as follow:
extract('2020-10-01', '2020-12-31')
extract('2021-01-01', '2021-12-31')
extract('2022-01-01', '2022-01-03')
So I'm trying to create loop where the start_date and end_date are computed dynamically. But being new to Python, I have no ideas for now how this can be done. Any help would be greatly appreciated.
EDIT
Answer to some comments here
Tried so far finding a solution starting from code like this so far:
from datetime import datetime
from dateutil import relativedelta
from datetime import datetime
from_date = datetime.strptime('2020-10-01', "%Y-%m-%d")
end_date = datetime.strptime('2022-01-03', "%Y-%m-%d")
# Get the interval between the two dates
diff = relativedelta.relativedelta(end_date, from_date)
Then I thought iterating accross the years using diff.years and adding some logic to build the start_date and end_date from there, but I thought there might be a much simplier approach.
Also saw others possibilities like here but still no final simple result found at the moment.
from_str = '2020-10-01'
end_str = '2022-01-03'
from_year = int(from_str[:4])
end_year = int(end_str[:4])
if from_year != end_year:
# from_date to end of first year
extract(from_str, f"{from_year}-12-31")
# full years
for y in range(from_year + 1, end_year):
extract(f"{y}-01-01", f"{y}-12-31")
# rest
extract(f"{end_year}-01-01", end_str)
else:
extract(from_str, end_str)
As mentioned in the comments, you can either use the datetime library or you can also use pandas if you want. The pandas version is the following (admittively not the most pretty, but it does the job):
import pandas as pd
import datetime
start = datetime.datetime(2020,10,1)
end = datetime.datetime(2022,1,3)
def extract(from_dt, to_dt):
print(f'Extracting from {from_dt} to {to_dt}')
prev_end = pd.to_datetime(start)
for next_end in pd.date_range(datetime.datetime(start.year, 12, 31), end, freq='y'):
if next_end < end:
extract(prev_end.strftime('%Y-%m-%d'), next_end.strftime('%Y-%m-%d'))
else:
extract(prev_end.strftime('%Y-%m-%d'), end.strftime('%Y-%m-%d'))
prev_end = next_end + datetime.timedelta(days=1)
if prev_end < end:
extract(prev_end.strftime('%Y-%m-%d'), end.strftime('%Y-%m-%d'))
If you need to parse the original dates from strings, check out datetime.strptime
This kind of problems are nice ones to resolve by recursion:
from datetime import datetime
start_date = '2020-10-01'
end_date = '2022-01-03'
def intervalcalc(datestart,dateend):
newdate=dateend[:4] + '-01-01'
startd = datetime.strptime(datestart, "%Y-%m-%d")
endd = datetime.strptime(newdate, "%Y-%m-%d")
if endd < startd:
print(datestart, dateend)
return True
else:
print(newdate, dateend)
previousyear=str(int(newdate[:4])-1) + '-12-31'
intervalcalc(datestart,previousyear)
intervalcalc(start_date, end_date)
output:
2022-01-01 2022-01-03
2021-01-01 2021-12-31
2020-10-01 2020-12-31
You just need to change the prints by calls to extract function.
As mentioned by #Wups the conversion to date is not really necessary, it could be an string compare as they are YYYYMMDD dates.
Also, this can be done the other way around and calculate from the start date year + '-12-31' and then compare dateend>end_date to determine the anchor for the recursion.
I want tot create a list of dates starting from 10/09/2020 with increments of 182 days until reaching 05/03/2020.
my code is this :
start_date="10/09/2020"
last_date="05/03/2020"
start_date=datetimedatetime.strptime(date,"%d/%m/%Y").date()
last_date=datetimedatetime.strptime(date,"%d/%m/%Y").date()
dates=[]
while dates[-1] != last_date:
i=star_date+timedelta(days=182)
dates.append(i)
dates[i]=i+timedelta(days=pago_cupon)
I don't know what's your problem.
You can try this code
from datetime import date, timedelta
start_day = date(year=2020, month=9, day=10)
end_day = date(year=2021, month=3, day=5)
one_data_delta = timedelta(days=1)
res = []
while end_day != start_day:
start_day += one_data_delta
res.append(start_day)
print(res)
A few problems:
You had a spelling mistake in your for loop star_date instead of start_date.
Also, you may want to change the comparison from != to less than equals or more than equals.
I am checking against (last_date - timedelta(days=182)) so that we won't exceed the last_date.
Your original start date is after your original end date.
As an example, I have adjusted the end day to be a couple years in the future.
I'm appending the dates as text.
from datetime import datetime, timedelta
start="10/09/2020"
last="05/03/2022"
start_date=datetime.strptime(start,"%d/%m/%Y")
last_date=datetime.strptime(last,"%d/%m/%Y")
dates=[]
while start_date <= (last_date - timedelta(days=182)):
start_date += timedelta(days=182)
dates.append(start_date.strftime("%d/%m/%Y"))
# not quite sure what you are trying to do here:
#dates[i]=i+timedelta(days=pago_cupon)
print(dates)
Output:
['11/03/2021', '09/09/2021']
I have a dictionary with many sorted dates. How could I write a loop in Python that checks if a certain date is in the dictionary and if not, it returns the closest date available? I want it to work that if after subtracting one day to the date, it checks again if now it exists in the dictionary and if not, it subtracts again until it finds a existing date.
Thanks in advance
from datetime import timedelta
def function(date):
if date not in dictio:
date -= timedelta(days=1)
return date
I've made a recursive function to solve your problem:
import datetime
def find_date(date, date_dict):
if date not in date_dict.keys():
return find_date(date-datetime.timedelta(days=1), date_dict)
else:
return date
I don't know what is the content of your dictionary but the following example should show you how this works:
import numpy as np
# creates a casual dates dictionary
months = np.random.randint(3,5,20)
days = np.random.randint(1,30,20)
dates = {
datetime.date(2019,m,d): '{}_{:02}_{:02}'.format(2019,m,d)
for m,d in zip(months,days)}
# select the date to find
target_date = datetime.date(2019, np.random.randint(3,5), np.random.randint(1,30))
# print the result
print("The date I wanted: {}".format(target_date))
print("The date I got: {}".format(find_date(target_date, dates)))
What you are looking for is possibly a while loop, although beware because if it will not find the date it will run to infinite. Perhaps you want to define a limit of attempts until the script should give up?
from datetime import timedelta, date
d1 = {
date(2019, 4, 1): None
}
def function(date, dictio):
while date not in dictio:
date -= timedelta(days=1)
return date
res_date = function(date.today(), d1)
print(res_date)
I am making a program where I input start date to dataStart(example 21.10.2000) and then input int days dateEnd and I convert it to another date (example 3000 = 0008-02-20)... Now I need to count these dates together, but I didn't managed myself how to do that. Here is my code.
from datetime import date
start=str(input("type start date (DD.MM.YYYY)"))
end=int(input("how many days from it?"))
dataStart=start.split(".")
days=int(dataStart[0])
months=int(dataStart[1])
years=int(dataStart[2])
endYears=0
endMonths=0
endDays=0
dateStart = date(years, months, days)
while end>=365:
end-=365
endYears+=1
else:
while end>=30:
end-=30
endMonths+=1
else:
while end>=1:
end-=1
endDays+=1
dateEnd = date(endYears, endMonths, endDays)
For adding days into date, you need to user datetime.timedelta
start=str(input("type start date (DD.MM.YYYY)"))
end=int(input("how many days from it?"))
date = datetime.strptime(start, "%d.%m.%Y")
modified_date = date + timedelta(days=end)
print(datetime.strftime(modified_date, "%d.%m.%Y"))
You may use datetime.timedelta to add certain units of time to your datetime object.
See the answers here for code snippets: Adding 5 days to a date in Python
Alternatively, you may wish to use the third-party dateutil library if you need support for time additions in units larger than weeks. For example:
>>> from datetime import datetime
>>> from dateutil import relativedelta
>>> one_month_later = datetime(2017, 5, 1) + relativedelta.relativedelta(months=1)
>>> one_month_later
>>> datetime.datetime(2017, 6, 1, 0, 0)
It will be easier to convert to datetime using datetime.datetime.strptime and for the part about adding days just use datetime.timedelta.
Below is a small snippet on how to use it:
import datetime
start = "21.10.2000"
end = 8
dateStart = datetime.datetime.strptime(start, "%d.%m.%Y")
dateEnd = dateStart + datetime.timedelta(days=end)
dateEnd.date() # to get the date format of the endDate
If you have any doubts please look at the documentation python3/python2.