Develop Reference

Python command parameter indicating time delta

I need to specify a parameter to a python script indicating time back from now. For example 1d is 1 day from now, 2h is 2 hours from now, 2d3h is 2 days, 3 hours from now. Similar to the journalctl --vacuum-time format (reference).
For example, this script will collect the data between now and 2 days and 3 hours in the past:
collect_data_script.py --start=2d3h
Is there some standard way to handle this parameter and a package that can process this time format? Or I will have to write it from scratch?

The ISO 8601 duration format like 2DT3H for "2 days (time designator) 3 hours" can be parsed using module isodate from Gerhard Weiss:
implements ISO 8601 date, time and duration parsing. The implementation follows ISO8601:2004 standard
parse_duration:
parses an ISO 8601 duration string into a timedelta or Duration object.
Example:
import isodate
isodate.parse_duration('p2d3h'.upper()) # prefix with 'p' for period and to uppercase
# will raise ValueError because ISO 8601 time designator 'T' missing
timedelta = isodate.parse_duration('p2dt3h'.upper()) # insert 'T' to work
# datetime.timedelta(2, 10800)
print(timedelta)
# 2 days, 3:00:00
Similar modules
arrow: offers a sensible and human-friendly approach to creating, manipulating, formatting and converting dates, times and timestamps.
duration: python time duration conversion module
iso8601: Simple module to parse ISO 8601 dates
See also:
How can I parse and compare ISO 8601 durations in Python?

#hc_dev solution will work. However, I used a slightly different one that needed fewer additional modules. Just posting here for reference:
from datetime import datetime, timedelta
from dateutil import parser
def process_cmd_time(time_str, bound) -> datetime:
"""
Convert passed string to the datetime object.
Will throw ParseError or ValueError if the string is invalid
Keyword arguments:
time_str - string representation of the time to set
bound - flag to force time bound to 24h from now
"""
time_str = time_str.strip()
go_back = time_str[0] == '-'
# Attempt to get the date. Will throw ParseError if invalid
input_date = parser.parse(time_str)
# If the time needs to go back, find the delta from the beginning of today,
# then subtract from now
if go_back:
delta_today = input_date - parser.parse("0h0m0s")
input_date = datetime.now() - delta_today
if bound:
delta_time = datetime.now() - input_date
if delta_time.days > 0:
input_date = datetime.now() - timedelta(hours=24, minutes=0, seconds=0)
return input_date

I have a list of dates and I want to subtract actual date from each of them to know how many day passed. Is there any fast way to do this?

I know I should import datetime to have actual date. But the rest is black magic for me right now.
ex.
dates = ['2019-010-11', '2013-05-16', '2011-06-16', '2000-04-22']
actual_date = datetime.datetime.now()
How can I subtract this and as a result have new list with days that passed by from dates to actual_date?

If I'm understanding correctly, you need to find the current date, and then find the number of days between the current date and the dates in your list?
If so, you could try this:
from datetime import datetime, date
dates = ['2019-10-11', '2013-05-16', '2011-06-16', '2000-04-22']
actual_date = date.today()
days = []
for date in dates:
date_object = datetime.strptime(date, '%Y-%m-%d').date()
days_difference = (actual_date - date_object).days
days.append(days_difference)
print(days)
What I am doing here is:
Converting the individual date strings to a "date" object
Subtracting the this date from the actual date. This gets you the time as well, so to strip that out we add .days.
Save the outcome to a list, although of course you could do whatever you wanted with the output.

Generate randomly formatted date strings for machine learning

For a NLP project in python I need to generate random dates for model training purpose. Particularly, the date format must be random and coherent with a set of language locales. The formats includes those with only numbers and formats with (partially) written out day and month names, and various common punctuations.
My best solution so far is the following algorithm:
generate a datetime() object with random values (nice solution here)
randomly select a locale, i.e. pick one of ['en_US','fr_FR','it_IT','de_DE'] where in this case this list is well known and short, so not a problem.
randomly select a format string for strftime(), i.e. ['%Y-%m-%d','%d %B %Y',...]. In my case the list should reflect potentially occuring date formats in the documents that will be exposed to the NLP model in the future.
generate a sting with strftime()
Especially for 3) i do not know a better version than to hardcode the list of what I saw manually within the training documents. I could not yet find a function that would turn ocr-dates into a format string, such that i could extend the list when yet-unseen date formats come by.
Do you have any suggestions on how to come up with better randomly formatted dates, or how to improve this approach?

USE random.randrange() AND datetime.timedelta() TO GENERATE A RANDOM DATE BETWEEN TWO DATES
Call datetime.date(year, month, day) to return a datetime object representing the time indicated by year, month, and day. Call this twice to define the start and end date. Subtract the start date from the end date to get the time between the two dates. Call datetime.timedelta.days to get the number of days from the previous result datetime.timedelta. Call random.randrange(days) to get a random integer less than the previous result days. Call datetime.timedelta(days=n) to get a datetime.timedelta representing the previous result n. Add this result to the start date.
start_date = datetime.date(2020, 1, 1)
end_date = datetime.date(2020, 2, 1)
time_between_dates = end_date - start_date
days_between_dates = time_between_dates.days
random_number_of_days = random.randrange(days_between_dates)
random_date = start_date + datetime.timedelta(days=random_number_of_days)
print(random_date)

Here is my solution. Concerning the local, all need to be available on your computer to avoid error
import random
from datetime import datetime, timedelta
import locale
LOCALE = ['en_US','fr_FR','it_IT','de_DE'] # all need to be available on your computer to avoid error
DATE_FORMAT = ['%Y-%m-%d','%d %B %Y']
def gen_datetime(min_year=1900, max_year=datetime.now().year):
# generate a datetime
start = datetime(min_year, 1, 1)
years = max_year - min_year + 1
end = start + timedelta(days=365 * years)
format_date = DATE_FORMAT[random.randint(0, len(DATE_FORMAT)-1)]
locale_date = LOCALE[random.randint(0, len(LOCALE)-1)]
locale.setlocale(locale.LC_ALL, locale_date) # generate error if local are not available on your computer
return (start + (end - start) * random.random()).strftime(format_date)
date = gen_datetime()
print(date)

Formatting string into datetime using Pandas - trouble with directives

I have a string that is the full year followed by the ISO week of the year (so some years have 53 weeks, because the week counting starts at the first full week of the year). I want to convert it to a datetime object using pandas.to_datetime(). So I do:
pandas.to_datetime('201145', format='%Y%W')
and it returns:
Timestamp('2011-01-01 00:00:00')
which is not right. Or if I try:
pandas.to_datetime('201145', format='%Y%V')
it tells me that %V is a bad directive.
What am I doing wrong?

I think that the following question would be useful to you: Reversing date.isocalender()
Using the functions provided in that question this is how I would proceed:
import datetime
import pandas as pd
def iso_year_start(iso_year):
"The gregorian calendar date of the first day of the given ISO year"
fourth_jan = datetime.date(iso_year, 1, 4)
delta = datetime.timedelta(fourth_jan.isoweekday()-1)
return fourth_jan - delta
def iso_to_gregorian(iso_year, iso_week, iso_day):
"Gregorian calendar date for the given ISO year, week and day"
year_start = iso_year_start(iso_year)
return year_start + datetime.timedelta(days=iso_day-1, weeks=iso_week-1)
def time_stamp(yourString):
year = int(yourString[0:4])
week = int(yourString[-2:])
day = 1
return year, week, day
yourTimeStamp = iso_to_gregorian( time_stamp('201145')[0] , time_stamp('201145')[1], time_stamp('201145')[2] )
print yourTimeStamp
Then run that function for your values and append them as date time objects to the dataframe.
The result I got from your specified string was:
2011-11-07

How can I generate POSIX values for yesterday and today at midnight in Python?

I've been struggling to determine how I can generate a POSIX (UNIX) time value for today and yesterday (midnight) via Python. I created this code, but keep stumbling with how to convert them to a POSIX value:
from datetime import datetime, timedelta
import time
today_string = datetime.now().strftime('%Y-%m-%d 00:00:00')
yesterday_string = (datetime.now() - timedelta(0)).strftime('%Y-%m-%d 00:00:00')
today = datetime.strptime(today_string, '%Y-%m-%d %H:%M:%S')
yesterday = datetime.strptime(yesterday_string, '%Y-%m-%d %H:%M:%S')
print time.mktime(today).timetuple()
This code yields an exception:
TypeError: argument must be 9-item sequence, not datetime.datetime
At this point, I'm at my wits end. Any help you can provide is appreciated.

You should apply the timetuple() method to the today object, not to the result of time.mktime(today):
>>> time.mktime(today.timetuple())
1345845600.0
By the way, I'm wrong or yesterday will be equal to today in your code?
edit:
To obtain the POSIX time for today you can simply do:
time.mktime(datetime.date.today().timetuple())

#Bakuriu is right here. But you are making this overcomplex.
Take a look at this:
from datetime import date, timedelta
import time
today = date.today()
today_unix = time.mktime(today.timetuple())
yesterday = today - timedelta(1)
yesterday_unix = time.mktime(yesterday.timetuple())
Since the date object doesn't hold time, it resets it to the midnight.
You could also replace the last part with:
yesterday_unix = today_unix - 86400
but note that it wouldn't work correctly across daylight saving time switches (i.e. you'll end up with 1 AM or 23 PM).

Getting a unix timestamp from a datetime object as a string and as a float:
datetime.now().strftime('%s')
'1345884732'
time.mktime(datetime.now().timetuple())
1345884732.0