To keep track of my when my files were backed up I want to have the filename of the backups as the datetime of when they were backed up. This will eventually be sorted and retrieved and sorted using python to allow me to get the most recent file based on the datetime filename.
The problem is, the automatic format of date time cant be saved like this:
2007-12-31 22:29:59
It can for example be saved like this:
2007-12-31 22-29-59
What is the best way to format the datetime so that I can easily sort by datetime on the name, and for bonus points, what is the python to show the datetime in that way.
You should have a look the documentation of the python time module: http://docs.python.org/2/library/time.html#module-time
If you go to the strftime() function, you will see that it accepts a string as input, which describes the format of the string you want to get as the return value.
Example (with hyphens between each date/time token):
>>> s = time.strftime('%Y-%m-%d-%H-%M-%S')
>>> print s
2012-12-08-14-55-44
The documentation contains a complete table of directives you can use to get different tokens.
What is the best way to format the datetime so that I can easily sort by datetime?
If you want to sort files according to datetimes names, you can consider that a biggest-to-lowest time specifier representation of a datetime (e.g.: YYYYMMDDhhmmss) preserves the same chronological and lexicographical order.
Related
This is my data :
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
I now want to convert from :
"2018-01-01" -> "Jan-01-2018" ["Monthname-day-year"] format
How to i do this ?
Is it possible to initialize this in the way we want to convert ?
Can i use something like:
for i in dates:
i = i.replace(i.month,i.strftime("%b"))
You can try this:
from datetime import datetime
dates = np.arange("2018-01-01", "2021-12-31", dtype="datetime64[D]")
result_dates = []
for date in dates.astype(datetime):
result_dates.append(date.strftime("%b-%d-%Y"))
But you will need to convert result dates as shown in the code
I feel compelled to elaborate on Silvio Mayolo's very relevant but ostensibly ignored comment above. Python stores a timestamp as structure (see How does Python store datetime internally? for more information) Hence, the DateTime does not as such have a 'format'. A format only becomes necessary when you want to print the date because you must first convert the timestamp to a string. Thus, you do NOT need to initialise any format. You only need to declare a format when the time comes to print the timestamp.
While you CAN store the date as a string in your dataframe index in a specific format, you CANNOT perform time related functions on it without first converting the string back to a time variable. ie current_time.hour will return an integer with the current hour if current_time is a datetime variable but will crash if it is a string formatted as a timestamp (such as "2023-01-15 17:23").
This is important to understand, because eventually you will need to manipulate the variables and need to understand whether you are working with a time or a string.
My website's articles are written using .md files, to get the created and modified times of these files I use the os.path.getctime() and os.path.getmtime() methods.
The output of these methods look like this:
1553541590.723329
1553541590.723329
While HTML requires this format:
2001-09-17T05:59:00+01:00
2013-09-16T19:08:47+01:00
I have two questions regarding this matter:
What's are the names of these two time formats?
How do I change the output of those methods to look like the required HTML format?
Thanks.
1) The os.path documentation indicates that both os.path.getctime() and os.path.getmtime() return a float indicating seconds since epoch. That seems consistent with the numbers you are getting.
2) The easiest thing to do would be to convert to an object to represent a date and then provide your desired format. Here, I used datetime with strftime() to output a string of desired format.
import datetime
>>>> datetime.datetime.fromtimestamp(1553541590.723329)
datetime.datetime(2019, 3, 25, 12, 19, 50, 723329)
>>>> datetime.datetime.fromtimestamp(1553541590.723329).strftime('%Y-%m-%dT%H:%M:%S')
'2019-03-25T12:19:50'
You may find it easiest to just add the time zone string on the end since adding a timezone to a datetime object is a little involved. If you do want to go through with it, you need to create a tzinfo object and use it to update the datetime object using datetime.astimezone(tz). Here's a pretty good resource for adding a timezone to a datetime object.
So this question is more of best way to handle this sort of input in python. Here is an example of input date 2018-12-31 23:59:59.999999. The millisecond part may or may not be part of input.
I am currently using this code to convert this to datetime
input_ts = datetime.datetime.strptime(input_str, '%Y-%m-%dT%H:%M:%S.%f')
But the problem in this case is that it will throw an exception if input string doesn't contain milliseconds part i.e., 2018-12-31 23:59:59
In Java, I could have approached this problem in two ways. (its a pseudo explanation, without taking into account of small boundary checks)
(preferred approach). Check the input string length. if its less than 19 then it is missing milliseconds. Append .000000 to it.
(not preferred). Let the main code parse the string, if it throws an exception, then parse it with new time format i.e., %Y-%m-%dT%H:%M:%S
The third approach could be just strip off milliseconds.
I am not sure if python has anything built-in to handle these kind of situations. Any suggestions?
You could use python-dateutil library, it is smart enough to parse most of the basic date formats.
import dateutil.parser
dateutil.parser.parse('2018-12-31 23:59:59.999999')
dateutil.parser.parse('2018-12-31 23:59:59')
In case you don't want to install any external libraries, you could iterate over list of different formats as proposed in this answer.
from datetime import datetime # import datetime class from datetime package
dt = datetime.now() # get current time
dt1 = dt1.strftime('%Y-%m-%d %H:%M:%S') # converting time to string
dt3 = dt2.strptime('2018/5/20','%Y/%m/%d') # converting a string to specified time
There is a widely-quoted method in the following answer for specifying date format when using pandas to_csv:
How to specify date format when using pandas.to_csv?
This answer describes:
date_format='%Y%m%d'
But I have a different requirement for which can find no information.
How can I specify a different date format for the actual year/month/day tokens?
...date_format='%Y%m%d'... translates to 2014/10/2 as of today's date. I can use this information to juggle the same data around -- eg 10/2/2014, but I cannot change the format itself.
I would like to output 02-Oct_2014. I tried '%dd%mmm%yyyy' but the extra letters are just added as extra letters -- no change in the date format.
Is it possible to specify formats other than permutations of '%Y%m%d'?
Pandas uses strftime, so use the format codes it specifies.
For 02-Oct_2014 it looks like you want %d-%b_%Y
>>> df = pd.DataFrame(list(range(5)), index=pd.date_range('10/1/14', periods=5))
>>> print(df.to_csv(date_format='%d-%b_%Y'))
,0
01-Oct_2014,0
02-Oct_2014,1
03-Oct_2014,2
04-Oct_2014,3
05-Oct_2014,4
The format you want is '%d-%b_%Y'. How did I figure this out? I looked at man strftime because that's what is being used under the hood (or an emulation of it). I searched the docs for "month" and found this:
%b is replaced by national representation of the abbreviated month name.
It's also shown in the Python docs here: https://docs.python.org/2/library/time.html#time.strftime
And finally, you can test many such formats directly on the *nix command line like so:
date +%d-%b_%Y
I would like a simple way to find and reformat text of the format 'DD/MM/YYYY' into 'YYYY/MM/DD' to be compatible with MySQL TIMESTAMPs, in a list of text items that may or may not contain a date atall, under python. (I'm thinking RegEx?)
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
Great thing about standards is that there are so many to choose from....
You can read the string into a datetime object and then output it back as a string using a different format. For e.g.
>>> from datetime import datetime
>>> datetime.strptime("31/12/2009", "%d/%m/%Y").strftime("%Y/%m/%d")
'2009/12/31'
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
If the input format is inconsistent, can vary, then you are better off with dateutil.
>>> from dateutil.parser import parse
>>> parse("31/12/2009").strftime("%Y/%m/%d")
'2009/12/31'
Dateutil can handle a lot of input formats automatically. To operate on a list you can map the a wrapper over the parse function over the list and convert the values appropriately.
If you're using the MySQLdb (also known as "mysql-python") module, for any datetime or timestamp field you can provide a datetime type instead of a string. This is the type that is returned, also and is the preferred way to provide the value.
For Python 2.5 and above, you can do:
from datetime import datetime
value = datetime.strptime(somestring, "%d/%m/%Y")
For older versions of python, it's a bit more verbose, but not really a big issue.
import time
from datetime import datetime
timetuple = time.strptime(somestring, "%d/%m/%Y")
value = datetime(*timetuple[:6])
The various format-strings are taken directly from what's accepted by your C library. Look up man strptime on unix to find other acceptable format values. Not all of the time formats are portable, but most of the basic ones are.
Note datetime values can contain timezones. I do not believe MySQL knows exactly what to do with these, though. The datetimes I make above are usually considered as "naive" datetimes. If timezones are important, consider something like the pytz library.