Develop Reference

How do I convert date from alphabetical to numeric format?

I want to convert date from 'Sep 17, 2021' format to '17.09.2021'. I made a function, but I can't apply it to the series. What am I doing wrong?
def to_normal_date(bad_date):
datetime.strptime(bad_date, '%b %d, %Y')
return s.strftime('%Y-%m-%d')
df['normal_date'] = df['date'].apply(to_normal_date)
I receive a ValueError when I'm trying to apply it to series. But it works fine with this:
to_normal_date('Sep 16, 2021')

Use pd.to_datetime to convert the "date" column to datetime format. Specifying errors="coerce" will convert dates that are not in the correct format to NaN values instead of raising errors.
Convert to the required format using .strftime with the .dt accessor.
df["normal_date"] = pd.to_datetime(df["date"], format="%b %d, %Y", errors="coerce").dt.strftime("%d.%m.%Y")
>>> df
date normal_date
0 Sep 17, 2021 17.09.2021
1 Oct 31, 2021 31.10.2021
2 Nov 19, 2021 19.11.2021
3 Dec 25, 2021 25.12.2021

Try:
pd.to_datetime(df['date'], format='%b %d, %Y').dt.strftime('%Y-%m-%d')
It should work provided all df['date'] entries match the date pattern of 'Sep 17, 2021'.

Converting a string month day, year to mm/dd/yyyy

I need to covert a string which contains date information (e.g., November 3, 2020) into date format (i.e., 11/03/2020).
I wrote
df['Date']=pd.to_datetime(df['Date']).map(lambda x: x.strftime('%m/%d/%y'))
where Date is
November 3, 2020
June 26, 2002
July 02, 2010
and many other dates, but I found the error ValueError: NaTType does not support strftime.

You can use pandas.Series.dt.strftime, which handles the NaT:
import pandas as pd
dates = ['November 3, 2020',
'June 26, 2002',
'July 02, 2010',
'NaT']
dates = pd.to_datetime(dates)
df = pd.DataFrame(dates, columns=['Date'])
df['Date'] = df['Date'].dt.strftime('%m/%d/%y')
Output:
Date
0 11/03/20
1 06/26/02
2 07/02/10
3 NaN

error reading date time from csv using pandas

I am using Pandas to read and process csv file. My csv file have date/time column that looks like:
11:59:50:322 02 10 2015 -0400 EDT
11:11:55:051 16 10 2015 -0400 EDT
00:38:37:106 02 11 2015 -0500 EST
04:15:51:600 14 11 2015 -0500 EST
04:15:51:600 14 11 2015 -0500 EST
13:43:28:540 28 11 2015 -0500 EST
09:24:12:723 14 12 2015 -0500 EST
13:28:12:346 28 12 2015 -0500 EST
How can I read this using python/pandas, so far what I have is this:
pd.to_datetime(pd.Series(df['senseStartTime']),format='%H:%M:%S:%f %d %m %Y %z %Z')
But this is not working, though previously I was able to use the same code for another format (with a different format specifier). Any suggestions?

The issue you're having is likely because versions of Python before 3.2 (I think?) had a lot of trouble with time zones, so your format string might be screwing up on the %z and %Z parts. For example, in Python 2.7:
In [187]: import datetime
In [188]: datetime.datetime.strptime('11:59:50:322 02 10 2015 -0400 EDT', '%H:%M:%S:%f %d %m %Y %z %Z')
ValueError: 'z' is a bad directive in format '%H:%M:%S:%f %d %m %Y %z %Z'
You're using pd.to_datetime instead of datetime.datetime.strptime but the underlying issues are the same, you can refer to this thread for help. What I would suggest is instead of using pd.to_datetime, do something like
In [191]: import dateutil
In [192]: dateutil.parser.parse('11:59:50.322 02 10 2015 -0400')
Out[192]: datetime.datetime(2015, 2, 10, 11, 59, 50, 322000, tzinfo=tzoffset(None, -14400))
It should be pretty simple to chop off the timezone at the end (which is redundant since you have the offset), and change the ":" to "." between the seconds and microseconds.

Since datetime.timezone has become available in Python 3.2, you can use %z with .strptime() (see docs). Starting with:
dateparse = lambda x: pd.datetime.strptime(x, '%H:%M:%S:%f %d %m %Y %z %Z')
df = pd.read_csv(path, parse_dates=['time_col'], date_parser=dateparse)
to get:
time_col
0 2015-10-02 11:59:50.322000-04:00
1 2015-10-16 11:11:55.051000-04:00
2 2015-11-02 00:38:37.106000-05:00
3 2015-11-14 04:15:51.600000-05:00
4 2015-11-14 04:15:51.600000-05:00
5 2015-11-28 13:43:28.540000-05:00
6 2015-12-14 09:24:12.723000-05:00
7 2015-12-28 13:28:12.346000-05:00

how to convert a string type to date format

My source data has a column including the date information but it is a string type.
Typical lines are like this:
04 13, 2013
07 1, 2012
I am trying to convert to a date format, so I used panda's to_datetime function:
df['ReviewDate_formated'] = pd.to_datetime(df['ReviewDate'],format='%mm%d, %yyyy')
But I got this error message:
ValueError: time data '04 13, 2013' does not match format '%mm%d, %yyyy' (match)
My questions are:
How do I convert to a date format?
I also want to extract to Month and Year and Day columns because I need to do some month over month comparison? But the problem here is the length of the string varies.

Your format string is incorrect, you want '%m %d, %Y', there is a reference that shows what the valid format identifiers are:
In [30]:
import io
import pandas as pd
t="""ReviewDate
04 13, 2013
07 1, 2012"""
df = pd.read_csv(io.StringIO(t), sep=';')
df
Out[30]:
ReviewDate
0 04 13, 2013
1 07 1, 2012
In [31]:
pd.to_datetime(df['ReviewDate'], format='%m %d, %Y')
Out[31]:
0 2013-04-13
1 2012-07-01
Name: ReviewDate, dtype: datetime64[ns]
To answer the second part, once the dtype is a datetime64 then you can call the vectorised dt accessor methods to get just the day, month, and year portions:
In [33]:
df['Date'] = pd.to_datetime(df['ReviewDate'], format='%m %d, %Y')
df['day'],df['month'],df['year'] = df['Date'].dt.day, df['Date'].dt.month, df['Date'].dt.year
df
Out[33]:
ReviewDate Date day month year
0 04 13, 2013 2013-04-13 13 4 2013
1 07 1, 2012 2012-07-01 1 7 2012

How to get datetime object from string "DD Month YYY" where month is in full name ex. January

I've a string, for ex '03 July 2012' and i want a datetime object from this. I'm looking for the most optimum way to do it.

Did you read the documentation?
>>> import datetime
>>> datetime.datetime.strptime('03 July 2012', '%d %B %Y')
datetime.datetime(2012, 7, 3, 0, 0)
%d stays for day (03), %B for full month day (July) and %Y for year (2012).