I am trying to format some dates with datetime, but for some reason it is ignoring my format call. I want day/month/Year format which is what the CSV file has the format is, but when I try this.
df = pd.read_csv('test.csv', parse_dates=['Date'],
date_parser=lambda x: pd.to_datetime(x, format='%d/%m/%Y'))
Result:
Why is it what I can only assume "defaulting" to %Y-%m-%d ???
This should work.
import datetime as dt
import pandas as pd
df = pd.read_csv('test.csv')
formatted_dates =[]
for old_date in df['Date']:
dt_obj = dt.datetime.strptime(old_date,'%d/%m/%Y')
new_date = """{}/{}/{}""".format(dt_obj.day,dt_obj.month,dt_obj.year)
formatted_dates.append(new_date)
df['Date'] = formatted_dates
Output:
18/1/2017
22/1/2017
31/1/2017
...
P.S. There's a bug with the parse_dates,date_parser in pd.read_csv which automatically changes the format to the YYYY-MM-DD.
Related
I'm trying to read a CSV file, where some columns have date or time values.
I started with this:
import pandas as pd
from datetime import datetime
timeparse = lambda x: datetime.strptime(x, '%H:%M:%S.%f')
lap_times = pd.read_csv(
'data/lap_times.csv',
parse_dates={'time_datetime': ['time']},
date_parser=timeparse
)
But sometimes the row of the column has a format %M:%S.%f and sometimes has %H:%M:%S.%f. So I got an error.
I thought about creating a function like this, but I can't see how I would pass an argument to the function to do the transformation for each row of the column passed as an argument.
def timeparse_1():
try:
return datetime.strptime(x, '%H:%M:%S.%f')
finally:
return datetime.strptime(x, '%M:%S.%f')
But I'm getting:
NameError: name 'x' is not defined
It would be easier if you post a sample of your CSV file, but something like this may work:
import pandas as pd
from datetime import datetime as dt
df = pd.DataFrame({'Time': ['12:34:56', '12:34:56.789']})
df.Time = df.Time.apply(lambda x: dt.strptime(x, '%H:%M:%S.%f') if len(x) > 8 else dt.strptime(x, '%H:%M:%S'))
Which will result in:
>>> df
0 1900-01-01 12:34:56.000
1 1900-01-01 12:34:56.789
Name: Time, dtype: datetime64[ns]
>>>
But there is a better way:
import pandas as pd
df = pd.DataFrame({'Time': ['12:34:56', '12:34:56.789']})
df.Time = df.Time.apply(pd.to_datetime)
Which results in the following:
>>> df
0 2022-11-20 12:34:56.000
1 2022-11-20 12:34:56.789
Name: Time, dtype: datetime64[ns]
>>>
Using the day of today to complete the datetime object.
How can I convert "2022-03-01 1:01:42 AM" to just 1:01:42?
I tried to strip just the time out and convert to datetime format, but it keeps adding the current date to the beginning. Otherwise, it doesn't properly convert to datetime format so I can plot it later. All I want is the time in datetime format.
def time():
df['Time'] = df['TIME'].apply(lambda x: x.split(' ')[1])
df['Time'] = pd.to_datetime(df.Time, format = '%H:%M:%S', errors='ignore').dt.time
How about using simple date format
from datetime import datetime
now = datetime.now()
print (now.strftime("%H:%M:%S"))
No need to split it. just simply:
import pandas as pd
df = pd.DataFrame({'ID':[1,2],
'TIME':['2022-03-01 1:01:42 AM', '2022-03-01 12:01:42 PM']})
df['Time'] = pd.to_datetime(df.TIME, errors='ignore').dt.time
Output:
df.iloc[0]['Time']
Out[1]: datetime.time(1, 1, 42)
I am trying to get the current date in 'YYYY-MM-DD' format, but it seems to be in a String format, and I want it to be in datetime64[ns] format.
So far, I have done this:
>>> import datetime
>>> from datetime import datetime
>>> todays_date = datetime.today().strftime('%Y-%m-%d')
>>> todays_date
'2022-06-06'
I have got the current date, but it is not in the format I need.
How do I convert it to datetime64[ns]?
Thanks.
If you are trying to convert your data to [ns] in pandas this will allow you to achieve the [ns] option
todays_date = datetime.datetime.today().strftime('%Y-%m-%d %X')
df = pd.DataFrame({
'Dates' : [todays_date]
})
df['Dates'] = pd.to_datetime(df['Dates'], infer_datetime_format=True)
df.dtypes
If for whatever reason you want just the date and not the time for your data you can do this to remove the hours/minutes/seconds and it will still be in [ns] format
todays_date = datetime.datetime.today().strftime('%Y-%m-%d 00:00:00')
df = pd.DataFrame({
'Dates' : [todays_date]
})
df['Dates'] = pd.to_datetime(df['Dates'], infer_datetime_format=True)
df.dtypes
pd.to_datetime("today").strftime("%Y/%m/%d")
I'm trying to convert the column 'DateTime' of my pandas df to DateTime format. The following doesn't work, how can I specify the format when my strings are of the form '12/2/14 0:01'?
df['DateTime'] = pd.to_datetime(df['DateTime'], format='%m/%d/%YY %H:%M').dt.time
Better yet, can I simply break it apart into two separate columns for date and time?
from datetime import datetime
df['DateTime'] = df['DateTime'].apply(lambda x: datetime.strptime(x, '%m/%d/%y %H:%M'))
I have a csv files with dates in the format displayed as dd-mmm-yy and i want to read in the format yyyy-mm-dd. parse dates option works but it not converting dates correct before 2000
Example: actual date is 01-Aug-1968. It is displayed as 01-Aug-68. Pandas date parase and correction=true reads the date as 01-Aug-2068.
Is there any option to read the date in pandas in the correct format for the dates before 2000.
from dateutil.relativedelta import relativedelta
import datetime
let's assume you have a csv like this:
mydates
18-Aug-68
13-Jul-45
12-Sep-00
20-Jun-10
15-Jul-60
Define your date format
d = lambda x: pd.datetime.strptime(x, '%d-%b-%y')
Put a constraint on them
dateparse = lambda x: d(x) if d(x) < datetime.datetime.now() else d(x) - relativedelta(years=100)
read your csv:
df = pd.read_csv("myfile.csv", parse_dates=['mydates'], date_parser=dateparse)
here is your result:
print df
mydates
0 1968-08-18
1 1945-07-13
2 2000-09-12
3 2010-06-20
4 1960-07-15
Voilà