Timestamp datetime64 to datetime in dataframe - python

I am confused with datetime64 and trying convert it to a normal time format.
I have a column with timestamp format: 2022.01.02D23:10:12.197164900.
Output expected is: 2022-01-02 23:10:12
I'm trying with:
df['executionTime'] = pd.to_datetime(df['executionTime'], format='%Y-%m-%d %H:%M:%S.%f', errors='coerce')

Try this:
df['executionTime'] = pd.to_datetime(df['executionTime'], format='%Y.%m.%dD%H:%M:%S.%f', errors='coerce')

Related

How to get the current date in 'YYYY-MM-DD' and 'datetime64[ns]' format?

I am trying to get the current date in 'YYYY-MM-DD' format, but it seems to be in a String format, and I want it to be in datetime64[ns] format.
So far, I have done this:
>>> import datetime
>>> from datetime import datetime
>>> todays_date = datetime.today().strftime('%Y-%m-%d')
>>> todays_date
'2022-06-06'
I have got the current date, but it is not in the format I need.
How do I convert it to datetime64[ns]?
Thanks.
If you are trying to convert your data to [ns] in pandas this will allow you to achieve the [ns] option
todays_date = datetime.datetime.today().strftime('%Y-%m-%d %X')
df = pd.DataFrame({
'Dates' : [todays_date]
})
df['Dates'] = pd.to_datetime(df['Dates'], infer_datetime_format=True)
df.dtypes
If for whatever reason you want just the date and not the time for your data you can do this to remove the hours/minutes/seconds and it will still be in [ns] format
todays_date = datetime.datetime.today().strftime('%Y-%m-%d 00:00:00')
df = pd.DataFrame({
'Dates' : [todays_date]
})
df['Dates'] = pd.to_datetime(df['Dates'], infer_datetime_format=True)
df.dtypes
pd.to_datetime("today").strftime("%Y/%m/%d")

change timestamp format pandas

Example dataHow to change the timestamp format which has the format of '2019-12-16-12-40-53' and I want it to convert to '2019-12-16 12:40:53'
I tried
df['timeStamp'] = df['timeStamp'].apply(lambda x:
dt.datetime.strptime(x,'%Y%b%d:%H:%M:%S'))
and I got the error
ValueError: time data '2019-12-16-12-40-53' does not match format '%Y%b%d:%H:%M:%S'
I am attaching an image of data that I am using as timestamp.
Use to_datetime and change format with %m for match months in numbers with - between parts of datetimes:
df['timeStamp'] = pd.to_datetime(df['timeStamp'], format='%Y-%m-%d-%H-%M-%S')

How to format datetime in a dataframe the way I want?

I cannot find the correct format for this datetime. I have tried several formats, %Y/%m/%d%I:%M:%S%p is the closest format I can find for the example below.
df['datetime'] = '2019-11-13 16:28:05.779'
df['datetime'] = pd.to_datetime(df['datetime'], format="%Y/%m/%d%I:%M:%S%p")
Result:
ValueError: time data '2019-11-13 16:28:05.779' does not match format '%Y/%m/%d%I:%M:%S%p' (match)
Before guessing yourself have pandas make the first guess
df['datetime'] = pd.to_datetime(df['datetime'], infer_datetime_format=True)
0 2019-11-13 16:28:05.779
Name: datetime, dtype: datetime64[ns]
You can solve this probably by using the parameter infer_datetime_format=True. Here's an example:
df = {}
df['datetime'] = '2019-11-13 16:28:05.779'
df['datetime'] = pd.to_datetime(df['datetime'], infer_datetime_format=True)
print(df['datetime'])
print(type(df['datetime'])
Output:
2019-11-13 16:28:05.779000
<class 'pandas._libs.tslibs.timestamps.Timestamp'>
Here is the pandas.to_datetime() call with the correct format string: pd.to_datetime(df['datetime'], format="%Y/%m/%d %H:%M:%S")
You were missing a space, %I is for 12-hour time (the example time you gave is 16:28, and %p is, to quote the docs, the Locale’s equivalent of either AM or PM.

pandas.to_datetime with different length date strings

I have a column of timestamps that I would like to convert to datetime in my pandas dataframe. The format of the dates is %Y-%m-%d-%H-%M-%S which pd.to_datetime does not recognize. I have manually entered the format as below:
df['TIME'] = pd.to_datetime(df['TIME'], format = '%Y-%m-%d-%H-%M-%S')
My problem is some of the times do not have seconds so they are shorter
(format = %Y-%m-%d-%H-%M).
How can I get all of these strings to datetimes?
I was thinking I could add zero seconds (-0) to the end of my shorter dates but I don't know how to do that.
try strftime and if you want the right format and if Pandas can't recognize your custom datetime format, you should provide it explicetly
from functools import partial
df1 = pd.DataFrame({'Date': ['2018-07-02-06-05-23','2018-07-02-06-05']})
newdatetime_fmt = partial(pd.to_datetime, format='%Y-%m-%d-%H-%M-%S')
df1['Clean_Date'] = (df1.Date.str.replace('-','').apply(lambda x: pd.to_datetime(x).strftime('%Y-%m-%d-%H-%M-%S'))
.apply(newdatetime_fmt))
print(df1,df1.dtypes)
output:
Date Clean_Date
0 2018-07-02-06-05-23 2018-07-02 06:05:23
1 2018-07-02-06-05 2018-07-02 06:05:00
Date object
Clean_Date datetime64[ns]

How to change format of data to '%Y%m%d' in Pandas?

I have a DF with first column showing as e.g. 2018-01-31 00:00:00.
I want to convert whole column (or during printing / saving to other variable) that date to 20180131 format.
NOT looking to do that during saving to a CSV file.
Tried this but it did not work:
df['mydate'] = pd.to_datetime(df['mydate'], format='%Y%m%d')
pd.to_datetime is used to convert your series to datetime:
s = pd.Series(['2018-01-31 00:00:00'])
s = pd.to_datetime(s)
print(s)
0 2018-01-31
dtype: datetime64[ns]
pd.Series.dt.strftime converts your datetime series to a string in your desired format:
s = s.dt.strftime('%Y%m%d')
print(s)
0 20180131
dtype: object
pd.to_datetime will convert a string to a date. You want to covert a date to a string
df['mydate'].dt.strftime('%Y%m%d')
Note that it's possible your date is already a string, but in the wrong format in which case you might have to convert it to a date first:
pd.to_datetime(df['mydate'], format='%Y-%m-%d %H:%M:%S').dt.strftime('%Y%m%d')
Convert the string column with 2018-01-31 00:00:00. to a datetime:
df['mydate'] = pd.to_datetime(df['mydate'])
#Get your preferred strings based on format:
df['mydate'].dt.strftime('%Y-%m-%d')
#Output: '2018-01-31'
df['mydate'].dt.strftime('%Y%m%d')
#output:'20180131'

Categories

Resources