I have a column of (created AT) in my DataFrame which has a timestamps like shown below:
Created AT
1) 2021-04-19T09:14:10.526Z
2) 2021-04-19T09:13:06.809Z
3) 2021-04-19T09:13:06.821Z
I want to extract the time only from above column etc . It should show like:
9:14:8 etc
How to extract this ?
If your date column is a string, you need to convert it to datetime and then take a substring of the time:
df = pd.DataFrame(data = {"Created At":["2021-04-19T09:14:10.526Z","2021-04-19T09:14:10.526Z"]})
df['Created At'] = pd.to_datetime(df['Created At'])
df['Created At'] = df['Created At'].dt.time.astype(str).str[:8]
df['time'] = pd.to_datetime(df['Created AT'])
print(df['time'].dt.time)
On the first line, convert the datetime to objects and write in a new column.
On the second, we get the time from datetime objects
I have a solution to your question. It can have multiple solutions but here I am giving some solution here using time, DateTime
you can get the string using
import time
import datetime
s = '2021-04-19T09:14:10.526Z'
t = s.split('T')[1].split('.')[0]
print(t)
and for getting time stamp of it do one more line
print(datetime.datetime.strptime(t,"%H:%M:%S"))
Convert to datetime and use strftime to format exactly as you like it.
data = ['2021-04-19T09:14:10.526Z',
'2021-04-19T09:13:06.809Z',
'2021-04-19T09:13:06.821Z']
df = pd.DataFrame(data=data, columns=['Created AT'])
df['Created AT'] = pd.to_datetime(df['Created AT']).dt.strftime('%H:%M:%S')
print(df)
Created AT
0 09:14:10
1 09:13:06
2 09:13:06
First convert the column to datetime format if not already in that format:
df['Created AT'] = pd.to_datetime(df['Created AT'])
Then, add the new column time with formatting by .dt.strftime() as follows (if you don't want the nano-second part):
df['time'] = df['Created AT'].dt.strftime('%H:%M:%S')
print(df)
Created AT time
0 2021-04-19 09:14:10.526000+00:00 09:14:10
1 2021-04-19 09:13:06.809000+00:00 09:13:06
2 2021-04-19 09:13:06.821000+00:00 09:13:06
Related
I have a numpy array (called dates) of dates (as strings) which I thought were in the form %Y-%m-%d %H:%M:%S. However, I get an error that I have dates such as 2021-05-11T00:00:00.0000000. Not sure where did that additional 'T' come and why is the time so precise.
I am trying to get rid of the time and only have the date.
My code is here:
dates = dataset.iloc[:,0].to_numpy()
newDates = []
for i in range(0,len(dates)):
newDates.append(datetime.strptime(dates[i], '%Y-%m-%dT%H:%M:%S.%f'))
newDates[i] = newDates[i].strftime('%Y-%m-%d')
dates = newDates
I get an error saying "ValueError: unconverted data remains: 0".
If I wrote instead
newDates.append(datetime.strptime(dates[i], '%Y-%m-%dT%H:%M:%S%f'))
I get an error "ValueError: unconverted data remains: .0000000".
In which format should the date be given?
If you have datetime in dataframe you can use pd.to_datetime and Series.dt.strftime for converting to desired format. pandas do all for you! (why convert values in dataframe to numpy.array.)
import pandas as pd
# example df
df = pd.DataFrame({'datetime': ['2021-05-11T00:00:00.0000000' ,
'2021-05-20T00:00:00.0000000' ,
'2021-06-24T00:00:00.0000000']})
df['datetime'] = pd.to_datetime(df['datetime']).dt.strftime('%Y-%m-%d')
print(df)
datetime
0 2021-05-11
1 2021-05-20
2 2021-06-24
Does this help? https://strftime.org/
The extra T can be seen after %Y-%m-%d
If you just want to get the date, just split the string like this.
date = date.split('T')[0]
this will first split the date string into to parts,
[2021-05-11','00:00:00.0000000]
then you can extract the first variable in the list by saving only index 0
then you are just left with
date = '2021-05-11'
dates = dataset.iloc[:,0].to_numpy()
newDates = []
for i in dates:
newDates.append(i.split('T')[0])
dates = newDates
assuming dates is a list
How to change format date from 12-Mar-2022 to , format='%d/%m/%Y' in python
so the problem is I read data from the google sheet where in the data contain multiple format, some of them is 12/03/2022 and some of them 12-Mar-2022.
I tried using this got error of couse because doesn't match for 12-Mar-2022
defectData_x['date'] = pd.to_datetime(defectData_x['date'], format='%d/%m/%Y')
Appreciate your help
defectData_x['date1'] = defectData_x['date'].dt.strftime('%d/%m/%Y')
don forget date1's dtype is not datetime but object
so it is better using date column and date1 column both before make final result
after final result, you can drop date column
add my example:
import pandas as pd
df = pd.DataFrame(["12/03/2022", "12-Mar-2022"], columns=["date"])
df["date1"] = pd.to_datetime(df["date"])
df['date2'] = df['date1'].dt.strftime('%d/%m/%Y')
I am calling some financial data from an API which is storing the time values as (I think) UTC (example below):
enter image description here
I cannot seem to convert the entire column into a useable date, I can do it for a single value using the following code so I know this works, but I have 1000's of rows with this problem and thought pandas would offer an easier way to update all the values.
from datetime import datetime
tx = int('1645804609719')/1000
print(datetime.utcfromtimestamp(tx).strftime('%Y-%m-%d %H:%M:%S'))
Any help would be greatly appreciated.
Simply use pandas.DataFrame.apply:
df['date'] = df.date.apply(lambda x: datetime.utcfromtimestamp(int(x)/1000).strftime('%Y-%m-%d %H:%M:%S'))
Another way to do it is by using pd.to_datetime as recommended by Panagiotos in the comments:
df['date'] = pd.to_datetime(df['date'],unit='ms')
You can use "to_numeric" to convert the column in integers, "div" to divide it by 1000 and finally a loop to iterate the dataframe column with datetime to get the format you want.
import pandas as pd
import datetime
df = pd.DataFrame({'date': ['1584199972000', '1645804609719'], 'values': [30,40]})
df['date'] = pd.to_numeric(df['date']).div(1000)
for i in range(len(df)):
df.iloc[i,0] = datetime.utcfromtimestamp(df.iloc[i,0]).strftime('%Y-%m-%d %H:%M:%S')
print(df)
Output:
date values
0 2020-03-14 15:32:52 30
1 2022-02-25 15:56:49 40
I have a dataframe with time column as string and I should convert it to a timestamp only with h:m:sec.ms . Here an example:
import pandas as pd
df=pd.DataFrame({'time': ['02:21:18.110']})
df.time= pd.to_datetime(df.time , format="%H:%M:%S.%f")
df # I get 1900-01-01 02:21:18.110
Without format flag, I get current day 2020-12-16. How can I get the stamp without year-month-day which seemingly always is included. Thanks!
If need processing values later by some datetimelike methods better is convert values to timedeltas by to_timedelta instead times:
df['time'] = pd.to_timedelta(df['time'])
print (df)
time
0 0 days 02:21:18.110000
You need this:
df=pd.DataFrame({'time': ['02:21:18.110']})
df['time'] = pd.to_datetime(df['time']).dt.time
In [1023]: df
Out[1023]:
time
0 02:21:18.110000
In a train data set, datetime column is an object . First row of this column : 2009-06-15 17:26:21 UTC . I tried splitting the data
train['Date'] = train['pickup_datetime'].str.slice(0,11)
train['Time'] = test['pickup_datetime'].str.slice(11,19)
So that I can split the Date and time as two variables and change them to datetime data type. Tried lot of methods but could not get the result.
train['Date']=pd.to_datetime(train['Date'], format='%Y-%b-%d')
Also tried spliting the date,time and UTC
train['DateTime'] = pd.to_datetime(train['DateTime'])
Please suggest a code for this. I am a begginer.
Thanks in advance
I would try the following
import pandas as pd
#create some random dates matching your formatting
df = pd.DataFrame({"date": ["2009-06-15 17:26:21 UTC", "2010-08-16 19:26:21 UTC"]})
#convert to datetime objects
df["date"] = pd.to_datetime(df["date"])
print(df["date"].dt.date) #returns the date part without tz information
print(df["date"].dt.time) #returns the time part
Output:
0 2009-06-15
1 2010-08-16
Name: date, dtype: object
0 17:26:21
1 19:26:21
Name: date, dtype: object
For further information feel free to consult the docs:
dt.date
dt.time
For your particular case:
#convert to datetime object
df['pickup_datetime']= pd.to_datetime(df['pickup_datetime'])
# seperate date and time
df['Date'] = df['pickup_datetime'].dt.date
df['Time'] = df['pickup_datetime'].dt.time