I have a timestamp column in my dataframe which is originally a str type. Some sample values:
'6/13/2015 6:45:58 AM'
'6/13/2015 7:00:37 PM'
I use the following code to convert this values into datetime with 24H format using this code:
df['timestampx'] = pd.to_datetime(df['timestamp'], format='%m/%d/%Y %H:%M:%S %p')
And, I obtain this result:
2015-06-13 06:45:58
2015-06-13 07:00:37
That means, the dates are NOT converted with 24H format and I am also loosing the AM/PM info. Any help?
You're reading it in as a 24 hour time, but really the current format isn't 24 hour time, it's 12 hour time. Read it in as 12 hour with the suffix (AM/PM), then you'll be OK to output in 24 hour time later if need be.
df = pd.DataFrame(['6/13/2015 6:45:58 AM','6/13/2015 7:00:37 PM'], columns = ['timestamp'])
df['timestampx'] = pd.to_datetime(df['timestamp'], format='%m/%d/%Y %I:%M:%S %p')
print df
timestamp timestampx
0 6/13/2015 6:45:58 AM 2015-06-13 06:45:58
1 6/13/2015 7:00:37 PM 2015-06-13 19:00:37
Related
Currently I'm capturing dates from a csv file, but the date field can come in any format.
I want to transform this dates to only %Y-%m-%d date format. But with strptime does not work.
For example:
Csv Dates ----> Transformation
2020/06/23 06:00
---> 2020-06-23
23/04/2020 05:00
---> 2020-04-23
11/4/2020 10:00
---> 2020-04-11
2022/1/24 11:00
---> 2022-01-24
Code:
fecha_csv = row[7]
fecha_csv = datetime.strptime(fecha_csv, '%Y-%m-%d %H:%M:%S')
fecha_csv = fecha_csv.date()
This is assuming the format of dates that you have given in your example. to format further dates this may need to be modified depending on the date given.
the problem you are having possibly is that you aren't converting it into a proper datetime object so that you can change it to the date format that you would like.
you can change the date format with time into just the date with a couple of methods. one is to just do string manipulation if the date is always formatted the same like the examples shown or you could convert it to datetime objects like the following.
fecha_csv = row[7]
if len(fecha_csv.split('/')[0]) > 2: # year is first
datetime.strptime(fecha_csv, '%Y/%m/%d %H:%M').strftime("%Y-%m-%d")
else: # year is last
datetime.strptime(fecha_csv, '%d/%m/%Y %H:%M').strftime("%Y-%m-%d")
A problem with your current code is that it was formatted to read dates in as 2020-06-23 06:00:00 when it should only be formatted to read in as 2020/06/23 06:00
Similarly, you could use a date parser -
from dateutil.parser import parse
fecha_csv = row[7]
csv_date = parse(fetch_csv).date()
How can I get another column with the hour only from a column whose data type is an object whose format is this 9/5/21 18:00 ?
I have tried converting it to datetime first but I can't seem to get it right
df['starttime'] = pd.to_datetime(df['starttime'], format='%m-%d-%y %H:%M')
the error is
time data '9/5/21 18:00' does not match format '%m-%d-%y %H:%M' (match)
You use - instead of / as a separator of the date part:
df = pd.DataFrame({'starttime': ['9/5/21 18:00']})
df['starttime'] = pd.to_datetime(df['starttime'], format='%m/%d/%y %H:%M')
df['hour'] = df['starttime'].dt.hour
df['time'] = df['starttime'].dt.time
print(df)
# Output
starttime hour time
0 2021-09-05 18:00:00 18 18:00:00
I have a dataset that has a column of that date written in the following format. How can I convert them to timestamp?
date = 1/1/2016 1:00:00 AM
you may need to use datetime library:
import datetime
datestr = "1/1/2016 1:00:00 AM"
datetm =datetime.datetime.strptime(
datestr, '%d/%m/%Y %H:%M:%S %p')
datetime.datetime.timestamp(datetm)
output:
1451610000.0
More info on formats, etc.: docs
i have a column hour which has value like '2019091300',"2019091301" the last two digits are hour value i want to transform it '2019-091-13 00:00:00', '2019-09-13 01:00:00' etc. '20190913' could be transformable by DateTime formatting . but I am stuck processing the last hour part . any solution would be helpful
Add parameter format with %Y for YYYY, %m for MM, %d for DD and %H for HH in to_datetime:
df['date'] = pd.to_datetime(df['date'], format='%Y%m%d%H')
I cannot find the correct format for this datetime. I have tried several formats, %Y/%m/%d%I:%M:%S%p is the closest format I can find for the example below.
df['datetime'] = '2019-11-13 16:28:05.779'
df['datetime'] = pd.to_datetime(df['datetime'], format="%Y/%m/%d%I:%M:%S%p")
Result:
ValueError: time data '2019-11-13 16:28:05.779' does not match format '%Y/%m/%d%I:%M:%S%p' (match)
Before guessing yourself have pandas make the first guess
df['datetime'] = pd.to_datetime(df['datetime'], infer_datetime_format=True)
0 2019-11-13 16:28:05.779
Name: datetime, dtype: datetime64[ns]
You can solve this probably by using the parameter infer_datetime_format=True. Here's an example:
df = {}
df['datetime'] = '2019-11-13 16:28:05.779'
df['datetime'] = pd.to_datetime(df['datetime'], infer_datetime_format=True)
print(df['datetime'])
print(type(df['datetime'])
Output:
2019-11-13 16:28:05.779000
<class 'pandas._libs.tslibs.timestamps.Timestamp'>
Here is the pandas.to_datetime() call with the correct format string: pd.to_datetime(df['datetime'], format="%Y/%m/%d %H:%M:%S")
You were missing a space, %I is for 12-hour time (the example time you gave is 16:28, and %p is, to quote the docs, the Locale’s equivalent of either AM or PM.