Develop Reference

How to convert string to datetime object then create If statement when time is on the hour

I'm reading (actually scraping) an RSS feed of NOAA buoy data. One of the data is the date and time the data was collected at the buoy. So the string I am extrapolating is in this format: January 10, 2023 9:48 am
But if the time is on the hour, say 'January 10, 2023 10:00 am', the feed produces an extra variable that throws my output off.
Thus, my code would check to see if the feed is on the hour and change the variables, like so:
air_temp = rows[7]
water_temp = rows[8]
if [time minutes = '00']:
air_temp = rows[7]
water_temp = rows[8]
I'm assuming I would need to change the time string to datetime in order to write the If statement? (Otherwise, I'm happy with the string format as is for my output.)

to check if the time is on the hour, you could do this to convert it to a datetime, then check if the minutes are zero:
from datetime import datetime as dt
date = 'January 10, 2023 10:00 am'
datetime = dt.strptime(s, '%B %d, %Y %H:%M %p')
minute = datetime.minute
if minute == 0:
do whatever
you could also do this which requires less thinking about the date format :)
import pandas as pd
pd.to_datetime(date)

You can use datetime for the same and convert your input time format to python's datetime data type.
import datetime
input_str = 'January 10, 2023 9:48 am'
input_time_format = '%B %d, %Y %I:%M %p'
datetime_str = datetime.datetime.strptime(input_str, input_time_format)
print(datetime_str.minute)
if datetime_str.minute == 0:
pass
You can check more details about input format here: https://docs.python.org/3/library/datetime.html

Converting TimeFormat 'DD/MM/YY HH:MM AM' from String in Python to Datetime 'DD/MM/YY HH:MM:SS'

sorry am new here and a total Python Rookie.
I pull Data with Python from Jira and put it into DataFrame. There I have a datetime as string in following format: DD/MM/YY HH:MM AM (or PM). Now I want to convert to Datetime to make it comparable with other datetimes to DD/MM/YY HH:MM:SS. I wanted to use datetime.strptime but it always fails. Any ideas? Thanks in advance!

Use a custom format: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
from datetime import datetime
datetime.strptime("05/11/22 07:40 AM", "%d/%m/%y %I:%M %p")
# datetime.datetime(2022, 11, 5, 7, 40)
datetime.strptime("05/11/22 07:40 PM", "%d/%m/%y %I:%M %p")
# datetime.datetime(2022, 11, 5, 19, 40)

You mention "DataFrame", so I guess you are using pandas. In that case, you should not have to do much work yourself. Just point pandas' read_* function to the column(s) that contain datetime-like strings. With argument dayfirst=True you make sure that dates are inferred as DD/MM/YY, not MM/DD/YY:
import pandas as pd
from io import StringIO
df = pd.read_csv(StringIO("""\
when,value
11/05/22 07:40 AM,42
10/05/22 09:42 AM,8
09/05/22 08:41 AM,15
"""), parse_dates=['when'], dayfirst=True)
This yields the following DataFrame, df.dtypes containing a nice datetime64[ns] for column "when".
when value
0 2022-05-11 07:40:00 42
1 2022-05-10 09:42:00 8
2 2022-05-09 08:41:00 15
If you already have the DataFrame with a string column, you can convert it after the fact using
df["when"] = pd.to_datetime(df["when"], dayfirst=True)

time data not matching format

time data '07/10/2019:08:00:00 PM' does not match format '%m/%d/%Y:%H:%M:%S.%f'
I am not sure what is wrong. Here is the code that I have been using:
import datetime as dt
df['DATE'] = df['DATE'].apply(lambda x: dt.datetime.strptime(x,'%m/%d/%Y:%H:%M:%S.%f'))
Here's a sample of the column:
Transaction_date
07/10/2019:08:00:00 PM
07/23/2019:08:00:00 PM
3/15/2021
8/15/2021
8/26/2021

Your format is incorrect. Try:
df["DATE"] = pd.to_datetime(df["DATE"], format="%m/%d/%Y:%I:%M:%S %p")
You should be using %I to specify a 12-hour format and %p for AM/PM.
Separately, just use pd.to_datetime instead of importing datetime and using apply.
Example:
>>> pd.to_datetime('07/10/2019:08:00:00 PM', format="%m/%d/%Y:%I:%M:%S %p")
Timestamp('2019-07-10 20:00:00')
Edit:
To handle multiple formats, you can use pd.to_datetime with fillna:
df["DATE"] = pd.to_datetime(df["DATE"], format="%m/%d/%Y:%I:%M:%S %p", errors="coerce").fillna(pd.to_datetime(df["DATE"], format="%m/%d/%Y", errors="coerce"))

Convert an unusual/custom time format to datetime object

I have an unusual datetime format in my dataset, which I need to convert to usable datetime object.
An example looks like: '1/3/2018 1:29:35 PM(UTC+0)'
I have tried to parse it with:
from dateutil.parser import parse
parse('1/3/2018 1:29:35 PM(UTC+0)')
but it doesn't recognize the format.
My current workaround is to parse the datetime column (the data is in pandas dataframe) using regex into two columns, like so:
and then depending on the value of the 'utc' column apply custom convert_to_eastern function.
I wonder if there is an easier way to accomplish it using datetime.datetime.strptime() ?
Following didn't work:
import datetime as dt
my_time='1/3/2018 1:29:35 PM(UTC+0)'
dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%z)')
Addition:
This is not a question: "How to convert UTC timezone into local timezone" My dataset has rows with UTC as well as Eastern time zone rows. The problem I have is that the format is not an ISO format, but some human-readable custom format.

Question: an easier way to accomplish it using datetime.datetime.strptime()
Split the datestring into parts: utc:[('1/3/2018 1:29:35 PM', '(UTC+0)', 'UTC', '+', '0')]
Rebuild the datestring, fixing the hour part padding with 0 to 2 digits.
I assume, there are no minutes in the UTC part, therefore defaults to 00.
If the datestring has more then 2 UTC digits, returns the unchanged datestring.
Note: The strptime format have to be %Z%z!
Documentation: strftime-and-strptime-behavior
from datetime import datetime
import re
def fix_UTC(s):
utc = re.findall(r'(.+?)(\((\w{3})(\+|\-)(\d{1,2})\))', s)
if utc:
utc = utc[0]
return '{}({}{}{})'.format(utc[0], utc[2], utc[3], '{:02}00'.format(int(utc[4])))
else:
return s
my_time = fix_UTC('1/3/2018 1:29:35 PM(UTC+0)')
date = datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')
print("{} {}".format(date, date.tzinfo))
Output:
2018-01-03 13:29:35+01:00 UTC
Tested with Python: 3.4.2

The problem is with '+0' for your timezone 'UTC+0'. datetime only takes utc offset in the form of HHMM. Possible workaround:
import datetime as dt
my_time = '1/3/2018 1:29:35 PM(UTC+0)'
my_time=my_time.replace('+0','+0000')
dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')

It should be something like that:
import datetime as dt
my_time='1/3/2018 1:29:35 PM(UTC+0000)'
tmp = dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')
print(tmp)
Big "Z" for timezone (UTC, GMT etc), small "z" for delta. Also you should add more zeros to delta.

Cannot convert dataframe column to 24-H format datetime

I have a timestamp column in my dataframe which is originally a str type. Some sample values:
'6/13/2015 6:45:58 AM'
'6/13/2015 7:00:37 PM'
I use the following code to convert this values into datetime with 24H format using this code:
df['timestampx'] = pd.to_datetime(df['timestamp'], format='%m/%d/%Y %H:%M:%S %p')
And, I obtain this result:
2015-06-13 06:45:58
2015-06-13 07:00:37
That means, the dates are NOT converted with 24H format and I am also loosing the AM/PM info. Any help?

You're reading it in as a 24 hour time, but really the current format isn't 24 hour time, it's 12 hour time. Read it in as 12 hour with the suffix (AM/PM), then you'll be OK to output in 24 hour time later if need be.
df = pd.DataFrame(['6/13/2015 6:45:58 AM','6/13/2015 7:00:37 PM'], columns = ['timestamp'])
df['timestampx'] = pd.to_datetime(df['timestamp'], format='%m/%d/%Y %I:%M:%S %p')
print df
timestamp timestampx
0 6/13/2015 6:45:58 AM 2015-06-13 06:45:58
1 6/13/2015 7:00:37 PM 2015-06-13 19:00:37