Develop Reference

Convert an unusual/custom time format to datetime object

I have an unusual datetime format in my dataset, which I need to convert to usable datetime object.
An example looks like: '1/3/2018 1:29:35 PM(UTC+0)'
I have tried to parse it with:
from dateutil.parser import parse
parse('1/3/2018 1:29:35 PM(UTC+0)')
but it doesn't recognize the format.
My current workaround is to parse the datetime column (the data is in pandas dataframe) using regex into two columns, like so:
and then depending on the value of the 'utc' column apply custom convert_to_eastern function.
I wonder if there is an easier way to accomplish it using datetime.datetime.strptime() ?
Following didn't work:
import datetime as dt
my_time='1/3/2018 1:29:35 PM(UTC+0)'
dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%z)')
Addition:
This is not a question: "How to convert UTC timezone into local timezone" My dataset has rows with UTC as well as Eastern time zone rows. The problem I have is that the format is not an ISO format, but some human-readable custom format.

Question: an easier way to accomplish it using datetime.datetime.strptime()
Split the datestring into parts: utc:[('1/3/2018 1:29:35 PM', '(UTC+0)', 'UTC', '+', '0')]
Rebuild the datestring, fixing the hour part padding with 0 to 2 digits.
I assume, there are no minutes in the UTC part, therefore defaults to 00.
If the datestring has more then 2 UTC digits, returns the unchanged datestring.
Note: The strptime format have to be %Z%z!
Documentation: strftime-and-strptime-behavior
from datetime import datetime
import re
def fix_UTC(s):
utc = re.findall(r'(.+?)(\((\w{3})(\+|\-)(\d{1,2})\))', s)
if utc:
utc = utc[0]
return '{}({}{}{})'.format(utc[0], utc[2], utc[3], '{:02}00'.format(int(utc[4])))
else:
return s
my_time = fix_UTC('1/3/2018 1:29:35 PM(UTC+0)')
date = datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')
print("{} {}".format(date, date.tzinfo))
Output:
2018-01-03 13:29:35+01:00 UTC
Tested with Python: 3.4.2

The problem is with '+0' for your timezone 'UTC+0'. datetime only takes utc offset in the form of HHMM. Possible workaround:
import datetime as dt
my_time = '1/3/2018 1:29:35 PM(UTC+0)'
my_time=my_time.replace('+0','+0000')
dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')

It should be something like that:
import datetime as dt
my_time='1/3/2018 1:29:35 PM(UTC+0000)'
tmp = dt.datetime.strptime(my_time, '%m/%d/%Y %I:%M:%S %p(%Z%z)')
print(tmp)
Big "Z" for timezone (UTC, GMT etc), small "z" for delta. Also you should add more zeros to delta.

Accounting for AM PM in strptime not working

I am following another search thread here, but it is not working. Anytime I include %p to get AM/PM, I get the following error:
ValueError: time data '11:30' does not match format '%I:%M%p'
This is true if I have '%I:%M %p' or '%I:%M%p'. If I leave the %p off it works fine, but that defeats the purpose of getting AM PM
# my code
from datetime import datetime
date_string = '11:30'
format = '%I:%M %p'
my_date = datetime.strptime(date_string, format)
my_date.strftime(format)

You just need to provide same format of your time in strptime function. And then use AM/PM format ('%I:%M %p') in strftime function.
from datetime import datetime
def getTime(time_string):
time_object = datetime.strptime(time_string,'%H:%M') #Covert string to time object
return time_object.strftime('%I:%M %p') #Convert time object to AM/PM format
getTime('11:30')
Output: 11:30 am
getTime('13:30')
Output: 01:30 pm

Yes, cause you need to add if it's AM or PM to match the format.
from datetime import datetime
date_string = '11:30 AM'
format = '%I:%M %p'
my_date = datetime.strptime(date_string, format)
my_date.strftime(format)
%I matches hour, %M matches minutes, %p matches AM/PM

The format strings provided strptime (...) must match exactly.
If you do not know which format your time is in, you can try multiple ones:
from datetime import datetime
def getTime(text, formats = ['%I:%M %p','%I:%M']):
"""Tries different patterns to create a time from text.
First format with match wins.
As default the time is parsed with am/pm, as fallback without it."""
for pattern in formats:
try:
return datetime.strptime(text, pattern)
except:
pass # catch all errors
# nothing matched, return None
raise ValueError("No format {} matched '{}'".format(formats,text))
a_time = getTime("11:42") # produces an am time
b_time = getTime("11:42 pm") # produces a pm time
print(a_time.strftime("%I:%M %p"))
print(b_time.strftime("%I:%M %p"))
try:
c_time = getTime("does not work")
except ValueError as e:
print(type(e),e)
Output:
11:42 AM
11:42 PM
<class 'ValueError'> No format ['%I:%M %p', '%I:%M'] matched 'does not work'

Create a datetime from a string representation in a CSV file

I have a CSV file with recorded datetimes with a particular format:
%Y-%m-%d %H:%M:%s %Z
Example:
2017-02-11 14:11:42 PST
I am trying to format the datetime to a friendlier value to use later on.
However, I have been unable to create a datetime object with my code so far.
Here is my code:
for r in row:
purchase_date.append(
datetime.strptime(row['purchase-date'], "%Y/%m/%d %H:%M:%S %Z")
)
This is the error received:
ValueError: time data '2017-02-11 14:11:42 PST' does not match format %Y/%m/%d %H:%M:%S %Z'

Timezones are often rather wonky when trying to convert from a string. It is often best to deal with the timezone string yourself. Here is a bit of code which separates the timezone from the timestamp, and then converts them separately.
Code:
import datetime as dt
import pytz
my_timezones = dict(
PST='US/Pacific',
)
def convert_my_datetime_str(dt_str):
# split into time and timezone
timestamp, tz_str = dt_str.rsplit(' ', 1)
# convert the date string to datetime
time = dt.datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S")
# get a timezone name
tz = pytz.timezone(my_timezones[tz_str])
# return a timezone aware datetime
return tz.localize(time)
Test Code:
print(convert_my_datetime_str('2017-02-11 14:11:42 PST'))
Results;
2017-02-11 14:11:42-08:00

You should be able to just change the format to match your date strings. In the error, your date string has dashes instead of slashes, so make the format string match:
for r in row:
purchase_date.append(
datetime.strptime(row['purchase-date'], "%Y-%m-%d %H:%M:%S %Z")
)

strptime with timestamp and AM/PM

I am trying to convert from string to timestamp using:
from datetime import datetime
date_object = datetime.strptime('09-MAR-15 12.54.45.000000000 AM', '%d-%b-%y %I.%M.%S.%f %p')
I get:
ValueError:
time data '09-MAR-15 12.54.45.000000000 AM' does not match format
'%d-%b-%y %I.%M.%S.%f %p'

The below will work as long as the the part after the decimal point always ends in 000. :-) %f captures microseconds, while I guess your timestamp uses nanoseconds?
date_object = datetime.strptime('09-MAR-15 12.54.45.000000000 AM',
'%d-%b-%y %I.%M.%S.%f000 %p')
You might consider just chopping off those three digits. E.g.
date_object = datetime.strptime(
re.sub(r'\d{3}( .M)$', r'\1', '09-MAR-15 12.54.45.000000000 AM'),
'%d-%b-%y %I.%M.%S.%f %p')

Python/Pandas Datetime Conversion of Date in Format DD-MON-YY HH:MM:SS.NS PM

Running into some issues with datetime conversion.
import pandas as pd
mydate = '12-AUG-03 04.16.41.000000 PM'
mydateconv = pd.to_datetime(mydate)
print mydateconv
"2003-08-12 12:00:00"
Is there a reason the time is being reset to 12:00:00?
I've also tried other formatting derivations without success.
mydateconv = pd.to_datetime(mydate, format = '%d-%m-%y %I:%M:%S.%f %p')
Any recommendations?
The raw data I'm receiving has dates in the above format, so I'm looking for suggestions on a solution which addresses dates in this format, whether it's the use of stock function or determination that I'll need something a bit more custom because of the format.
Many thanks in advance for any thoughts.

Your format string needs to be: '%d-%b-%y %I.%M.%S.%f %p', see the docs:
In [35]:
pd.to_datetime('12-AUG-03 04.16.41.000000 PM', format = '%d-%b-%y %I.%M.%S.%f %p')
Out[35]:
Timestamp('2003-08-12 16:16:41')
You had several errors in your format string '%d-%m-%y %I:%M:%S.%f %p'.
Firstly your months are abbreviated so you should use b instead of m.
Secondly your time components had dot (.)separators not colon (:) separators.