I have a time string like this: 2013-08-22 16:56:19 Etc/GMT
I need to parse it to a datetime object. I'm getting hung up on the posix-style time zone, which I can't get Python to natively grok.
Here are a couple of attempts and their failures. I'm including the timezone-stripped version first to show the parsing is otherwise correct.
datetime.strptime
>>> datetime.datetime.strptime("2013-08-22 16:56:19 UTC", "%Y-%m-%d %H:%M:%S %Z")
datetime.datetime(2013, 8, 22, 16, 56, 19)
>>> datetime.datetime.strptime("2013-08-22 16:56:19 Etc/GMT", "%Y-%m-%d %H:%M:%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '2013-08-22 16:56:19 Etc/GMT' does not match format '%Y-%m-%d %H:%M:%S %Z'
time.strptime
>>> time.strptime("2013-08-22 16:56:19 UTC", "%Y-%m-%d %H:%M:%S %Z")
time.struct_time(tm_year=2013, tm_mon=8, tm_mday=22, tm_hour=16, tm_min=56, tm_sec=19, tm_wday=3, tm_yday=234, tm_isdst=0)
>>> time.strptime("2013-08-22 16:56:19 Etc/GMT", "%Y-%m-%d %H:%M:%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '2013-08-22 16:56:19 Etc/GMT' does not match format '%Y-%m-%d %H:%M:%S %Z'
dateutil.parser
>>> dateutil.parser.parse("2013-08-22 16:56:19")
datetime.datetime(2013, 8, 22, 16, 56, 19)
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
possible solutions/avenues tried
tzinfos
It seems like dateutil's tzinfos argument should be perfect for this, but it doesn't work either... or I'm misreading the trace and doing something wrong. (I used this as an example)
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT", tzinfos={ 'Etc/GMT': pytz.timezone('UTC') })
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
tzinfos, round two
It seems like this stackoverflow answer is getting towards the usage of tzinfos that I might need. I tried the simplified version of the above (where value=offset seconds). Still fails.
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT", tzinfos={ 'Etc/GMT': 0 })
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
do it the wrong way
I could always use something like regex or string matching to find and change this timezone, but it feels wrong.
red herrings
The common problem with the posix ETC/xxx timezones is that they have reversed signs. This is a UTC ("no offset") posix time zone, and many of the questions I've found that deal with "etc" have to do with this reversed offset.
How about this:
parse date string without the timezone part into datetime object via strptime()
parse timezone string into pytz timezone
update tzinfo on the datetime object via replace()
from datetime import datetime
import pytz
date_string = "2013-08-22 16:56:19"
tz_string = "Etc/GMT"
dt = datetime.strptime(date_string, "%Y-%m-%d %H:%M:%S")
dt = dt.replace(tzinfo=pytz.timezone('Etc/GMT'))
print dt
prints:
2013-08-22 16:56:19+00:00
This actually correctly understands and reverses signs in the POSIX timezone format, for example:
dt = dt.replace(tzinfo=pytz.timezone('Etc/GMT-1'))
print dt # prints 2013-08-22 16:56:19+01:00
Related
There is a source column of values from Excel that I would like to convert to dates (python).
df['DATE_'] = pd.to_datetime(df['DATE_'], format='%Y%m%d.0')
Source column from Excel:
'2010-06-16 00:00:00'
Unfortunately, system generates the following error.
Traceback (most recent call last):
File "C:/Users/103925alf1/PycharmProjects/p03/venv/Include/p03.py", line 5, in <module>
df['DATE_'] = pd.to_datetime(df['DATE_'], format='%Y%m%d.0')
File "C:\Users\103925alf1\PycharmProjects\p03\venv\lib\site-packages\pandas\core\tools\datetimes.py", line 728, in to_datetime
values = convert_listlike(arg._values, format)
File "C:\Users\103925alf1\PycharmProjects\p03\venv\lib\site-packages\pandas\core\tools\datetimes.py", line 435, in _convert_listlike_datetimes
raise e
File "C:\Users\103925alf1\PycharmProjects\p03\venv\lib\site-packages\pandas\core\tools\datetimes.py", line 400, in _convert_listlike_datetimes
arg, format, exact=exact, errors=errors
File "pandas\_libs\tslibs\strptime.pyx", line 142, in pandas._libs.tslibs.strptime.array_strptime
ValueError: time data '2010-06-16 00:00:00' does not match format '%Y%m%d.0' (match)
Process finished with exit code 1
I am trying to check if the string entered by the user contains date using python.
something like this.
user_input = ("Enter date")
if user_input==#type of date(yyyy-mm-dd):
print(okay)
else:
print("failed")
The dateutil library is your best friend when it comes to parsing dates:
>>> from dateutil.parser import *
>>> parse('2003-12-23')
datetime.datetime(2003, 12, 23, 0, 0)
>>> parse('2003-12-32')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1356, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 653, in parse
ret = self._build_naive(res, default)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1227, in _build_naive
naive = default.replace(**repl)
ValueError: day is out of range for month
>>>
If you want to be more restrictive, ie. only accept dates with the YYYY-MM-DD format, then checking with a regex first might be your thing:
def is_date(ds):
if re.match(r'\d{4}-\d{2}-\d{2}', ds):
return bool(parse(ds))
return False
>>> is_date('2003-12-23')
True
>>> is_date('2003-12-32')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in is_date
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1356, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 653, in parse
ret = self._build_naive(res, default)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1227, in _build_naive
naive = default.replace(**repl)
ValueError: day is out of range for month
If you don't want to be quite as restrictive, but accept all iso-formatted dates, then:
from dateutil.parser import isoparse
>>> isoparse('2003-12-23')
datetime.datetime(2003, 12, 23, 0, 0)
>>> isoparse('20031223')
datetime.datetime(2003, 12, 23, 0, 0)
This is what you want:
>>> import datetime
>>> def validate(date_text):
try:
datetime.datetime.strptime(date_text, '%Y-%m-%d')
except ValueError:
raise ValueError("Incorrect data format, should be YYYY-MM-DD")
>>> validate('2003-12-23')
>>> validate('2003-12-32')
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
validate('2003-12-32')
File "<pyshell#18>", line 5, in validate
raise ValueError("Incorrect data format, should be YYYY-MM-DD")
ValueError: Incorrect data format, should be YYYY-MM-DD
I'm trying to dynamically convert a string seconds like below to a string date.
'1545239561 +0100'
The problem is the timezone inserted at the end, and I can't find any python time object method using the good format to retrieve the date from this string.
My tries :
>>>seconds = '1545239561 +0100'
>>>time.strftime('%y%m%d-%H%M%S-%f', datetime.datetime.fromtimestamp(seconds)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type str)
>>>time.strptime(seconds)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 559, in _strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '1545239561 +0100' does not match format '%a %b %d %H:%M:%S %Y'
>>>time.strptime(seconds, "%S +%Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 559, in _strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '1545239561 +0100' does not match format '%S +%Z'
I would try to process both those value separately and merge them into single datetime:
>>>from datetime import datetime
>>>s = '1545239561 +0100'
>>>seconds, offset = s.split()
>>>datetime.fromtimestamp(int(seconds)).replace(tzinfo=datetime.strptime(offset, "%z").tzinfo)
datetime.datetime(2018, 12, 19, 17, 12, 41, tzinfo=datetime.timezone(datetime.timedelta(0, 3600)))
Yes #mfrackwiak...
I did this
>>> epoch = "1545239561 +0100"
>>> seconds, offset = epoch.split()
>>> datetime.fromtimestamp(int(seconds)).replace(tzinfo=datetime.strptime(offset, "%z").tzinfo).strftime('%Y-%m-%d %H:%M:%S-%Z')
'2018-12-19 18:12:41-UTC+01:00'
>>>
You can try the below example :
from datetime import datetime,timedelta
# split the timevalue and offset value
ss = '1545239561 +0100'.split()
format = "%A, %B %d, %Y %I:%M:%S"
# calculate the hour and min in the offset
hour = int(ss[1][0:2])
min = int(ss[1][2:])
# calculate the time from the sec and convert it to datetime object
time_from_sec = datetime.strptime(datetime.fromtimestamp(int(ss[0])).strftime(
format), format)
# add the offset delta value to the time calculated
time_with_delta_added = time_from_sec + timedelta(hours=hour,minutes=min)
print(time_with_delta_added)
Output :
2018-12-19 12:22:41
I'm trying to learn the functions of Python but there is one thing I came across which I cannot figure out.
calculated_time = '2014.03.08 11:43:12'
>>> calculated_time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
>>> print calculated_time
2017-09-22 15:59:34
Now when I run:
cmpDate = datetime.strptime(calculated_time, '%Y.%m.%d %H:%M:%S')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data '2017-09-22 16:35:12' does not match format'%Y.%m.%d %H:%M:%S'
I can't understand why, if I directly pass this date then it is running but when I pass it after storing in a variable then I've got an error.
They are not the same formats :
'%Y.%m.%d %H:%M:%S'
'%Y-%m-%d %H:%M:%S'
Notice the different separators between year, month and day?
You need to use a consistent datetime format between strftime and strptime.
As an example :
>>> from datetime import datetime
>>> datetime.now().strftime("%Y-%m-%d %H:%M:%S")
'2017-09-22 15:06:51'
>>> datetime.strptime(datetime.now().strftime("%Y-%m-%d %H:%M:%S"), "%Y-%m-%d %H:%M:%S")
datetime.datetime(2017, 9, 22, 15, 3, 52)
>>> datetime.strptime(datetime.now().strftime("%Y-%m-%d %H:%M:%S"), "%Y.%m.%d %H:%M:%S")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/media/steam/anaconda3/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/media/steam/anaconda3/lib/python3.6/_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '2017-09-22 15:03:57' does not match format '%Y.%m.%d %H:%M:%S'
>>> datetime.now().strftime("%Y.%m.%d %H:%M:%S")
'2017.09.22 15:06:55'
>>> datetime.strptime(datetime.now().strftime("%Y.%m.%d %H:%M:%S"), "%Y.%m.%d %H:%M:%S")
datetime.datetime(2017, 9, 22, 15, 4, 3)
because you specified a format of '%Y.%m.%d %H:%M:%S' but you put '-' between the date elements instead of "."
The Problem
This code
#!/usr/bin/env python
import pynotify
import time
import datetime
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
produces
5/1/12 1:15 PM
Traceback (most recent call last):
File "tmp.py", line 9, in <module>
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: PM
Removing the import pynotify,
#!/usr/bin/env python
import time
import datetime
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
Removes the error.
5/1/12 1:15 PM
WHY?!!?!
Python Version
Python 2.7.2+ (default, Oct 4 2011, 20:06:09)
[GCC 4.6.1] on linux2
pynotify.file
I added print calls for pynotify.__file__ and datetime.__file__
/usr/lib/python2.7/lib-dynload/datetime.so
/usr/lib/python2.7/dist-packages/gtk-2.0/pynotify/__init__.pyc
5/1/12 1:15 PM
Traceback (most recent call last):
File "a.py", line 11, in <module>
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: PM
PDB
5/1/12 1:15 PM
> /usr/lib/python2.7/_strptime.py(324)_strptime()
-> found = format_regex.match(data_string)
(Pdb) format
'%d/%m/%y %H:%M %p'
(Pdb) continue
> /usr/lib/python2.7/_strptime.py(329)_strptime()
-> if len(data_string) != found.end():
(Pdb) continue
> /usr/lib/python2.7/_strptime.py(331)_strptime()
-> raise ValueError("unconverted data remains: %s" %
(Pdb) len(data_string)
14
(Pdb) found.end()
12
(Pdb) found.group(0)
'5/1/12 1:15 '
It would appear that '%d/%m/%y %H:%M %p' isn't capturing ALL of '5/1/12 1:15 PM'
That's a fun problem. I'd wager what's happening is that pynotify is changing your locale settings, which is breaking strptime's interpretation of your timestamp string.
Here's your code with a couple of debugging print statements that will illustrate the theory:
#!/usr/bin/env python
import time
import datetime
import locale
print locale.getlocale()
import pynotify
print locale.getlocale()
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
On my system, I get the following:
(None, None)
('en_US', 'UTF8')
5/1/12 1:15 PM
I am not getting your error, but it's possible pynotify is setting your locale to something completely silly which is confusing strptime.
Maybe have a look at that and fiddle a bit with your locale settings, either unset it before calling strptime (and set it back after, no knowing what kind of assumptions pynotify makes) or set it to something sane if you discover it's set to something silly.