Develop Reference

ValueError: unconverted data remains: 02

how must I change an element from a list that is in this form: 05/06/2020, to get a date object like:
date_object = datetime.strptime(listelement, '%m/%d/%y') ?
Here is my Code:
daten = {}
with open("Observed_Monthly_Rain_Gauge_Accumulations_-_Oct_2002_to_May_2017.csv", 'r') as csvfile:
regen_csv = csv.reader(csvfile)
next(regen_csv, None)
for rows in regen_csv:
keys = rows[:1]
strt= str(keys[0]) in
date_object = datetime.strptime(strt, '%m/%d/%y')
values = rows[1:]
daten[strt] = values
print(daten)
Traceback (most recent call last):
File *directory*, line 16, in <module>
date_object = datetime.strptime(strt, '%m/%d/%y')
File "C:\Program Files\Python37\lib\_strptime.py", line 577, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "C:\Program Files\Python37\lib\_strptime.py", line 362, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: 02
Heres a link to the list with the data:
https://data.seattle.gov/api/views/rdtp-hzy3/rows.csv?accessType=DOWNLOAD

You need to use %Y for years in the form 2020. A full list of format codes can be found here.
date_object = datetime.strptime(strt, '%m/%d/%Y')

Check if input string contains date(specially in the format yy-mm-dd) using python

I am trying to check if the string entered by the user contains date using python.
something like this.
user_input = ("Enter date")
if user_input==#type of date(yyyy-mm-dd):
print(okay)
else:
print("failed")

The dateutil library is your best friend when it comes to parsing dates:
>>> from dateutil.parser import *
>>> parse('2003-12-23')
datetime.datetime(2003, 12, 23, 0, 0)
>>> parse('2003-12-32')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1356, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 653, in parse
ret = self._build_naive(res, default)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1227, in _build_naive
naive = default.replace(**repl)
ValueError: day is out of range for month
>>>
If you want to be more restrictive, ie. only accept dates with the YYYY-MM-DD format, then checking with a regex first might be your thing:
def is_date(ds):
if re.match(r'\d{4}-\d{2}-\d{2}', ds):
return bool(parse(ds))
return False
>>> is_date('2003-12-23')
True
>>> is_date('2003-12-32')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in is_date
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1356, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 653, in parse
ret = self._build_naive(res, default)
File "c:\srv\venv\dev\lib\site-packages\dateutil\parser\_parser.py", line 1227, in _build_naive
naive = default.replace(**repl)
ValueError: day is out of range for month
If you don't want to be quite as restrictive, but accept all iso-formatted dates, then:
from dateutil.parser import isoparse
>>> isoparse('2003-12-23')
datetime.datetime(2003, 12, 23, 0, 0)
>>> isoparse('20031223')
datetime.datetime(2003, 12, 23, 0, 0)

This is what you want:
>>> import datetime
>>> def validate(date_text):
try:
datetime.datetime.strptime(date_text, '%Y-%m-%d')
except ValueError:
raise ValueError("Incorrect data format, should be YYYY-MM-DD")
>>> validate('2003-12-23')
>>> validate('2003-12-32')
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
validate('2003-12-32')
File "<pyshell#18>", line 5, in validate
raise ValueError("Incorrect data format, should be YYYY-MM-DD")
ValueError: Incorrect data format, should be YYYY-MM-DD

Convert string second including timezone to a formated string date

I'm trying to dynamically convert a string seconds like below to a string date.
'1545239561 +0100'
The problem is the timezone inserted at the end, and I can't find any python time object method using the good format to retrieve the date from this string.
My tries :
>>>seconds = '1545239561 +0100'
>>>time.strftime('%y%m%d-%H%M%S-%f', datetime.datetime.fromtimestamp(seconds)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type str)
>>>time.strptime(seconds)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 559, in _strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '1545239561 +0100' does not match format '%a %b %d %H:%M:%S %Y'
>>>time.strptime(seconds, "%S +%Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 559, in _strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py", line 362, in _strptime
(data_string, format))
ValueError: time data '1545239561 +0100' does not match format '%S +%Z'

I would try to process both those value separately and merge them into single datetime:
>>>from datetime import datetime
>>>s = '1545239561 +0100'
>>>seconds, offset = s.split()
>>>datetime.fromtimestamp(int(seconds)).replace(tzinfo=datetime.strptime(offset, "%z").tzinfo)
datetime.datetime(2018, 12, 19, 17, 12, 41, tzinfo=datetime.timezone(datetime.timedelta(0, 3600)))

Yes #mfrackwiak...
I did this
>>> epoch = "1545239561 +0100"
>>> seconds, offset = epoch.split()
>>> datetime.fromtimestamp(int(seconds)).replace(tzinfo=datetime.strptime(offset, "%z").tzinfo).strftime('%Y-%m-%d %H:%M:%S-%Z')
'2018-12-19 18:12:41-UTC+01:00'
>>>

You can try the below example :
from datetime import datetime,timedelta
# split the timevalue and offset value
ss = '1545239561 +0100'.split()
format = "%A, %B %d, %Y %I:%M:%S"
# calculate the hour and min in the offset
hour = int(ss[1][0:2])
min = int(ss[1][2:])
# calculate the time from the sec and convert it to datetime object
time_from_sec = datetime.strptime(datetime.fromtimestamp(int(ss[0])).strftime(
format), format)
# add the offset delta value to the time calculated
time_with_delta_added = time_from_sec + timedelta(hours=hour,minutes=min)
print(time_with_delta_added)
Output :
2018-12-19 12:22:41

can't parse parse time string with posix-style timezone in python

I have a time string like this: 2013-08-22 16:56:19 Etc/GMT
I need to parse it to a datetime object. I'm getting hung up on the posix-style time zone, which I can't get Python to natively grok.
Here are a couple of attempts and their failures. I'm including the timezone-stripped version first to show the parsing is otherwise correct.
datetime.strptime
>>> datetime.datetime.strptime("2013-08-22 16:56:19 UTC", "%Y-%m-%d %H:%M:%S %Z")
datetime.datetime(2013, 8, 22, 16, 56, 19)
>>> datetime.datetime.strptime("2013-08-22 16:56:19 Etc/GMT", "%Y-%m-%d %H:%M:%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '2013-08-22 16:56:19 Etc/GMT' does not match format '%Y-%m-%d %H:%M:%S %Z'
time.strptime
>>> time.strptime("2013-08-22 16:56:19 UTC", "%Y-%m-%d %H:%M:%S %Z")
time.struct_time(tm_year=2013, tm_mon=8, tm_mday=22, tm_hour=16, tm_min=56, tm_sec=19, tm_wday=3, tm_yday=234, tm_isdst=0)
>>> time.strptime("2013-08-22 16:56:19 Etc/GMT", "%Y-%m-%d %H:%M:%S %Z")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '2013-08-22 16:56:19 Etc/GMT' does not match format '%Y-%m-%d %H:%M:%S %Z'
dateutil.parser
>>> dateutil.parser.parse("2013-08-22 16:56:19")
datetime.datetime(2013, 8, 22, 16, 56, 19)
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
possible solutions/avenues tried
tzinfos
It seems like dateutil's tzinfos argument should be perfect for this, but it doesn't work either... or I'm misreading the trace and doing something wrong. (I used this as an example)
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT", tzinfos={ 'Etc/GMT': pytz.timezone('UTC') })
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
tzinfos, round two
It seems like this stackoverflow answer is getting towards the usage of tzinfos that I might need. I tried the simplified version of the above (where value=offset seconds). Still fails.
>>> dateutil.parser.parse("2013-08-22 16:56:19 Etc/GMT", tzinfos={ 'Etc/GMT': 0 })
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 720, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_dateutil-2.1-py2.7.egg/dateutil/parser.py", line 310, in parse
raise ValueError("unknown string format")
ValueError: unknown string format
do it the wrong way
I could always use something like regex or string matching to find and change this timezone, but it feels wrong.
red herrings
The common problem with the posix ETC/xxx timezones is that they have reversed signs. This is a UTC ("no offset") posix time zone, and many of the questions I've found that deal with "etc" have to do with this reversed offset.

How about this:
parse date string without the timezone part into datetime object via strptime()
parse timezone string into pytz timezone
update tzinfo on the datetime object via replace()
from datetime import datetime
import pytz
date_string = "2013-08-22 16:56:19"
tz_string = "Etc/GMT"
dt = datetime.strptime(date_string, "%Y-%m-%d %H:%M:%S")
dt = dt.replace(tzinfo=pytz.timezone('Etc/GMT'))
print dt
prints:
2013-08-22 16:56:19+00:00
This actually correctly understands and reverses signs in the POSIX timezone format, for example:
dt = dt.replace(tzinfo=pytz.timezone('Etc/GMT-1'))
print dt # prints 2013-08-22 16:56:19+01:00

Pynotify screws up datetime, why?

The Problem
This code
#!/usr/bin/env python
import pynotify
import time
import datetime
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
produces
5/1/12 1:15 PM
Traceback (most recent call last):
File "tmp.py", line 9, in <module>
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: PM
Removing the import pynotify,
#!/usr/bin/env python
import time
import datetime
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
Removes the error.
5/1/12 1:15 PM
WHY?!!?!
Python Version
Python 2.7.2+ (default, Oct 4 2011, 20:06:09)
[GCC 4.6.1] on linux2
pynotify.file
I added print calls for pynotify.__file__ and datetime.__file__
/usr/lib/python2.7/lib-dynload/datetime.so
/usr/lib/python2.7/dist-packages/gtk-2.0/pynotify/__init__.pyc
5/1/12 1:15 PM
Traceback (most recent call last):
File "a.py", line 11, in <module>
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
File "/usr/lib/python2.7/_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: PM
PDB
5/1/12 1:15 PM
> /usr/lib/python2.7/_strptime.py(324)_strptime()
-> found = format_regex.match(data_string)
(Pdb) format
'%d/%m/%y %H:%M %p'
(Pdb) continue
> /usr/lib/python2.7/_strptime.py(329)_strptime()
-> if len(data_string) != found.end():
(Pdb) continue
> /usr/lib/python2.7/_strptime.py(331)_strptime()
-> raise ValueError("unconverted data remains: %s" %
(Pdb) len(data_string)
14
(Pdb) found.end()
12
(Pdb) found.group(0)
'5/1/12 1:15 '
It would appear that '%d/%m/%y %H:%M %p' isn't capturing ALL of '5/1/12 1:15 PM'

That's a fun problem. I'd wager what's happening is that pynotify is changing your locale settings, which is breaking strptime's interpretation of your timestamp string.
Here's your code with a couple of debugging print statements that will illustrate the theory:
#!/usr/bin/env python
import time
import datetime
import locale
print locale.getlocale()
import pynotify
print locale.getlocale()
c='5/1/12 1:15 PM'
print c
dt = time.strptime(c, "%d/%m/%y %H:%M %p")
On my system, I get the following:
(None, None)
('en_US', 'UTF8')
5/1/12 1:15 PM
I am not getting your error, but it's possible pynotify is setting your locale to something completely silly which is confusing strptime.
Maybe have a look at that and fiddle a bit with your locale settings, either unset it before calling strptime (and set it back after, no knowing what kind of assumptions pynotify makes) or set it to something sane if you discover it's set to something silly.