I have some dates in a json files, and I am searching for those who corresponds to today's date :
import os
import time
from datetime import datetime
from pytz import timezone
input_file = file(FILE, "r")
j = json.loads(input_file.read().decode("utf-8-sig"))
os.environ['TZ'] = 'CET'
for item in j:
lt = time.strftime('%A %d %B')
st = item['start']
st = datetime.strptime(st, '%A %d %B')
if st == lt :
item['start'] = datetime.strptime(st,'%H:%M')
I had an error like this :
File "/home/--/--/--/app/route.py", line 35, in file.py
st = datetime.strptime(st, '%A %d %B')
File "/usr/lib/python2.7/_strptime.py", line 328, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: 02:05
Do you have any suggestions ?
The value of st at st = datetime.strptime(st, '%A %d %B') line something like 01 01 2013 02:05 and the strptime can't parse this. Indeed, you get an hour in addition of the date... You need to add %H:%M at your strptime.
Best answer is to use the from dateutil import parser.
usage:
from dateutil import parser
datetime_obj = parser.parse('2018-02-06T13:12:18.1278015Z')
print datetime_obj
# output: datetime.datetime(2018, 2, 6, 13, 12, 18, 127801, tzinfo=tzutc())
You have to parse all of the input string, you cannot just ignore parts.
from datetime import date, datetime
for item in j:
st = datetime.strptime(item['start'], '%A %d %B %H:%M')
if st.date() == date.today():
item['start'] = st.time()
Here, we compare the date to today's date by using more datetime objects instead of trying to use strings.
The alternative is to only pass in part of the item['start'] string (splitting out just the time), but there really is no point here, not when you could just parse everything in one step first.
Well it was very simple. I was missing the format of the date in the json file, so I should write :
st = datetime.strptime(st, '%A %d %B %H %M')
because in the json file the date was like :
"start": "Friday 06 December 02:05",
timeobj = datetime.datetime.strptime(my_time, '%Y-%m-%d %I:%M:%S')
File "/usr/lib/python2.7/_strptime.py", line 335, in _strptime
data_string[found.end():])
ValueError: unconverted data remains:
In my case, the problem was an extra space in the input date string. So I used strip() and it started to work.
just cut the string that match the format, do something like:
st = datetime.strptime(st[:-6], '%A %d %B')
ValueError: unconverted data remains: 02:05 means that part of your date including time is not in the datetime.strptime used pattern my suggestion is to make simple trick and check if your string date has time or not eg. len(date_string) > 10:
from datetime import datetime
date_strings = ['2022-12-31 02:05:00', '2022-12-31', '2023-01-01 05:30:00', '2023-01-01']
dates = []
for date_string in date_strings:
if len(date_string) > 10:
# String has time information
date = datetime.strptime(date_string, "%Y-%m-%d %H:%M:%S")
else:
# String has no time information
date = datetime.strptime(date_string, "%Y-%m-%d")
dates.append(date)
print(dates)
Related
I'm reading (actually scraping) an RSS feed of NOAA buoy data. One of the data is the date and time the data was collected at the buoy. So the string I am extrapolating is in this format: January 10, 2023 9:48 am
But if the time is on the hour, say 'January 10, 2023 10:00 am', the feed produces an extra variable that throws my output off.
Thus, my code would check to see if the feed is on the hour and change the variables, like so:
air_temp = rows[7]
water_temp = rows[8]
if [time minutes = '00']:
air_temp = rows[7]
water_temp = rows[8]
I'm assuming I would need to change the time string to datetime in order to write the If statement? (Otherwise, I'm happy with the string format as is for my output.)
to check if the time is on the hour, you could do this to convert it to a datetime, then check if the minutes are zero:
from datetime import datetime as dt
date = 'January 10, 2023 10:00 am'
datetime = dt.strptime(s, '%B %d, %Y %H:%M %p')
minute = datetime.minute
if minute == 0:
do whatever
you could also do this which requires less thinking about the date format :)
import pandas as pd
pd.to_datetime(date)
You can use datetime for the same and convert your input time format to python's datetime data type.
import datetime
input_str = 'January 10, 2023 9:48 am'
input_time_format = '%B %d, %Y %I:%M %p'
datetime_str = datetime.datetime.strptime(input_str, input_time_format)
print(datetime_str.minute)
if datetime_str.minute == 0:
pass
You can check more details about input format here: https://docs.python.org/3/library/datetime.html
I am trying to do scraping from https://finansial.bisnis.com/read/20210506/90/1391096/laba-bank-mega-tumbuh-dua-digit-kuartal-i-2021-ini-penopangnya. I am trying to scrape the date of news, here's my code:
news['tanggal'] = newsScrape['date']
dates = []
for x in news['tanggal']:
x = listToString(x)
x = x.strip()
x = x.replace('\r', '').replace('\n', '').replace(' \xa0|\xa0', ',').replace('|', ', ')
dates.append(x)
dates = listToString(dates)
dates = dates[0:20]
if len(dates) == 0:
continue
news['tanggal'] = dt.datetime.strptime(dates, '%d %B %Y, %H:%M')
but I got this error:
ValueError: time data '06 Mei 2021, 11:32 ' does not match format '%d %B %Y, %H:%M'
My assumption is because Mei is in Indonesian language, meanwhile the format need May which is in English. How to change Mei to be May? I have tried dates = dates.replace('Mei', 'May') but it doesnt work on me. When I tried it, I got error ValueError: unconverted data remains: The type of dates is string. Thanks
You can try with the following
import datetime as dt
import requests
from bs4 import BeautifulSoup
import urllib.request
url="https://finansial.bisnis.com/read/20210506/90/1391096/laba-bank-mega-tumbuh-dua-digit-kuartal-i-2021-ini-penopangnya"
r = requests.get(url, verify=False)
soup = BeautifulSoup(r.content, 'html.parser')
info_soup= soup.find(class_="new-description")
x=info_soup.find('span').get_text(strip=True)
x = x.strip()
x = x.replace('\r', '').replace('\n', '').replace(' \xa0|\xa0', ',').replace('|', ', ')
x = x[0:20]
x = x.rstrip()
date= dt.datetime.strptime(x.replace('Mei', 'May'), '%d %B %Y, %H:%M')
print(date)
result:
2021-05-06 11:45:00
Your assumption regarding the May -> Mei change is correct, the reason you're likely facing a problem after the replacement are the trailing spaces in your string, which are not accounted for in your format. You can use string.rstrip() to remove these spaces.
import datetime as dt
dates = "06 Mei 2021, 11:32 "
dates = dates.replace("Mei", "May") # The replacement will have to be handled for all months, this is only an example
dates = dates.rstrip()
date = dt.datetime.strptime(dates, "%d %B %Y, %H:%M")
print(date) # 2021-05-06 11:32:00
While
this does fix the problem here, it's messy to have to shorten the string like this after dates = dates[0:20]. Consider using regex to gain the appropriate format at once.
The problem seems to be just the trailing white space you have, which explains the error ValueError: unconverted data remains: . It is complaining that it is unable to convert the remaining data (whitespace).
s = '06 Mei 2021, 11:32 '.replace('Mei', 'May').strip()
datetime.strptime(s, '%d %B %Y, %H:%M')
# Returns datetime.datetime(2021, 5, 6, 11, 32)
Also, to convert all the Indonesian months to English, you can use a dictionary:
id_en_dict = {
...,
'Mei': 'May',
...
}
I am following another search thread here, but it is not working. Anytime I include %p to get AM/PM, I get the following error:
ValueError: time data '11:30' does not match format '%I:%M%p'
This is true if I have '%I:%M %p' or '%I:%M%p'. If I leave the %p off it works fine, but that defeats the purpose of getting AM PM
# my code
from datetime import datetime
date_string = '11:30'
format = '%I:%M %p'
my_date = datetime.strptime(date_string, format)
my_date.strftime(format)
You just need to provide same format of your time in strptime function. And then use AM/PM format ('%I:%M %p') in strftime function.
from datetime import datetime
def getTime(time_string):
time_object = datetime.strptime(time_string,'%H:%M') #Covert string to time object
return time_object.strftime('%I:%M %p') #Convert time object to AM/PM format
getTime('11:30')
Output: 11:30 am
getTime('13:30')
Output: 01:30 pm
Yes, cause you need to add if it's AM or PM to match the format.
from datetime import datetime
date_string = '11:30 AM'
format = '%I:%M %p'
my_date = datetime.strptime(date_string, format)
my_date.strftime(format)
%I matches hour, %M matches minutes, %p matches AM/PM
The format strings provided strptime (...) must match exactly.
If you do not know which format your time is in, you can try multiple ones:
from datetime import datetime
def getTime(text, formats = ['%I:%M %p','%I:%M']):
"""Tries different patterns to create a time from text.
First format with match wins.
As default the time is parsed with am/pm, as fallback without it."""
for pattern in formats:
try:
return datetime.strptime(text, pattern)
except:
pass # catch all errors
# nothing matched, return None
raise ValueError("No format {} matched '{}'".format(formats,text))
a_time = getTime("11:42") # produces an am time
b_time = getTime("11:42 pm") # produces a pm time
print(a_time.strftime("%I:%M %p"))
print(b_time.strftime("%I:%M %p"))
try:
c_time = getTime("does not work")
except ValueError as e:
print(type(e),e)
Output:
11:42 AM
11:42 PM
<class 'ValueError'> No format ['%I:%M %p', '%I:%M'] matched 'does not work'
I have string time in the following format
2016-12-10T13:54:15.294
I am using the following method to format the time:
time.strptime(ts, '%b %d %H:%M:%S %Y')
Which throws an error:
time data did not match format: data=2016-12-10T13:54:15.294 fmt=%a %b %d %H:%M:%S %Y
Any ideas where I am going wrong?
You need to first parse the string as its formatted, then print it out the way you want.
>>> import datetime
>>> ts = "2016-12-10T13:54:15.294"
>>> parsed = datetime.datetime.strptime(ts, '%Y-%m-%dT%H:%M:%S.%f')
>>> parsed
datetime.datetime(2016, 12, 10, 13, 54, 15, 294000)
>>> parsed.strftime('%b %d %H:%M:%S %Y')
'Dec 10 13:54:15 2016'
I think your date format is incorrectly specified in string. This should work:
import datetime
a = '2016-12-10T13:54:15.294'
b= datetime.datetime.strptime(a,'%Y-%m-%dT%H:%M:%S.%f')
print b
The error is not wrong, the format string is not even close to the string you're trying to parse.
You have {year}-{month}-{day}T{hour}:{minute}:{second}.{milliseconds} but trying to parse it with {weekday name} {month name} {day} {hour}:{minute}:{second} {year}. Did you copy this from somewhere?
According to the documentation, your format string should look more like %Y-%m-%dT%H:%M:%S.%f.
>>> time.strptime('2016-12-10T13:54:15.294', '%Y-%m-%dT%H:%M:%S.%f')
time.struct_time(tm_year=2016, tm_mon=12, tm_mday=10, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=5, tm_yday=345, tm_isdst=-1)
Your format string is not correct.
You can check format string just using strftime method of date object. For example:
d = datetime.datetime.now()
print(d.strftime('%Y-%d-%mT%H:%M:%S'))
Output:
Dec 16 11:02:46 2016
But you have string in following format 2016-12-10T13:54:15.294, so you just need to change format string:
print(time.strptime(ts, '%Y-%d-%mT%H:%M:%S.%f'))
output:
time.struct_time(tm_year=2016, tm_mon=10, tm_mday=12, tm_hour=13, tm_min=54, tm_sec=15, tm_wday=2, tm_yday=286, tm_isdst=-1)
I am trying to parse this "For The Year Ending December 31, 2015" and convert it to 2015-12-31 using the datetime lib. How would I go about partitioning and then converting the date? My program is looking through an excel file with multiple sheets and combining them into one; however, there is need now to add a date column, but I can only get it write the full value to the cell. So my data column currently has "For The Year Ending December 31, 2015" in all the rows.
Thanks in advance!
Here is the code block that is working now. Thanks all! edited to account for text that could vary.
if rx > (options.startrow-1):
ws.write(rowcount, 0, sheet.name)
date_value = sheet.cell_value(4,0)
s = date_value.split(" ")
del s[-1]
del s[-1]
del s[-1]
string = ' '.join(s)
d = datetime.strptime(date_value, string + " %B %d, %Y")
result = datetime.strftime(d, '%Y-%m-%d')
ws.write(rowcount, 9, result)
for cx in range(sheet.ncols):
Simply include the hard-coded portion and then use the proper identifiers:
>>> import datetime
>>> s = "For The Year Ending December 31, 2015"
>>> d = datetime.datetime.strptime(s, 'For The Year Ending %B %d, %Y')
>>> result = datetime.datetime.strftime(d, '%Y-%m-%d')
>>> print(result)
2015-12-31
from datetime import datetime
date_string = 'For The Year Ending December 31, 2015'
date_string_format = 'For The Year Ending %B %d, %Y'
date_print_class = datetime.strptime(date_string, date_string_format)
wanted_date = datetime.strftime(date_print_class, '%Y-%m-%d')
print(wanted_date)