I'm trying to get my code to store messages within txt file using json. Each time a new message comes in, it will add the new message to the array.
The structure will be:
{
"Messages": {
"Test Contact 2": {
"0": "\"Message 1"
},
"Test Contact 1": {
"0": "\"Message 1\"",
"1": "\"Message 2\""
}
}
}
And here is my current code:
class PluginOne(IPlugin):
def process(self):
try:
print("Database")
data_store('Test contact', 'Text Message')
pass
except Exception as exc:
print("Error in database: " + exc.args)
def data_store(key_id, key_info):
try:
with open('Plugins/Database/messages.txt', 'r+') as f:
data = json.load(f)
data[key_id] = key_info
f.seek(0)
json.dump(data, f)
f.truncate()
pass
except Exception as exc:
print("Error in data store: " + exc.args)
when I try to run the code, I get the following error
Can't convert 'tuple' object to str implicitly
In your exception handler, you're adding exc.args to a string. The args attribute is a tuple, which can't be converted to a string implicitly. You could...
# print it seperately
print("Error in data store")
print(exc.args)
# or alternatively
print("Error in data store: " + str(exc.args))
# or alternatively
print("Error in data store: " + str(exc))
However, this being a problem in the exception handler, the root cause of the problem is something else, and your current exception handler isn't that great at handling it:
without your exception handler, Python would show a complete traceback of the root cause of the exception, and halt your program.
with your exception handler, only your message is printed and the program continues. This might not be what you want.
It would perhaps be better to only catch the specific exceptions you know you can recover from.
Related
try:
msg_json = json.loads(message_string)
if "task" in msg_json:
job_type = msg_json["task"]
return (job_type, msg_json)
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I have this code where i am trying to load some JSON formatted message string. After looking back at this piece of code i feel i maybe using try and catch in the wrong way and i was looking for suggestions as i am new to python and there may be cleaner approaches. There is no bug here but this is more for me to learn the "Cleaner" approach. As such i am open to all suggestions that explain the cleaner and more correct approach.
You could handle both of your error cases in catch blocks, which makes your "happy path" code a bit cleaner and neatly groups all the error handling in one place:
try:
msg_json = json.loads(message_string)
return (msg_json["task"], msg_json)
except KeyError:
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I am facing this problem while I try to loop tweet_id using the API and write it to tweet_json.txt, the output for all data is Failed which I know is wrong
Before it was working good but when I try to Run all the code again it starts to show failed
for tweet_id in df['tweet_id']:
try:
tweet = api.get_status(tweet_id, tweet_mode = 'extended')
with open('tweet_json.txt', 'a+') as file:
json.dump(tweet._json, file)
file.write('\n')
print (tweet_id, 'success')
except:
print (tweet_id, 'Failed')
Your except is swallowing whatever exception is causing your code to die. Until you comment out the except or make it more specific you won't know if your problem is the Twitter API or file I/O or something else. Good luck!
A quick step forward would be to adjust your exception handler so that it writes the exception. I like to use the format_exc function to get my stack traces so i can write it with a logger, or however i want to handle it.
from traceback import format_exc
try:
a = "" + 1
except Exception as ex:
print("Exception encountered! \n %s " % format_exc())
itemis a python dictionary
print item.get('body')
gives the following output in some cases:
"1211V1"
however, item.get('body') mostly has a unicode string of the format:
u'{"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"}'
I needed to convert the above unicode string to a python dictionary. So I am doing this:
try:
body_dic = json.loads(item.get('body'))
body_string = ""
for body_item in body_dic.keys():
body_string += body_item + ": {'required': True, 'type': 'resource', 'value': " + str(body_dic.get('body_item')) + "\n\t\t\t\t"
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
and then a bunch of code after this. So in the above the moment item.get('body') comes out to be "1211V1" a ValueError Exception should be raised and the execution flow should get into the except block. Am I right ?
It does not get raised however and the execution flow continues to go onto the next line which is :
for body_item in body_dic.keys():
and then the following exception gets raised:
AttributeError
("'unicode' object has no attribute 'keys'",)
which I get to know if I change the except block in the above to catch a generic exception as :
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
Please help me understand this. In my opinion the moment the first exception gets raised (which in our case should be the ValueError Exception) the control flow should go into the catch block. Why does it go to the next line of code and then when the Attribute Exception gets raised does it get caught.
Assuming that, as you wrote
print item.get("body")
returns literally
"1211V1"
then the quotation marks are part of the string itself.
So you effectively calling
json.loads('"1211V1"')
where you are loading a JSON string literal--perfectly valid. Then, of course, you get an AttributeError for trying to call .keys() on a unicode object.
If you're using print to debug a problem it might mislead you in this way--you're better off often, if you really want to be sure what the object is that you're having trouble with, writing print(repr(obj)). In this case that would tell you that item.get("body") is u'"1211V1"'.
So your problem is that you have a sequence of dicts, whose attribute body is a JSON string. It may either be:
"1211V1"
or:
{
"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"
}
This is, a JSON string or a JSON object. By json.loads()ing this string you are always getting a valid Python value, either a Python str or a Python dict, respectively. What you want to do is detect if its one or another:
json_body = json.loads(item['body'])
if type(json_body) is dict:
for key, value in json_body.items():
json_body[key] = {'required': True, 'type': 'resource', 'value': value}
body_string = json.dumps(json_body)
else:
pass # Handle the "1211V1"
I am writing a code to parse through a bunch of xml files. It basically looks like this:
for i in range(0, 20855):
urlb = str(i)
url = urla + urlb
trys=0
t=0
while (trys < 3):
try:
cfile = UR.urlopen(url)
trys = 3
except urllib.error.HTTPError as e:
t=t+1
print('error at '+str(time.time()-tstart)+' seconds')
print('typeID = '+str(i))
print(e.code)
print(e.read())
time.sleep (0.1)
trys=0+t
tree = ET.parse(cfile) ##parse xml file
root = tree.getroot()
...do a bunch of stuff with i and the file data
I'm having a problem with some of the urls I'm calling not actually containing an xml file which breaks my code. I have a list of all the actual numbers that I use instead of the range shown but i really don't want to go through all 21000 and remove each number that fails. Is there an easier way to get around this? I get an error from the while loop (which i have to deal with timeouts really) that looks like this:
b'A non-marketable type was given'
error at 4.321678161621094 seconds
typeID = 31
400
So I was thinking there has to be a good way to bail out of that iteration of the for-loop if my while-loop returns three errors but i can't use break. Maybe an if/else-loop under the while-loop that just passes if the t variable is 3?
You might try this:
for i in range(0, 20855):
url = '%s%d' % (urla, i)
for trys in range(3):
try:
cfile = UR.urlopen(url)
break
except urllib.error.HTTPError as e:
print('error at %s seconds' % (time.time()-tstart))
print('typeID = %i'%i)
print(e.code)
print(e.read())
time.sleep(0.1)
else:
print "retry failed 3 times"
continue
try:
tree = ET.parse(cfile) ##parse xml file
except Exception, e:
print "cannot read xml"
print e
continue
root = tree.getroot()
...do a bunch of stuff with i and the file data
Regarding your "algorithmic" problem: You can always set an error state (as simple as e.g. last_iteration_successful = False) in the while body, then break out of the while body, then check the error state in the for body, and conditionally break out of the for body, too.
Regarding architecture: Prepare your code for all relevant errors that might occur, via proper exception handling with try/except blocks. It might also make sense to define custom Exception types, and then raise them manually. Raising an exception immediately interrupts the current control flow, it could save many breaks.
In my application users enter a url and I try to open the link and get the title of the page. But I realized that there can be many different kinds of errors, including unicode characters or newlines in titles and AttributeError and IOError. I first tried to catch each error, but now in case of a url fetch error I want to redirect to an error page where the user will enter the title manually. How do I catch all possible errors? This is the code I have now:
title = "title"
try:
soup = BeautifulSoup.BeautifulSoup(urllib.urlopen(url))
title = str(soup.html.head.title.string)
if title == "404 Not Found":
self.redirect("/urlparseerror")
elif title == "403 - Forbidden":
self.redirect("/urlparseerror")
else:
title = str(soup.html.head.title.string).lstrip("\r\n").rstrip("\r\n")
except UnicodeDecodeError:
self.redirect("/urlparseerror?error=UnicodeDecodeError")
except AttributeError:
self.redirect("/urlparseerror?error=AttributeError")
#https url:
except IOError:
self.redirect("/urlparseerror?error=IOError")
#I tried this else clause to catch any other error
#but it does not work
#this is executed when none of the errors above is true:
#
#else:
# self.redirect("/urlparseerror?error=some-unknown-error-caught-by-else")
UPDATE
As suggested by #Wooble in the comments I added try...except while writing the title to database:
try:
new_item = Main(
....
title = unicode(title, "utf-8"))
new_item.put()
except UnicodeDecodeError:
self.redirect("/urlparseerror?error=UnicodeDecodeError")
This works. Although the out-of-range character —is still in title according to the logging info:
***title: 7.2. re — Regular expression operations — Python v2.7.1 documentation**
Do you know why?
You can use except without specifying any type to catch all exceptions.
From the python docs http://docs.python.org/tutorial/errors.html:
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except IOError as (errno, strerror):
print "I/O error({0}): {1}".format(errno, strerror)
except ValueError:
print "Could not convert data to an integer."
except:
print "Unexpected error:", sys.exc_info()[0]
raise
The last except will catch any exception that has not been caught before (i.e. a Exception which is not of IOError or ValueError.)
You can use the top level exception type Exception, which will catch any exception that has not been caught before.
http://docs.python.org/library/exceptions.html#exception-hierarchy
try:
soup = BeautifulSoup.BeautifulSoup(urllib.urlopen(url))
title = str(soup.html.head.title.string)
if title == "404 Not Found":
self.redirect("/urlparseerror")
elif title == "403 - Forbidden":
self.redirect("/urlparseerror")
else:
title = str(soup.html.head.title.string).lstrip("\r\n").rstrip("\r\n")
except UnicodeDecodeError:
self.redirect("/urlparseerror?error=UnicodeDecodeError")
except AttributeError:
self.redirect("/urlparseerror?error=AttributeError")
#https url:
except IOError:
self.redirect("/urlparseerror?error=IOError")
except Exception, ex:
print "Exception caught: %s" % ex.__class__.__name__