try:
msg_json = json.loads(message_string)
if "task" in msg_json:
job_type = msg_json["task"]
return (job_type, msg_json)
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I have this code where i am trying to load some JSON formatted message string. After looking back at this piece of code i feel i maybe using try and catch in the wrong way and i was looking for suggestions as i am new to python and there may be cleaner approaches. There is no bug here but this is more for me to learn the "Cleaner" approach. As such i am open to all suggestions that explain the cleaner and more correct approach.
You could handle both of your error cases in catch blocks, which makes your "happy path" code a bit cleaner and neatly groups all the error handling in one place:
try:
msg_json = json.loads(message_string)
return (msg_json["task"], msg_json)
except KeyError:
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
Related
I'm new to Python, but as much as I search - I don't find a "jsonfiy" method for printing an error to the logs.
the way I'm printing today:
except Exception as e:
print(type(e))
print(e)
print(traceback.format_exc())
This way I get all needed data from the exception, but I must represent it in a JSON format.
Any help will be appreciated!
I finally found the answer to my question, using python-json-logger, in my case I used it within Flask app, so the way to configure logger to use JSON format is this:
formatter = jsonlogger.JsonFormatter('%(asctime) %(levelname) %(message)')
app.logger.handlers[0].setFormatter(formatter)
Then I used app logger to log an error with JSON format like this:
except Exception as e:
app.logger.error({"error": type(e).__name__, "message": e, "at": traceback.format_exc()})
I am facing this problem while I try to loop tweet_id using the API and write it to tweet_json.txt, the output for all data is Failed which I know is wrong
Before it was working good but when I try to Run all the code again it starts to show failed
for tweet_id in df['tweet_id']:
try:
tweet = api.get_status(tweet_id, tweet_mode = 'extended')
with open('tweet_json.txt', 'a+') as file:
json.dump(tweet._json, file)
file.write('\n')
print (tweet_id, 'success')
except:
print (tweet_id, 'Failed')
Your except is swallowing whatever exception is causing your code to die. Until you comment out the except or make it more specific you won't know if your problem is the Twitter API or file I/O or something else. Good luck!
A quick step forward would be to adjust your exception handler so that it writes the exception. I like to use the format_exc function to get my stack traces so i can write it with a logger, or however i want to handle it.
from traceback import format_exc
try:
a = "" + 1
except Exception as ex:
print("Exception encountered! \n %s " % format_exc())
itemis a python dictionary
print item.get('body')
gives the following output in some cases:
"1211V1"
however, item.get('body') mostly has a unicode string of the format:
u'{"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"}'
I needed to convert the above unicode string to a python dictionary. So I am doing this:
try:
body_dic = json.loads(item.get('body'))
body_string = ""
for body_item in body_dic.keys():
body_string += body_item + ": {'required': True, 'type': 'resource', 'value': " + str(body_dic.get('body_item')) + "\n\t\t\t\t"
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
and then a bunch of code after this. So in the above the moment item.get('body') comes out to be "1211V1" a ValueError Exception should be raised and the execution flow should get into the except block. Am I right ?
It does not get raised however and the execution flow continues to go onto the next line which is :
for body_item in body_dic.keys():
and then the following exception gets raised:
AttributeError
("'unicode' object has no attribute 'keys'",)
which I get to know if I change the except block in the above to catch a generic exception as :
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
Please help me understand this. In my opinion the moment the first exception gets raised (which in our case should be the ValueError Exception) the control flow should go into the catch block. Why does it go to the next line of code and then when the Attribute Exception gets raised does it get caught.
Assuming that, as you wrote
print item.get("body")
returns literally
"1211V1"
then the quotation marks are part of the string itself.
So you effectively calling
json.loads('"1211V1"')
where you are loading a JSON string literal--perfectly valid. Then, of course, you get an AttributeError for trying to call .keys() on a unicode object.
If you're using print to debug a problem it might mislead you in this way--you're better off often, if you really want to be sure what the object is that you're having trouble with, writing print(repr(obj)). In this case that would tell you that item.get("body") is u'"1211V1"'.
So your problem is that you have a sequence of dicts, whose attribute body is a JSON string. It may either be:
"1211V1"
or:
{
"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"
}
This is, a JSON string or a JSON object. By json.loads()ing this string you are always getting a valid Python value, either a Python str or a Python dict, respectively. What you want to do is detect if its one or another:
json_body = json.loads(item['body'])
if type(json_body) is dict:
for key, value in json_body.items():
json_body[key] = {'required': True, 'type': 'resource', 'value': value}
body_string = json.dumps(json_body)
else:
pass # Handle the "1211V1"
If I catch a KeyError, how can I tell what lookup failed?
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError:
# How can I tell what key ("FastestMan" or "FastestWoman") caused the error?
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key in JSON")
Take the current exception (I used it as e in this case); then for a KeyError the first argument is the key that raised the exception. Therefore we can do:
except KeyError as e: # One would do it as 'KeyError, e:' in Python 2.
cause = e.args[0]
With that, you have the offending key stored in cause.
Expanding your sample code, your log might look like this:
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError as e:
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key '"
e.args[0]
"' in JSON")
It should be noted that e.message works in Python 2 but not Python 3, so it shouldn't be used.
Not sure if you're using any modules to assist you - if the JSON is coming in as a dict, one can use dict.get() towards a useful end.
def POIJSON2DOM (location_node, POI_JSON):
man_JSON = POI_JSON.get("FastestMan", 'No Data for fastest man')
woman_JSON = POI_JSON.get("FastestWoman", 'No Data for fastest woman')
#work with the answers as you see fit
dict.get() takes two arguments - the first being the key you want, the second being the value to return if that key does not exist.
If you import the sys module you can get exception info with sys.exc_info()
like this:
def POIJSON2DOM (location_node, POI_JSON):
try:
man_JSON = POI_JSON["FastestMan"]
woman_JSON = POI_JSON["FastestWoman"]
except KeyError:
# you can inspect these variables for error information
err_type, err_value, err_traceback = sys.exc_info()
REDI.LogErrorMessage ("POIJSON2DOM", "Can't find mandatory key in JSON")
I have a function with a try catch block where I have:
def testfunction():
try:
good = myfunction()
return good
except ExceptionOne:
error="Error ExceptionOne".
return error
except ExceptionTwo:
error="Error ExceptionTwo".
return error
except ExceptionThree:
error="Error ExceptionThree".
return error
Is there a way such that I could structure this in such a way where have a single return statement for all the exceptions?
What about something like:
def testfunction():
try:
good = myfunction() # or `return myfunction()` if the case is actually this simple
except (ExceptionOne, ExceptionTwo, ExceptionThree) as err:
error = 'Error %s' % err.__class__.__name__
return error
return good
Of course, I'm not exactly sure what the purpose of this is. Why not just let the exception propogate and handle it higher up? As it is, you need logic to check the return value to see if it's good or not anyway, I think it would be cleaner if that logic was bound up in exception handling rather than in type checking or string value checking ...