I'm new to Python, but as much as I search - I don't find a "jsonfiy" method for printing an error to the logs.
the way I'm printing today:
except Exception as e:
print(type(e))
print(e)
print(traceback.format_exc())
This way I get all needed data from the exception, but I must represent it in a JSON format.
Any help will be appreciated!
I finally found the answer to my question, using python-json-logger, in my case I used it within Flask app, so the way to configure logger to use JSON format is this:
formatter = jsonlogger.JsonFormatter('%(asctime) %(levelname) %(message)')
app.logger.handlers[0].setFormatter(formatter)
Then I used app logger to log an error with JSON format like this:
except Exception as e:
app.logger.error({"error": type(e).__name__, "message": e, "at": traceback.format_exc()})
Related
try:
msg_json = json.loads(message_string)
if "task" in msg_json:
job_type = msg_json["task"]
return (job_type, msg_json)
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I have this code where i am trying to load some JSON formatted message string. After looking back at this piece of code i feel i maybe using try and catch in the wrong way and i was looking for suggestions as i am new to python and there may be cleaner approaches. There is no bug here but this is more for me to learn the "Cleaner" approach. As such i am open to all suggestions that explain the cleaner and more correct approach.
You could handle both of your error cases in catch blocks, which makes your "happy path" code a bit cleaner and neatly groups all the error handling in one place:
try:
msg_json = json.loads(message_string)
return (msg_json["task"], msg_json)
except KeyError:
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I am facing this problem while I try to loop tweet_id using the API and write it to tweet_json.txt, the output for all data is Failed which I know is wrong
Before it was working good but when I try to Run all the code again it starts to show failed
for tweet_id in df['tweet_id']:
try:
tweet = api.get_status(tweet_id, tweet_mode = 'extended')
with open('tweet_json.txt', 'a+') as file:
json.dump(tweet._json, file)
file.write('\n')
print (tweet_id, 'success')
except:
print (tweet_id, 'Failed')
Your except is swallowing whatever exception is causing your code to die. Until you comment out the except or make it more specific you won't know if your problem is the Twitter API or file I/O or something else. Good luck!
A quick step forward would be to adjust your exception handler so that it writes the exception. I like to use the format_exc function to get my stack traces so i can write it with a logger, or however i want to handle it.
from traceback import format_exc
try:
a = "" + 1
except Exception as ex:
print("Exception encountered! \n %s " % format_exc())
I've uploaded a json from the user and now I'm trying to compare that json to a schema using the jsonschema validator. I'm getting an error, ValidationError: is not of type u'object'
Failed validating u'type' in schema
This is my code so far:
from __future__ import unicode_literals
from django.shortcuts import render, redirect
import jsonschema
import json
import os
from django.conf import settings
#File to store all the parsers
def jsonVsSchemaParser(project, file):
baseProjectURL = 'src\media\json\schema'
projectSchema = project.lower()+'.schema'
projectPath = os.path.join(baseProjectURL,projectSchema)
filePath = os.path.join(settings.BASE_DIR,'src\media\json', file)
actProjectPath = os.path.join(settings.BASE_DIR,projectPath)
print filePath, actProjectPath
schemaResponse = open(actProjectPath)
schema = json.load(schemaResponse)
response = open(filePath)
jsonFile = json.load(response)
jsonschema.validate(jsonFile, schema)
I'm trying to do something similar to this question except instead of using a url I'm using my filepath.
Also I'm using python 2.7 and Django 1.11 if that is helpful at all.
Also I'm pretty sure I don't have a problem with my filepaths because I printed them and it outputted what I was expecting. I also know that my schema and json can be read by jsonschema since I used it on the command line as well.
EDIT: that validation error seemed to be a fluke. the actual validation error I'm consistently getting is "-1 is not of type u'string'". The annoying thing is that's supposed to be like that. It is wrong that sessionid isn't a string but I want that to be handled by the jsonschema but I don't want my validation errors to be given in this format: . What I want to do is collect all validation errors in an array and then post it to the user in the next page.
I just ended up putting a try-catch around my validate method. Here's what it looks like:
validationErrors = []
try:
jsonschema.validate(jsonFile, schema)
except jsonschema.exceptions.ValidationError as error:
validationErrors.append(error)
EDIT: This solution only works if you have one error because after the validation error is called it breaks out of the validate method. In order to present every error you need to use lazy validation. This is how it looks in my code if you need another example:
v = jsonschema.Draft4Validator(schema)
for error in v.iter_errors(jsonFile):
validationErrors.append(error)
✓ try-except-else-finally statement is a great way to catch and handle exceptions(Run time errors) in Python.
✓So if you want to catch and store Exceptions in an array then the great solution for you is to use try-except statement. In this way you can catch and store in any data structure like lists etc. and your program with continue with its execution, it will not terminate.
✓ Below is a modified code where I have used a for loop which catches error 5 times and stores in list.
validationErrors = []
for i in range(5):
try:
jsonschema.validate(jsonFile, schema)
except jsonschema.exceptions.ValidationError as error:
validationErrors.append(error)
✓ Finally, you can have a look at the below code sample where I have stored ZeroDivisionError and it's related string message in 2 different lists by iterating over a for loop 5 times.
You can use the 2nd list ZeroDivisionErrorMessagesList to pass to template, if you want to print messages on web page (if you want). You can use 1st also.
ZeroDivisionErrorsList = [];
ZeroDivisionErrorMessagesList = list(); # list() is same as [];
for i in range(5):
try:
a = 10 / 0; # it will raise exception
print(a);. # it will not execute
except ZeroDivisionError as error:
ZeroDivisionErrorsList.append(error)
ZeroDivisionErrorMessagesList.append(str(error))
print(ZeroDivisionErrorsList);
print(); # new line
print(ZeroDivisionErrorMessagesList);
» Output:
[ZeroDivisionError('division by zero',),
ZeroDivisionError('division by zero',),
ZeroDivisionError('division by zero',),
ZeroDivisionError('division by zero',),
ZeroDivisionError('division by zero',)]
['division by zero', 'division by zero', 'division by zero', 'division by zero', 'division by zero']
itemis a python dictionary
print item.get('body')
gives the following output in some cases:
"1211V1"
however, item.get('body') mostly has a unicode string of the format:
u'{"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"}'
I needed to convert the above unicode string to a python dictionary. So I am doing this:
try:
body_dic = json.loads(item.get('body'))
body_string = ""
for body_item in body_dic.keys():
body_string += body_item + ": {'required': True, 'type': 'resource', 'value': " + str(body_dic.get('body_item')) + "\n\t\t\t\t"
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
and then a bunch of code after this. So in the above the moment item.get('body') comes out to be "1211V1" a ValueError Exception should be raised and the execution flow should get into the except block. Am I right ?
It does not get raised however and the execution flow continues to go onto the next line which is :
for body_item in body_dic.keys():
and then the following exception gets raised:
AttributeError
("'unicode' object has no attribute 'keys'",)
which I get to know if I change the except block in the above to catch a generic exception as :
except Exception as e:
print "futt gayaa"
print type(e).__name__
print e.args
body_string = item.get('body')
Please help me understand this. In my opinion the moment the first exception gets raised (which in our case should be the ValueError Exception) the control flow should go into the catch block. Why does it go to the next line of code and then when the Attribute Exception gets raised does it get caught.
Assuming that, as you wrote
print item.get("body")
returns literally
"1211V1"
then the quotation marks are part of the string itself.
So you effectively calling
json.loads('"1211V1"')
where you are loading a JSON string literal--perfectly valid. Then, of course, you get an AttributeError for trying to call .keys() on a unicode object.
If you're using print to debug a problem it might mislead you in this way--you're better off often, if you really want to be sure what the object is that you're having trouble with, writing print(repr(obj)). In this case that would tell you that item.get("body") is u'"1211V1"'.
So your problem is that you have a sequence of dicts, whose attribute body is a JSON string. It may either be:
"1211V1"
or:
{
"points_token_id":"327727a0-3909-4132-8fa2-ee45146add1e"
}
This is, a JSON string or a JSON object. By json.loads()ing this string you are always getting a valid Python value, either a Python str or a Python dict, respectively. What you want to do is detect if its one or another:
json_body = json.loads(item['body'])
if type(json_body) is dict:
for key, value in json_body.items():
json_body[key] = {'required': True, 'type': 'resource', 'value': value}
body_string = json.dumps(json_body)
else:
pass # Handle the "1211V1"
If I catch a KeyError, how can I tell what lookup failed?
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError:
# How can I tell what key ("FastestMan" or "FastestWoman") caused the error?
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key in JSON")
Take the current exception (I used it as e in this case); then for a KeyError the first argument is the key that raised the exception. Therefore we can do:
except KeyError as e: # One would do it as 'KeyError, e:' in Python 2.
cause = e.args[0]
With that, you have the offending key stored in cause.
Expanding your sample code, your log might look like this:
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError as e:
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key '"
e.args[0]
"' in JSON")
It should be noted that e.message works in Python 2 but not Python 3, so it shouldn't be used.
Not sure if you're using any modules to assist you - if the JSON is coming in as a dict, one can use dict.get() towards a useful end.
def POIJSON2DOM (location_node, POI_JSON):
man_JSON = POI_JSON.get("FastestMan", 'No Data for fastest man')
woman_JSON = POI_JSON.get("FastestWoman", 'No Data for fastest woman')
#work with the answers as you see fit
dict.get() takes two arguments - the first being the key you want, the second being the value to return if that key does not exist.
If you import the sys module you can get exception info with sys.exc_info()
like this:
def POIJSON2DOM (location_node, POI_JSON):
try:
man_JSON = POI_JSON["FastestMan"]
woman_JSON = POI_JSON["FastestWoman"]
except KeyError:
# you can inspect these variables for error information
err_type, err_value, err_traceback = sys.exc_info()
REDI.LogErrorMessage ("POIJSON2DOM", "Can't find mandatory key in JSON")