I have a small python code in which I am using exception handling.
def handler(event):
try:
client = boto3.client('dynamodb')
response = client.scan(TableName=os.environ["datapipeline_table"])
return response
except Exception as error:
logging.exception("GetPipelinesError: %s",json.dumps(error))
raise GetPipelinesError(json.dumps({"httpStatus": 400, "message": "Unable to fetch Pipelines"}))
class GetPipelinesError(Exception):
pass
pylint warning gives me " Consider explicitly re-raising using the 'from' keyword ".
I saw few other posts, where they used from and raised an error. I made modifications like this
except Exception as GetPipelinesError:
logging.exception("GetPipelinesError: %s",json.dumps(GetPipelinesError))
raise json.dumps({"httpStatus": 400, "message": "Unable to fetch Pipelines"}) from GetPipelinesError
Is this the right way to do ?
No. The purpose of raise-from is to chain exceptions. The correct syntax in your case is:
except Exception as error:
raise GetPipelinesError(json.dumps(
{"httpStatus": 400, "message": "Unable to fetch Pipelines"})) from error
The expressions that follow raise and from must be exception classes or instances.
Related
According to the documentation for PyJWT, the class jwt.exceptions.InvalidTokenError is the base error when the decode method fails. However, the following code still breaks with different exceptions:
try:
jwt.decode(jwt_token, os.environ['SECRET'], algorithms="HS256")
except jwt.exceptions.InvalidTokenError:
pass
My thinking was that since InvalidTokenError is the base error, this except block should catch all the other possible PyJWT errors such as InvalidSignatureError, DecodeError etc. My question is if there is a base error for PyJwt I could use. I know using except Exception is always an option but that's bad form so I'd like to avoid it if possible. Thanks!
I got this micro example working. In general, don't pass an exception, at least print the error message.
#pip install pyjwt
import os, jwt, hashlib
jwt_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.VUI28NztICBIB9m6kZolIEzYJojrw0eUr_4bVoQ1Ong"
os.environ['SECRET'] = hashlib.sha256(b"0").hexdigest()
try:
print(jwt.decode(jwt_token, os.environ['SECRET'], algorithms="HS256"))
except jwt.exceptions.InvalidTokenError as e:
print(repr(e))
except Exception as e:
print("WARNING NORMAL EXCEPTION CAUGHT")
print(repr(e))
Output:
{'sub': '1234567890', 'name': 'John Doe', 'iat': 1516239022}
Which error was raised? My best guess is that you have another problem. KeyError for Secret, is not an error related to jwt:
WARNING NORMAL EXCEPTION CAUGHT
KeyError('SECRET')
If your token is incorrect, you get this:
DecodeError('Not enough segments')
And if your signare is not correct, this:
InvalidSignatureError('Signature verification failed')
I want to log my error messages as well as raise an exception with the same message.
For eg:
try:
foo()
except Exception as error:
error_message = "Custom error message"
logger.error(error_message)
raise RunttimeError(error_message)
I want to do something like this
try:
foo()
except Exception as error:
raise RunttimeError(logger.error("Custom error message"))
I want the logger.error() to return the error message.(This is not only for logger.error(), but for all other logger functions)
Is it possible?
I tried creating a Custom Handler for python logger which returns the msg provided to it, but that did not work. It behaves similar to Python StreamHandler
Currently I am trying to implement a function call that sends failed messages from a converter to a DLQ topic with Kafka. As part of the DLQ message I want to include the exception error that we also log.
the code:
except json.decoder.JSONDecodeError:
log.error('failed to parse json',
topic=msg.topic(),
partition=msg.partition(),
offset=msg.offset()
)
produce_dlq_message(converters.get("DLQ"), msg, error_message)
I need to get the value of that latest log.error() call and assign it to variable: error_message
I call this exact same function at another exception block but it has a different value in log.error() call so I need something that gets the last/latest error message.
While there might be a way to achieve that with multistructlog, processors and whatnot, I would recommend to take the low-tech route:
try:
...
except json.decoder.JSONDecodeError:
em = {
"event": "failed to parse json",
"topic": msg.topic(),
"partition": msg.partition(),
"offset": msg.offset(),
}
log.error(**em)
produce_dlq_message(converters.get("DLQ"), msg, json.dumps(em))
It means em gets serialized twice, but I think overall the simplicity makes it worth it.
I used this solution in the end.
try:
...
except json.decoder.JSONDecodeError as e:
log.error("failed to parse json",
topic=msg.topic(),
partition=msg.partition(),
offset=msg.offset(),
)
error_message = str(e)
produce_dlq_message(converters.get("DLQ"), msg, error_message)
There are three situations when I need to handle exceptions.
When data validation raised exception
When library/module functions raised exceptions (e.g. database connection abort)
When business logic raises exception such as 500, 503, 401, 403 and 404
def library_func():
try:
...
except HTTPException:
raise TwitterServiceException("Twitter is down!")
#view_config(route_name="home", renderer="json")
#validator
#authorization
def home_view(request):
try:
tweets = library_func()
return {"tweets": tweets}
except TwitterServiceException as e:
LOG.critical(e.msg)
raise ParnterServcieError(e.msg) # this is probably a 503 error
def validator(args):
# I will show the high level of this decorator
try:
decode input as JSON
verify data format
except ValueError as err:
error = {'error': "Missing required parameters."}
except json.JSONDecodeError as err:
error = {'error': "Failed to decode the incoming JSON payload."}
if error is not None:
return HTTPBadRequest(body=json.dumps(error),
content_type='application/json')
def authorization(args):
# very similar to validator except it performs authorization and if failed
# 401 is raised with some helpful message.
The doc suggests Custom Exception Views. In my PoC above, I will tie ParnterServcieError as one. I can even generalize HTTPBadRequest and all praymid.httpexceptions using custom exception so that I no longer need to repeat json.dumps and content_type. I can set a boilerplate error body before I return request.response object.
Idea:
#view_config(context=ParnterServcieError)
def 503_service_error_view(e, request):
request.response.status = 503
request.response.json_body = {"error": e.msg}
return request.response
I can generalize one for all uncaught, unspecified exceptions (which results in 500 Internal Server Error) called 500_internal_server_error_view.
Does this seem sane and clean to people? Is my way of handling high and low level of exceptions proper and Pythonic?
I applied this strategy to ToDoPyramid and could encapsulate error handling in a single custom exception view that was repeated multiple times in the application before. Until you could even improve it, you got a great idea. Pyramid rocks.
References
Catching database connection error in ToDoPyramid
In my GAE application, I have several request handlers that return JSON-formated response. When one of these is called, if an error occurs (exception, or programming error), the output is not JSON: it is the stack trace.
What I need is:
Output without error:
{
"foo" : 1
"bar" : 2
"status" : "OK"
}
Output when an error occurs:
{
"status" : "ERR"
"errorMessage" : "An error occurred!"
}
My question is: What is the best practice to make sure that, in any case, the output will be a JSON-formated response? Of course, a common solution for every request handlers would be great.
Any help would be appreciated.
Sure - use the ereporter class (described here: https://stackoverflow.com/a/4296664/336505), but create a custom BaseHandler that formats your uncaught exceptions as JSON output:
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(etc, etc) # format the exception
If an error occurs, to avoid getting a stack trace or other ugly output, you will need to employ a try ... except: http://docs.python.org/tutorial/errors.html
e.g.
try:
# ... your code ...
except TypeError as e:
# ... do something with this error type
except:
# ... generic exception catchall
# output that JSON response