I'm writing a web-application based on the webapp2 framework. I'm using a common base exception for all the errors I'm explicitly throwing, like
class MyBaseException(Exception):
def __init__(self, status, code, message):
self.status_code = status
self.error_code = code
self.error_message = message
and have a BaseRequestHandler that outputs a simple JSON response for errors, defaulting to a 500/Generic error for unexpected exceptions.
class BaseHandler(webapp2.RequestHandler):
def handle_exception(self, e, debug):
logger.exception(e)
status = e.status_code if isinstance(e, MyBaseException) else 500
code = e.error_code if isinstance(e, MyBaseException) else 'GENERIC_ERROR'
message = e.error_message if isinstance(e, MyBaseException) else 'Blah blah blah'
self.response.set_status(status)
self.response.content_type = 'application/json'
self.response.write(json.encode({"code": code, "message": message})
Those isinstance checks look ugly to me, so I'm thinking there has to be a better way. Suggestions?
EDIT What has this to do with downcasting?
In java, my "native" language, I'd do something like
MyBaseException b = (MyBaseException) e;
JSONObject j = new JSONObject();
j.put("error_code", e.getCode());
j.put("error_message", e.getErrorMessage());
...
but python has no explicit type casts so... is it possible to do something like that?
The usual Python idiom is to ignore the class of the object altogether, so you'd just do:
status = e.status_code
code = e.error_code
message = e.error_message
This is referred to as duck typing.
If you may need to handle other types of exceptions (where, in your Java example you'd get a ClassCastException) the standard Python idiom is to just wrap the whole thing in a try...catch:
try:
status = e.status_code
etc.
catch AttributeError: # i.e. e doesn't have one of the listed attributes
status = "500"
etc.
Related
I have the model object that looks like this
class CatalogModel(BaseModel):
#property
def custom_service(self):
return CustomService()
async def get_offers(self, catalog_name):
try:
svc_response = self.custom_service.get_offers(catalog_name=catalog_name)()
except BaseException:
raise SalesForceException()
return CustomOfferResponse().dump(svc_response)
I am trying to write a test for that get_offers function (that uses custom_services which is connecting to Salesforce)
My test is looking like this. I am using pytest, pytest vcr etc.
class TestCatalogModel:
catalog_name = 'CATALOG_1'
#freeze_time("2021-07-12")
async def test_get_offers(self, loop, offers, offers_response):
with MockUser(ident="test_model_get_offers"):
with patch(
"com.services.client.CustomService.get_offers", new=offers
):
eo_offers = await CatalogModel().get_offers(self.catalog_name)
assert offers_response == eo_offers
However when executing the test it fails with the error
E vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/test_api/recordings/2021-07-12/test_get_offers_model/salesforce/auth/client/services_oauth2_token.yaml') in your current record mode ('none').
E No match for the request (<Request (POST) https://server.salesforce.com/services/oauth2/token>) was found.
E No similar requests, that have not been played, found.
During handling of the above exception, another exception occurred:
...model.py:17: in test_get_offers
eo_offers = await CatalogModel().get_offers(self.catalog_name)
model.py:24: in get_offers
raise SalesForceException()
E ...SalesForceException: Error from Salesforce.
As far as I understand it is trying to connect to real Salesforce service, rather than using a mock. What is the problem?
I would avoid trying to partially patch methods on classes. Instead I would use inversion of control to allow mock instances to be used during tests. This avoids having to do any patching at all.
class CatalogModel(BaseModel):
def __init__(self, custom_service=None):
if custom_service is None:
custom_service = CustomService()
self.custom_service = custom_service
async def get_offers(self, catalog_name):
try:
svc_response = self.custom_service.get_offers(catalog_name=catalog_name)()
except BaseException:
raise SalesForceException()
return CustomOfferResponse().dump(svc_response)
Now writing the test becomes much simpler.
class TestCatalogModel:
catalog_name = 'CATALOG_1'
#freeze_time("2021-07-12")
async def test_get_offers(self, loop, offers, offers_response):
with MockUser(ident="test_model_get_offers"):
custom_service = mock.Mock()
custom_service.get_offers.return_value = offers
model = CatalogModel(custom_service)
eo_offers = await model.get_offers(self.catalog_name)
assert offers_response == eo_offers
I am working a project in Python were I am gathering rain data and in this case temperature data from the Netatmo weather stations (Its basically just a private weather station you can set up in you garden and it will collect rain data, temperature, wind etc.)
When using the patatmo API you need a user with credential, a client. This client then has 500 requests pr. Hour which can be used on different requests, among these are the client.GetPublicdata request and the client.Getmeassure request. The Getmeassure request requires a station id and a module id, which I get from the the Getpuclicdat request. If I run out of requests I will catch that error using the except ApiResponseError: and I will then change the client credentials since I am not alone in this project and have credentials from two other people. My issues are:
If a station or module ID is not found using the Getmeassure request it will return a different type of ApiResponseError, which in my current code also is caught in the previously mentioned except, and this results in an endless loop where the code just changes credential all the time.
The code looks like this:
from patatmo.api.errors import ApiResponseError
...
...
...
a_test1 = 0
while a_test1 == 0:
try:
Outdoor_data = client.Getmeasure(
device_id = stations_id ,
module_id = modul_ID ,
type = typ ,
real_time = True ,
date_begin = Start ,
date_end = End
)
time.sleep(p)
a_test1 = 1
except ApiResponseError:
credentials = cred_dict[next(ns)]
client = patatmo.api.client.NetatmoClient()
client.authentication.credentials = credentials
client.authentication.tmpfile = 'temp_auth.json'
print('Changeing credentials')
credError = credError + 1
if credError > 4:
time.sleep(600)
credError = 0
pass
except:
print('Request Error')
time.sleep(p)
The documentation for the error.py script, that was made by someone else, looks like this:
class ApiResponseError(BaseException):
pass
class InvalidCredentialsError(ApiResponseError):
pass
class InvalidApiInputError(BaseException):
pass
class InvalidRegionError(InvalidApiInputError):
def __init__(self):
message = \
("'region' required keys: "
"lat_ne [-85;85], lat_sw [-85;85], "
"lon_ne [-180;180] and lon_sw [-180;180] "
"with lat_ne > lat_sw and lon_ne > lon_sw")
super().__init__(message)
class InvalidRequiredDataError(InvalidApiInputError):
def __init__(self):
message = "'required_data' must be None or in {}".format(
GETPUBLICDATA_ALLOWED_REQUIRED_DATA)
super().__init__(message)
class InvalidApiRequestInputError(BaseException):
pass
class InvalidPayloadError(InvalidApiRequestInputError):
pass
API_ERRORS = {
"invalid_client": InvalidCredentialsError("wrong credentials"),
"invalid_request": ApiResponseError("invalid request"),
"invalid_grant": ApiResponseError("invalid grant"),
"Device not found": ApiResponseError("Device not found - Check Device ID "
"and permissions")
}
What I want to do is catch the error depending what type of error I get, and I just doesn't seem to have any luck doing so
Which of those exceptions subclasses do you get on request overrun? Use only that one to drive the auth swap. If it’s not a particular one but is shared with other bad events, you will need to examine the exception’s variables. Also you will need to figure out which exception to work around - bad station id might mean someone is offline so ignore and try later. Vs logic flaws in your program, abend on those, fix, retry.
Exception handling is on a first-catch, first-handled basis. Your most specific classes have to do the “except” first to match first, else a generic one would grab it and handle it. Watch your APIs exception hierarchy carefully!
maxexc = 1000
countexc = 1
while ...
# slow your loop
time.sleep(p)
try:
... what you normally do
# dont use the too generic ApiResponseError here yet
except requestoverrunexception as e:
... swap credendentials
countexc+=1
except regionexception as e:
... ignore this region for a while
countexc += 1
# whoops abend and fix
except ApiResponseError as e:
print vars(e)
raise
except Exception as e:
print(e)
raise
I would bail after too many exceptions, thats what countexc is about.
Also 500 requests an hour seems generous. Don’t try to fudge that unless you have a real need. Some providers may even have watchdogs and get rid of you if you abuse them.
I need to validate a Flask rule in a REST API that is built using Flask-Rebar. I've tried the following method:
#registry.handles(
rule='/horses/<int:horse_id>',
method='GET',
marshal_schema=GetHorseByIdSchema()
)
def getHorseById(horse_id):
return {"horse_id": horse_id}
This is using the <int:my_var> syntax as specified here. When entering an integer value as the horse_id everything works correctly. The issue comes from entering a non-integer value, i.e. a; this throws a 404 Not Found status code when I was expecting a 400 Bad Request.
I don't think I can use any of the marshalling schemas for this so I'm unsure as to what to try next, any help is appreciated.
Thanks,
Adam
This is how Flask/Werkzeug behaves, so its a bit beyond Flask-Rebar's control. That is, the following will also return a 404 for /horses/a:
app = Flask(__name__)
#app.route('/horses/<int:horse_id>')
def getHorseById(horse_id):
return str(horse_id)
With that, here are some workarounds:
(1) Custom URL converter: http://flask.pocoo.org/docs/1.0/api/#flask.Flask.url_map
This would look something like:
import flask
from werkzeug.routing import BaseConverter
class StrictIntegerConverter(BaseConverter):
def to_python(self, value):
try:
return int(value)
except ValueError:
flask.abort(400)
app = flask.Flask(__name__)
app.url_map.converters['strict_integer'] = StrictIntegerConverter
#registry.handles(
rule='/horses/<strict_integer:horse_id>',
method='GET',
marshal_schema=GetHorseByIdSchema()
)
def getHorseById(horse_id):
return {'horse_id': horse_id}
However, routing is done outside of application context, so we can't use flask.jsonify nor Flask-Rebar's errors to raise nice JSON errors.
(2) Check the type inside the handler function
from flask_rebar.errors import BadRequest
#registry.handles(
rule='/horses/<horse_id>',
method='GET',
marshal_schema=GetHorseByIdSchema()
)
def getHorseById(horse_id):
try:
horse_id = int(horse_id)
except ValueError:
raise BadRequest('horse_id must be an integer')
return {'horse_id': horse_id}
This is a little less elegant, but it works.
The Swagger document will default to a string type for the horse_id parameter, but we can work around that as well:
from werkzeug.routing import UnicodeConverter
class StrDocumentedAsIntConverter(UnicodeConverter):
pass
app.url_map.converters['str_documented_as_int'] = StrDocumentedAsIntConverter
registry.swagger_generator.register_flask_converter_to_swagger_type('str_documented_as_int', 'int')
#registry.handles(rule='/horses/<str_documented_as_int:horse_id>')
...
(3) Accept Flask/Werkzeug behavior
Depending on how badly you need 400 instead of 404, it might be most practical to do nothing and just give in to how Flask/Werkzeug do it.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have a tiny web-server written in Python 3 using http.server which calls the function translate() in method do_GET() like this:
class httpd(BaseHTTPRequestHandler):
def do_GET(self):
self.wfile.write(bytes(f'{translate(var[0])}', 'utf-8'))
Now in this translate() function I have several conditional statements and try and except blocks roughly like this:
def translate(param):
try:
# do something
except SomeError as some_err:
print("Error: " % some_err)
return ""
if True:
try:
# do something
except SomeOtherError as some_other_err:
print("Error: " % some_other_err)
return ""
except SomeThirdError as some_third_err:
print("Third error: " % some_third_err)
return ""
else:
# additional try and except blocks which print an error and
# return an empty string
The code above is simplified, but in principle I return an empty string if an exception happens and thus my web server returns nothing to client if an exception happens.
Is there a more manageable way to handle this? Specifically, I'm looking to:
Avoid catching each error via a separate except section, while still supporting an error message dependent on error type.
Avoid writing multiple try / except statements, often nested, within my function.
Note: This is a copy of this now deleted question. The solution from that post is included below, but other answers are welcome.
I'm not sure if having nested try blocks is absolutely necessary in your logic, but I'd try a single try block with a custom Exception class. Something like this:
class MyException(Exception):
"""Generic error message."""
def __str__(self):
return self.__doc__
class SomeError(MyException):
"""SomeError message."""
pass
class SomeOtherError(MyException):
"""SomeOtherError message."""
pass
class SomeThirdError(MyException):
"""SomeThirdError message."""
pass
def translate(param):
try:
# do something
...
if cond1:
raise SomeError()
...
if cond2:
raise SomeOtherError()
...
if cond3:
raise SomeThirdError()
...
except MyException as err:
print(err)
return ""
How about contextmanager? To alleviate your concern about custom error messages, you can feed a dictionary mapping error classes to messages of your choice.
Since different operations require different errors to be handled, you can use multiple with statements, feeding different errors as arguments each time.
Here's a contrived example:
from contextlib import contextmanager
#contextmanager
def error_handling(msg, *exceptions):
try:
yield
except exceptions as my_error:
print(msg[my_error.__class__], my_error)
return ''
def do_stuff(d, key, index):
custom_msg = {IndexError: 'You have an Index Error!',
KeyError: 'You have a Key Error!'}
with error_handling(custom_msg, IndexError, KeyError):
return d[key][index]
# example prints "You have an Index Error! list index out of range"; then returns ''
do_stuff({'a': [0, 1, 2]}, 'a', 10)
How about to define a function that has a dictionary contained the whole errors and responses. in this case you can catch an exception one time and send it to a handler
UPDATE :
def handle (e) :
exp = {'IOError' : 'NO such a file in dir ...! ' ,
'KeyboardInterrupt' : 'Exiting ... (Keyboard interruption)',
'IndexError' : ' a sequence subscript is out of range' ,
'NameError' : 'a local or global name is not found'}
for key , value in exp.items() :
if e == key :
print (value) #or do whatever you want
def test() :
try :
f = open('no-file' , 'r')
except Exception as e :
handle (type(e).__name__)
if __name__ == "__main__" :
test()
I'm rather new to Python, but have grown to like it. I am starting our first Python project and am doing some prototyping. The "Python philosophy" confuses me in terms of typing and exceptions. Can someone please shoot at this excerpt? Am I over engineering or missing some fundamental Python methodology?
class URIPartError(Exception):
pass
class WCFClient(object):
def __init__(self, host, scheme='http', port=80, path='/', user=None, password=None):
super(WCFClient, self).__init__()
#Store our variables
try:
self.__host = str(host).lower()
self.__scheme = str(scheme).lower()
self.__port = int(port)
self.__path = str(path)
self.__user = str(user) if user else None
self.__password = str(password) if password else None
except (TypeError, ValueError), e:
raise URIPartError('Invalid URI part')
#Are our inputs valid?
if not self.__scheme == 'http' and not self.__scheme == 'https':
raise URIPartError('Invalid URI scheme')
if not path.startswith('/') or not path.endswith('/'):
raise URIPartError('Invalid URI path')
#Generate valid URI for baseurl
if (self.__scheme == 'http' and self.__port == 80) or (self.__scheme == 'https' and self.__port == 443):
self.__baseurl = '{0}://{1}{2}'.format(self.__scheme, self.__host, self.__path)
else:
self.__baseurl = '{0}://{1}:{2}{3}'.format(self.__scheme, self.__host, self.__port, self.__path)
def baseurl(self):
return self.__baseurl
Thanks!
There's nothing wrong here in terms of typing. You're not insisting on types, you're doing exactly the right thing by checking values - if a user passes invalid parameters, it's perfectly valid to raise exceptions.
The only comment I would make here is that it's very unPythonic to use "private" double-underscore variables together with getters. Rather than setting self.__baseurl and then providing a baseurl() method, just set self.baseurl directly.
Seeing as how your WCFClient inherits from object (which btw only makes sense if you are using Python 2.x, since in Python 3 this is default), it does not really make sense to call the constructor of the object class. This means that you can remove the line
super(WCFClient, self).__init__()
For the rest it is all ok :)