Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have a tiny web-server written in Python 3 using http.server which calls the function translate() in method do_GET() like this:
class httpd(BaseHTTPRequestHandler):
def do_GET(self):
self.wfile.write(bytes(f'{translate(var[0])}', 'utf-8'))
Now in this translate() function I have several conditional statements and try and except blocks roughly like this:
def translate(param):
try:
# do something
except SomeError as some_err:
print("Error: " % some_err)
return ""
if True:
try:
# do something
except SomeOtherError as some_other_err:
print("Error: " % some_other_err)
return ""
except SomeThirdError as some_third_err:
print("Third error: " % some_third_err)
return ""
else:
# additional try and except blocks which print an error and
# return an empty string
The code above is simplified, but in principle I return an empty string if an exception happens and thus my web server returns nothing to client if an exception happens.
Is there a more manageable way to handle this? Specifically, I'm looking to:
Avoid catching each error via a separate except section, while still supporting an error message dependent on error type.
Avoid writing multiple try / except statements, often nested, within my function.
Note: This is a copy of this now deleted question. The solution from that post is included below, but other answers are welcome.
I'm not sure if having nested try blocks is absolutely necessary in your logic, but I'd try a single try block with a custom Exception class. Something like this:
class MyException(Exception):
"""Generic error message."""
def __str__(self):
return self.__doc__
class SomeError(MyException):
"""SomeError message."""
pass
class SomeOtherError(MyException):
"""SomeOtherError message."""
pass
class SomeThirdError(MyException):
"""SomeThirdError message."""
pass
def translate(param):
try:
# do something
...
if cond1:
raise SomeError()
...
if cond2:
raise SomeOtherError()
...
if cond3:
raise SomeThirdError()
...
except MyException as err:
print(err)
return ""
How about contextmanager? To alleviate your concern about custom error messages, you can feed a dictionary mapping error classes to messages of your choice.
Since different operations require different errors to be handled, you can use multiple with statements, feeding different errors as arguments each time.
Here's a contrived example:
from contextlib import contextmanager
#contextmanager
def error_handling(msg, *exceptions):
try:
yield
except exceptions as my_error:
print(msg[my_error.__class__], my_error)
return ''
def do_stuff(d, key, index):
custom_msg = {IndexError: 'You have an Index Error!',
KeyError: 'You have a Key Error!'}
with error_handling(custom_msg, IndexError, KeyError):
return d[key][index]
# example prints "You have an Index Error! list index out of range"; then returns ''
do_stuff({'a': [0, 1, 2]}, 'a', 10)
How about to define a function that has a dictionary contained the whole errors and responses. in this case you can catch an exception one time and send it to a handler
UPDATE :
def handle (e) :
exp = {'IOError' : 'NO such a file in dir ...! ' ,
'KeyboardInterrupt' : 'Exiting ... (Keyboard interruption)',
'IndexError' : ' a sequence subscript is out of range' ,
'NameError' : 'a local or global name is not found'}
for key , value in exp.items() :
if e == key :
print (value) #or do whatever you want
def test() :
try :
f = open('no-file' , 'r')
except Exception as e :
handle (type(e).__name__)
if __name__ == "__main__" :
test()
Related
I have this code structure:
#########file1.py#############
def newsCSVwriter(fileName):
try:
newsCleaner(fileName)
except Exception as e:
print "Exception: ", e
########file1.py#############
def newsCleaner(newsFile):
....
#########file2.py###########
try:
df1['newsFile'].apply(newsCSVwriter)
except Exception as e:
print "exception:",e
I want to write a csv that has a status column value of yes or no depending on whether newsCleaner(fileName) returns a value or exception. Should I implement the logic in file1 or file2? Also, an example will be great.
Assuming you don't actually need the returned value, in your newCSVWriter function do this:
try:
newsCleaner(fileName)
except:
return 'no'
else:
return 'yes'
How you structure your code in terms of files is dependant on what all of it does, but you've only posted part of it.
Stylistically I would rename them to something more informative than 'file1' and 'file2'. I would also have the function return a bool (True or False) instead, but that's up to you.
I have always wondered who takes the control of the program after an exception has thrown. I was seeking for a clear answer but did not find any. I have the following functions described, each one executes an API call which involves a network request, therefore I need to handle any possible errors by a try/except and possibly else block (JSON responses must be parsed/decoded as well):
# This function runs first, if this fails, none of the other functions will run. Should return a JSON.
def get_summary():
pass
# Gets executed after get_summary. Should return a string.
def get_block_hash():
pass
# Gets executed after get_block_hash. Should return a JSON.
def get_block():
pass
# Gets executed after get_block. Should return a JSON.
def get_raw_transaction():
pass
I wish to implement a kind of retry functionality on each function, so if it fails due to a timeout error, connection error, JSON decode error etc., it will keep retrying without compromising the flow of the program:
def get_summary():
try:
response = request.get(API_URL_SUMMARY)
except requests.exceptions.RequestException as error:
logging.warning("...")
#
else:
# Once response has been received, JSON should be
# decoded here wrapped in a try/catch/else
# or outside of this block?
return response.text
def get_block_hash():
try:
response = request.get(API_URL + "...")
except requests.exceptions.RequestException as error:
logging.warning("...")
#
else:
return response.text
def get_block():
try:
response = request.get(API_URL + "...")
except requests.exceptions.RequestException as error:
logging.warning("...")
#
else:
#
#
#
return response.text
def get_raw_transaction():
try:
response = request.get(API_URL + "...")
except requests.exceptions.RequestException as error:
logging.warning("...")
#
else:
#
#
#
return response.text
if __name__ == "__main__":
# summary = get_summary()
# block_hash = get_block_hash()
# block = get_block()
# raw_transaction = get_raw_transaction()
# ...
I want to keep clean code on the outermost part of it (block after if __name__ == "__main__":), I mean, I don't want to fill it with full of confused try/catch blocks, logging, etc.
I tried to call a function itself when an exception threw on any of those functions but then I read about stack limit and thought it was a bad idea, there should be a better way to handle this.
request already retries by itself N number of times when I call the get method, where N is a constant in the source code, it is 100. But when the number of retries has reached 0 it will throw an error I need to catch.
Where should I decode JSON response? Inside each function and wrapped by another try/catch/else block? or in the main block? How can I recover from an exception and keep trying on the function it failed?
Any advice will be grateful.
You could keep those in an infinite loop (to avoid recursion) and once you get the expected response just return:
def get_summary():
while True:
try:
response = request.get(API_URL_SUMMARY)
except requests.exceptions.RequestException as error:
logging.warning("...")
#
else:
# As winklerrr points out, try to return the transformed data as soon
# as possible, so you should be decoding JSON response here.
try:
json_response = json.loads(response)
except ValueError as error: # ValueError will catch any error when decoding response
logging.warning(error)
else:
return json_response
This function keeps executing until it receives the expected result (reaches return json_response) otherwise it will be trying again and again.
You can do the following
def my_function(iteration_number=1):
try:
response = request.get(API_URL_SUMMARY)
except requests.exceptions.RequestException:
if iteration_number < iteration_threshold:
my_function(iteration_number+1)
else:
raise
except Exception: # for all other exceptions, raise
raise
return json.loads(resonse.text)
my_function()
Where should I decode JSON response?
Inside each function and wrapped by another try/catch/else block or in the main block?
As a rule thumb: try to transform data as soon as possible into the format you want it to be. It makes the rest of your code easier if you don't have to extract everything again from a response object all the time. So just return the data you need, in the easiest format you need it to be.
In your scenario: You call that API in every function with the same call to requests.get(). Normally all the responses from an API have the same format. So this means, you could write an extra function which does that call for you to the API and directly loads the response into a proper JSON object.
Tip: For working with JSON make use of the standard library with import json
Example:
import json
def call_api(api_sub_path):
repsonse = requests.get(API_BASE_URL + api_sub_path)
json_repsonse = json.loads(repsonse.text)
# you could verify your result here already, e.g.
if json_response["result_status"] == "successful":
return json_response["result"]
# or maybe throw an exception here, depends on your use case
return json_response["some_other_value"]
How can I recover from an exception and keep trying on the function it failed?
You could use a while loop for that:
def main(retries=100): # default value if no value is given
result = functions_that_could_fail(retries)
if result:
logging.info("Finished successfully")
functions_that_depend_on_result_from_before(result)
else:
logging.info("Finished without result")
def functions_that_could_fail(retry):
while(retry): # is True as long as retry is bigger than 0
try:
# call all functions here so you just have to write one try-except block
summary = get_summary()
block_hash = get_block_hash()
block = get_block()
raw_transaction = get_raw_transaction()
except Exception:
retry -= 1
if retry:
logging.warning("Failed, but trying again...")
else:
# else gets only executed when no exception was raised in the try block
logging.info("Success")
return summary, block_hash, block, raw_transaction
logging.error("Failed - won't try again.")
result = None
def functions_that_depend_on_result_from_before(result):
[use result here ...]
So with the code from above you (and maybe also some other people who use your code) could start your program with:
if __name__ == "__main__":
main()
# or when you want to change the number of retries
main(retries=50)
I am creating a Bot for Reddit. I currently only have 1 very large function and I am looking to create sub-functions to make it more readable.
Here is a rough break-down of what it does
def replybot():
submissions = reversed(list(subreddit.get_new(limit=MAXPOSTS)))
for post in submissions:
try:
author = post.author.name
except AttributeError:
print "AttributeError: Author is deleted"
continue # Author is deleted. We don't care about this post.
# DOES PID EXIST IN DB? IF NOT ADD IT
cur.execute('SELECT * FROM oldposts WHERE ID=?', [pid])
sql.commit()
if cur.fetchone(): # Post is already in the database
continue
cur.execute('INSERT INTO oldposts VALUES(?)', [pid])
sql.commit()
...
I am looking to break the code up into segments i.e. put
try:
author = post.author.name
except AttributeError:
print "AttributeError: Author is deleted"
continue # Author is deleted. We don't care about this post.
in it's own function and call it from within replybot() but I run into the issue of calling continue. I get SyntaxError: 'continue' not properly in loop
Is there a way for me to do this?
If you take the inner part of a loop and convert it to its own function, it's no longer in a loop. The equivalent of continue in a loop, for a function, is return (i.e. terminate this iteration (which is now a function call) early).
Raise the error again instead of trying to continue. Either simply let it bubble to the main loop, or if you need better error handling, make your own custom error. For instance:
class MyNotFatalError(Exception):
pass
def do_something():
try:
a, b, c = 1, 2
except ValueError:
raise MyNotFatalError('Something went wrong')
# In your main function
for post in submissions:
try:
do_something()
do_some_other_thing()
do_some_more()
except MyNotFatalError as err:
continue # we could print out the error text here
do_some_last_thing()
It is probably better that way because you only catch errors you know you want to catch, and still let the program crash when there are actual bugs.
If you had simply caught ValueError that would also intercept and hide all other possible sources of the same kind of error.
as Claudiu said, when you broke inner commands into it's own function; It's not no longer in the loop and your code will be look like this:
def isNotAuthorDeleted(post):
try:
author = post.author.name
return author
except AttributeError:
print "AttributeError: Author is deleted"
return false
and your loop will be:
for post in submissions:
if not isNotAuthorDeleted(post):
continue
If I catch a KeyError, how can I tell what lookup failed?
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError:
# How can I tell what key ("FastestMan" or "FastestWoman") caused the error?
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key in JSON")
Take the current exception (I used it as e in this case); then for a KeyError the first argument is the key that raised the exception. Therefore we can do:
except KeyError as e: # One would do it as 'KeyError, e:' in Python 2.
cause = e.args[0]
With that, you have the offending key stored in cause.
Expanding your sample code, your log might look like this:
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError as e:
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key '"
e.args[0]
"' in JSON")
It should be noted that e.message works in Python 2 but not Python 3, so it shouldn't be used.
Not sure if you're using any modules to assist you - if the JSON is coming in as a dict, one can use dict.get() towards a useful end.
def POIJSON2DOM (location_node, POI_JSON):
man_JSON = POI_JSON.get("FastestMan", 'No Data for fastest man')
woman_JSON = POI_JSON.get("FastestWoman", 'No Data for fastest woman')
#work with the answers as you see fit
dict.get() takes two arguments - the first being the key you want, the second being the value to return if that key does not exist.
If you import the sys module you can get exception info with sys.exc_info()
like this:
def POIJSON2DOM (location_node, POI_JSON):
try:
man_JSON = POI_JSON["FastestMan"]
woman_JSON = POI_JSON["FastestWoman"]
except KeyError:
# you can inspect these variables for error information
err_type, err_value, err_traceback = sys.exc_info()
REDI.LogErrorMessage ("POIJSON2DOM", "Can't find mandatory key in JSON")
I have a function with a try catch block where I have:
def testfunction():
try:
good = myfunction()
return good
except ExceptionOne:
error="Error ExceptionOne".
return error
except ExceptionTwo:
error="Error ExceptionTwo".
return error
except ExceptionThree:
error="Error ExceptionThree".
return error
Is there a way such that I could structure this in such a way where have a single return statement for all the exceptions?
What about something like:
def testfunction():
try:
good = myfunction() # or `return myfunction()` if the case is actually this simple
except (ExceptionOne, ExceptionTwo, ExceptionThree) as err:
error = 'Error %s' % err.__class__.__name__
return error
return good
Of course, I'm not exactly sure what the purpose of this is. Why not just let the exception propogate and handle it higher up? As it is, you need logic to check the return value to see if it's good or not anyway, I think it would be cleaner if that logic was bound up in exception handling rather than in type checking or string value checking ...