Python: statically detect unhandled exceptions - python

I'm trying to write a highly-reliable piece of code in Python. The most common issue I run into is that after running for a while, some edge case will occur and raise an exception that I haven't handled. This most often happens when using external libraries - without reading the source code, I don't know of an easy way to get a list of all exceptions that might be raised when using a specific library function, so it's hard to know what to handle. I understand that it's bad practice to use catch-all handlers, so I haven't been doing that.
Is there a good solution for this? It seems like it should be possible for a static analysis tool to check that all exceptions are handled, but I haven't found one. Does this exist? If not, why? (is it impossible? a bad idea? etc) I especially would like it to analyze imported code for the reason explained above.

"it's bad practice to use catch-all handlers" to ignore exceptions:
Our web service has an except which wraps the main loop.
except:
log_exception()
attempt_recovery()
This is good, as it notifies us (necessary) of the unexpected error and then tries to recover (not necessary). We can then go look at those logs and figure out what went wrong so we can prevent it from hitting our general exception again.
This is what you want to avoid:
except:
pass
Because it ignores the error... then you don't know an error happened and your data may be corrupted/invalid/gone/stolen by bears. Your server may be up/down/on fire. We have no idea because we ignored the exception.
Python doesn't require registering of what exceptions might be thrown, so there are no checks for all exceptions a module might throw, but most will give you some idea of what you should be ready to handle in the docs. Depending on your service, when it gets an unhandled exception, you might want to:
Log it and crash
Log it and attempt to continue
Log it and restart
Notice a trend? The action changes, but you never want to ignore it.

Great question.
You can try approaching the problem statically (by, for instance, introducing a custom flake8 rule?), but, I think, it's a problem of testing and test coverage scope. I would approach the problem by adding more "negative path"/"error handling" checks/tests to the places where third-party packages are used, introducing mock side effects whenever needed and monitoring coverage reports at the same time.
I would also look into Mutation Testing idea (check out Cosmic Ray Python package). I am not sure if mutants can be configured to also throw exceptions, but see if it could be helpful.

Instead of trying to handle all exceptions, which is very hard as you described, why don't you catch all, but exclude some, for example the KeyboardInterrupt you mentioned in the comments?
This may help

Related

How can I know if a Python will potentially throw an exception in the Spyder IDE?

When running some code I had written I was surprised to see that a function threw an exception that was not caught and so my code crashed. The thing is, Spyder never informed me that the function could even throw an exception.
Is there a setting somewhere that I have to turn on to be informed of this?
Python isn't Java. Your IDE will not warn you about uncaught exceptions, because Python's dynamic nature means that exceptions could be raised (or caught) almost anywhere and there's no amount of static analysis that'll work for every -- or even most -- cases. Often when you develop Flask or Django applications, you actually want exceptions to float all the way up to the "root" exception handlers.
Bottom line: No Python IDE is going to do this, and it would not be expected or considered generally desirable in Python programming.

Best practices for creating custom exception handling without losing traceback?

I've seen some questions that kind of get at what I'm going for, but haven't found anything that quite satisfies my question.
I'm creating a fairly large and complex web server using blueprints in Python Flask. I would like to, when I encounter an error (try... except...), be able to log the error, among other things. Writing the code to log the error and traceback in each exception seems to violate DRY.
I know I could just define a normal method that has inputs for Strings and whatnot, where I could pass in my exception and traceback, but this feels wrong to me for some reason.
So my question is, is there a way to be able to, when something triggers an error, raise another exception while preserving the traceback, where I can have my error handling logic (logging, showing errors to user, etc)?
I'd like my web server code to look like this:
try:
# some error-prone process
except:
raise errorHandler # where "errorHandler" is my custom error handler
Additionally, this project is Python 3, so I would prefer to use something that plays nice with Python 3.
Thanks!

Python Exception Handling : Is there a method to know what type of exceptions my code can possibly throw?

I have a code, lets say :
'''
try:
somecode()
except Exception as e:
somelog()
'''
Is there a way to find out all the possible exceptions somecode() can throw so that I can handle them in an appropriate order.
While you may not always be able to know every error that may happen, you can do quite a bit by thinking of common cases. This link is a good starter's guide with examples:
https://www.pythonforbeginners.com/error-handling/exception-handling-in-python1
For raising exceptions you predict in your own functions, this is a good starter guide:
https://www.programiz.com/python-programming/user-defined-exception
Finally, when you're working with built in functions or packages, they usually document what exceptions they raise. For example, look at the built in page for Python
https://docs.python.org/3/library/functions.html
And ctrl-f ValueError. A lot of docs will tell you what exceptions they raise but beyond that it's up to you to anticipate and guess based on your implementation and usage.
Hope that helps!
There are not so many exception types which you may have to consider for a single case. In case you are trying to access a file or accessing the database, the options are very few. The best practice is to keep track with the documentation. This will not take much time to know the name of the exception.
https://docs.python.org/3/tutorial/errors.html

Catching all exceptions in order to debug more easily once my application is deployed

Novice programmer here. I'm phrasing this question in terms of Python, but I think it makes sense for any programming language that has the concept of Exception Handling.
Here it is: I'm writing a software for a business client, who will interact with the software by using a GUI.
I'm thinking about putting all of the core business logic in a try/except block in order to catch all exceptions that I haven't planned for (and which I catch and resolve before they bubble up). These could be bugs in other libraries that I'm using, my own bugs, or other unforeseen events. The output of those exceptions I want then to display in a GUI error message.
The reasoning behind this will be that if such an error were to occur once my software is deployed and the client would call me telling me "hey, I got this error message that says X", then I would have at least some hint from X as to what went wrong and could immediately start debugging - rather then the GUI just exiting silently and the client calling me and telling "your app just died".
1) Would this be a good idea? Since this software will be used in a "safe setting" there is no issue of sensitive information leaking into the error message, which then is displayed. But I'm still unsure, of there aren't any other problems I might run into with this approach.
2) If I should go down this route, should I use except BaseException or except Exception?
Even after consulting Python's exception hierarchy I don't really get when the three additional exceptions that I'd get if I were to use the first option are really thrown. My application won't use the keyboard, so I think I'm safe regarding KeyboardInterrupt, but I'm unsure about the other two,SystemExit and GeneratorExit.
Show the error messages in your gui only if the user needs to know, such as when the program cannot recover from an error.
Use except BaseException to avoid catching exceptions that need to be handled differently. If you need to catch every single type of exception, try to make as many except blocks as possible in order to recover from as many errors as possible.

Treating language errors and runtime errors differently in python

I have a decent sized code in python which I changed a bit and now I see that when executing the code, the script does not bail on language errors like missing function definitions.
I did not think it was possible to continue running a script with missing function definition. I used a function in one place with the intention of actually defining the function before running but forgot about it and ran the code. To my surprise I just got the following line printed to the console --
Worker instance has no attribute update_parse_status
Where Worker is the class I was supposed to add the update_parse_status() call to.
I realized that I had a try and a generic catch all exception handler around the code in question like this --
try:
Worker.update_parse_status()
except Exception as e:
print e
So python just throws a AttributeError and I was unknowingly catching it. It taught me a valuable lesson to not have catch all exception handlers. Also coming from a compiled language is there a better way to handle this? I mean, can I some how make sure python just exits on language errors? It will be very helpful to atleast weed out the obvious bugs (though I understand that bad code got me into this situation).
In python all names are looked up at runtime. Therefor, what you refer to as "language" errors are no different than runtime errors.
This makes python different than many other languages. One of the advantages of that fact is that you can easily customize the way names are looked up (e.g. by overriding class's __getattr__) for example.
You can make use of analyzers (e.g. pyflakes is pretty cool, but there are many others) to detect some of the errors before running your program, but no tool would be able to detect all.
You already answered your own question.
Don't catch exceptions if you don't need too ...
except Exception is dangerous
if you must catch the exception but use some logging mechanism to track it down.
Finally, make sure python just exits on errors: don't catch exceptions where it is not needed to guarantee the flow of the program.
Python, being an interpreter, does this. The best way to counter this is unit testing. The unittest module can help here, and setuptools can run these tests for you. You can package these with your source distribution, as well.
Of course, as you discovered, never use the catch-all exception. That also masks errors that unit tests could catch. You can also run your code under a debugger when developing it so this can be tracked more easily as well. Using the catch-all also makes debugging harder (or even impossible).
You can also do static analysys, using tools such as pylint or pyflakes.

Categories

Resources