How can I wrap a boto.storage_uri() call in python so I can handle possible exceptions?
Your question about Boto is a good one, not not easy to answer. The Boto exception hierarchy is poorly designed, and ultimately the only way to determine what the exception you want to trap is requires looking at the boto source code.
For example if you look at (on Ubuntu) /usr/share/pyshared/boto/exception.py you will see that there are two broad classes:
boto.exception.BotoClientError
boto.exception.BotoServerError
Many of the exceptions are derived from these two, though the concept of "Client" and "Server" is not very well defined and you probably would want to check for both to be sure as many exceptions can happen unexpected (as usual). However, exceptions such as boto.exception.NoAuthHandlerFound is derived directly from Exception and therefore you would have to check for it separately.
Unfortunately from looking at the code, it appears that there is neither consistency nor much care in defining the exception hierarchy in Boto, which is a flaw in Boto's design which unfortunately requires that you rely on more broad exception checking than would normally be recommended.
The first question is what exceptions is this call likely to generate? You do not want to make a blanket exception handler in any language. You should first take a look at the Boto documentation to see if it documents the exceptions you might see from a given call, but if not I would first try:
try:
uri = boto.storage_uri()
except Exception, e:
print e
Or log the exception (with the logging package exception method), but either way you want to take note of what types of exceptions you see while you're testing and whether you should handle any of them specially. You also may want to review the Python Tutorial section on Exceptions and Errors.
Looking at the boto3 source, Boto3Error is declared as the base of all boto errors. So, you can probably do this:
try:
boto.storage_uri()
except Boto3Error:
# handle errors
Related
I've seen some questions that kind of get at what I'm going for, but haven't found anything that quite satisfies my question.
I'm creating a fairly large and complex web server using blueprints in Python Flask. I would like to, when I encounter an error (try... except...), be able to log the error, among other things. Writing the code to log the error and traceback in each exception seems to violate DRY.
I know I could just define a normal method that has inputs for Strings and whatnot, where I could pass in my exception and traceback, but this feels wrong to me for some reason.
So my question is, is there a way to be able to, when something triggers an error, raise another exception while preserving the traceback, where I can have my error handling logic (logging, showing errors to user, etc)?
I'd like my web server code to look like this:
try:
# some error-prone process
except:
raise errorHandler # where "errorHandler" is my custom error handler
Additionally, this project is Python 3, so I would prefer to use something that plays nice with Python 3.
Thanks!
I have a code, lets say :
'''
try:
somecode()
except Exception as e:
somelog()
'''
Is there a way to find out all the possible exceptions somecode() can throw so that I can handle them in an appropriate order.
While you may not always be able to know every error that may happen, you can do quite a bit by thinking of common cases. This link is a good starter's guide with examples:
https://www.pythonforbeginners.com/error-handling/exception-handling-in-python1
For raising exceptions you predict in your own functions, this is a good starter guide:
https://www.programiz.com/python-programming/user-defined-exception
Finally, when you're working with built in functions or packages, they usually document what exceptions they raise. For example, look at the built in page for Python
https://docs.python.org/3/library/functions.html
And ctrl-f ValueError. A lot of docs will tell you what exceptions they raise but beyond that it's up to you to anticipate and guess based on your implementation and usage.
Hope that helps!
There are not so many exception types which you may have to consider for a single case. In case you are trying to access a file or accessing the database, the options are very few. The best practice is to keep track with the documentation. This will not take much time to know the name of the exception.
https://docs.python.org/3/tutorial/errors.html
I'm trying to write a highly-reliable piece of code in Python. The most common issue I run into is that after running for a while, some edge case will occur and raise an exception that I haven't handled. This most often happens when using external libraries - without reading the source code, I don't know of an easy way to get a list of all exceptions that might be raised when using a specific library function, so it's hard to know what to handle. I understand that it's bad practice to use catch-all handlers, so I haven't been doing that.
Is there a good solution for this? It seems like it should be possible for a static analysis tool to check that all exceptions are handled, but I haven't found one. Does this exist? If not, why? (is it impossible? a bad idea? etc) I especially would like it to analyze imported code for the reason explained above.
"it's bad practice to use catch-all handlers" to ignore exceptions:
Our web service has an except which wraps the main loop.
except:
log_exception()
attempt_recovery()
This is good, as it notifies us (necessary) of the unexpected error and then tries to recover (not necessary). We can then go look at those logs and figure out what went wrong so we can prevent it from hitting our general exception again.
This is what you want to avoid:
except:
pass
Because it ignores the error... then you don't know an error happened and your data may be corrupted/invalid/gone/stolen by bears. Your server may be up/down/on fire. We have no idea because we ignored the exception.
Python doesn't require registering of what exceptions might be thrown, so there are no checks for all exceptions a module might throw, but most will give you some idea of what you should be ready to handle in the docs. Depending on your service, when it gets an unhandled exception, you might want to:
Log it and crash
Log it and attempt to continue
Log it and restart
Notice a trend? The action changes, but you never want to ignore it.
Great question.
You can try approaching the problem statically (by, for instance, introducing a custom flake8 rule?), but, I think, it's a problem of testing and test coverage scope. I would approach the problem by adding more "negative path"/"error handling" checks/tests to the places where third-party packages are used, introducing mock side effects whenever needed and monitoring coverage reports at the same time.
I would also look into Mutation Testing idea (check out Cosmic Ray Python package). I am not sure if mutants can be configured to also throw exceptions, but see if it could be helpful.
Instead of trying to handle all exceptions, which is very hard as you described, why don't you catch all, but exclude some, for example the KeyboardInterrupt you mentioned in the comments?
This may help
I just wonder, when using try.. except, what's the difference between using the Base class "Except" and using a specific exception like "ImportError" or "IOError" or any other specific exception. Are there pros and cons between one and the other?
Never catch the base exception. Always capture only the specific exceptions that you know how to handle. Everything else should be left alone; otherwise you are potentially hiding important errors.
Of course it has advantages using the correct exception for the corresponding issue. However Python already defined all possible errors for coding problems. But you can make your own exception classes by inheriting from the Exception class. This way you can make more meaningful errors for spesific parts of your code. You can even make the expections print errors like this.
SomeError: 10 should have been 5.
Provides easier debugging of the code.
For more info.
In Python, I've read that it's better (and easier) to catch execptions, rather than check first, so that's what I'm trying to do.
My script opens and parses an XMLs file using
xml.dom.minidom.parse(xml_file_path)
so I'm catching
xml.parsers.expat.ExpatError
but if the file doesn't exist, I get a FileNotFoundError exception, so I obviously need to catch that too.
I know that I shouldn't really be catching all exceptions, but how can I know what exceptions I should be catching for a function like parse()?
You can consult documentation of library you use. And even better you can write a test where you trigger exception first. Then you will know exactly what exception you need to catch (and have another test to guard you in the future).
A good place to start are the 'Built-in Exceptions' of Python. With some study you can tell what operation can cause which exception. The documentation is also particularly useful as it provides practical examples.
Specifically in your case, you are performing file Input/Output operations (IO) and IO operations are handled generally as:
try:
with open('path_to_file', 'permission') as f:
#do something with the file
except IOError as e:
print e