In Python, I've read that it's better (and easier) to catch execptions, rather than check first, so that's what I'm trying to do.
My script opens and parses an XMLs file using
xml.dom.minidom.parse(xml_file_path)
so I'm catching
xml.parsers.expat.ExpatError
but if the file doesn't exist, I get a FileNotFoundError exception, so I obviously need to catch that too.
I know that I shouldn't really be catching all exceptions, but how can I know what exceptions I should be catching for a function like parse()?
You can consult documentation of library you use. And even better you can write a test where you trigger exception first. Then you will know exactly what exception you need to catch (and have another test to guard you in the future).
A good place to start are the 'Built-in Exceptions' of Python. With some study you can tell what operation can cause which exception. The documentation is also particularly useful as it provides practical examples.
Specifically in your case, you are performing file Input/Output operations (IO) and IO operations are handled generally as:
try:
with open('path_to_file', 'permission') as f:
#do something with the file
except IOError as e:
print e
Related
Just a quick little question about coding in general.
Say you're using a try-catch block, but all you do with the exception is print it to stderr. In this case, are you better off simply letting the error happen and letting it print on its own? Or is it still better to catch the exception so that it is documented for other coders?
In languages like Java, there is a "throws exception", but as far as I can tell Python has nothing of that kind.
Thanks
If we don't catch exception, then the normal flow of control will break where the exception occurred and will jump to the point where the exception is caught or the program will just terminate after printing the exception.
Not catching exception might be ok for simple programs, but for complex applications where we have multiple methods and libraries, we would wish to catch the exception where in the part of the program where it occurs and handle it according to our need. In web applications we may handle the exception and display user understandable error message.
In general, its always better to handle the exception even if we just print it to the logs.
I would always handle and catch the exception. No matter, whether small or large project. Because by catching an exception, you can give the user precise information WHAT he e.g. entered incorrectly (too small / too large a number ...) So the entire program code does not always have to be run through, but it left at the point where the exception is thrown.
In python you can throw like in java (throw exception) but with raise exception.
Here a small, understanding introduce for python exception. So I learned the exception in python. Good luck!
I have a code, lets say :
'''
try:
somecode()
except Exception as e:
somelog()
'''
Is there a way to find out all the possible exceptions somecode() can throw so that I can handle them in an appropriate order.
While you may not always be able to know every error that may happen, you can do quite a bit by thinking of common cases. This link is a good starter's guide with examples:
https://www.pythonforbeginners.com/error-handling/exception-handling-in-python1
For raising exceptions you predict in your own functions, this is a good starter guide:
https://www.programiz.com/python-programming/user-defined-exception
Finally, when you're working with built in functions or packages, they usually document what exceptions they raise. For example, look at the built in page for Python
https://docs.python.org/3/library/functions.html
And ctrl-f ValueError. A lot of docs will tell you what exceptions they raise but beyond that it's up to you to anticipate and guess based on your implementation and usage.
Hope that helps!
There are not so many exception types which you may have to consider for a single case. In case you are trying to access a file or accessing the database, the options are very few. The best practice is to keep track with the documentation. This will not take much time to know the name of the exception.
https://docs.python.org/3/tutorial/errors.html
I have some code that is calling several methods from multiple functions. Some of these methods may raise an exception. Each can raise a different type of exception. In my main script, I would like to log the exception and then just exit. So, no matter what the exception type, log it and exit
My question is this: is it better to list all the exceptions that might be generated, or just catch a generic Exception? Which is more pythonic?
example:
try:
some_stuff()
except (Exc1, Exc2, Exc3) as exc:
loger.exception(exc)
or this:
try:
some_stuff()
except Exception as exc:
loger.exception(exc)
Your plan to catch exception in main code, log it and terminate is good one.
There could be exceptions, which are fine and do not mean, you shall consider them as problem, e.g. KeyboardInterrupt
The strategy could be:
first, catch all the exceptions, which you expect to be fine and pass
then catch general Exception, log it and terminate.
The code could look like:
try:
some(stuff) # ...
# First, resolve expected exceptions
except KeyboardInterrupt:
pass
# Finally, log unexpected ones
except Exception as exc:
logger.exception(exc)
return # time to terminate
When is catching exceptions explicitly a failure
The advice to be better by explicitly catching all expected exceptions comes short in case an unexpected exception happens. Your plan to catch whatever comes to log file sounds good and provide enough information for resolving the problems later on.
Imagine, you have a daemon, which shall run and run. Under some conditions, it might fail.
With only expecting explicit exception, it may happen, unexpected exception happens, no expect
would have a chance to log this to a log file, stacktrace would be printed to stdout and forgotten
and program terminates.
This is clear and very reasonable use case for catching exception generally.
From the documentation:
Because except: catches all exceptions, including SystemExit,
KeyboardInterrupt, and GeneratorExit (which is not an error and should
not normally be caught by user code), using a bare except: is almost
never a good idea. In situations where you need to catch all “normal”
errors, such as in a framework that runs callbacks, you can catch the
base class for all normal exceptions, Exception. Unfortunately in Python 2.x it is
possible for third-party code to raise exceptions that do not inherit from Exception, so
in Python 2.x there are some cases where you may have to use a bare except: and manually > re-raise the exceptions you don’t want to catch.
In general, it is better to catch explicit exceptions. In Python 2, how you are doing it can result in exceptions you still don't catch if an external module is throwing something that doesn't inherit exception.
By catching explicit exceptions you are able to handle errors you know can occur. If you catch all, your application could do something unexpected and you may handle it incorrectly.
Additionally, do you really want to catch someone using Ctrl+C to end your program?
In all languages, times, and places it is better to specify specific exceptions. That way you won't mask a condition you didn't expect to. NEVER catch Exception in production code unless you have a very, very good reason, or your handler is truly generic. An example of a suitably generic handler is one which logs, performs cleanup, and reraises.
Take a note out of Tim Peter's book:
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
...
Explicit is better than implicit. It's more "pythonic" to write out the possible exceptions.
When using PyCharm IDE the use of except: without an exception type triggers a reminder from the IDE that this exception clause is Too broad.
Should I be ignoring this advice? Or is it Pythonic to always specific the exception type?
It's almost always better to specify an explicit exception type. If you use a naked except: clause, you might end up catching exceptions other than the ones you expect to catch - this can hide bugs or make it harder to debug programs when they aren't doing what you expect.
For example, if you're inserting a row into a database, you might want to catch an exception that indicates that the row already exists, so you can do an update.
try:
insert(connection, data)
except:
update(connection, data)
If you specify a bare except:, you would also catch a socket error indicating that the database server has fallen over. It's best to only catch exceptions that you know how to handle - it's often better for the program to fail at the point of the exception than to continue but behave in weird unexpected ways.
One case where you might want to use a bare except: is at the top-level of a program you need to always be running, like a network server. But then, you need to be very careful to log the exceptions, otherwise it'll be impossible to work out what's going wrong. Basically, there should only be at most one place in a program that does this.
A corollary to all of this is that your code should never do raise Exception('some message') because it forces client code to use except: (or except Exception: which is almost as bad). You should define an exception specific to the problem you want to signal (maybe inheriting from some built-in exception subclass like ValueError or TypeError). Or you should raise a specific built-in exception. This enables users of your code to be careful in catching just the exceptions they want to handle.
You should not be ignoring the advice that the interpreter gives you.
From the PEP-8 Style Guide for Python :
When catching exceptions, mention specific exceptions whenever
possible instead of using a bare except: clause.
For example, use:
try:
import platform_specific_module
except ImportError:
platform_specific_module = None
A bare except: clause will catch SystemExit and KeyboardInterrupt exceptions, making it harder to
interrupt a program with Control-C, and can disguise other problems.
If you want to catch all exceptions that signal program errors, use
except Exception: (bare except is equivalent to except
BaseException:).
A good rule of thumb is to limit use of bare 'except' clauses to two
cases:
If the exception handler will be printing out or logging the
traceback; at least the user will be aware that an error has occurred.
If the code needs to do some cleanup work, but then lets the exception
propagate upwards with raise. try...finally can be a better way to
handle this case.
Not specfic to Python this.
The whole point of exceptions is to deal with the problem as close to where it was caused as possible.
So you keep the code that could in exceptional cirumstances could trigger the problem and the resolution "next" to each other.
The thing is you can't know all the exceptions that could be thrown by a piece of code. All you can know is that if it's a say a file not found exception, then you could trap it and to prompt the user to get one that does or cancel the function.
If you put try catch round that, then no matter what problem there was in your file routine (read only, permissions, UAC, not really a pdf, etc), every one will drop in to your file not found catch, and your user is screaming "but it is there, this code is crap"
Now there are a couple of situation where you might catch everything, but they should be chosen consciously.
They are catch, undo some local action (such as creating or locking a resource, (opening a file on disk to write for instance), then you throw the exception again, to be dealt with at a higher level)
The other you is you don't care why it went wrong. Printing for instance. You might have a catch all round that, to say There is some problem with your printer, please sort it out, and not kill the application because of it. Ona similar vain if your code executed a series of separate tasks using some sort of schedule, you wouldnlt want the entire thing to die, because one of the tasks failed.
Note If you do the above, I can't recommend some sort of exception logging, e.g. try catch log end, highly enough.
Always specify the exception type, there are many types you don't want to catch, like SyntaxError, KeyboardInterrupt, MemoryError etc.
You will also catch e.g. Control-C with that, so don't do it unless you "throw" it again. However, in that case you should rather use "finally".
Here are the places where i use except without type
quick and dirty prototyping
That's the main use in my code for unchecked exceptions
top level main() function, where i log every uncaught exception
I always add this, so that production code does not spill stacktraces
between application layers
I have two ways to do it :
First way to do it : when a higher level layer calls a lower level function, it wrap the calls in typed excepts to handle the "top" lower level exceptions. But i add a generic except statement, to detect unhandled lower level exceptions in the lower level functions.
I prefer it this way, i find it easier to detect which exceptions should have been caught appropriately : i "see" the problem better when a lower level exception is logged by a higher level
Second way to do it : each top level functions of lower level layers have their code wrapped in a generic except, to it catches all unhandled exception on that specific layer.
Some coworkers prefer this way, as it keeps lower level exceptions in lower level functions, where they "belong".
Try this:
try:
#code
except ValueError:
pass
I got the answer from this link, if anyone else run into this issue Check it out
How can I wrap a boto.storage_uri() call in python so I can handle possible exceptions?
Your question about Boto is a good one, not not easy to answer. The Boto exception hierarchy is poorly designed, and ultimately the only way to determine what the exception you want to trap is requires looking at the boto source code.
For example if you look at (on Ubuntu) /usr/share/pyshared/boto/exception.py you will see that there are two broad classes:
boto.exception.BotoClientError
boto.exception.BotoServerError
Many of the exceptions are derived from these two, though the concept of "Client" and "Server" is not very well defined and you probably would want to check for both to be sure as many exceptions can happen unexpected (as usual). However, exceptions such as boto.exception.NoAuthHandlerFound is derived directly from Exception and therefore you would have to check for it separately.
Unfortunately from looking at the code, it appears that there is neither consistency nor much care in defining the exception hierarchy in Boto, which is a flaw in Boto's design which unfortunately requires that you rely on more broad exception checking than would normally be recommended.
The first question is what exceptions is this call likely to generate? You do not want to make a blanket exception handler in any language. You should first take a look at the Boto documentation to see if it documents the exceptions you might see from a given call, but if not I would first try:
try:
uri = boto.storage_uri()
except Exception, e:
print e
Or log the exception (with the logging package exception method), but either way you want to take note of what types of exceptions you see while you're testing and whether you should handle any of them specially. You also may want to review the Python Tutorial section on Exceptions and Errors.
Looking at the boto3 source, Boto3Error is declared as the base of all boto errors. So, you can probably do this:
try:
boto.storage_uri()
except Boto3Error:
# handle errors