What are the implications of using assert vs. raise Exception - python

Are there any significant differences with the following?
raise Exception("some exception")
assert False, "some exception"

Assertions can be disabled with the -O flag when starting Python. For this reason, use assertions only for sanity-checking, not for checking that is part of your program logic.
Besides this, there is of course the difference that assertions raise AssertionError, which you really shouldn't catch. When you raise an exception, you can make the type of the exception appropriate to the error and catch it later.

To extend a bit on #svk's answer (which already describes the technical differences and shortly addresses the conceptual difference): Assertions and exceptions are meant to be used for different purposes.
You can think of an assertion as a construct that is meant to provide information to developers. That is, if properly used, an assertion will inform a reader of the code (or, a static checker) about a condition that has to be fulfilled at run-time for the code to be correct. Similarly, an assertion that fails at run-time will convey the information that somewhere in the code there is a bug that needs fixing. Since the detection of a bug means that the software is unreliable (there is no way to predict what will happen if you continue), and since there is no way for the software itself to handle it (the code can't fix itself), it is often wise to just terminate the execution - and that is why you should not catch the AssertionError and continue.
In contrast, an exception would normally not be used to identify the presence of bugs in the software, but to identify exceptional situations for which there is likely some way of handling them on a higher level (where the exception is caught).

Related

Exceptions vs Errors in Python

I come from Java where Exceptions and Errors are quite different things and they both derive from something called Throwable.
In Java normally you should never try to catch an Error.
In Python though it seems the distinction is blurred.
So far after reading some docs and checking the hierarchy I have the following questions:
There are syntax errors which of course cause your program not to be able to start at all. Right?
"Errors detected during execution are called exceptions and are not unconditionally fatal" (per the tutorial). What does "fatal" mean here? Also, some objects like AttributeError are (by the above definition) actually exceptions even though they contain Error in their names, is that conclusion correct?
Some classes derive from Exception but contain Error in their name. Isn't this confusing? But even so it means that Error in the name is in no way special, it's still an Exception. Or not... ?
"All built-in, non-system-exiting exceptions are derived from [Exception]" (quote from here)
So which ones are system-exiting exceptions and which ones are not? It is not immediately clear. All user-defined exceptions should also be derived from Exception. So basically as a beginner do I need to worry about anything else but Exception? Seems like not.
Warnings also derive from Exception. So are warnings fatal or system-exiting or none of these?
Where does the AssertionError fit into all of this? Is it fatal or system exiting?
How does one know or specify that some Exception class represents fatal or system-exiting exception?
Yes. SyntaxError isn't catchable except in cases of dynamically executed code (via eval/exec), because it occurs before the code is actually running.
"Fatal" means "program dies regardless of what the code says"; that doesn't happen with exceptions in Python, they're all catchable. os._exit can forcibly kill the process, but it does so by bypassing the exception mechanism.
There is no difference between exceptions and errors, so the nomenclature doesn't matter.
System-exiting exceptions derive from BaseException, but not Exception. But they can be caught just like any other exception
Warnings behave differently based on the warnings filter, and deriving from Exception means they're not in the "system-exiting" category
AssertionError is just another Exception child class, so it's not "system exiting". It's just tied to the assert statement, which has special semantics.
Things deriving from BaseException but not Exception (e.g. SystemExit, KeyboardInterrupt) are "not reasonable to catch" (or if you do catch them, it should almost always be to log/perform cleanup and rethrow them), everything else (derived from Exception as well) is "conditionally reasonable to catch". There is no other distinction.
To be clear, "system-exiting" is just a way of saying "things which except Exception: won't catch"; if no except blocks are involved, all exceptions (aside from warnings, which as noted, behave differently based on the warnings filter) are "system-exiting".
Exceptions are designed for the programmer to know how to handle them such as outOfRange
Once an exception arises the programmer has to decide how to handle it and the code can continue to operate relatively smoothly
On the other hand an error indicates a problem that the programmer could not foresee as an import error or a memory error
Errors can still be addressed and ensure that the software continues to run but apparently not everything will be able to continue to work smoothly.

Python: statically detect unhandled exceptions

I'm trying to write a highly-reliable piece of code in Python. The most common issue I run into is that after running for a while, some edge case will occur and raise an exception that I haven't handled. This most often happens when using external libraries - without reading the source code, I don't know of an easy way to get a list of all exceptions that might be raised when using a specific library function, so it's hard to know what to handle. I understand that it's bad practice to use catch-all handlers, so I haven't been doing that.
Is there a good solution for this? It seems like it should be possible for a static analysis tool to check that all exceptions are handled, but I haven't found one. Does this exist? If not, why? (is it impossible? a bad idea? etc) I especially would like it to analyze imported code for the reason explained above.
"it's bad practice to use catch-all handlers" to ignore exceptions:
Our web service has an except which wraps the main loop.
except:
log_exception()
attempt_recovery()
This is good, as it notifies us (necessary) of the unexpected error and then tries to recover (not necessary). We can then go look at those logs and figure out what went wrong so we can prevent it from hitting our general exception again.
This is what you want to avoid:
except:
pass
Because it ignores the error... then you don't know an error happened and your data may be corrupted/invalid/gone/stolen by bears. Your server may be up/down/on fire. We have no idea because we ignored the exception.
Python doesn't require registering of what exceptions might be thrown, so there are no checks for all exceptions a module might throw, but most will give you some idea of what you should be ready to handle in the docs. Depending on your service, when it gets an unhandled exception, you might want to:
Log it and crash
Log it and attempt to continue
Log it and restart
Notice a trend? The action changes, but you never want to ignore it.
Great question.
You can try approaching the problem statically (by, for instance, introducing a custom flake8 rule?), but, I think, it's a problem of testing and test coverage scope. I would approach the problem by adding more "negative path"/"error handling" checks/tests to the places where third-party packages are used, introducing mock side effects whenever needed and monitoring coverage reports at the same time.
I would also look into Mutation Testing idea (check out Cosmic Ray Python package). I am not sure if mutants can be configured to also throw exceptions, but see if it could be helpful.
Instead of trying to handle all exceptions, which is very hard as you described, why don't you catch all, but exclude some, for example the KeyboardInterrupt you mentioned in the comments?
This may help

Python determine whether an exception was thrown (regardless of whether it is caught or not)

I am writing tests for some legacy code that is littered with catch-all constructs like
try:
do_something()
do_something_else()
for x in some_huge_list():
do_more_things()
except Exception:
pass
and I want to tell whether an exception was thrown inside the try block.
I want to avoid introducing changes into the codebase just to support a few tests and I don't want to make the except cases more specific for fear of unintentionally introducing regressions.
Is there a way of extracting information about exceptions that were raised and subsequently handled from the runtime? Or some function with a similar API to eval/exec/apply/call that either records information on every raised exception, lets the user supply an exception handler that gets run first, or lets the user register a callback that gets run on events like an exception being raised or caught.
If there isn't a way to detect whether an exception was thrown without getting under the (C)Python runtime in a really nasty way, what are some good strategies for testing code with catch-all exceptions inside the units you're testing?
Your only realistic option is to instrument the except handlers.
Python does record exception information, which is retrievable with sys.exc_info(), but this information is cleared when a function exits (Python 2) or the try statement is done (Python 3).
A good strategy would be testing observable behaviour. Since exceptions were explicitly excluded from the observable behaviour I do not think you should be testing whether an exception was raised or not.

Python - Why is an exception type needed to be put after an 'except'? [duplicate]

This question already has answers here:
Should I always specify an exception type in `except` statements?
(7 answers)
How to properly ignore exceptions
(12 answers)
Closed 8 years ago.
When you use try/except in python, why do you need an exception type to be named after except? Wouldn't it be easier to just catch all exceptions?
try:
#dosomething
except Exception:
#dosomething
Why is the 'Exception' needed after except?
Because you might handle different exceptions different ways.
For example, if you're attempting a network operation, and the network address you're trying to reach can't be resolved, that's likely due to user error, and you'll want to get the user involved, while some other kinds of errors can simply be retried after a short wait.
It's a good practice in exception handling to handle only the narrowest set of exceptions you expect at any one point in the code, and to only catch those exceptions that you're sure you know how to handle. A catch-all exception handler violates this principle.
From a purely syntactic point of view, this is acceptable code:
try:
# Do something
except:
print "Something went wrong."
HOWEVER, it's not a very good idea to do this a lot of the time. By catching all exceptions and not even saving the name, you're losing all information about where the error was. Just seeing Something went wrong. printed out is both useless and frustrating. So even if you don't want to handle each exception individually, you'd want to save the exception information at the very least.
try:
# Do something.
except Exception, e:
print "Encountered error " + str(e) + " during execution. Exiting gracefully."
The above code is something you might do if you absolutely can't let your program exit abruptly, for example.
EDIT: changed the answer to clarify that it's a bad idea, though possible.
Why and how to catch exceptions
Exceptions are really helpful, but they shall be handled in different manner depending on what code
you write.
Core distinction is, if your code is top level one (last resort to handle exceptions) or inner one.
Another aspect is, if some exceptions are excepted or unexpected.
Expected exceptions (like file, you are trying to use is missing) shall be handled, if the code has
a chance to do anything about it.
Unexpected exceptions shall not be handled unless you have to do so in top level code.
Handling exceptions in top level code
If it does not matter, that your code throws a bit ugly stack trace under some circumstances, simply
ingore the unexpected exceptions. This is mostly very efficient, as the code is kept simple, and
stack trace give you good chance to find out, what went wrong.
If you have to make your script "well behaving" - you could catch the exception and print some nice
looking excuse for what went wrong.
Handling exceptions in lower level code (modules, functions)
In your lower level code, you shall catch all expected exceptions, and the rest shall be ignored and
thrown up to higher levels, where is better chance to handle it properly.
If you have no expected exception, simply do not use try .. except block.
Printing some excuses from lower level code is mostly inappropriate (higher level code has no chance
t silence your printouts.
To your question - why except Exception
except with explicitly mentioned type of exception is the only solution to use for expected types
of exceptions. Without mentioning the type (or types), you are catching all and this is bad habit
unless you are in top level code.
As usual, there are exceptions to the recommendations above, but they are occurring less often than one
usually expects.
Different exceptions require different fixing. For example, when I was writing a python irc bot, i would have one exception for invalid access to a string, and that code in the except would try to remedy it. I also had one for bad sockets that would try to deduce why it went bad and fix it. I can't have these under one exception because they are fixed differently

Should I always specify an exception type in `except` statements?

When using PyCharm IDE the use of except: without an exception type triggers a reminder from the IDE that this exception clause is Too broad.
Should I be ignoring this advice? Or is it Pythonic to always specific the exception type?
It's almost always better to specify an explicit exception type. If you use a naked except: clause, you might end up catching exceptions other than the ones you expect to catch - this can hide bugs or make it harder to debug programs when they aren't doing what you expect.
For example, if you're inserting a row into a database, you might want to catch an exception that indicates that the row already exists, so you can do an update.
try:
insert(connection, data)
except:
update(connection, data)
If you specify a bare except:, you would also catch a socket error indicating that the database server has fallen over. It's best to only catch exceptions that you know how to handle - it's often better for the program to fail at the point of the exception than to continue but behave in weird unexpected ways.
One case where you might want to use a bare except: is at the top-level of a program you need to always be running, like a network server. But then, you need to be very careful to log the exceptions, otherwise it'll be impossible to work out what's going wrong. Basically, there should only be at most one place in a program that does this.
A corollary to all of this is that your code should never do raise Exception('some message') because it forces client code to use except: (or except Exception: which is almost as bad). You should define an exception specific to the problem you want to signal (maybe inheriting from some built-in exception subclass like ValueError or TypeError). Or you should raise a specific built-in exception. This enables users of your code to be careful in catching just the exceptions they want to handle.
You should not be ignoring the advice that the interpreter gives you.
From the PEP-8 Style Guide for Python :
When catching exceptions, mention specific exceptions whenever
possible instead of using a bare except: clause.
For example, use:
try:
import platform_specific_module
except ImportError:
platform_specific_module = None
A bare except: clause will catch SystemExit and KeyboardInterrupt exceptions, making it harder to
interrupt a program with Control-C, and can disguise other problems.
If you want to catch all exceptions that signal program errors, use
except Exception: (bare except is equivalent to except
BaseException:).
A good rule of thumb is to limit use of bare 'except' clauses to two
cases:
If the exception handler will be printing out or logging the
traceback; at least the user will be aware that an error has occurred.
If the code needs to do some cleanup work, but then lets the exception
propagate upwards with raise. try...finally can be a better way to
handle this case.
Not specfic to Python this.
The whole point of exceptions is to deal with the problem as close to where it was caused as possible.
So you keep the code that could in exceptional cirumstances could trigger the problem and the resolution "next" to each other.
The thing is you can't know all the exceptions that could be thrown by a piece of code. All you can know is that if it's a say a file not found exception, then you could trap it and to prompt the user to get one that does or cancel the function.
If you put try catch round that, then no matter what problem there was in your file routine (read only, permissions, UAC, not really a pdf, etc), every one will drop in to your file not found catch, and your user is screaming "but it is there, this code is crap"
Now there are a couple of situation where you might catch everything, but they should be chosen consciously.
They are catch, undo some local action (such as creating or locking a resource, (opening a file on disk to write for instance), then you throw the exception again, to be dealt with at a higher level)
The other you is you don't care why it went wrong. Printing for instance. You might have a catch all round that, to say There is some problem with your printer, please sort it out, and not kill the application because of it. Ona similar vain if your code executed a series of separate tasks using some sort of schedule, you wouldnlt want the entire thing to die, because one of the tasks failed.
Note If you do the above, I can't recommend some sort of exception logging, e.g. try catch log end, highly enough.
Always specify the exception type, there are many types you don't want to catch, like SyntaxError, KeyboardInterrupt, MemoryError etc.
You will also catch e.g. Control-C with that, so don't do it unless you "throw" it again. However, in that case you should rather use "finally".
Here are the places where i use except without type
quick and dirty prototyping
That's the main use in my code for unchecked exceptions
top level main() function, where i log every uncaught exception
I always add this, so that production code does not spill stacktraces
between application layers
I have two ways to do it :
First way to do it : when a higher level layer calls a lower level function, it wrap the calls in typed excepts to handle the "top" lower level exceptions. But i add a generic except statement, to detect unhandled lower level exceptions in the lower level functions.
I prefer it this way, i find it easier to detect which exceptions should have been caught appropriately : i "see" the problem better when a lower level exception is logged by a higher level
Second way to do it : each top level functions of lower level layers have their code wrapped in a generic except, to it catches all unhandled exception on that specific layer.
Some coworkers prefer this way, as it keeps lower level exceptions in lower level functions, where they "belong".
Try this:
try:
#code
except ValueError:
pass
I got the answer from this link, if anyone else run into this issue Check it out

Categories

Resources