Develop Reference

Python Exception Handling : Is there a method to know what type of exceptions my code can possibly throw?

I have a code, lets say :
'''
try:
somecode()
except Exception as e:
somelog()
'''
Is there a way to find out all the possible exceptions somecode() can throw so that I can handle them in an appropriate order.

While you may not always be able to know every error that may happen, you can do quite a bit by thinking of common cases. This link is a good starter's guide with examples:
https://www.pythonforbeginners.com/error-handling/exception-handling-in-python1
For raising exceptions you predict in your own functions, this is a good starter guide:
https://www.programiz.com/python-programming/user-defined-exception
Finally, when you're working with built in functions or packages, they usually document what exceptions they raise. For example, look at the built in page for Python
https://docs.python.org/3/library/functions.html
And ctrl-f ValueError. A lot of docs will tell you what exceptions they raise but beyond that it's up to you to anticipate and guess based on your implementation and usage.
Hope that helps!

There are not so many exception types which you may have to consider for a single case. In case you are trying to access a file or accessing the database, the options are very few. The best practice is to keep track with the documentation. This will not take much time to know the name of the exception.
https://docs.python.org/3/tutorial/errors.html

Chained Conditions that require order to not produce order

Whenever I chain conditions in Python (or any other language tbh) I stumble upon asking myself this, kicking me out of the productive "Zone".
When I chain conditions I can, by ordering them correctly, check conditions that without checking for the other conditions first, may produce an Error.
As an example lets assume the following snippet:
if "attr" in some_dictionary and some_value in some_dictionary["attr"]:
print("whooohooo")
If the first condition wasnt in the first place or even absent, the second condition my produce an KeyError
I do this pretty often to simply save space in the code, but I always wondered, if this is good style, if it comes with a risk or if its simply "pythonic".

A more Pythonic way is to "ask for forgivness rather than permission". In other words, use a try-except block:
try:
if some_value in some_dictionary["attr"]:
print("Woohoo")
except KeyError:
pass

Python is a late binding language, which is reflected in these kind of checks. The behavior is called short-circuiting. One thing I often do is:
def do(condition_check=None):
if condition_check is not None and condition_check():
# do stuff
Now, many people will argue that try: except: is more appropriate. This really depends on the use case!
if expressions are faster when the check is likely to fail, so use them when you know what is happening.
try expressions are faster when the check is likely to succeed, so use them to safeguard against exceptional circumstances.
if is explicit, so you know precisely what you are checking. Use it if you know what is happening, i.e. strongly typed situations.
try is implicit, so you only have to care about the outcome of a call. Use it when you don't care about the details, i.e. in weakly typed situations.
if works in a well-defined scope - namely right where you are performing the check. Use it for nested relations, where you want to check the top-most one.
try works on the entire contained call stack - an exception may be thrown several function calls deeper. Use it for flat or well-defined calls.
Basically, if is a precision tool, while try is a hammer - sometimes you need precision, and sometimes you just have nails.

What is the pythonic way to bubble up error conditions

I have been working on a Python project that has grown somewhat large, and has several layers of functions. Due to some boneheaded early decisions I'm finding that I have to go a fix a lot of crashers because the lower level functions are returning a type I did not expect in the higher level functions (usually None).
Before I go through and clean this up, I got to wondering what is the most pythonic way of indicating error conditions and handling them in higher functions?
What I have been doing for the most part is if a function can not complete and return its expected result, I'll return None. This gets a little gross, as you end up having to always check for None in all the functions that call it.
def lowLevel():
## some error occurred
return None
## processing was good, return normal result string
return resultString
def highLevel():
resultFromLow = lowLevel()
if not resultFromLow:
return None
## some processing error occurred
return None
## processing was good, return normal result string
return resultString
I guess another solution might be to throw exceptions. With that you still get a lot of code in the calling functions to handle the exception.
Nothing seems super elegant. What do other people use? In obj-c a common pattern is to return an error parameter by reference, and then the caller checks that.

It really depends on what you want to do about the fact this processing can't be completed. If these are really exceptions to the ordinary control flow, and you need to clean up, bail out, email people etc. then exceptions are what you want to throw. In this case the handling code is just necessary work, and is basically unavoidable, although you can rethrow up the stack and handle the errors in one place to keep it tidier.
If these errors can be tolerated, then one elegant solution is to use the Null Object pattern. Rather than returning None, you return an object that mimics the interface of the real object that would normally be returned, but just doesn't do anything. This allows all code downstream of the failure to continue to operate, oblivious to the fact there was a failure. The main downside of this pattern is that it can make it hard to spot real errors, since your code will not crash, but may not produce anything useful at the end of its run.
A common example of the Null Object pattern in Python is returning an empty list or dict when you're lower level function has come up empty. Any subsequent function using this returned value and iterating through elements will just fall through silently, without the need for error checking. Of course, if you require the list to have at least one element for some reason, then this won't work, and you're back to handling an exceptional situation again.

On the bright side, you have discovered exactly the problem with using a return value to indicate an error condition.
Exceptions are the pythonic way to deal with problems. The question you have to answer (and I suspect you already did) is: Is there a useful default that can be returned by low_level functions? If so, take it and run with it; otherwise, raise an exception (`ValueError', 'TypeError' or even a custom error).
Then, further up the call stack, where you know how to deal with the problem, catch the exception and deal with it. You don't have to catch exceptions immediately -- if high_level calls mid-level calls low_level, it's okay to not have any try/except in mid_level and let high_level deal with it. It may be that all you can do is have a try/except at the top of your program to catch and log all uncaught and undealt-with errors, and that can be okay.

This is not necessarily Pytonic as such but experience has taught me to let exceptions "lie where they lie".
That is to say; Don't unnecessarily hide them or re-raise a different exception.
It's sometimes good practice to let the callee fail rather than trying to capture and hide all kinds of error conditions.
Obviously this topic is and can be a little subjective; but if you don't hide or raise a different exception, then it's much easier to debug your code and much easier for the callee of your functions or api to understand what went wrong.
Note: This answer is not complete -- See comments. Some or all of the answers presented in this Q&A should probably be combined in a nice way presneting the various problems and solutions in a clear and concise manner.

python isinstance vs hasattr vs try/except: What is better?

I am trying to figure out the tradeoffs between different approaches of determining whether or not with object obj you can perform action do_stuff(). As I understand, there are three ways of determining if this is possible:
# Way 1
if isinstance(obj, Foo):
obj.do_stuff()
# Way 2
if hasattr(obj, 'do_stuff'):
obj.do_stuff()
# Way 3
try:
obj.do_stuff()
except:
print 'Do something else'
Which is the preferred method (and why)?

I believe that the last method is generally preferred by Python coders because of a motto taught in the Python community: "Easier to ask for forgiveness than permission" (EAFP).
In a nutshell, the motto means to avoid checking if you can do something before you do it. Instead, just run the operation. If it fails, handle it appropriately.
Also, the third method has the added advantage of making it clear that the operation should work.
With that said, you really should avoid using a bare except like that. Doing so will capture any/all exceptions, even the unrelated ones. Instead, it is best to capture exceptions specifically.
Here, you will want to capture for an AttributeError:
try:
obj.do_stuff() # Try to invoke do_stuff
except AttributeError:
print 'Do something else' # If unsuccessful, do something else

Checking with isinstance runs counter to the Python convention of using duck typing.
hasattr works fine, but is Look Before you Leap instead of the more Pythonic EAFP.
Your implementation of way 3 is dangerous, since it catches any and all errors, including those raised by the do_stuff method. You could go with the more precise:
try:
_ds = obj.do_stuff
except AttributeError:
print('Do something else')
else:
_ds()
But in this case, I'd prefer way 2 despite the slight overhead - it's just way more readable.

The correct answer is 'neither'
hasattr delivers functionality however it is possibly the worst of all options.
We use the object oriented nature of python because it works. OO analysis is never accurate and often confuses however we use class hierarchies because we know they help people do better work faster. People grasp objects and a good object model helps coders change things more quickly and with less errors. The right code ends up clustered in the right places. The objects:
Can just be used without considering which implementation is present
Make it clear what needs to be changed and where
Isolate changes to some functionality from changes to some other functionality – you can fix X without fearing you will break Y
hasattr vs isinstance
Having to use isinstance or hasattr at all indicates the object model is broken or we are using it incorrectly. The right thing to do is to fix the object model or change how we are using it.
These two constructs have the same effect and in the imperative ‘I need the code to do this’ sense they are equivalent. Structurally there is a huge difference. On meeting this method for the first time (or after some months of doing other things), isinstance conveys a wealth more information about what is actually going on and what else is possible. Hasattr does not ‘tell’ you anything.
A long history of development lead us away from FORTRAN and code with loads of ‘who am I’ switches. We choose to use objects because we know they help make the code easier to work with. By choosing hasattr we deliver functionality however nothing is fixed, the code is more broken than it was before we started. When adding or changing this functionality in the future we will have to deal with code that is unequally grouped and has at least two organising principles, some of it is where it ‘should be’ and the rest is randomly scattered in other places. There is nothing to make it cohere. This is not one bug but a minefield of potential mistakes scattered over any execution path that passes through your hasattr.
So if there is any choice, the order is:
Use the object model or fix it or at least work out what is wrong
with it and how to fix it
Use isinstance
Don’t use hasattr

Cheap exception handling in Python?

I read in an earlier answer that exception handling is cheap in Python so we shouldn't do pre-conditional checking.
I have not heard of this before, but I'm relatively new to Python. Exception handling means a dynamic call and a static return, whereas an if statement is static call, static return.
How can doing the checking be bad and the try-except be good, seems to be the other way around. Can someone explain this to me?

Don't sweat the small stuff. You've already picked one of the slower scripting languages out there, so trying to optimize down to the opcode is not going to help you much. The reason to choose an interpreted, dynamic language like Python is to optimize your time, not the CPU's.
If you use common language idioms, then you'll see all the benefits of fast prototyping and clean design and your code will naturally run faster as new versions of Python are released and the computer hardware is upgraded.
If you have performance problems, then profile your code and optimize your slow algorithms. But in the mean time, use exceptions for exceptional situations since it will make any refactoring you ultimately do along these lines a lot easier.

You might find this post helpful: Try / Except Performance in Python: A Simple Test where Patrick Altman did some simple testing to see what the performance is in various scenarios pre-conditional checking (specific to dictionary keys in this case) and using only exceptions. Code is provided as well if you want to adapt it to test other conditionals.
The conclusions he came to:
From these results, I think it is fair
to quickly determine a number of
conclusions:
If there is a high likelihood that the element doesn't exist, then
you are better off checking for it
with has_key.
If you are not going to do anything with the Exception if it is
raised, then you are better off not
putting one have the except
If it is likely that the element does exist, then there is a very
slight advantage to using a try/except
block instead of using has_key,
however, the advantage is very slight.

Putting aside the performance measurements that others have said, the guiding principle is often structured as "it is easier to ask forgiveness than ask permission" vs. "look before you leap."
Consider these two snippets:
# Look before you leap
if not os.path.exists(filename):
raise SomeError("Cannot open configuration file")
f = open(filename)
vs.
# Ask forgiveness ...
try:
f = open(filename)
except IOError:
raise SomeError("Cannot open configuration file")
Equivalent? Not really. OSes are multi-taking systems. What happens if the file was deleted between the test for 'exists' and 'open' call?
What happens if the file exists but it's not readable? What if it's a directory name instead of a file. There can be many possible failure modes and checking all of them is a lot of work. Especially since the 'open' call already checks and reports all of those possible failures.
The guideline should be to reduce the chance of inconsistent state, and the best way for that is to use exceptions instead of test/call.

"Can someone explain this to me?"
Depends.
Here's one explanation, but it's not helpful. Your question stems from your assumptions. Since the real world conflicts with your assumptions, it must mean your assumptions are wrong. Not much of an explanation, but that's why you're asking.
"Exception handling means a dynamic call and a static return, whereas an if statement is static call, static return."
What does "dynamic call" mean? Searching stack frames for a handler? I'm assuming that's what you're talking about. And a "static call" is somehow locating the block after the if statement.
Perhaps this "dynamic call" is not the most costly part of the operation. Perhaps the if-statement expression evaluation is slightly more expensive than the simpler "try-it-and-fail".
Turns out that Python's internal integrity checks are almost the same as your if-statement, and have to be done anyway. Since Python's always going to check, your if-statement is (mostly) redundant.
You can read about low-level exception handling in http://docs.python.org/c-api/intro.html#exceptions.
Edit
More to the point: The if vs. except debate doesn't matter.
Since exceptions are cheap, do not label them as a performance problem.
Use what makes your code clear and meaningful. Don't waste time on micro-optimizations like this.

With Python, it is easy to check different possibilities for speed - get to know the timeit module :
... example session (using the command line) that compare the cost of using hasattr() vs. try/except to test for missing and present object attributes.
% timeit.py 'try:' ' str.__nonzero__' 'except AttributeError:' ' pass'
100000 loops, best of 3: 15.7 usec per loop
% timeit.py 'if hasattr(str, "__nonzero__"): pass'
100000 loops, best of 3: 4.26 usec per loop
% timeit.py 'try:' ' int.__nonzero__' 'except AttributeError:' ' pass'
1000000 loops, best of 3: 1.43 usec per loop
% timeit.py 'if hasattr(int, "__nonzero__"): pass'
100000 loops, best of 3: 2.23 usec per loop
These timing results show in the hasattr() case, raising an exception is slow, but performing a test is slower than not raising the exception. So, in terms of running time, using an exception for handling exceptional cases makes sense.
EDIT: The command line option -n will default to a large enough count so that the run time is meaningful. A quote from the manual:
If -n is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds.

I am a python beginner as well. While I cannot say why exactly Exception handling has been called cheap in the context of that answer, here are my thoughts:
Note that checking with if-elif-else has to evaluate a condition every time. Exception handling, including the search for an exception handler occurs only in an exceptional condition, which is likely to be rare in most cases. That is a clear efficiency gain.
As pointed out by Jay, it is better to use conditional logic rather than exceptions when there is a high likelihood of the key being absent. This is because if the key is absent most of the time, it is not an exceptional condition.
That said, I suggest that you don't worry about efficiency and rather about meaning. Use exception handling to detect exceptional cases and checking conditions when you want to decide upon something. I was reminded about the importance of meaning by S.Lott just yesterday.
Case in point:
def xyz(key):
dictOb = {x:1, y:2, z:3}
#Condition evaluated every time
if dictOb.has_key(key): #Access 1 to dict
print dictOb[key] #Access 2
Versus
#Exception mechanism is in play only when the key isn't found.
def xyz(key):
dictOb = {x:1, y:2, z:3}
try:
print dictOb[key] #Access 1
except KeyError:
print "Not Found"
Overall, having some code that handles something,like a missing key, just in case needs exception handling, but in situations like when the key isn't present most of the time, what you really want to do is to decide if the key is present => if-else. Python emphasizes and encourages saying what you mean.
Why Exceptions are preferred to if-elif ->
It expresses the meaning more clearly when you are looking foe exceptional aka unusual/unexpected conditions in your code.
It is cleaner and a whole lot more readable.
It is more flexible.
It can be used to write more concise code.
Avoids a lot of nasty checking.
It is more maintainable.
Note
When we avoid using try-except, Exceptions continue being raised. Exceptions which aren't handled simply go to the default handler. When you use try-except, you can handle the error yourself. It might be more efficient because if-else requires condition evaluation, while looking for an exception handler may be cheaper. Even if this is true, the gain from it will be too minor to bother thinking about.
I hope my answer helps.

What are static versus dynamic calls and returns, and why do you think that calls and returns are any different in Python depending on if you are doing it in a try/except block? Even if you aren't catching an exception, Python still has to handle the call possibly raising something, so it doesn't make a difference to Python in regards to how the calls and returns are handled.
Every function call in Python involves pushing the arguments onto the stack, and invoking the callable. Every single function termination is followed by the caller, in the internal wiring of Python, checking for a successful or exception termination, and handles it accordingly. In other words, if you think that there is some additional handling when you are in a try/except block that is somehow skipped when you are not in one, you are mistaken. I assume that is what you "static" versus "dynamic" distinction was about.
Further, it is a matter of style, and experienced Python developers come to read exception catching well, so that when they see the appropriate try/except around a call, it is more readable than a conditional check.

The general message, as S.Lott said, is that try/except doesn't hurt so you should feel free to use it whenever it seems appropriate.
This debate is often called "LBYL vs EAFP" – that's "look before you leap" vs "easier to ask forgiveness than permission". Alex Martelli weighs forth on the subject here: http://mail.python.org/pipermail/python-list/2003-May/205182.html This debate is almost six years old, but I don't think the basic issues have changed very much.