Develop Reference

How to catch a custom exception not defined by me in python

I have the following try block:
try:
# depending on conditions, this can generate several types of errors
mymodule.do_stuff()
except Exception as e:
print("something went wrong, the error was :" + type(e).__name__)
I would like to catch potential errors from do_stuff(). After trial and error I was able to generate a list of potential errors that can be triggered by do_stuff() by printing their type(e).__name__ value:
DoStuffInsufficientMemoryException
DoStuffInsufficientCPUException
DoStuffInsufficientDiskException
but if I try do modify my except statement from except Exception as e to except DoStuffInsufficientMemoryException, I will get the error that DoStuffInsufficientMemoryException is not defined.
I tried defining a class that extends Exception for it, as most tutorials / questions in here suggest, basically:
class WAFInvalidParameterException(Exception):
pass
so now that variable is recognized, but since I can't control the error that do_sutff() will raise, I can't really raise this exception in my initial try block.
Ideally I would like to have 1 except block for each error so I would like to have something like:
try:
mymodule.do_stuff()
except DoStuffInsufficientMemoryException:
free_memory()
except DoStuffInsufficientCPUException:
kill_processes()
except DoStuffInsufficientDiskException:
free_disk_space()
but of course this doesn't work as these variables are not defined.

Just like you can't reference do_stuff without its module specifier, you have to specify in which module namespace these exceptions are defined.
try:
mymodule.do_stuff()
except mymodule.DoStuffInsufficientMemoryException:
free_memory()
except mymodule.DoStuffInsufficientCPUException:
kill_processes()
except mymodule.DoStuffInsufficientDiskException:
free_disk_space()
If free_memory is also in the mymodule namespace, of course you need to specify it there as well.
Alternatively, when you import mymodule, you can explicitly import selected symbols into the current namespace:
from mymodule import do_stuff, DoStuffInsufficientMemoryException, ...
and then, because they are in the current package, you can (or indeed must) refer to them without the package prefix mymodule.
A well-designed module will export selected symbols so you can refer to them without the package prefix, but whether this makes sense for your own package depends on its general design and intended audience. Some large packages define a separate subpackage for exceptions so you can say
import bigpackage.exceptions
to import them all. You will probably still want to explore the package's documentation (or, if it's lacking, its source code) to discover which exceptions exist and how they are organized. Many packages define a base exception class from which all its other exceptions are subclasses so that you can easily catch them all with just one symbol, like
try:
bigpackage.heavy_wizardry()
except bigpackage.BigBaseException:
print("you are turned into a frog")

EDIT : you can import other methods instead of creating your own, of course
The try/except block will try to execute the code and if an error is raised and specified in the except statement, it will stop the execution of the code located in the try block and execute the other code located in the except block. So, to catch your custom error, you have to raise it in the first place.
If you didn't know, you can raise errors using the raise statement. Here, I've made a simple chunk of code. I have a custom error, a variable x initialized at 2, and a method that adds 1 to the variable given in argument. The method will raise a CustomError if the variable becomes 3.
# Here, I define the custom error and allow a custom message to be displayed
# using the super() method
class CustomError(Exception):
def __init__(self, msg):
super().__init__(msg)
# I initialize x at 2
x = 2
# I create the method that will add 1 to the variable given in argument
# and raise a CustomError if the variable becomes 3
# This is completely random, and you can make whatever checks you want before raising
# Your custom error
def do_stuff(x):
x += 1
if x == 3:
raise CustomError("x is 3")
# Now, I write the try/except block. I raise e (the CustomError) if it is
# being raised in the method do_stuff(x)
try:
do_stuff(x)
except CustomError as e:
raise e
Feel free to try the code out to get a better understanding of it !

Usually if a function
module.foo()
throws an exception DoStuffInsufficientMemoryException it would be also importable as
from module import DoStuffInsufficientMemoryException
If this results in ImportError then you need the fullname function from this answer; use it with e (it takes an instance and returns the class name). If it gives
foo.bar.DoStuffInsufficientMemoryException
then you can import the exception as
from foo.bar import DoStuffInsufficientMemoryException
The above might not work for all cases. One notable case is Boto 3 AWS client library that does not make the exceptions importable - instead they will be attributes on the client instance.

unit test for custom exception (chained exception python 3)

Working on testing custom based exceptions within python3, within the client code I have:
class myCustomException(Exception):
pass
def someFunc():
try:
mathCheck = 2/0
print(mathCheck)
except ZeroDivisionError as e:
raise myCustomException from e
On the test side:
def testExceptionCase(self):
with self.assertRaises(ZeroDivisionError) as captureException:
self.someFunc()
My question is:
How to essentially capture the chained exception i.e. the myCustomException using unittest (so proving that the custom exception did get called and raised from the base exception which is ZeroDivisonError), assume I have already done the import of unittest, and imports within client-test files.
Is there a way to say we were able to keep track of the traceback chaining from ZeroDivisionError and myCustomException. Basically, this test should also fail if it didn't raise the myCustomException. Appreciate any help!

from client import MyCustomException
def testExceptionCase(self):
with self.assertRaises(MyCustomException):
self.someFunc()
Also you may want to use UpperCamelCase for Exceptions and Classes in general

Handle c library error in python?

If I'm using a lib written in c in python, such as zlib, and I get an error like:
error: Error -3 while decompressing: invalid distance too far back
How do I handle this exception is python? presumably the exception is defined in the c module and there is no exception class to catch on like except SomeException?

To call a C function from Python, the C function must be wrapped by a Python function. For CPython, the wrapper can be written in C with the help of a package such as Swig, in Cython, or in Python with the help of the ctypes module. The wrapper must translate Python inputs to C values, define C output varibles, call the C function, check the return code, and either translate the C output to Python and return it or translate the C code to a Python exception and raise it.
It appears that you are using Python's zlib, which wraps a C zlib. It defines a Python exception class error.
>>> import zlib
>>> zlib.error
<class 'zlib.error'>
>>> issubclass(zlib.error, Exception)
True
You catch it like any other exception.
>>> try:
raise zlib.error('Error 99: unknown')
except zlib.error as e:
print(e)
Error 99: unknown
To be consistent with the exception naming convention, I would have named itZlibError.

Catch Built-in Exception (rather than self defined)

I want to patch a library to catch the built-in ConnectionError (which inherits from OSError).
So far so good. As it happens, the library has a "self-defined" Exception that is also called ConnectionError:
class LibraryError(Exception):
pass
class ConnectionError(LibraryError):
pass
I guess, if I now tried to catch a ConnectionError, doing something like
try:
do_something()
except ConnectionError as e:
try_to_get_it_right_again()
I would only catch the self-defined ConnectionError, which inherits from LibraryError. (Disclaimer: I have to admit, I haven't tested that myself, as I didn't know how).
How would I get Python to catch the built-in ConnectionError?

Use the builtins module, the explicit name for the namespace where built-in names like int and ConnectionError live.
import builtins
try:
...
except builtins.ConnectionError:
...
In Python 2, this would be __builtin__, although Python 2 doesn't have ConnectionError. Note that __builtins__ is its own weird thing; even if it looks like what you want, it's not.
If you want code that works in both Python 2 and Python 3... well, the exception hierarchy looks pretty different in Python 2, and ConnectionError doesn't even exist, so it's not as simple as deciding whether to use builtins or __builtin__. The builtins/__builtin__ thing is easy enough to solve, at least.
To import the right module depending on Python version, you can catch the ImportError and import the other module:
try:
import builtins
except ImportError:
import __builtin__ as builtins
Pretending for a moment that Python 2 has ConnectionError, you could save a reference to the built-in ConnectionError before shadowing the name:
_builtin_ConnectionError = ConnectionError
class ConnectionError(LibraryError):
...

Use the the ConnectionError defined along with the other exceptions in the builtins library:
import builtins
try:
# connection error raised
except builtins.ConnectionError as conerr:
# handle stuff

What is the correct way to identify an Exception?

If exceptions are always fatal, making use of them in Python is easy
# moduleB.py
import moduleC
But evaluating an exception requires more than just it's type, we often need to determine where an exception came from
# moduleA.py
try:
import moduleB
except ImportError as e:
print e
if str(e) == "No module named moduleB":
pass
else:
raise
In some projects this pattern results in a lot of code that is not easy to read. Is this the best way to ensure that I'm catching a local exception? I would like to be able write
import moduleB else pass

Sorry, but the example you posted is the canonical Way Of Doing It. Python doesn't have any syntax for catching the exception raised by the import statement but not by something below it.
Just a small warning, though: str(e) can cause unicode errors if e's message is unicode. You can fix that by using repr(e).

No, don't try to analyse the error string. You can log the exception for debugging if you want, but if it's "import if you can", then this will suffice:
try:
import foo
except ImportError:
pass # or foo = None or whatever

In your specific case, parsing the string is the only viable solution, but note that you don't need to do it: what is the difference if the moduleB or if it is a module imported by the moduleB not to be found?
If you are throwing custom exceptions, you can provide additional information when you raise them:
raise Exception(12) # 12 is the error code
and then get it through the args property:
if e.args[0] == 12:
handle()
A better way may be to subclass Exception and provide your own properties (for example error_code or module_name).

try:
import moduleB
except ImportError:
pass
This ought to be both sufficient and succinct enough.