I have a module that depends on some system settings. For example, to work properly it needs to have an environment variable FOO set. I would like the module to raise an Exception if this condition is not met at import time.
# mymodule.py
if 'FOO' not in sys.environ:
raise SomeException('ensure that FOO is provided')
I would like to know:
Is it the best practice to check those conditions at import time, or maybe it would be better to do it later (when)
What type of exception should I raise? Should it be my own class MyModuleImportError(Exception) or maybe some built-in exceptions are more suitable and commonly used? One candidate would be ImportError, but it seems to be reserved for situation when python "fails to find the module definition"
Is this what you are looking for?
>>> import os
>>> if not os.getenv('FOO', False):
raise OSError('FOO not in envs')
Yes, you should check these kind of things at the start of your code. Else there is no reason for it to continue running.
an OSError (as above)
Related
I need to force an exception to be raised outside a function that does this:
def foo(x):
try:
some_calculation(x)
except:
print("ignore exception")
Is there a way to override the catch-all inside foo? I would like to raise an exception inside some_calculation(x) that can be caught or detected outside foo.
FYI, foo is a third party function I have no control over.
No. Your options are:
submit a fix to the library maintainers
fork the library, and provide your own vendorised version
monkey patch the library on import
The last is perhaps the easiest and quickest to get up and running.
Example:
main.py
# before doing anything else import and patch the third party library
# we need to patch foo before anyone else has a chance to import or use it
import third_party_library
# based off of third_party_library version 1.2.3
# we only catch Exception rather than a bare except
def foo(x):
try:
some_calculation(x)
except Exception:
print("ignore exception")
third_party_library.foo = foo
# rest of program as usual
...
Things might be slightly more complicated than that if foo() is re-exported across several different modules (if the third party library has its own from <x> import foo statements. But if just requires monkey patching more attributes of the various re-exporting modules.
Technically it would be possible to force an exception to be raised, but it would involve setting an execution trace and forcing an exception to be thrown in the exception handling code of the foo(). It would be weird, the exception would appear to come from print("ignore exception") rather than
some_calculation(x). So don't do that.
I have the following try block:
try:
# depending on conditions, this can generate several types of errors
mymodule.do_stuff()
except Exception as e:
print("something went wrong, the error was :" + type(e).__name__)
I would like to catch potential errors from do_stuff(). After trial and error I was able to generate a list of potential errors that can be triggered by do_stuff() by printing their type(e).__name__ value:
DoStuffInsufficientMemoryException
DoStuffInsufficientCPUException
DoStuffInsufficientDiskException
but if I try do modify my except statement from except Exception as e to except DoStuffInsufficientMemoryException, I will get the error that DoStuffInsufficientMemoryException is not defined.
I tried defining a class that extends Exception for it, as most tutorials / questions in here suggest, basically:
class WAFInvalidParameterException(Exception):
pass
so now that variable is recognized, but since I can't control the error that do_sutff() will raise, I can't really raise this exception in my initial try block.
Ideally I would like to have 1 except block for each error so I would like to have something like:
try:
mymodule.do_stuff()
except DoStuffInsufficientMemoryException:
free_memory()
except DoStuffInsufficientCPUException:
kill_processes()
except DoStuffInsufficientDiskException:
free_disk_space()
but of course this doesn't work as these variables are not defined.
Just like you can't reference do_stuff without its module specifier, you have to specify in which module namespace these exceptions are defined.
try:
mymodule.do_stuff()
except mymodule.DoStuffInsufficientMemoryException:
free_memory()
except mymodule.DoStuffInsufficientCPUException:
kill_processes()
except mymodule.DoStuffInsufficientDiskException:
free_disk_space()
If free_memory is also in the mymodule namespace, of course you need to specify it there as well.
Alternatively, when you import mymodule, you can explicitly import selected symbols into the current namespace:
from mymodule import do_stuff, DoStuffInsufficientMemoryException, ...
and then, because they are in the current package, you can (or indeed must) refer to them without the package prefix mymodule.
A well-designed module will export selected symbols so you can refer to them without the package prefix, but whether this makes sense for your own package depends on its general design and intended audience. Some large packages define a separate subpackage for exceptions so you can say
import bigpackage.exceptions
to import them all. You will probably still want to explore the package's documentation (or, if it's lacking, its source code) to discover which exceptions exist and how they are organized. Many packages define a base exception class from which all its other exceptions are subclasses so that you can easily catch them all with just one symbol, like
try:
bigpackage.heavy_wizardry()
except bigpackage.BigBaseException:
print("you are turned into a frog")
EDIT : you can import other methods instead of creating your own, of course
The try/except block will try to execute the code and if an error is raised and specified in the except statement, it will stop the execution of the code located in the try block and execute the other code located in the except block. So, to catch your custom error, you have to raise it in the first place.
If you didn't know, you can raise errors using the raise statement. Here, I've made a simple chunk of code. I have a custom error, a variable x initialized at 2, and a method that adds 1 to the variable given in argument. The method will raise a CustomError if the variable becomes 3.
# Here, I define the custom error and allow a custom message to be displayed
# using the super() method
class CustomError(Exception):
def __init__(self, msg):
super().__init__(msg)
# I initialize x at 2
x = 2
# I create the method that will add 1 to the variable given in argument
# and raise a CustomError if the variable becomes 3
# This is completely random, and you can make whatever checks you want before raising
# Your custom error
def do_stuff(x):
x += 1
if x == 3:
raise CustomError("x is 3")
# Now, I write the try/except block. I raise e (the CustomError) if it is
# being raised in the method do_stuff(x)
try:
do_stuff(x)
except CustomError as e:
raise e
Feel free to try the code out to get a better understanding of it !
Usually if a function
module.foo()
throws an exception DoStuffInsufficientMemoryException it would be also importable as
from module import DoStuffInsufficientMemoryException
If this results in ImportError then you need the fullname function from this answer; use it with e (it takes an instance and returns the class name). If it gives
foo.bar.DoStuffInsufficientMemoryException
then you can import the exception as
from foo.bar import DoStuffInsufficientMemoryException
The above might not work for all cases. One notable case is Boto 3 AWS client library that does not make the exceptions importable - instead they will be attributes on the client instance.
The questions:
Is there a robust way to distinguish the import error of the current call to __import__ from the nested import errors?
Is there another robust way to check the module existense and accessibility except of try statement?
The story:
I've found this code in python2.7/unittest/loader.py.
parts = name.split('.')
if module is None:
parts_copy = parts[:]
while parts_copy:
try:
module = __import__('.'.join(parts_copy))
break
except ImportError:
del parts_copy[-1]
if not parts_copy:
raise
It tries to import module by it's name. If failed then it removes the last part from the list of parts and tries again, until the list is empty.
The code relies on the ImportError exception. It is based on assumption that the exception means the specified name is not a module name, but a name of some object in that module (maybe function or class).
The problem is that the ImportError exception may be raised by a nested import including my own modules. If so, I'd not like to have this error be suppressed and handled as a normal situation by this code. I want to know why my code does not work without deep dive into the library source code. But this kind of problems makes it hard and wastes the developer's time. I'd like to propose a change to this library, but I'm not sure what is the good solution in this case.
If exceptions are always fatal, making use of them in Python is easy
# moduleB.py
import moduleC
But evaluating an exception requires more than just it's type, we often need to determine where an exception came from
# moduleA.py
try:
import moduleB
except ImportError as e:
print e
if str(e) == "No module named moduleB":
pass
else:
raise
In some projects this pattern results in a lot of code that is not easy to read. Is this the best way to ensure that I'm catching a local exception? I would like to be able write
import moduleB else pass
Sorry, but the example you posted is the canonical Way Of Doing It. Python doesn't have any syntax for catching the exception raised by the import statement but not by something below it.
Just a small warning, though: str(e) can cause unicode errors if e's message is unicode. You can fix that by using repr(e).
No, don't try to analyse the error string. You can log the exception for debugging if you want, but if it's "import if you can", then this will suffice:
try:
import foo
except ImportError:
pass # or foo = None or whatever
In your specific case, parsing the string is the only viable solution, but note that you don't need to do it: what is the difference if the moduleB or if it is a module imported by the moduleB not to be found?
If you are throwing custom exceptions, you can provide additional information when you raise them:
raise Exception(12) # 12 is the error code
and then get it through the args property:
if e.args[0] == 12:
handle()
A better way may be to subclass Exception and provide your own properties (for example error_code or module_name).
try:
import moduleB
except ImportError:
pass
This ought to be both sufficient and succinct enough.
Quick background: writing a module. One of my objects has methods that may or may not be successfully completed - depending on the framework used underneath my module. So a few methods first need to check what framework they actually have under their feet. Current way of tackling this is:
def framework_dependent_function():
try:
import module.that.may.not.be.available
except ImportError:
# the required functionality is not available
# this function can not be run
raise WrongFramework
# or should I just leave the previous exception reach higher levels?
[ ... and so on ... ]
Yet something in my mind keeps telling me that doing imports in the middle of a file is a bad thing. Can't remember why, can't even come up with a reason - apart from slightly messier code, I guess.
So, is there anything downright wrong about doing what I'm doing here? Perhaps other ways of scouting what environment the module is running in, somewhere near __init__?
This version may be faster, because not every call to the function needs to try to import the necessary functionality:
try:
import module.that.may.not.be.available
def framework_dependent_function():
# whatever
except ImportError:
def framework_dependent_function():
# the required functionality is not available
# this function can not be run
raise NotImplementedError
This also allows you to do a single attempt to import the module, then define all of the functions that might not be available in a single block, perhaps even as
def notimplemented(*args, **kwargs):
raise NotImplementedError
fn1 = fn2 = fn3 = notimplemented
Put this at the top of your file, near the other imports, or in a separate module (my current project has one called utils.fixes). If you don't like function definition in a try/except block, then do
try:
from module.that.may.not.be.available import what_we_need
except ImportError:
what_we_need = notimplemented
If these functions need to be methods, you can then add them to your class later:
class Foo(object):
# assuming you've added a self argument to the previous function
framework_dependent_method = framework_dependent_function
Similar to larsmans suggestion but with a slight change
def NotImplemented():
raise NotImplementedError
try:
import something.external
except ImportError:
framework_dependent_function = NotImplemented
def framework_dependent_function():
#whatever
return
I don't like the idea of function definitions in the try: except: of the import
You could also use imp.find_module (see here) in order to check for the presence of a specific module.