Develop Reference

What is the difference between except and except BaseException

What is the difference between those two:
except:
# do something
and
except BaseException as be:
print(be)
I mean in the first case all possible exception are caught, but is this true for the second?
Also can the error message be printed using the first case?

The accepted answer is incorrect incomplete (at least for Python 3.6 and above).
By catching Exception you catch most errors - basically all the errors that any module you use might throw.
By catching BaseException, in addition to all the above exceptions, you also catch exceptions of the types SystemExit, KeyboardInterrupt, and GeneratorExit.
By catching KeyboardInterrupt, for example, you may stop your code from exiting after an initiated exit by the user (like pressing ^C in the console, or stopping launched application on some interpreters). This could be a wanted behavior (for example - to log an exit), but should be used with extreme care!
In the above example, by catching BaseException, you may cause your application to hang when you want it to exit.

Practically speaking, there is no difference between except: and except BaseException:, for any current Python release.
That's because you can't just raise any type of object as an exception. The raise statement explicitly disallows raising anything else:
[...] raise evaluates the first expression as the exception object. It must be either a subclass or an instance of BaseException.
Bold emphasis mine. This has not always been the case however, in older Python releases (2.4 and before) you could use strings as exceptions too.
The advantage then is that you get to have easy access to the caught exception. In order to be able to add as targetname, you must catch a specific class of exceptions, and only BaseException is going to do that.
You can still access the currently active exception by using sys.exc_info() though:
except:
be = sys.exc_info()[1]
Pick what you feel is more readable for your future self and for your colleagues.

Python: Make exceptions 'exiting'

In Python, is there any (proper) way to change the the default exception handling behaviour so that any uncaught exception will terminate/exit the program?
I don't want to wrap the entire program in a generic try-except block:
try:
// write code here
except Exception:
sys.exit(1)
For those asking for more specificity and/or claiming this is already the case, it's my understanding that not all Python exceptions are system-exiting: docs
Edit: It looks like I have forked processes complicating matters so won't be posting any specific details about my own mess.
If you're looking for an answer to the original question, Dmitry's comment is interesting and useful, references the 2nd answer to this question

You can use Specific exception instead of Exception because Exception is a Base class for all exceptions. For more details refer Exception tutorial
You can write your script like this-
try:
# write code here
except OverflowError:
raise SystemExit
except ArithmeticError:
sys.exit()
except IOError:
quit()
Try this different approaches to find what is exactly you are missing.
Edit 1 - Maintain Program Execution
In order to maintain your program execution try this one-
consider error_handler function is raising SystemExit exception then In your main method you need to add below code so you can maintain your program execution.
try:
error_handler()
except SystemExit:
print "sys.exit was called but I'm proceeding anyway (so there!-)."

Robust endless loop for server written in Python

I write a server which handles events and uncaught exceptions during handling the event must not terminate the server.
The server is a single non-threaded python process.
I want to terminate on these errors types:
KeyboardInterrupt
MemoryError
...
The list of built in exceptions is long: https://docs.python.org/2/library/exceptions.html
I don't want to re-invent this exception handling, since I guess it was done several times before.
How to proceed?
Have a white-list: A list of exceptions which are ok and processing the next event is the right choice
Have a black-list: A list of exceptions which indicate that terminating the server is the right choice.
Hint: This question is not about running a unix daemon in background. It is not about double fork and not about redirecting stdin/stdout :-)

I would do this in a similar way you're thinking of, using the 'you shall not pass' Gandalf exception handler except Exception to catch all non-system-exiting exceptions while creating a black-listed set of exceptions that should pass and end be re-raised.
Using the Gandalf handler will make sure GeneratorExit, SystemExit and KeyboardInterrupt (all system-exiting exceptions) pass and terminate the program if no other handlers are present higher in the call stack. Here is where you can check with type(e) that a __class__ of a caught exception e actually belongs in the set of black-listed exceptions and re-raise it.
As a small demonstration:
import exceptions # Py2.x only
# dictionary holding {exception_name: exception_class}
excptDict = vars(exceptions)
exceptionNames = ['MemoryError', 'OSError', 'SystemError'] # and others
# set containing black-listed exceptions
blackSet = {excptDict[exception] for exception in exceptionNames}
Now blackSet = {OSError, SystemError, MemoryError} holding the classes of the non-system-exiting exceptions we want to not handle.
A try-except block can now look like this:
try:
# calls that raise exceptions:
except Exception as e:
if type(e) in blackSet: raise e # re-raise
# else just handle it
An example which catches all exceptions using BaseException can help illustrate what I mean. (this is done for demonstration purposes only, in order to see how this raising will eventually terminate your program). Do note: I'm not suggesting you use BaseException; I'm using it in order to demonstrate what exception will actually 'pass through' and cause termination (i.e everything that BaseException catches):
for i, j in excptDict.iteritems():
if i.startswith('__'): continue # __doc__ and other dunders
try:
try:
raise j
except Exception as ex:
# print "Handler 'Exception' caught " + str(i)
if type(ex) in blackSet:
raise ex
except BaseException:
print "Handler 'BaseException' caught " + str(i)
# prints exceptions that would cause the system to exit
Handler 'BaseException' caught GeneratorExit
Handler 'BaseException' caught OSError
Handler 'BaseException' caught SystemExit
Handler 'BaseException' caught SystemError
Handler 'BaseException' caught KeyboardInterrupt
Handler 'BaseException' caught MemoryError
Handler 'BaseException' caught BaseException
Finally, in order to make this Python 2/3 agnostic, you can try and import exceptions and if that fails (which it does in Python 3), fall-back to importing builtins which contains all Exceptions; we search the dictionary by name so it makes no difference:
try:
import exceptions
excDict = vars(exceptions)
except ImportError:
import builtins
excDict = vars(builtins)
I don't know if there's a smarter way to actually do this, another solution might be instead of having a try-except with a signle except, having 2 handlers, one for the black-listed exceptions and the other for the general case:
try:
# calls that raise exceptions:
except tuple(blackSet) as be: # Must go first, of course.
raise be
except Exception as e:
# handle the rest

The top-most exception is BaseException. There are two groups under that:
Exception derived
everything else
Things like Stopiteration, ValueError, TypeError, etc., are all examples of Exception.
Things like GeneratorExit, SystemExit and KeyboardInterrupt are not descended from Execption.
So the first step is to catch Exception and not BaseException which will allow you to easily terminate the program. I recommend also catching GeneratorExit as 1) it should never actually be seen unless it is raised manually; 2) you can log it and restart the loop; and 3) it is intended to signal a generator has exited and can be cleaned up, not that the program should exit.
The next step is to log each exception with enough detail that you have the possibility of figuring out what went wrong (when you later get around to debugging).
Finally, you have to decide for yourself which, if any, of the Exception derived exceptions you want to terminate on: I would suggest RuntimeError and MemoryError, although you may be able to get around those by simply stopping and restarting your server loop.
So, really, it's up to you.
If there is some other error (such as IOError when trying to load a config file) that is serious enough to quit on, then the code responsible for loading the config file should be smart enough to catch that IOError and raise SystemExit instead.
As far as whitelist/blacklist -- use a black list, as there should only be a handful, if any, Exception-based exceptions that you need to actually terminate the server on.

Which exceptions i should catch and which should not in Python

For example i have a program with this structure:
Domain logic module -> Settings module -> Settings store backend
Next is a part of Settings module.
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise SettingsLoadDataException('Error loading json')
Need I a custom exception SettingsLoadDataException here, or I could just skip catching json.loads errors?
def load_from_json(self, json_str):
self.load_from_dict(json.loads(json_str))
Update.
Also good variant is:
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise ValueError('Error loading json')

That is a problem only you can answer. You could catch all exceptions, or you could let the program crash if it throws an exception you don't handle. If it is vital that the program doesn't crash, catch the exception. However, you should implement a recovery method then. If the Json doesn't load properly, can your program do anything useful without it ? If it can, I would catch the exception, otherwise you could just display an error and terminate.

You should work with exceptions in such a way, that seeing a stack trace explains the problem to you immediately.
I am no Python expert, but won't you loose the piece of information that it was actually ValueError, that caused program crash? You will see only SettingsLoadDataException in a trace without any real reason of it, right?
Also, if you do not rethrow exceptions, you should catch only those, you know how to deal with. It is always better to have your program crash, than to leave it in an unexpected state.

How to prevent "too broad exception" in this case?

I have a list of functions that may fail and, if one fails, I don't want the script to stop, but to continue with next function.
I am executing it with something like this :
list_of_functions = [f_a, f_b, f_c]
for current_function in list_of_functions:
try:
current_function()
except Exception:
print(traceback.format_exc())
It's working fine, but it is not PEP8 compliant:
When catching exceptions, mention specific exceptions whenever
possible instead of using a bare except: clause.
For example, use:
try:
import platform_specific_module
except ImportError:
platform_specific_module = None
A bare except: clause will catch SystemExit and KeyboardInterrupt
exceptions, making it harder to interrupt a program with Control-C,
and can disguise other problems. If you want to catch all exceptions
that signal program errors, use except Exception: (bare except is
equivalent to except BaseException: ).
A good rule of thumb is to limit use of bare 'except' clauses to two
cases:
If the exception handler will be printing out or logging the traceback; at least the user will be aware that an error has occurred.
If the code needs to do some cleanup work, but then lets the exception propagate upwards with raise . try...finally can be a better
way to handle this case.
How can I do this the good way?

The PEP8 guide you quote suggests that it is okay to use a bare exception in your case provided you are logging the errors. I would think that you should cover as many exceptions as you can/know how to deal with and then log the rest and pass, e.g.
import logging
list_of_functions = [f_a,f_b,f_c]
for current_function in list_of_functions:
try:
current_function()
except KnownException:
raise
except Exception as e:
logging.exception(e)

Use this to cheat PEP8:
try:
"""code"""
except (Exception,):
pass

I think in some rare cases catching general exception is just justified and there is a way to trick PEP8 inspection:
list_of_functions = [f_a,f_b,f_c]
for current_function in list_of_functions:
try:
current_function()
except (ValueError, Exception):
print(traceback.format_exc())
You can replace ValueError by any other. It works for me (at least in PyCharm).

You can just put a comment like except Exception as error: # pylint: disable=broad-except that's worked for me actually. I hope it could be work for you.

From issue PY-9715 on yourtrack.jetbrains.com:
From pep-0348:
BaseException
The superclass that all exceptions must inherit from. It's name was
chosen to reflect that it is at the base of the exception hierarchy
while being an exception itself. "Raisable" was considered as a name,
it was passed on because its name did not properly reflect the fact
that it is an exception itself.
Direct inheritance of BaseException is not expected, and will be
discouraged for the general case. Most user-defined exceptions should
inherit from Exception instead. This allows catching Exception to
continue to work in the common case of catching all exceptions that
should be caught. Direct inheritance of BaseException should only be
done in cases where an entirely new category of exception is desired.
But, for cases where all exceptions should be caught blindly, except
BaseException will work.

You can avoid the error if you then re-raise the Exception. This way you are able to do damage control and not endanger loosing track of its occurance.

Do you perhaps mean that each function can raise different exceptions? When you name the exception type in the except clause it can be any name that refers to an exception, not just the class name.
eg.
def raise_value_error():
raise ValueError
def raise_type_error():
raise TypeError
def raise_index_error():
doesnt_exist
func_and_exceptions = [(raise_value_error, ValueError), (raise_type_error, TypeError),
(raise_index_error, IndexError)]
for function, possible_exception in func_and_exceptions:
try:
function()
except possible_exception as e:
print("caught", repr(e), "when calling", function.__name__)
prints:
caught ValueError() when calling raise_value_error
caught TypeError() when calling raise_type_error
Traceback (most recent call last):
File "run.py", line 14, in <module>
function()
File "run.py", line 8, in raise_index_error
doesnt_exist
NameError: name 'doesnt_exist' is not defined
Of course that leaves you with not knowing what to do when each exception occurs. But since you just want to ignore it and carry on then that's not a problem.

First, generate the pylintrc using the below command
pylint --generate-rcfile > .pylintrc
For reference:
https://learn.microsoft.com/en-us/visualstudio/python/linting-python-code?view=vs-2022
Search for disable (uncomment if needed) in the generate pylintrc file and add the below exception.
broad-except
Rerun the pylint command and see the magic