I am printing Python exception messages to a log file with logging.error:
import logging
try:
1/0
except ZeroDivisionError as e:
logging.error(e) # ERROR:root:division by zero
Is it possible to print more detailed information about the exception and the code that generated it than just the exception string? Things like line numbers or stack traces would be great.
logger.exception will output a stack trace alongside the error message.
For example:
import logging
try:
1/0
except ZeroDivisionError:
logging.exception("message")
Output:
ERROR:root:message
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: integer division or modulo by zero
#Paulo Cheque notes, "be aware that in Python 3 you must call the logging.exception method just inside the except part. If you call this method in an arbitrary place you may get a bizarre exception. The docs alert about that."
Using exc_info options may be better, to allow you to choose the error level (if you use exception, it will always be at the error level):
try:
# do something here
except Exception as e:
logging.critical(e, exc_info=True) # log exception info at CRITICAL log level
One nice thing about logging.exception that SiggyF's answer doesn't show is that you can pass in an arbitrary message, and logging will still show the full traceback with all the exception details:
import logging
try:
1/0
except ZeroDivisionError:
logging.exception("Deliberate divide by zero traceback")
With the default (in recent versions) logging behaviour of just printing errors to sys.stderr, it looks like this:
>>> import logging
>>> try:
... 1/0
... except ZeroDivisionError:
... logging.exception("Deliberate divide by zero traceback")
...
ERROR:root:Deliberate divide by zero traceback
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: integer division or modulo by zero
Quoting
What if your application does logging some other way – not using the logging module?
Now, traceback could be used here.
import traceback
def log_traceback(ex, ex_traceback=None):
if ex_traceback is None:
ex_traceback = ex.__traceback__
tb_lines = [ line.rstrip('\n') for line in
traceback.format_exception(ex.__class__, ex, ex_traceback)]
exception_logger.log(tb_lines)
Use it in Python 2:
try:
# your function call is here
except Exception as ex:
_, _, ex_traceback = sys.exc_info()
log_traceback(ex, ex_traceback)
Use it in Python 3:
try:
x = get_number()
except Exception as ex:
log_traceback(ex)
You can log the stack trace without an exception.
https://docs.python.org/3/library/logging.html#logging.Logger.debug
The second optional keyword argument is stack_info, which defaults to False. If true, stack information is added to the logging message, including the actual logging call. Note that this is not the same stack information as that displayed through specifying exc_info: The former is stack frames from the bottom of the stack up to the logging call in the current thread, whereas the latter is information about stack frames which have been unwound, following an exception, while searching for exception handlers.
Example:
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> logging.getLogger().info('This prints the stack', stack_info=True)
INFO:root:This prints the stack
Stack (most recent call last):
File "<stdin>", line 1, in <module>
>>>
If you use plain logs - all your log records should correspond this rule: one record = one line. Following this rule you can use grep and other tools to process your log files.
But traceback information is multi-line. So my answer is an extended version of solution proposed by zangw above in this thread. The problem is that traceback lines could have \n inside, so we need to do an extra work to get rid of this line endings:
import logging
logger = logging.getLogger('your_logger_here')
def log_app_error(e: BaseException, level=logging.ERROR) -> None:
e_traceback = traceback.format_exception(e.__class__, e, e.__traceback__)
traceback_lines = []
for line in [line.rstrip('\n') for line in e_traceback]:
traceback_lines.extend(line.splitlines())
logger.log(level, traceback_lines.__str__())
After that (when you'll be analyzing your logs) you could copy / paste required traceback lines from your log file and do this:
ex_traceback = ['line 1', 'line 2', ...]
for line in ex_traceback:
print(line)
Profit!
This answer builds up from the above excellent ones.
In most applications, you won't be calling logging.exception(e) directly. Most likely you have defined a custom logger specific for your application or module like this:
# Set the name of the app or module
my_logger = logging.getLogger('NEM Sequencer')
# Set the log level
my_logger.setLevel(logging.INFO)
# Let's say we want to be fancy and log to a graylog2 log server
graylog_handler = graypy.GELFHandler('some_server_ip', 12201)
graylog_handler.setLevel(logging.INFO)
my_logger.addHandler(graylog_handler)
In this case, just use the logger to call the exception(e) like this:
try:
1/0
except ZeroDivisionError, e:
my_logger.exception(e)
If "debugging information" means the values present when exception was raised, then logging.exception(...) won't help. So you'll need a tool that logs all variable values along with the traceback lines automatically.
Out of the box you'll get log like
2020-03-30 18:24:31 main ERROR File "./temp.py", line 13, in get_ratio
2020-03-30 18:24:31 main ERROR return height / width
2020-03-30 18:24:31 main ERROR height = 300
2020-03-30 18:24:31 main ERROR width = 0
2020-03-30 18:24:31 main ERROR builtins.ZeroDivisionError: division by zero
Have a look at some pypi tools, I'd name:
tbvaccine
traceback-with-variables
better-exceptions
Some of them give you pretty crash messages:
But you might find some more on pypi
A little bit of decorator treatment (very loosely inspired by the Maybe monad and lifting). You can safely remove Python 3.6 type annotations and use an older message formatting style.
fallible.py
from functools import wraps
from typing import Callable, TypeVar, Optional
import logging
A = TypeVar('A')
def fallible(*exceptions, logger=None) \
-> Callable[[Callable[..., A]], Callable[..., Optional[A]]]:
"""
:param exceptions: a list of exceptions to catch
:param logger: pass a custom logger; None means the default logger,
False disables logging altogether.
"""
def fwrap(f: Callable[..., A]) -> Callable[..., Optional[A]]:
#wraps(f)
def wrapped(*args, **kwargs):
try:
return f(*args, **kwargs)
except exceptions:
message = f'called {f} with *args={args} and **kwargs={kwargs}'
if logger:
logger.exception(message)
if logger is None:
logging.exception(message)
return None
return wrapped
return fwrap
Demo:
In [1] from fallible import fallible
In [2]: #fallible(ArithmeticError)
...: def div(a, b):
...: return a / b
...:
...:
In [3]: div(1, 2)
Out[3]: 0.5
In [4]: res = div(1, 0)
ERROR:root:called <function div at 0x10d3c6ae8> with *args=(1, 0) and **kwargs={}
Traceback (most recent call last):
File "/Users/user/fallible.py", line 17, in wrapped
return f(*args, **kwargs)
File "<ipython-input-17-e056bd886b5c>", line 3, in div
return a / b
In [5]: repr(res)
'None'
You can also modify this solution to return something a bit more meaningful than None from the except part (or even make the solution generic, by specifying this return value in fallible's arguments).
In your logging module(if custom module) just enable stack_info.
api_logger.exceptionLog("*Input your Custom error message*",stack_info=True)
If you look at the this code example (which works for Python 2 and 3) you'll see the function definition below which can extract
method
line number
code context
file path
for an entire stack trace, whether or not there has been an exception:
def sentry_friendly_trace(get_last_exception=True):
try:
current_call = list(map(frame_trans, traceback.extract_stack()))
alert_frame = current_call[-4]
before_call = current_call[:-4]
err_type, err, tb = sys.exc_info() if get_last_exception else (None, None, None)
after_call = [alert_frame] if err_type is None else extract_all_sentry_frames_from_exception(tb)
return before_call + after_call, err, alert_frame
except:
return None, None, None
Of course, this function depends on the entire gist linked above, and in particular extract_all_sentry_frames_from_exception() and frame_trans() but the exception info extraction totals less than around 60 lines.
Hope that helps!
I wrap all functions around my custom designed logger:
import json
import timeit
import traceback
import sys
import unidecode
def main_writer(f,argument):
try:
f.write(str(argument))
except UnicodeEncodeError:
f.write(unidecode.unidecode(argument))
def logger(*argv,logfile="log.txt",singleLine = False):
"""
Writes Logs to LogFile
"""
with open(logfile, 'a+') as f:
for arg in argv:
if arg == "{}":
continue
if type(arg) == dict and len(arg)!=0:
json_object = json.dumps(arg, indent=4, default=str)
f.write(str(json_object))
f.flush()
"""
for key,val in arg.items():
f.write(str(key) + " : "+ str(val))
f.flush()
"""
elif type(arg) == list and len(arg)!=0:
for each in arg:
main_writer(f,each)
f.write("\n")
f.flush()
else:
main_writer(f,arg)
f.flush()
if singleLine==False:
f.write("\n")
if singleLine==True:
f.write("\n")
def tryFunc(func, func_name=None, *args, **kwargs):
"""
Time for Successfull Runs
Exception Traceback for Unsuccessful Runs
"""
stack = traceback.extract_stack()
filename, codeline, funcName, text = stack[-2]
func_name = func.__name__ if func_name is None else func_name # sys._getframe().f_code.co_name # func.__name__
start = timeit.default_timer()
x = None
try:
x = func(*args, **kwargs)
stop = timeit.default_timer()
# logger("Time to Run {} : {}".format(func_name, stop - start))
except Exception as e:
logger("Exception Occurred for {} :".format(func_name))
logger("Basic Error Info :",e)
logger("Full Error TraceBack :")
# logger(e.message, e.args)
logger(traceback.format_exc())
return x
def bad_func():
return 'a'+ 7
if __name__ == '__main__':
logger(234)
logger([1,2,3])
logger(['a','b','c'])
logger({'a':7,'b':8,'c':9})
tryFunc(bad_func)
My approach was to create a context manager, to log and raise Exceptions:
import logging
from contextlib import AbstractContextManager
class LogError(AbstractContextManager):
def __init__(self, logger=None):
self.logger = logger.name if isinstance(logger, logging.Logger) else logger
def __exit__(self, exc_type, exc_value, traceback):
if exc_value is not None:
logging.getLogger(self.logger).exception(exc_value)
with LogError():
1/0
You can either pass a logger name or a logger instance to LogError(). By default it will use the base logger (by passing None to logging.getLogger).
One could also simply add a switch for raising the error or just logging it.
If you can cope with the extra dependency then use twisted.log, you don't have to explicitly log errors and also it returns the entire traceback and time to the file or stream.
A clean way to do it is using format_exc() and then parse the output to get the relevant part:
from traceback import format_exc
try:
1/0
except Exception:
print 'the relevant part is: '+format_exc().split('\n')[-2]
Regards
Related
When I want to log some specific Exception, but otherwise ignore it, I can do that like so:
import logging
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
logger.error("It failed with:", exc_info=True)
(This is in fact an MRE, as something_that_may_fail hasn't been defined, so the try block will raise NameError with message name 'something_that_may_fail' is not defined. 😉)
This however will also log the stack trace:
It failed with:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
NameError: name 'something_that_may_fail' is not defined
Sometimes that isn't what I want: In some cases, I already know that exception type and exception message (together with my custom log message) will suffice, and don't want to expand the log with stack traces that don't tell me anything new. So I'd want a log entry that simply is
It failed with:
NameError: name 'something_that_may_fail' is not defined
I can achieve that by passing a 3-tuple as the exc_info, with the stack trace component replaced by None:
import logging
import sys
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
exc_type, exc_value, _trace = sys.exc_info()
logger.error("It failed with:", exc_info=(exc_type, exc_value, None))
But I'm not sure how reliable that is. (The documentation doesn't mention how the tuple may or may not deviate from one returned by sys.exc_info().)
Examining the exception myself with
...
except NameError as e:
...
comes with its own problems:
f"{type(e)}" gives string <class 'NameError'> instead of just string NameError
The proper solution to get the fully qualified type name, including packages/modules but without builtin. is rather unwieldy and not something I'd want in exception handling code. See the currently accepted answer to Get fully qualified class name of an object in Python.
Can I rely on the message always being e.args[0]? (I might have uses for other exceptions (with more sub-types) than just NameError, which I've used here only as an example.)
So what is the proper way to log exception type and message without the stack trace? Is there a cleaner way than my make-the-trace-None hack above?
traceback.format_exception_only can be used for that:
import logging
import sys
import traceback
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError:
exc_type, exc_value, _trace = sys.exc_info()
exc_desc_lines = traceback.format_exception_only(exc_type, exc_value)
exc_desc = ''.join(exc_desc_lines).rstrip()
logger.error(f"It failed with:\n{exc_desc}")
or without sys:
import logging
import traceback
logger = logging.getLogger(__name__)
try:
something_that_may_fail()
except NameError as e:
exc_desc_lines = traceback.format_exception_only(type(e), e)
exc_desc = ''.join(exc_desc_lines).rstrip()
logger.error(f"It failed with:\n{exc_desc}")
(Found this by looking how the logging module actually extracts and formats information from exc_info. There traceback.print_exception is being used, so I looked what else is available in the traceback module.)
I write a lot of Python code that uses external libraries. Frequently I will write a bug, and when I run the code I get a big long traceback in the Python console. 99.999999% of the time it's due to a coding error in my code, not because of a bug in the package. But the traceback goes all the way to the line of error in the package code, and either it takes a lot of scrolling through the traceback to find the code I wrote, or the traceback is so deep into the package that my own code doesn't even appear in the traceback.
Is there a way to "black-box" the package code, or somehow only show traceback lines from my code? I'd like the ability to specify to the system which directories or files I want to see traceback from.
In order to print your own stacktrace, you would need to handle all unhandled exceptions yourself; this is how the sys.excepthook becomes handy.
The signature for this function is sys.excepthook(type, value, traceback) and its job is:
This function prints out a given traceback and exception to sys.stderr.
So as long as you can play with the traceback and only extract the portion you care about you should be fine. Testing frameworks do that very frequently; they have custom assert functions which usually does not appear in the traceback, in other words they skip the frames that belong to the test framework. Also, in those cases, the tests usually are started by the test framework as well.
You end up with a traceback that looks like this:
[ custom assert code ] + ... [ code under test ] ... + [ test runner code ]
How to identify your code.
You can add a global to your code:
__mycode = True
Then to identify the frames:
def is_mycode(tb):
globals = tb.tb_frame.f_globals
return globals.has_key('__mycode')
How to extract your frames.
skip the frames that don't matter to you (e.g. custom assert code)
identify how many frames are part of your code -> length
extract length frames
def mycode_traceback_levels(tb):
length = 0
while tb and is_mycode(tb):
tb = tb.tb_next
length += 1
return length
Example handler.
def handle_exception(type, value, tb):
# 1. skip custom assert code, e.g.
# while tb and is_custom_assert_code(tb):
# tb = tb.tb_next
# 2. only display your code
length = mycode_traceback_levels(tb)
print ''.join(traceback.format_exception(type, value, tb, length))
install the handler:
sys.excepthook = handle_exception
What next?
You could adjust length to add one or more levels if you still want some info about where the failure is outside of your own code.
see also https://gist.github.com/dnozay/b599a96dc2d8c69b84c6
As others suggested, you could use sys.excepthook:
This function prints out a given traceback and exception to sys.stderr.
When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.
(emphasis mine)
It's possible to filter a traceback extracted by extract_tb (or similar functions from the traceback module) based on specified directories.
Two functions that can help:
from os.path import join, abspath
from traceback import extract_tb, format_list, format_exception_only
def spotlight(*show):
''' Return a function to be set as new sys.excepthook.
It will SHOW traceback entries for files from these directories. '''
show = tuple(join(abspath(p), '') for p in show)
def _check_file(name):
return name and name.startswith(show)
def _print(type, value, tb):
show = (fs for fs in extract_tb(tb) if _check_file(fs.filename))
fmt = format_list(show) + format_exception_only(type, value)
print(''.join(fmt), end='', file=sys.stderr)
return _print
def shadow(*hide):
''' Return a function to be set as new sys.excepthook.
It will HIDE traceback entries for files from these directories. '''
hide = tuple(join(abspath(p), '') for p in hide)
def _check_file(name):
return name and not name.startswith(hide)
def _print(type, value, tb):
show = (fs for fs in extract_tb(tb) if _check_file(fs.filename))
fmt = format_list(show) + format_exception_only(type, value)
print(''.join(fmt), end='', file=sys.stderr)
return _print
They both use the traceback.extract_tb. It returns "a list of “pre-processed” stack trace entries extracted from the traceback object"; all of them are instances of traceback.FrameSummary (a named tuple). Each traceback.FrameSummary object has a filename field which stores the absolute path of the corresponding file. We check if it starts with any of the directory paths provided as separate function arguments to determine if we'll need to exclude the entry (or keep it).
Here's an Example:
The enum module from the standard library doesn't allow reusing keys,
import enum
enum.Enum('Faulty', 'a a', module=__name__)
yields
Traceback (most recent call last):
File "/home/vaultah/so/shadows/main.py", line 23, in <module>
enum.Enum('Faulty', 'a a', module=__name__)
File "/home/vaultah/cpython/Lib/enum.py", line 243, in __call__
return cls._create_(value, names, module=module, qualname=qualname, type=type, start=start)
File "/home/vaultah/cpython/Lib/enum.py", line 342, in _create_
classdict[member_name] = member_value
File "/home/vaultah/cpython/Lib/enum.py", line 72, in __setitem__
raise TypeError('Attempted to reuse key: %r' % key)
TypeError: Attempted to reuse key: 'a'
We can restrict stack trace entries to our code (in /home/vaultah/so/shadows/main.py).
import sys, enum
sys.excepthook = spotlight('/home/vaultah/so/shadows')
enum.Enum('Faulty', 'a a', module=__name__)
and
import sys, enum
sys.excepthook = shadow('/home/vaultah/cpython/Lib')
enum.Enum('Faulty', 'a a', module=__name__)
give the same result:
File "/home/vaultah/so/shadows/main.py", line 22, in <module>
enum.Enum('Faulty', 'a a', module=__name__)
TypeError: Attempted to reuse key: 'a'
There's a way to exclude all site directories (where 3rd party packages are installed - see site.getsitepackages)
import sys, site, jinja2
sys.excepthook = shadow(*site.getsitepackages())
jinja2.Template('{%}')
# jinja2.exceptions.TemplateSyntaxError: unexpected '}'
# Generates ~30 lines, but will only display 4
Note: Don't forget to restore sys.excepthook from sys.__excepthook__. Unfortunately, you won't be able to "patch-restore" it using a context manager.
the traceback.extract_tb(tb) would return a tuple of error frames in the format(file, line_no, type, error_statement) , you can play with that to format the traceback. Also refer https://pymotw.com/2/sys/exceptions.html
import sys
import traceback
def handle_exception(ex_type, ex_info, tb):
print ex_type, ex_info, traceback.extract_tb(tb)
sys.excepthook = handle_exception
I've seen similar questions to this one but none of them really address the trackback.
If I have a class like so
class Stop_if_no_then():
def __init__(self, value one, operator, value_two, then, line_or_label, line_number):
self._firstvalue = value_one
self._secondvalue = value_two
self._operator = operator
self._gohere = line_or_label
self._then = then
self._line_number = line_number
def execute(self, OtherClass):
"code comparing the first two values and making changes etc"
What I want my execute method to be able to do is if self._then is not equal to the string "THEN" (in allcaps) then I want it to raise a custom error message and terminate the whole program while also not showing a traceback.
If the error is encountered the only thing that should print out would look something like (I'm using 3 as an example, formatting is not a problem) this.
`Syntax Error (Line 3): No -THEN- present in the statement.`
I'm not very picky about it actually being an exception class object, so there's no issue in that aspect. Since I will be using this in a while loop, simple if, elif just repeats the message over and over (because obviously I am not closing the loop). I have seen sys.exit() but that also prints out a giant block of red text, unless I am not using it correctly. I don't want to catch the exception in my loop because there are other classes in the same module in which I need to implement something like this.
You can turn off the traceback by limiting its depth.
Python 2.x
import sys
sys.tracebacklimit = 0
Python 3.x
In Python 3.5.2 and 3.6.1, setting tracebacklimit to 0 does not seem to have the intended effect. This is a known bug. Note that -1 doesn't work either. Setting it to None does however seem to work, at least for now.
In Python 3.6.2 and above you should set tracebacklimit to 0 or -1, as setting it to None does not disable the traceback output.
Python 3.6.1 and bellow results:
>>> import sys
>>> sys.tracebacklimit = 0
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
>>> sys.tracebacklimit = -1
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
>>> sys.tracebacklimit = None
>>> raise Exception
Exception
Python 3.6.2 and above results:
>>> import sys
>>> sys.tracebacklimit = 0
>>> raise Exception
Exception
>>> sys.tracebacklimit = -1
>>> raise Exception
Exception
>>> sys.tracebacklimit = None
>>> raise Exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception
Nevertheless, for better or worse, if multiple exceptions are raised, they can all still be printed. For example:
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
You can use SystemExit exception:
except Exception as err:
raise SystemExit(err)
https://docs.python.org/3/library/exceptions.html
You can use a try: and then except Exception as inst:
What that will do is give you your error message in a variable named inst and you can print out the arguments on the error with inst.args. Try printing it out and seeing what happens, and is any item in inst.args is the one you are looking for.
EDIT Here is an example I tried with pythons IDLE:
>>> try:
open("epik.sjj")
except Exception as inst:
d = inst
>>> d
FileNotFoundError(2, 'No such file or directory')
>>> d.args
(2, 'No such file or directory')
>>> d.args[1]
'No such file or directory'
>>>
EDIT 2: as for closing the program you can always raise and error or you can use sys.exit()
The cleanest way that I know is to use sys.excepthook.
You implement a three argument function that accepts type, value, and traceback and does whatever you like (say, only prints the value) and assign that function to sys.excepthook.
Here is an example:
import sys
def excepthook(type, value, traceback):
print(value)
sys.excepthook = excepthook
raise ValueError('hello')
This is available in both python 2 and python 3.
If you want to get rid of any traceback for customs exceptions and have line number,
you can do this trick
Python 3
import sys
import inspect
class NoTraceBackWithLineNumber(Exception):
def __init__(self, msg):
try:
ln = sys.exc_info()[-1].tb_lineno
except AttributeError:
ln = inspect.currentframe().f_back.f_lineno
self.args = "{0.__name__} (line {1}): {2}".format(type(self), ln, msg),
sys.exit(self)
class MyNewError(NoTraceBackWithLineNumber):
pass
raise MyNewError("Now TraceBack Is Gone")
Will give this output, and make the raise keyword useless
MyNewError (line 16): Now TraceBack Is Gone
"Exception chaining can be disabled by using from None " - Python docs
>>> try:
... open('database.sqlite')
... except IOError:
... raise RuntimeError from None
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
In general, if you want to catch any exception except SystemExit, and exit with the exception's message without the traceback, define your main function as below:
>>> import sys
>>> def main():
... try:
... # Run your program from here.
... raise RandomException # For testing
... except (Exception, KeyboardInterrupt) as exc:
... sys.exit(exc)
...
>>> main()
name 'RandomException' is not defined
$ echo $?
1
Note that in the case of multiple exceptions being raised, only one message is printed.
This answer is meant to improve upon the one by The-IT.
Sentry can detect additional data associated with an exception such as:
How do you raise such an exception from Python (it's a Django app) with your own additional data fields?.
I log exceptions using the logging library so after debugging the code a bit, I noticed the extra parameter:
import logging
logger = logging.getLogger('my_app_name')
def do_something():
try:
#do some stuff here that might break
except Exception, e:
logger.error(e, exc_info=1, extra={'extra-data': 'blah', })
Passing exc_info=1 is the same as calling logger.exception. However, exception() does not accept kwargs, which are required for using the extra parameter.
These values will show up in the 'Additional Data' section of the Sentry Error dashboard.
wes' answer didn't help me because I want to actually raise an exception, not only log it.
Here's what I did (client is the Raven Sentry client):
client.extra_context({'foo': 'bar'})
raise RuntimeError('Whoops, something went wrong!')
You might try one of these two approaches:
>>> # Raise the exception with the data you want.
>>> raise Exception('extra information')
Traceback (most recent call last):
File "<pyshell#64>", line 1, in <module>
raise Exception('extra information')
Exception: extra information
>>> # Catch an exception and add extra arguments.
>>> try:
raise Exception()
except Exception as error:
error.args += ('extra information',)
raise
Traceback (most recent call last):
File "<pyshell#68>", line 2, in <module>
raise Exception()
Exception: extra information
>>>
You can add as many additional data fields as you want by adding more arguments.
The Sentry handler adds that info in your screenshot when capturing the message for an exception, and takes that information from the traceback, not the exception itself.
You can add extra fields by passing extra keyword arguments to .capture(); the Django client does so for you if you pass in the request object, for example.
Currently, no other data is taken from exceptions. You'd have to expand the exception handling yourself to add such a facility.
None of the existing answers served my exact use case well (which was to add additional context from the django Request object into the sentry data). What ended up working very well for that after some digging was overriding the client using the SENTRY_CLIENT setting.
Here's a complete simple use case:
from raven.contrib.django.raven_compat import DjangoClient
class CustomSentryClient(DjangoClient):
def get_data_from_request(self, request):
result = super(EToolsSentryClient, self).get_data_from_request(request)
if getattr(request, 'custom_field', None):
if 'extra' not in result:
result['extra'] = {}
result['extra']['custom_field'] = request.custom_field
return result
and then in settings.py you would just add
SENTRY_CLIENT = 'myapp.CustomSentryClient'
This is a followup to my question Hang in Python script using SQLAlchemy and multiprocessing. As discussed in that question, pickling exceptions is problematic in Python. This is usually not a issue, but one case when it is, is when errors occur in the python multiprocessing module. Since multiprocessing moves objects around by pickling, if an error occurs inside a multiprocessing process, the entire process may hang, as demonstrated in that question.
One possible approach is to fix all the problematic exceptions, as discussed in that question. This is not easy, since one cannot easily know in advance which exceptions may be called. An alternative approach, which was suggested by lbolla in an answer to the question, is to catch the exception, construct an equivalent harmless exception, and then rethrow.
However, I'm not sure of exactly how to do this. Consider the following code.
class BadExc(Exception):
def __init__(self, message, a):
'''Non-optional param in the constructor.'''
Exception.__init__(self, message)
self.a = a
import sys
try:
try:
#print foo
raise BadExc("bad exception error message", "a")
except Exception, e:
raise Exception(e.__class__.__name__ + ": " +str(e)), None, sys.exc_info()[2]
except Exception, f:
pass
import cPickle
a = cPickle.dumps(f)
l = cPickle.loads(a)
print "raising error"
raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]
This code pickles and unpickles the exception, and then throws it, giving the error
raising error
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
Exception: BadExc: bad exception error message
Credits to Glenn Maynard's answer to "“Inner exception” (with traceback) in Python?".
This has the important stuff, namely the traceback, the error message, and the exception type, so this might be the best one can do. But ideally I'd like something that looks exactly like the original exception, namely
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
__main__.BadExc: bad exception error message
or more generally, with the name of the exception in the front, rather than Exception. Is this possible?
Alternatively, instead of the BadExc class, one can use the print foo statement instead, which gives a NameError. However, this exception does not require special handling.
You can override sys.excepthook to achieve what you want. It at least works for this example, but it's pretty hacky so please test and no promises :-)
import sys
def excepthook_wrapper(type, value, traceback):
if len(value.args) == 2:
name, msg = value.args
value.args = (msg,)
sys.__excepthook__(name, value, traceback)
else:
sys.__excepthook__(type, value, traceback)
sys.excepthook = excepthook_wrapper
(Edit: I'm not really happy with this because now 'normal' Exceptions with two arguments will get handled differently too. Possible solution, 'tag' your special Exceptions by passing "PICKLED" as a first argument and then check for that, instead of checking for the length of the args.)
And then create the Exception with two arguments, the name (__module__.__class__) and the Exception message (str(e)):
try:
try:
#print foo
raise BadExc("bad exception error message", "a")
except Exception, e:
cls = e.__class__
if hasattr(cls, '__module__'):
name = '{0}.{1}'.format(cls.__module__, cls.__name__)
else:
name = cls.__name__
raise Exception(name, str(e)), None, sys.exc_info()[2]
except Exception, f:
pass
Then this:
import cPickle
a = cPickle.dumps(f)
l = cPickle.loads(a)
print "raising error"
raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]
Prints:
raising error
Traceback (most recent call last):
File "test.py", line 18, in <module>
raise BadExc("bad exception error message", "a")
__main__.BadExc: bad exception error message