My largest issue is the fancy IPython exceptions. I want them look like normal python
exceptions, but when I try to reset sys.excepthook it does not work:
In [31]: import sys; sys.excepthook = sys.__excepthook__; (sys.excepthook, sys.__excepthook__)
Out[31]:
(<bound method TerminalInteractiveShell.excepthook of <IPython.terminal.interactiveshell.TerminalInteractiveShell object at 0x27c8e10>>,
<function sys.excepthook>)
IPython replaces sys.excepthook each time you execute a line of code, rendering it pointless to change it each time. In addition to this, IPython catches all exceptions and handles them itself, without having to invoke sys.excepthook.
This answer to a related question provides a way of overriding this behaviour: Basically, you've to override Ipython's showtraceback function with one that will format and display your exception the way you want it.
def showtraceback(self):
traceback_lines = traceback.format_exception(*sys.exc_info())
del traceback_lines[1]
message = ''.join(traceback_lines)
sys.stderr.write(message)
import sys
import traceback
import IPython
IPython.core.interactiveshell.InteractiveShell.showtraceback = showtraceback
Related
Situation
The xlwings package provides a convenient way to call python functions from an excel VBA module. The xlwings documentation gives the following basic example:
Write the code below into a VBA module.
Sub HelloWorld()
RunPython ("import hello; hello.world()")
End Sub
This calls the following code in hello.py:
# hello.py
import numpy as np
import xlwings as xw
def world():
wb = xw.Book.caller()
wb.sheets[0].range('A1').value = 'Hello World!'
Trying to run the python function world() directly (instead of calling it from excel VBA) gives the following error message:
Exception: Book.caller() must not be called directly. Call through
Excel or set a mock caller first with Book.set_mock_caller().
Question
I would like to modify the world() function such that it raises a custom exception instead when being run directly. In order to achieve this I first need to determine programmatically whether the world() function is being run directly or being called from excel VBA (at least that's what I'd think). How could I do this?
You can catch the exception and then raise your own:
def world():
try:
wb = xw.Book.caller()
except Exception:
raise CustomException(custom_message)
wb.sheets[0].range('A1').value = 'Hello World!'
You are worried that Exception is too generic, and rightly so. But that's xlwings' fault, not yours. If it is raising a generic Exception that's all you are left with to catch. You could check the exception message to make sure that you are not catching the wrong exception, but that would be brittle. Error messages are usually undocumented and not to be regarded as public, stable API.
Alternatively you can fix the problem where the problem is, xlwings' source code, and make it do what looks to me like the right thing to do: raising a more specific exception.
class NotFromExcelError(Exception):
pass
And at the end of caller:
raise NotFromExcelError('Book.caller() must not be called directly. Call through Excel '
'or set a mock caller first with Book.set_mock_caller().')
I hope a pull request like this would be accepted because raising a bare Exception like it currently does looks really wrong.
I have a few long-running experiments in my Jupyter Notebooks. Because I don't know when they will finish, I add an email function to the last cell of the notebook, so I automatically get an email, when the notebook is done.
But when there is a random exception in one of the cells, the whole notebook stops executing and I never get any email. So I'm wondering if there is some magic function that could execute a function in case of an exception / kernel stop.
Like
def handle_exception(stacktrace):
send_mail_to_myself(stacktrace)
%%in_case_of_notebook_exception handle_exception # <--- this is what I'm looking for
The other option would be to encapsulate every cell in try-catch, right? But that's soooo tedious.
Thanks in advance for any suggestions.
Such a magic command does not exist, but you can write it yourself.
from IPython.core.magic import register_cell_magic
#register_cell_magic('handle')
def handle(line, cell):
try:
exec(cell)
except Exception as e:
send_mail_to_myself(e)
raise # if you want the full trace-back in the notebook
It is not possible to load the magic command for the entire notebook automatically, you have to add it at each cell where you need this feature.
%%handle
some_code()
raise ValueError('this exception will be caught by the magic command')
#show0k gave the correct answer to my question (in regards to magic methods). Thanks a lot! :)
That answer inspired me to dig a little deeper and I came across an IPython method that lets you define a custom exception handler for the whole notebook.
I got it to work like this:
from IPython.core.ultratb import AutoFormattedTB
# initialize the formatter for making the tracebacks into strings
itb = AutoFormattedTB(mode = 'Plain', tb_offset = 1)
# this function will be called on exceptions in any cell
def custom_exc(shell, etype, evalue, tb, tb_offset=None):
# still show the error within the notebook, don't just swallow it
shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)
# grab the traceback and make it into a list of strings
stb = itb.structured_traceback(etype, evalue, tb)
sstb = itb.stb2text(stb)
print (sstb) # <--- this is the variable with the traceback string
print ("sending mail")
send_mail_to_myself(sstb)
# this registers a custom exception handler for the whole current notebook
get_ipython().set_custom_exc((Exception,), custom_exc)
So this can be put into a single cell at the top of any notebook and as a result it will do the mailing in case something goes wrong.
Note to self / TODO: make this snippet into a little python module that can be imported into a notebook and activated via line magic.
Be careful though. The documentation contains a warning for this set_custom_exc method: "WARNING: by putting in your own exception handler into IPython’s main execution loop, you run a very good chance of nasty crashes. This facility should only be used if you really know what you are doing."
Since notebook 5.1 you can use a new tag: raises-exception
This will indicate that exception in the specific cell is expected and jupyter will not stop the execution.
(In order to set a tag you have to choose from the main menu: View -> Cell Toolbar -> Tags)
Why exec is not always the solution
It's some years later and I had a similar issue trying to handle errors with Jupyter magics. However, I needed variables to persist in the actual Jupyter notebook.
%%try_except print
a = 12
raise ValueError('test')
In this example, I want the error to print (but could be anything such as e-mail as in the opening post), but also a == 12 to be true in the next cell. For that reason, the method exec suggested does not work when you define the magic in a different file. The solution I found is to use the IPython functionalities.
How you can solve it
from IPython.core.magic import line_magic, cell_magic, line_cell_magic, Magics, magics_class
#magics_class
class CustomMagics(Magics):
#cell_magic
def try_except(self, line, cell):
""" This magic wraps a cell in try_except functionality """
try:
self.shell.ex(cell) # This executes the cell in the current namespace
except Exception as e:
if ip.ev(f'callable({how})'): # check we have a callable handler
self.shell.user_ns['error'] = e # add error to namespace
ip.ev(f'{how}(error)') # call the handler with the error
else:
raise e
# Register
from IPython import get_ipython
ip = get_ipython()
ip.register_magics(CustomMagics)
I don't think there is an out-of-the-box way to do that not using a try..except statement in your cells. AFAIK a 4 years old issue mentions this, but is still in open status.
However, the runtools extension may do the trick.
We can define an alias in ipython with the %alias magic function, like this:
>>> d
NameError: name 'd' is not defined
>>> %alias d date
>>> d
Fri May 15 00:12:20 AEST 2015
This escapes to the shell command date when you type d into ipython.
But I want to define an alias to execute some python code, in the current interpreter scope, rather than a shell command. Is that possible? How can we make this kind of alias?
I work in the interactive interpreter a lot, and this could save me a lot of commands I find myself repeating often, and also prevent some common typos.
The normal way to do this would be to simply write a python function, with a def. But if you want to alias a statement, rather than a function call, then it's actually a bit tricky.
You can achieve this by writing a custom magic function. Here is an example, which effectively aliases the import statement to get, within the REPL.
from IPython.core.magic import register_line_magic
#register_line_magic
def get(line):
code = f"import {line}"
print("-->", code)
exec(code, globals())
del get # in interactive mode-remove from scope so function doesn't shadow magic
edit: below is the previous code, for older versions of IPython
from IPython.core.magic_arguments import argument, magic_arguments
#magic_arguments()
#argument('module')
def magic_import(self, arg):
code = 'import {}'.format(arg)
print('--> {}'.format(code))
self.shell.run_code(code)
ip = get_ipython()
ip.define_magic('get', magic_import)
Now it is possible to execute get statements which are aliased to import statements.
Demo:
In [1]: get json
--> import json
In [2]: json.loads
Out[2]: <function json.loads>
In [3]: get potato
--> import potato
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<string> in <module>()
ImportError: No module named potato
In [4]:
Of course, this is extendible to arbitrary python code, and optional arguments are supported aswell.
I don't know since when IPython provides with macro. And now you can simply do this:
ipy = get_ipython()
ipy.define_macro('d', 'date')
You can put this code into any file located in ~/.ipython/profile_default/startup/, and then this macro will be automatically available when you start IPython.
However, a macro doesn't accept arguments. So pleaes keep this in mind before you choose to define a macro.
Is there a way to launch an IPython shell or prompt when my program runs a line that raises an exception?
I'm mostly interested in the context, variables, in the scope (and subscopes) where the exception was raised. Something like Visual Studio's debugging, when an exception is thrown but not caught by anyone, Visual Studio will halt and give me the call stack and the variables present at every level.
Do you think there's a way to get something similar using IPython?
EDIT: The -pdb option when launching IPython doesn't seem do what I want (or maybe I don't know how to use it properly, which is entirely possible). I run the following script :
def func():
z = 2
g = 'b'
raise NameError("This error will not be caught, but IPython still"
"won't summon pdb, and I won't be able to consult"
"the z or g variables.")
x = 1
y = 'a'
func()
Using the command :
ipython -pdb exceptionTest.py
Which stops execution when the error is raised, but brings me an IPython prompt where I have access to the global variables of the script, but not the local variables of function func. pdb is only invoked when I directly type a command in ipython that causes an error, i.e. raise NameError("This, sent from the IPython prompt, will trigger pdb.").
I don't necessarily need to use pdb, I'd just like to have access to the variables inside func.
EDIT 2: It has been a while, IPython's -pdb option is now working just as I want it to. That means when I raise an exception I can go back in the scope of func and read its variables z and g without any problem. Even without setting the -pdb option, one can run IPython in interactive mode then call the magic function %debug after the program has exit with error -- that will also drop you into an interactive ipdb prompt with all scopes accessibles.
Update for IPython v0.13:
import sys
from IPython.core import ultratb
sys.excepthook = ultratb.FormattedTB(mode='Verbose',
color_scheme='Linux', call_pdb=1)
Doing:
ipython --pdb -c "%run exceptionTest.py"
kicks off the script after IPython initialises and you get dropped into the normal IPython+pdb environment.
You can try this:
from ipdb import launch_ipdb_on_exception
def main():
with launch_ipdb_on_exception():
# The rest of the code goes here.
[...]
ipdb integrates IPython features into pdb. I use the following code to throw my apps into the IPython debugger after an unhanded exception.
import sys, ipdb, traceback
def info(type, value, tb):
traceback.print_exception(type, value, tb)
ipdb.pm()
sys.excepthook = info
#snapshoe's answer does not work on newer versions of IPython.
This does however:
import sys
from IPython import embed
def excepthook(type, value, traceback):
embed()
sys.excepthook = excepthook
#Adam's works like a charm except that IPython loads a bit slowly(800ms on my machine). Here I have a trick to make the load lazy.
class ExceptionHook:
instance = None
def __call__(self, *args, **kwargs):
if self.instance is None:
from IPython.core import ultratb
self.instance = ultratb.FormattedTB(mode='Verbose',
color_scheme='Linux', call_pdb=1)
return self.instance(*args, **kwargs)
sys.excepthook = ExceptionHook()
Now we don't need to wait at the very beginning. Only when the program crashes will cause IPython to be imported.
You can do something like the following:
import sys
from IPython.Shell import IPShellEmbed
ipshell = IPShellEmbed()
def excepthook(type, value, traceback):
ipshell()
sys.excepthook = excepthook
See sys.excepthook and Embedding IPython.
If you want to both get the traceback and open a IPython shell with the environment at the point of the exception:
def exceptHook(*args):
'''A routine to be called when an exception occurs. It prints the traceback
with fancy formatting and then calls an IPython shell with the environment
of the exception location.
'''
from IPython.core import ultratb
ultratb.FormattedTB(call_pdb=False,color_scheme='LightBG')(*args)
from IPython.terminal.embed import InteractiveShellEmbed
import inspect
frame = inspect.getinnerframes(args[2])[-1][0]
msg = 'Entering IPython console at {0.f_code.co_filename} at line {0.f_lineno}'.format(frame)
savehook = sys.excepthook # save the exception hook
InteractiveShellEmbed()(msg,local_ns=frame.f_locals,global_ns=frame.f_globals)
sys.excepthook = savehook # reset IPython's change to the exception hook
import sys
sys.excepthook = exceptHook
Note that it is necessary to pull than namespace information from the last frame referenced by the traceback (arg[2])
This man page says iPython has --[no]pdb option to be passed at command line to start iPython for uncaught exceptions. Are you looking for more?
EDIT:
python -m pdb pythonscript.py can launch pdb. Not sure about similar thing with iPython though. If you are looking for the stack trace and general post-mortem of the abnormal exit of program, this should work.
Do you actually want to open a pdb session at every exception point? (as I think a pdb session opened from ipython is the same as the one open in the normal shell). If that's the case, here's the trick:
http://code.activestate.com/recipes/65287-automatically-start-the-debugger-on-an-exception/
Is there a convenient way to get a more detailed stack trace on a Python exception? I'm hoping to find a wrapper utility/module or some other way to get a bit more info from the stack trace without having to actually modify the Python script that generates it. I'd like to be able to use this when running unit tests, or doctests, or when running utilities or inline scripts from the shell.
Specifically I think I'd like to have the values of local variables, or maybe just the values of the arguments passed to the innermost function in the stack trace. Some options to set the detail level would be nifty.
Not specifically related to your problem, but you might find this code useful -- automatically starts up the python debugger when a fatal exception occurs. Good for working with interactive code. It's originally from ActiveState
# code snippet, to be included in 'sitecustomize.py'
import sys
def info(type, value, tb):
if hasattr(sys, 'ps1') or not sys.stderr.isatty():
# we are in interactive mode or we don't have a tty-like
# device, so we call the default hook
sys.__excepthook__(type, value, tb)
else:
import traceback, pdb
# we are NOT in interactive mode, print the exception...
traceback.print_exception(type, value, tb)
print
# ...then start the debugger in post-mortem mode.
pdb.pm()
sys.excepthook = info
Did you have a look at traceback module?
http://docs.python.org/library/traceback.html
Also on SO:
Showing the stack trace from a running Python application
As mentionned by pyfunc, you can use the function in the traceback module but you only get a stacktrace.
If you want to inspect the stack you have to use the sys.exc_info() function and walk the traceback member and dump information from its frame (tb_frame). See the python Reference Manual for further information on these types.
Here is an example:
def killit(a):
a[10000000000000] = 1
def test(a):
killit(a)
def iterate_traceback(tb):
while tb is not None:
yield tb
tb = tb.tb_next
try:
test(tuple())
except Exception as e:
import sys
exception_info = sys.exc_info()
traceback = exception_info[2]
for tb in iterate_traceback(traceback):
print "-" * 10
print tb.tb_frame.f_code
print tb.tb_frame.f_locals
print tb.tb_frame.f_globals