By default, ipython uses ipdb as debugger with %pdb or %debug magics.
However, I much prefer pdb++... Is there a way of changing the debugger called with these magics ? (I am aware I can simply use pdb.xpm() on exception with pdb++, but I'd like to make it work with ipython magic commands so that I don't have to wrap the code each time...)
So at least for limited circumstances and not in a way I'd necessarily recommend, the answer here is yes. I can't promise the below will work outside the confines of what I did, but it might give you enough insight to play around with it yourself. Caution is warranted because it involves changing undocumented attributes of the ipython shell class at runtime. TLDR: I hunted down how ipython calls the debugger when the %pdb magic is on or when you call the %debug magic, and I updated it to use the debugger I wanted. Skip the next two paragraphs if you just want the approach that worked for me and don't care about the hunt.
Long version: when you run ipython it starts an instance of TerminalInteractiveShell, which has a debugger_cls attribute telling you the debugger that ipython will launch. Unfortunately, at the level of TerminalInteractiveShell, debugger_cls is actually a property of the class, and has no setter that lets you modify it. Rather, it either gets set to Pdb (actually a more featureful ipython Pdb than the traditional pdb) or TerminalPdb (even more features).
If you dig deeper, however, you find that debugger_cls gets passed up to InteractiveShell to initialize how tracebacks are handled. There it seems to disappear into the initialization of InteractiveShell's InteractiveTB property, but actually just ends up as the debugger_cls attribute of that (InteractiveTB) class (by setting the inherited attribute from TBTools). Finally, this debugger_cls attribute only gets used to set the pdb attribute (more or less by doing TBToolsInstance.pdb = TBToolsInstance.debugger_cls()) in one of several places. In any case, it turns out that these attributes can be changed! And if you change them correctly they will percolate to the shell itself! Importantly, this relies on the fact that ipython makes use of the Traitlets package to create a Singleton object for the shell, and this allows you to gain access to that object from within the terminal itself. More on that below.
Below I show the code you can run in the ipython shell to achieve your desired result. As an example, I'm replacing the default debugger (TerminalPdb) with a modified version I created that deals more nicely with certain list comprehensions (LcTerminalPdb). The process (which you can run in the ipython shell) is as follows.
# import the TerminalInteractiveShell class
from IPython.terminal.interactiveshell import TerminalInteractiveShell
# grab the specific instance of the already initialized ipython
shl = TerminalInteractiveShell().instance()
# grab the InteractiveTB attribute (which is a class)
tbClass = shl.InteractiveTB
# load the debugger class you want to use; I assume it's accessible on your path
from LcTerminalPdb import LcTerminalPdb
# change tbClass's debugger_cls to the debugger you want (for consistency
# with the next line)
tbClass.debugger_cls = LcTerminalPdb
# more importantly, set the pdb attribute to an instance of the class
tbClass.pdb = tbClass.debugger_cls()
# The above line is necessary if you already have the terminal running
# (and have entered pdb at least once); otherwise, ipython will run it on
# its own
That's it! Note that because you call the instance() method of TerminalInteractiveShell, you are grabbing the object for the currently running shell, which is why the modifications will affect the shell itself and so all following debugs. For a bonus, you can add these lines of code to your ipython_config.py file, so the debugger you want (LcTerminalPdb here) is always loaded with ipython:
c.InteractiveShellApp.exec_lines = [
'%pdb on',
'from LcTerminalPdb import LcTerminalPdb',
'from IPython.terminal.interactiveshell import TerminalInteractiveShell',
'shl = TerminalInteractiveShell().instance().InteractiveTB',
'shl.debugger_cls = LcTerminalPdb',
]
Note that above I don't need to write the extra shl.pdb = shl.debugger_cls() line, as ipython will take care of it the first time a debug point is entered. But feel free to, to be sure.
NOTES:
I have only tested this using LcTerminalPdb, and only briefly, but it seems to work appropriately
I suspect as long as other pdb debuggers have the same API as pdb (i.e. if they can be used by the PYTHONBREAKPOINT environment variable) then it should work
It's really unclear to me whether changing such deep attributes will have unexpected effects, so not sure how much I recommend this approach
Related
I have a function in maya that imports in other functions and creates a shelf with buttons for specific functions. I have a function that has a scriptJob command that works fine. if I import that file in manually and not through the shelf button, but gives a NameError when using the shelf script to run it.
This is an example of the script
myShelf.py file:
import loopingFunction
loopingFunction.runThis()
loopingFunction.py file:
import maya.cmds as mc
def setSettings():
#have some settings set before running this
runThis()
def runThis():
print "yay this ran"
evalDeferred(mc.scriptJob(ro=True,ac=["'someMesh.outMesh',runThis"])
if I run this through the shelf function, I get a runThis nameError is not defined and if I try modifying the scriptJob command to loopingFunction.runThis, I get a nameError loopingFunction is not defined(not sure if using loopingFunction.runThis is even correct, to be honest)
Not sure how I can get around this without having to manually import in the function rather than through the shelf file.
Using string references for callback functions like this often leads to scope problems. (More on that, and why not to use strings, here)
If you pass the function directly to the callback as an object, instead of using a string, it should work correctly as along as you have actually imported the code.
in this case you want an evalDeferred -- do you actually need it? -- so it helps to add a little function around the actual code so that the scriptjob creation actually happens later -- otherwise it will get evaluated before the deferral is scheduled.
def runThis():
print "callback ran"
def do_scriptjob():
cmds.scriptJob(ro=True, ac=('someMesh.outMesh', runThis)
cmds.evalDeferred(do_scriptjob)
In both runThis and do_scriptjob I did not add the parens -- we are letting the evalDeferred and the scriptJob have the function objects to call when they are ready. Using the parens would pass the results of the functions which is not what you want here.
Incidentally it looks like you're trying to create a new copy of the scriptJob inside the scriptJob itself. It'd be better to just drop the runOnce flag and leave the scriptJob lying around -- if something in runThis actually affected the someMesh.outMesh attribute, your maya will go into an infinite loop. I did not change the structure in my example, but I would not recommend writing this kind of self-replicating code if you can possibly avoid it.
You have a problem of nested/maya scope variables
mc.scriptJob(ro=True,ac=["'someMesh.outMesh',runThis"]
This line is a maya command string that is evaluated in the main maya scope (like a global)
As your function have a namespace with the import : 'loopingFunction', you need to enforce it in the string command.
import loopingFunction
loopingFunction.runThis()
You should write
evalDeferred(mc.scriptJob(ro=True,ac=["'someMesh.outMesh',loopingFunction.runThis"])
If you want something more general, you can do :
def runThis(ns=''):
print "yay this ran"
if ns != '':
ns += '.'
evalDeferred(mc.scriptJob(ro=True,ac=["'someMesh.outMesh',{}runThis".format(ns)])
and then run in shelf :
import loopingFunction
loopingFunction.runThis('loopingFunction')
like this you can write any formof namepsaces :
import loopingFunction as loopF
loopF.runThis('loopF')
In older (I believe pre-5.0) versions of IPython, if I was working on a line/block, and suddenly discovered I needed to investigate something else to finish it, my approach was to hit Ctrl-C, which left the incomplete line/block on screen, but unexecuted, and gave me a fresh prompt. That is, I'd see something like:
In [1]: def foo():
...: stuff^C # <-- Just realized I needed to check something on stuff usage
In [2]: # <-- cursor goes to new line, but old stuff still on terminal
In newer IPython (which seems to have switched from readline to prompt_toolkit as the "CLI support framework"), the behavior of Ctrl-C differs; now, instead of giving me a newline, it just resets the current one, discarding everything I've typed and returning the cursor to the beginning of the line.
# Before:
In [1]: def foo():
...: stuff
# After Ctrl-C:
In [1]: # Hey, where'd everything go?
This is extremely annoying, since I can no longer see or copy/paste the code I was working on to resume my work after I've done whatever side task precipitated the need for a fresh prompt.
My question is: Is there any way to restore the old IPython behavior, where Ctrl-C does the following things:
Does not execute the line/block typed so far
Leaves it on the screen
Ability to choose (at config time is fine) whether to add to the history (this would be personal preference; do you want half-formed stuff in the history, or just on the terminal for copy/paste?)
Provides me with a fresh prompt below the text typed so far
I've searched everywhere, and the most I've found is a bug report comment that mentions this new behavior in passing as "...a change from earlier versions of IPython, but it is intentional."
I haven't been able to find anything documented about modifying the behavior in the IPython or prompt_toolkit documentation; I've found where a lot of these handlers are installed, but attempts at monkey-patching to alter the current behavior have failed (and frankly, monkey-patching undocumented code means I risk it breaking every upgrade, so I'd like to find some semi-supported fix for this; failing that, hacky monkey-patching is acceptable though).
And after more research, I found what appears to be a supported approach, relying on the IPython keyboard shortcuts documentation (documented slightly differently for 5.x and 6.x).
The solution is to create a file in ~/.ipython/profile_default/startup (any name, ending with .py or ipy is fine, e.g. fixctrlc.py), and add the following:
def fix_ctrlc():
'''Set up bindings so IPython 5.0+ responds to Ctrl-C as in pre-5.0
Specific behavior is to have Ctrl-C leave existing typed command on
screen and preserved in history. Easily modified to not put in history.
Since this is run as a startup script, all work, including imports,
is done in this function to avoid polluting global namespace.
Updates made as needed at https://stackoverflow.com/a/45600868/364696
'''
from IPython import get_ipython
from prompt_toolkit.enums import DEFAULT_BUFFER
from prompt_toolkit.keys import Keys
from prompt_toolkit.filters import HasFocus, ViInsertMode, EmacsInsertMode
ip = get_ipython()
# Determine if we're on a version of IPython that needs a fix,
# acquire the key bindings registry from the appropriate location,
# and establish closure state appropriate to that version of IPython
try:
try:
# IPython 5-6; render_as_done doesn't exist, but manual print works
registry = ip.pt_cli.application.key_bindings_registry
redraw_args = {}
doprint = True
except AttributeError:
# IPython 7+ (tested through 8.0.1)
# render_as_done necessary, and removes need for print
registry = ip.pt_app.key_bindings
redraw_args = {'render_as_done': True}
doprint = False
except AttributeError:
# On an old version of IPython that doesn't need the fix, or
# a new version that changed the registry location. Nothing to do.
return
def on_ctrlc(event):
text = event.cli.current_buffer.text.rstrip()
if text:
# Update cursor position to last non-space char in buffer (so Ctrl-C
# with cursor in middle of block doesn't lose text typed after cursor)
event.cli.current_buffer.cursor_position = len(text)
event.cli.current_buffer.text = text
# Redraw so cursor in correct position before print
event.cli._redraw(**redraw_args)
# (Optional) Put non-empty partial commands in history, not just left on screen
# Delete to leave them on screen, but not in history
event.cli.current_buffer.append_to_history()
# Print a newline to move us past currently typed text so it's not
# replaced on redraw
if doprint:
print()
# Reset/redraw prompt
event.cli.reset()
# Clear active buffer, leaving you with fresh, empty prompt
event.cli.current_buffer.reset()
registry.add_binding(
Keys.ControlC,
filter=(HasFocus(DEFAULT_BUFFER) & (ViInsertMode() | EmacsInsertMode()))
)(on_ctrlc)
fix_ctrlc()
del fix_ctrlc # Avoid polluting global namespace
Please feel free to contribute if you find a better solution.
My python scripts often contain "executable code" (functions, classes, &c) in the first part of the file and "test code" (interactive experiments) at the end.
I want python, py_compile, pylint &c to completely ignore the experimental stuff at the end.
I am looking for something like #if 0 for cpp.
How can this be done?
Here are some ideas and the reasons they are bad:
sys.exit(0): works for python but not py_compile and pylint
put all experimental code under def test():: I can no longer copy/paste the code into a python REPL because it has non-trivial indent
put all experimental code between lines with """: emacs no longer indents and fontifies the code properly
comment and uncomment the code all the time: I am too lazy (yes, this is a single key press, but I have to remember to do that!)
put the test code into a separate file: I want to keep the related stuff together
PS. My IDE is Emacs and my python interpreter is pyspark.
Use ipython rather than python for your REPL It has better code completion and introspection and when you paste indented code it can automatically "de-indent" the pasted code.
Thus you can put your experimental code in a test function and then paste in parts without worrying and having to de-indent your code.
If you are pasting large blocks that can be considered individual blocks then you will need to use the %paste or %cpaste magics.
eg.
for i in range(3):
i *= 2
# with the following the blank line this is a complete block
print(i)
With a normal paste:
In [1]: for i in range(3):
...: i *= 2
...:
In [2]: print(i)
4
Using %paste
In [3]: %paste
for i in range(10):
i *= 2
print(i)
## -- End pasted text --
0
2
4
In [4]:
PySpark and IPython
It is also possible to launch PySpark in IPython, the enhanced Python interpreter. PySpark works with IPython 1.0.0 and later. To use IPython, set the IPYTHON variable to 1 when running bin/pyspark:1
$ IPYTHON=1 ./bin/pyspark
Unfortunately, there is no widely (or any) standard describing what you are talking about, so getting a bunch of python specific things to work like this will be difficult.
However, you could wrap these commands in such a way that they only read until a signifier. For example (assuming you are on a unix system):
cat $file | sed '/exit(0)/q' |sed '/exit(0)/d'
The command will read until 'exit(0)' is found. You could pipe this into your checkers, or create a temp file that your checkers read. You could create wrapper executable files on your path that may work with your editors.
Windows may be able to use a similar technique.
I might advise a different approach. Separate files might be best. You might explore iPython notebooks as a possible solution, but I'm not sure exactly what your use case is.
Follow something like option 2.
I usually put experimental code in a main method.
def main ():
*experimental code goes here *
Then if you want to execute the experimental code just call the main.
main()
With python-mode.el mark arbitrary chunks as section - for example via py-sectionize-region.
Than call py-execute-section.
Updated after comment:
python-mode.el is delivered by melpa.
M-x list-packages RET
Look for python-mode - the built-in python.el provides 'python, while python-mode.el provides 'python-mode.
Developement just moved hereto: https://gitlab.com/python-mode-devs/python-mode
I think the standard ('Pythonic') way to deal with this is to do it like so:
class MyClass(object):
...
def my_function():
...
if __name__ == '__main__':
# testing code here
Edit after your comment
I don't think what you want is possible using a plain Python interpreter. You could have a look at the IEP Python editor (website, bitbucket): it supports something like Matlab's cell mode, where a cell can be defined with a double comment character (##):
## main code
class MyClass(object):
...
def my_function():
...
## testing code
do_some_testing_please()
All code from a ##-beginning line until either the next such line or end-of-file constitutes a single cell.
Whenever the cursor is within a particular cell and you strike some hotkey (default Ctrl+Enter), the code within that cell is executed in the currently running interpreter. An additional feature of IEP is that selected code can be executed with F9; a pretty standard feature but the nice thing here is that IEP will smartly deal with whitespace, so just selecting and pasting stuff from inside a method will automatically work.
I suggest you use a proper version control system to keep the "real" and the "experimental" parts separated.
For example, using Git, you could only include the real code without the experimental parts in your commits (using add -p), and then temporarily stash the experimental parts for running your various tools.
You could also keep the experimental parts in their own branch which you then rebase on top of the non-experimental parts when you need them.
Another possibility is to put tests as doctests into the docstrings of your code, which admittedly is only practical for simpler cases.
This way, they are only treated as executable code by the doctest module, but as comments otherwise.
I would like to script some behaviour in GDB using Python: given a regular expression describing a set of functions, instantiate a subclass of gdb.Breakpoint (eg. MyBreakpoint) for each function matched.
There is no equivalent of rbreak in GDB's Python module. I had thought of doing this:
gdb.execute('rbreak {:s}'.format(regexp))
breakpoints = gdb.breakpoints()
# Extract breakpoint strings, delete existing breakpoints, and
# recreate them using my subclass.
for bp in breakpoints:
loc = bp.location
bp.delete()
MyBreakpoint(loc)
...however this suffers from the problem that there might already be some user defined breakpoints, and this would destroy them.
My next idea was to iterate over all possible functions to break on, and do the matching using Python's re module. However, there doesn't seem to be any way to list functions available for breaking from within Python.
My question is: could either of these approaches be salvaged so that they will work reliably and not clobber state set by a user in an interactive session; or is there some other way to achieve this? Or will I have to compromise on "not clobbering user state?"
Since rbreak creates new breakpoint objects, even if the breakpoints are for the same locations as pre-existing breakpoints, you can run gdb.breakpoints() before and after the execution of rbreak to see which breakpoints were added.
obreakpoints = gdb.breakpoints();
gdb.execute('rbreak {:s}'.format(regexp))
breakpoints = set(gdb.breakpoints()).difference(set(obreakpoints))
In my python file, I have made a GUI widget that takes some inputs from user. I have imported a python module in my python file that takes some input using raw_input(). I have to use this module as it is, I have no right to change it. When I run my python file, it ask me for the inputs (due to raw_input() of imported module). I want to use GUI widget inputs in that place.
How can I pass the user input (that we take from widget) as raw_input() of imported module?
First, if importing it directly into your script isn't actually a requirement (and it's hard to imagine why it would be), you can just run the module (or a simple script wrapped around it) as a separate process, using subprocess or pexpect.
Let's make this concrete. Say you want to use this silly module foo.py:
def bar():
x = raw_input("Gimme a string")
y = raw_input("Gimme another")
return 'Got two strings: {}, {}'.format(x, y)
First write a trivial foo.wrapper.py:
import foo
print(foo.bar())
Now, instead of calling foo.do_thing() directly in your real script, run foo_wrapper as a child process.
I'm going to assume that you already have the input you want to send it in a string, because that makes the irrelevant parts of the answer simpler (in fact, it makes them possible—if you wanted to use some GUI code for that, there's really no way I could show you how unless you first tell us which GUI library you're using).
So:
foo_input = 'String 1\nString 2\n'
with subprocess.Popen([sys.executable, 'foo_wrapper.py'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE) as p:
foo_output, _ = p.communicate(foo_input)
Of course in real life you'll want to use an appropriate path for foo_wrapper.py instead of assuming that it's in the current working directory, but this should be enough to illustrate the idea.
Meanwhile, if "I have no right to change it" just means "I don't (and shouldn't) have checkin rights to the foo project's github site or the relevant subtree on our company's P4 server" or whatever, there's a really easy answer: Fork it, and change the fork.
Even if it's got a weak copyleft license like LGPL: fork it, change the fork, publish your fork under the same license as the original, then use your fork.
If you're depending on the foo package being installed on every target system, and can't depend on your replacement foo being installed instead, that's a bit more of a problem. But if the function or method that actually calls raw_input is just a small fraction of the actual code in foo, you can fix that by monkeypatching foo at runtime.
And that leads to the last-ditch possibility: You can always monkeypatch raw_input itself.
Again, I'm going to assume that you already have the input you need to give it to make things simpler.
So, first you write a replacement function:
foo_input = ['String 1\n', 'String 2\n']
def fake_raw_input(prompt):
global foo_input
return foo_input.pop()
Now, there are two ways you can patch this in. Usually, you want to do this:
import foo
foo.raw_input = fake_raw_input
This means any code in foo that calls raw_input will see the function you crammed into its module globals instead of the normal builtin. Unless it does something really funky (like looking up the builtin directly and copying it to a local variable or something), this is the answer.
If you need to handle one of those really funky edge cases, and you don't mind doing something questionable, you can do this:
import __builtin__
__builtin__.raw_input = fake_raw_input
You must do this before the first import foo anywhere in your problem. Also, it's not clear whether this is intentionally guaranteed to work, accidentally guaranteed to work (and should be fixed in the future), or not guaranteed to work. But it does work (at least for CPython 2.5-2.7, which is what you're probably using).