I'm trying to find a way to pause a script's execution (one that is not timeout) that would allow me to e.g. check variable values at that point, input additional commands from the command line and then resume the execution once I'm ready. In IDL (from which I switched to Python very recently) I was able to do that by including "stop" at the chosen code line and then typing ".cont". Does something similar exist in Python?
Many thanks in advance for the help!
(This is my first post here, so I hope the above description meets the standards but if not I'd welcome any critical feedback!)
I believe you are looking for the Python Debugger (pdb):
The module pdb defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame. It also supports post-mortem debugging and can be called under program control.
I'd suggest you to start with set_trace():
The typical usage to break into the debugger from a running program is to insert
import pdb; pdb.set_trace()
at the location you want to break into the debugger. You can then step through the code following this statement, and continue running without the debugger using the continue command.
Be sure to take a look at the other ways to enter the Debugger and the Debugger Commands section.
Related
I have found that the pycharm debugger will not stop at breakpoints set in work functions of custom python GR blocks. However, it will stop at breakpoints set in a block's constructor.
Is there a way to stop at breakpoints in a work function? I know that the function is being run because when I put print statements in the work function, I see the results of the print when the flowgraph is run.
That's to be expected: the work() is called from C++ context; your debugger simply doesn't "control" the execution.
I'm sure it's possible to allow attaching breakpoints, but I don't know of a specific way. It would be relatively involved, because you need to instrument python in a GNU Radio-specific way, I guess.
When I execute a python program, the results starts to appear quickly and I can't read it all. It just flushes over my screen.
When the execution ends, I can no longer see the first displays, because the terminal display space is limited.
How save the output, so I can read all of it?
You have a few options here.
Add a breakpoint and learn how to use the debugger. Once you add this command (import pdb;pdb.set_trace() # this will take some learning so look up what pdb is online. actually, i prefer 'ipdb' instead.), the code will stop at that specific point when you execute it.
Save it to a file (python file.py > filename.txt) and then read it afterwards. Bonus: Before you ask yourself, where are my outputs? https://askubuntu.com/questions/625224/how-to-redirect-stderr-to-a-file
(More advanced) Your code is spitting out too much garbage output. You can remove some of the code or use python logging filters.
May be platform dependant.
On Linux you can also pipe your program output into your favorite pager (less for example) if you don't want to write it to a file.
python file.py | less
I am running Emacs 23.2 with python.el and debugging some Python code with pdb.
My code spawns a sibling thread using the threading module and I set a breakpoint at the start of the run() method, but the break is never handled by pdb even though the code definitely runs and works for all intents and purposes.
I was under the impression that I could use pdb to establish breakpoints in any thread, even though full multi-threaded debugging is in fact not supported.
Am I wrong in assuming pdb within an M-x pdb invocation can break within any thread? If you don't believe me try this minimal example for yourself.
import threading
class ThreadTest(threading.Thread):
def __init__(self,):
threading.Thread.__init__(self)
def run(self):
print "Type M-x pdb, set a breakpoint here then type c <RET>..."
print "As you can see it does not break!"
if __name__ == '__main__':
tt = ThreadTest()
tt.start()
Thanks to Pierre and the book text he refers to, I tried the option to include pdb.set_trace() as follows:
def run(self):
import pdb; pdb.set_trace()
print "Set a breakpoint here then M-x pdb and type c..."
But this only breaks and offers pdb controls for step, next, continue etcetera, if it is executed from a console and run directly within the Python interpreter, and crucially not via M-x pdb - at least with my Emacs and pdb configuration.
So my original question could do with being rephrased:
Is there a way to invoke a Python program from within Emacs, where that program uses inlined invocation of pdb (thereby supporting breaks in multi-threaded applications), and for there to be a pdb comint control buffer established auto-magically?
or
If I run my Python application using M-x pdb and it contains an inline invocation of pdb, how best to handle the fact that this results in a pdb-session-within-a-pdb-session with the associated loss of control?
Are you using the default python.el? I've given up on that and started using python-mode.el. Then type M-x shell, from the prompt type python myproblem.py (replace with your program name of course) and it will stop at the set_trace line. It works out of the box with pdb integration. (And it works on your program).
See http://heather.cs.ucdavis.edu/~matloff/158/PLN/ParProcBook.pdf, there's a section on multithreaded debugging.
3.6.1 Using PDB to Debug Threaded Programs
Using PDB is a bit more complex when threads are involved. One cannot, for instance, simply do something
like this:
pdb.py buggyprog.py
because the child threads will not inherit the PDB process from the main thread. You can still run PDB in
the latter, but will not be able to set breakpoints in threads.
What you can do, though, is invoke PDB from within the function which is run by the thread, by calling
pdb.set trace() at one or more points within the code:
import pdb
pdb.set_trace()
In essence, those become breakpoints.
For example, in our program srvr.py in Section 3.1.1, we could add a PDB call at the beginning of the loop
in serveclient():
while 1:
import pdb
pdb.set_trace()
# receive letter from client, if it is still connected
k = c.recv(1)
if k == ’’: break
You then run the program directly through the Python interpreter as usual, NOT through PDB, but then the
program suddenly moves into debugging mode on its own. At that point, one can then step through the code
using the n or s commands, query the values of variables, etc.
PDB’s c (“continue”) command still works. Can one still use the b command to set additional breakpoints?
Yes, but it might be only on a one-time basis, depending on the context. A breakpoint might work only once,
due to a scope problem. Leaving the scope where we invoked PDB causes removal of the trace object. Thus
I suggested setting up the trace inside the loop above.
I'm developing my own Python code interpreter using the Python C API, as described in the Python documentation. I've taken a look on the Python source code and I tried to follow the same steps that are carried out in the standard interpreter when executing a py file. These steps (sequence of C API function calls) are basically:
PyRun_AnyFileExFlags()
PyRun_SimpleFileExFlags()
PyRun_FileExFlags()
PyArena_New()
PyParser_ASTFromFile()
run_mod()
PyAST_Compile()
PyEval_EvalCode()
PyEval_EvalCodeEx()
PyThreadState_GET()
PyFrame_New()
PyEval_EvalFrameEx()
The only difference in my code is that I do manually the AST compilation, frame creation, etc. and then I call PyEval_EvalFrame.
With this, I am able to execute an arbitrary .py file with my program, as if it were the normal Python interpreter. My problem comes when the code that my program is executing makes use of the time module: all time module operations get blocked in the GIL! For example, if the Python code calls time.sleep(1), this call is blocked and never gets executed.
Obviously I am doing something wrong that blocks the GIL (and therefore blocks the time module) but I dont know how to correct it. The last statement in my code where I have control is in PyEval_EvalFrameEx, and from that point on, everything runs "as in regular Python interpreter", I think.
Anybody had a similar problem? What am I doing wrong, so that I block the time module?
Hope somebody can help me...
Thanks for your time. Best regards,
R.
You need to provide more detail.
How does your interpreter's behavior differ from the standard interpreter?
If you just want to run arbitrary source files, why are you not calling one of the higher level interfaces, like PyRun_SimpleFile? Did your code call Py_Initialize?
I'm using the PyDev for Eclipse plugin, and I'm trying to set a break point in some code that gets run in a background thread. The break point never gets hit even though the code is executing. Here's a small example:
import thread
def go(count):
print 'count is %d.' % count # set break point here
print 'calling from main thread:'
go(13)
print 'calling from bg thread:'
thread.start_new_thread(go, (23,))
raw_input('press enter to quit.')
The break point in that example gets hit when it's called on the main thread, but not when it's called from a background thread. Is there anything I can do, or is that a limitation of the PyDev debugger?
Update
Thanks for the work arounds. I submitted a PyDev feature request, and it has been completed. It should be released with version 1.6.0. Thanks, PyDev team!
The problem is that there's no API in the thread module to know when a thread starts.
What you can do in your example is set the debugger trace function yourself (as Alex pointed) as in the code below (if you're not in the remote debugger, the pydevd.connected = True is currently required -- I'll change pydev so that this is not needed anymore). You may want to add a try..except ImportError for the pydevd import (which will fail if you're not running in the debugger)
def go(count):
import pydevd
pydevd.connected = True
pydevd.settrace(suspend=False)
print 'count is %d.' % count # set break point here
Now, on a second thought, I think that pydev can replace the start_new_thread method in the thread module providing its own function which will setup the debugger and later call the original function (just did that and it seems to be working, so, if you use the nightly that will be available in some hours, which will become the future 1.6.0, it should be working without doing anything special).
The underlying issue is with sys.settrace, the low-level Python function used to perform all tracing and debugging -- as the docs say,
The function is thread-specific; for a
debugger to support multiple threads,
it must be registered using settrace()
for each thread being debugged.
I believe that when you set a breakpoint in PyDev, the resulting settrace call is always happening on the main thread (I have not looked at PyDev recently so they may have added some way to work around that, but I don't recall any from the time when I did look).
A workaround you might implement yourself is, in your main thread after the breakpoint has been set, to use sys.gettrace to get PyDev's trace function, save it in a global variable, and make sure in all threads of interest to call sys.settrace with that global variable as the argument -- a tad cumbersome (more so for threads that already exist at the time the breakpoint is set!), but I can't think of any simpler alternative.
On this question, I found a way to start the command-line debugger:
import pdb; pdb.set_trace()
It's not as easy to use as the Eclipse debugger, but it's better than nothing.
For me this worked according to one of Fabio's posts, after setting the trace with setTrace("000.000.000.000") # where 0's are the IP of your computer running Eclipse/PyDev
threading.settrace(pydevd.GetGlobalDebugger().trace_dispatch)