How to quickly debug misbehaving script in Python without pdb? - python

There is a high level logic error deep within my Python script, and pdb doesn't help to debug it. Is there any other way to see what is being executed after I run my script?
NOTE: Using pdb is too slow and inconvenient - I wish I could grep over all cases when my function is executed, instead of inspecting manually each and every call, set/unset breakpoints. The state is lost when I exit pdb and its user interface is more confusing than helpful - requires docs at hand.
UPDATE: made it clear that pdb is not an option, so the most popular Python debugging tips can not be applied

I would recommend using pdb. You can use
import pdb
at the top of your script, and then add the line
pdb.set_trace()
somewhere in the code where you want to trace the problem. When the script gets to that line, you will have an interactive console where you can check variable values, run your own checks, and see what is going on. You can use n to execute the next line, or c to continue to the next occurrence of set_trace(). Full documentation is here: http://docs.python.org/2/library/pdb.html.
Let me know if you have any specific questions!

No, there's no magic formula.
My best suggestion is to get a good IDE with a debugger, like JetBrains' PyCharm, and step through your code to see where you went wrong.
Most of the time these situations happen because you make assumptions about behavior that aren't true. Get a debugger, step through, and check your assumptions.

I found a way to do this using excellent trace module that comes with Python.
An example how to troubleshoot module installation problem:
python -m trace -t setup.py install > execution.log
This will dump all source line of setup.py install execution to execution.log. I find this more useful than pdb, because usability of pdb command line interface is very poor.

Related

Any command like '%debug' (in jupyter) when running python script?

In jupyter, when an error occurs, we can continuously debug the error with the command %debug. I want to know if there is the similar way in running python script (.py) in the shell.
I know that I can use pdb to make some break points, but I just want to know the way without such a pre-processing (because re-running the code until the error costs a lot of time).
In general, no: it depends on "the shell" that you are running. Jupyter launches with a lot of instrumentation in support of its debugger, assuming that you're using Jupyter because you want those capabilities at the ready.
I presume that you're using some UNIX shell (since you mention pdb); implicitly loading superfluous software is antithetical to the UNIX philosophy.
I think that what you'll need is one of the "after" debugger modes, although that will still leave you without information from just before the error point: those packages cannot do much to trace the history of problem variables.

While using pylint I found that it also has an option of silent run. Why is it there and what is the use of it?

Why does pylint give an option to silent run I dont understand. The documentation doesnt providde
reason for the same.
Also, does anyone know how can one generate .pylintrc file by running a python program. I know that it can be generated from command prompt. I just wanted to know how to generate the same by running a python program.
As stated in the User Guide you can run pylint from a python script using:
import pylint.lint
pylint_opts = ['--version']
pylint.lint.Run(pylint_opts)
# or
from pylint import epylint as lint
(pylint_stdout, pylint_stderr) = lint.py_run('module_name.py', return_std=True)
from the linked page
Then you can just pass in the --generate-rcfile option.
As for your first question, please clarify the silent run?
Edit:
My understanding is that the "silent" run is not so silent (as in no output). It is just a different way of running pylint with no exit code. However you get the standard out and standard error as return values which is almost equivalent of what you get in the other run modes. You can then work with these outputs and parse them or store them somewhere etc.

python debugging point equivalent to Perl $DB::single=1

I am a Perl programmer learning Python. I am writing my code in emacs debugging with python -m pdb script.py using Python 2.7.3.
I would like to know what is the python equivalent to in Perl adding a $DB::single=1;1; to a specific line of python code, so that when running the debugger, it will stop there, even if it's a different source code file from where the execution started (e.g. a line of code in a library being used by script.py).
Any ideas?
EDITED: after looking at pdb.set_trace() or ipdb.set_trace(), I consider them good solutions but not 100% identical to the behaviour of $DB::single=1;1;. This is, I would like the breakpoint to be on the set_trace line, instead of the next line. This is accomplished in Perl's $DB::single=1; by adding another statement in the same line: 1;, which makes it $DB::single=1;1;.
Using set_trace(), I get the breakpoint at the line after the statement, even if I add 1; after it. Still not fully understanding how Python treats multi-statement lines in comparison to Perl.
Anybody?
Any ideas?
Is the following satisfying your needs ?
import ipdb; ipdb.set_trace()
just write it somewhere in your code and run your script with python script.py.
you need the ipython debugger (ipython is an enhanced python interpreter):
pip install ipdb
edit: did you know that if you run M-x pdb RET pdb myscript.py RET, you'll have a pdb prompt and emacs will track the source code in another buffer, but it doesn't stop where you defined ipdb.set_trace() ?
Virtual Env ?
if you use virtual envs, you have a couple of options. I recommand installing virtualenvwrapper from ELPA and run M-x venv-workon.
Python comes with debugger called pdb. To stop a script at given point in code put the following
import pdb; pdb.set_trace()
Since you are using emacs, you would may want to try out the command pdb provided by gud.el (correction: You do not need to preload 'python-mode' to run pdb, thanks #Andreas Röhler for correction) . Start it by pdb name_of_script.py, then you can set breakpoint from emacs by pressing C-xSPACE at the line you want to set breakpoint at. I recommend you to use menu-bar to explore the commands provided by the debugger (GUD). You can also use the usual pdb commands in the *gud-pdb* buffer started by emacs.

Python shell automatically load changes

I really love python because I love interactive development. There's one area where python appears to fall short, however, and that's in the area of automatically reloading changed files. Basically, what I want to have happen is to be able to modify a python file on-disk and then have my running python instance automatically reload the changed module to allow me to immediately access my changes in the REPL so I can test them out. Basically, I want some sort of watch command.
I happen to use the bpython shell because I think it's the best one available, but this feature is so important to me that I'd be willing to switch to any other python shell that does it right. Is it possible?
Something like tail -f in python + reload().
I really think they should make the tags OS and Python version mandatory though.
If you're trying to "test out" your code, perhaps you should be looking into doing automated unit tests instead of testing your code repeatedly and manually. It'll allow you to test more code quicker and waste less of your precious, precious development time.
Personally, I use unittest with py.test as the runner.

How can I save / copy classes & functions I've written in the python interpreter?

How can you save functions/ classes you've writing in a python interactive session to a file? Specifically, is there a way in pydev / eclipse's interactive session (on a mac) to do this?
I just started learning python - and am enjoying using the interpreter's interactive session for testing and playing with modules I've written. However, I find myself writing functions in the interpreter, which I think, oh it would be cool to save that to my script files. How do I do this?
I tried:
import pickle
pickle.dump(my_function, open("output.p", "w"))
But it seems to be more of a binary serialization, or at least nothing that I could copy and paste into my code...
Are there ways to see the code behind classes & functions I've defined in the interpreter? And then copy them out of the interpreter?
Update:
Ok, here's what I've learned so far:
I missed the easiest of all - PyDev's interactive session in eclipse allows you to right click and save your session. Still have to remove >>>'s, but gets the job done.
IPython is apparently the way to go to do this.
How to save a Python interactive session? has more details.
The best environment for interactive coding sessions has to be IPython, in my opinion. It's built on and extends the basic Python interpreter with a lot of magic, including history. For example, you can issue the command %logstart to dump all subsequent input to a file, which still needs to be edited afterward before it will be a script, but gives you a lot to work with.
When installing IPython, don't forget pyreadline.
In general, however, it is best to write code in an IDE and then run it. IPython helps here as well. If you write and save the script, then use the IPython "run" command to run it, the entire global namespace of the script will be available for inspection in your IPython session. Additionally, you can use the -d argument to run to trigger the pdb debugger immediately on any unhandled exception.
If you're more of a straightlaced IDE and debugger kind of guy, then the easiest and best lightweight environment has to be PyScripter.
I think the answer is to change your workflow.
What I do is write my functions in an editor (emacs), and then press a key combination (Ctrl-c Ctrl-e) to send the region of text to the (i)python interpreter.
That way I can save the function if I want, and also play with it in an interpreter.
Emacs is central to how I do it, but I'm sure there must be similar approaches with many editors (vim, gedit, etc) and IDEs.
PS. Finding a good editor is crucial when working with Python. The editor must be able to move blocks of code to the left and right easily, or the whitespace issue becomes too onerous.
I dislike typing blocks of code in the python interpreter because it doesn't allow me to shift blocks easily. You'll like Python even more when you find the right editor.
You can setup a python history file which stores everything you type into the interpreter.
Here's how:
http://docs.python.org/tutorial/interactive.html
I think it can't be done.
Python can perform instrospection with the inspect module, but the inspect.getsource function won't work without a source file.

Categories

Resources