Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
So, I started learning to code in Python and later Django. The first times it was hard looking at tracebacks and actually figure out what I did wrong and where the syntax error was. Some time has passed now and some way along the way, I guess I got a routine in debugging my Django code. As this was done early in my coding experience, I sat down and wondered if how I was doing this was ineffective and could be done faster. I usually manage to find and correct the bugs in my code, but I wonder if I should be doing it faster?
I usually just use the debug info Django gives when enabled. When things do end up as I thought it would, I break the code flow a lot with a syntax error, and look at the variables at that point in the flow to figure out, where the code does something other than what I wanted.
But can this be improved? Are there some good tools or better ways to debug your Django code?
There are a bunch of ways to do it, but the most straightforward is to simply
use the Python debugger. Just add following line in to a Django view function:
import pdb; pdb.set_trace()
or
breakpoint() #from Python3.7
If you try to load that page in your browser, the browser will hang and you get a prompt to carry on debugging on actual executing code.
However there are other options (I am not recommending them):
* return HttpResponse({variable to inspect})
* print {variable to inspect}
* raise Exception({variable to inspect})
But the Python Debugger (pdb) is highly recommended for all types of Python code. If you are already into pdb, you'd also want to have a look at IPDB that uses ipython for debugging.
Some more useful extension to pdb are
pdb++, suggested by Antash.
pudb, suggested by PatDuJour.
Using the Python debugger in Django, suggested by Seafangs.
I really like Werkzeug's interactive debugger. It's similar to Django's debug page, except that you get an interactive shell on every level of the traceback. If you use the django-extensions, you get a runserver_plus managment command which starts the development server and gives you Werkzeug's debugger on exceptions.
Of course, you should only run this locally, as it gives anyone with a browser the rights to execute arbitrary python code in the context of the server.
A little quickie for template tags:
#register.filter
def pdb(element):
import pdb; pdb.set_trace()
return element
Now, inside a template you can do {{ template_var|pdb }} and enter a pdb session (given you're running the local devel server) where you can inspect element to your heart's content.
It's a very nice way to see what's happened to your object when it arrives at the template.
There are a few tools that cooperate well and can make your debugging task easier.
Most important is the Django debug toolbar.
Then you need good logging using the Python logging facility. You can send logging output to a log file, but an easier option is sending log output to firepython. To use this you need to use the Firefox browser with the firebug extension. Firepython includes a firebug plugin that will display any server-side logging in a Firebug tab.
Firebug itself is also critical for debugging the Javascript side of any app you develop. (Assuming you have some JS code of course).
I also liked django-viewtools for debugging views interactively using pdb, but I don't use it that much.
There are more useful tools like dozer for tracking down memory leaks (there are also other good suggestions given in answers here on SO for memory tracking).
I use PyCharm (same pydev engine as eclipse). Really helps me to visually be able to step through my code and see what is happening.
Almost everything has been mentioned so far, so I'll only add that instead of pdb.set_trace() one can use ipdb.set_trace() which uses iPython and therefore is more powerful (autocomplete and other goodies). This requires ipdb package, so you only need to pip install ipdb
I've pushed django-pdb to PyPI.
It's a simple app that means you don't need to edit your source code every time you want to break into pdb.
Installation is just...
pip install django-pdb
Add 'django_pdb' to your INSTALLED_APPS
You can now run: manage.py runserver --pdb to break into pdb at the start of every view...
bash: manage.py runserver --pdb
Validating models...
0 errors found
Django version 1.3, using settings 'testproject.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
GET /
function "myview" in testapp/views.py:6
args: ()
kwargs: {}
> /Users/tom/github/django-pdb/testproject/testapp/views.py(7)myview()
-> a = 1
(Pdb)
And run: manage.py test --pdb to break into pdb on test failures/errors...
bash: manage.py test testapp --pdb
Creating test database for alias 'default'...
E
======================================================================
>>> test_error (testapp.tests.SimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../django-pdb/testproject/testapp/tests.py", line 16, in test_error
one_plus_one = four
NameError: global name 'four' is not defined
======================================================================
> /Users/tom/github/django-pdb/testproject/testapp/tests.py(16)test_error()
-> one_plus_one = four
(Pdb)
The project's hosted on GitHub, contributions are welcome of course.
The easiest way to debug python - especially for programmers that are used to Visual Studio - is using PTVS (Python Tools for Visual Studio).
The steps are simple:
Download and install it from https://microsoft.github.io/PTVS/
Set breakpoints and press F5.
Your breakpoint is hit, you can view/change the variables as easy as debugging C#/C++ programs.
That's all :)
If you want to debug Django using PTVS, you need to do the following:
In Project settings - General tab, set "Startup File" to "manage.py", the entry point of the Django program.
In Project settings - Debug tab, set "Script Arguments" to "runserver --noreload". The key point is the "--noreload" here. If you don't set it, your breakpoints won't be hit.
Enjoy it.
I use pyDev with Eclipse really good, set break points, step into code, view values on any objects and variables, try it.
I use PyCharm and stand by it all the way. It cost me a little but I have to say the advantage that I get out of it is priceless. I tried debugging from console and I do give people a lot of credit who can do that, but for me being able to visually debug my application(s) is great.
I have to say though, PyCharm does take a lot of memory. But then again, nothing good is free in life. They just came with their latest version 3. It also plays very well with Django, Flask and Google AppEngine. So, all in all, I'd say it's a great handy tool to have for any developer.
If you are not using it yet, I'd recommend to get the trial version for 30 days to take a look at the power of PyCharm. I'm sure there are other tools also available, such as Aptana. But I guess I just also like the way PyCharm looks. I feel very comfortable debugging my apps there.
From my perspective, we could break down common code debugging tasks into three distinct usage patterns:
Something has raised an exception: runserver_plus' Werkzeug debugger to the rescue. The ability to run custom code at all the trace levels is a killer. And if you're completely stuck, you can create a Gist to share with just a click.
Page is rendered, but the result is wrong: again, Werkzeug rocks. To make a breakpoint in code, just type assert False in the place you want to stop at.
Code works wrong, but the quick look doesn't help. Most probably, an algorithmic problem. Sigh. Then I usually fire up a console debugger PuDB: import pudb; pudb.set_trace(). The main advantage over [i]pdb is that PuDB (while looking as you're in 80's) makes setting custom watch expressions a breeze. And debugging a bunch of nested loops is much simpler with a GUI.
Ah, yes, the templates' woes. The most common (to me and my colleagues) problem is a wrong context: either you don't have a variable, or your variable doesn't have some attribute. If you're using debug toolbar, just inspect the context at the "Templates" section, or, if it's not sufficient, set a break in your views' code just after your context is filled up.
So it goes.
Add import pdb; pdb.set_trace() or breakpoint() (form python3.7) at the corresponding line in the Python code and execute it. The execution will stop with an interactive shell. In the shell you can execute Python code (i.e. print variables) or use commands such as:
c continue execution
n step to the next line within the same function
s step to the next line in this function or a called function
q quit the debugger/execution
Also see: https://poweruser.blog/setting-a-breakpoint-in-python-438e23fe6b28
Sometimes when I wan to explore around in a particular method and summoning pdb is just too cumbersome, I would add:
import IPython; IPython.embed()
IPython.embed() starts an IPython shell which have access to the local variables from the point where you call it.
I just found wdb (http://www.rkblog.rk.edu.pl/w/p/debugging-python-code-browser-wdb-debugger/?goback=%2Egde_25827_member_255996401). It has a pretty nice user interface / GUI with all the bells and whistles. Author says this about wdb -
"There are IDEs like PyCharm that have their own debuggers. They offer similar or equal set of features ... However to use them you have to use those specific IDEs (and some of then are non-free or may not be available for all platforms). Pick the right tool for your needs."
Thought i'd just pass it on.
Also a very helpful article about python debuggers:
https://zapier.com/engineering/debugging-python-boss/
Finally, if you'd like to see a nice graphical printout of your call stack in Django, checkout:
https://github.com/joerick/pyinstrument. Just add pyinstrument.middleware.ProfilerMiddleware to MIDDLEWARE_CLASSES, then add ?profile to the end of the request URL to activate the profiler.
Can also run pyinstrument from command line or by importing as a module.
I highly recommend epdb (Extended Python Debugger).
https://bitbucket.org/dugan/epdb
One thing I love about epdb for debugging Django or other Python webservers is the epdb.serve() command. This sets a trace and serves this on a local port that you can connect to. Typical use case:
I have a view that I want to go through step-by-step. I'll insert the following at the point I want to set the trace.
import epdb; epdb.serve()
Once this code gets executed, I open a Python interpreter and connect to the serving instance. I can analyze all the values and step through the code using the standard pdb commands like n, s, etc.
In [2]: import epdb; epdb.connect()
(Epdb) request
<WSGIRequest
path:/foo,
GET:<QueryDict: {}>,
POST:<QuestDict: {}>,
...
>
(Epdb) request.session.session_key
'i31kq7lljj3up5v7hbw9cff0rga2vlq5'
(Epdb) list
85 raise some_error.CustomError()
86
87 # Example login view
88 def login(request, username, password):
89 import epdb; epdb.serve()
90 -> return my_login_method(username, password)
91
92 # Example view to show session key
93 def get_session_key(request):
94 return request.session.session_key
95
And tons more that you can learn about typing epdb help at any time.
If you want to serve or connect to multiple epdb instances at the same time, you can specify the port to listen on (default is 8080). I.e.
import epdb; epdb.serve(4242)
>> import epdb; epdb.connect(host='192.168.3.2', port=4242)
host defaults to 'localhost' if not specified. I threw it in here to demonstrate how you can use this to debug something other than a local instance, like a development server on your local LAN. Obviously, if you do this be careful that the set trace never makes it onto your production server!
As a quick note, you can still do the same thing as the accepted answer with epdb (import epdb; epdb.set_trace()) but I wanted to highlight the serve functionality since I've found it so useful.
One of your best option to debug Django code is via wdb:
https://github.com/Kozea/wdb
wdb works with python 2 (2.6, 2.7), python 3 (3.2, 3.3, 3.4, 3.5) and pypy. Even better, it is possible to debug a python 2 program with a wdb server running on python 3 and vice-versa or debug a program running on a computer with a debugging server running on another computer inside a web page on a third computer!
Even betterer, it is now possible to pause a currently running python process/thread using code injection from the web interface. (This requires gdb and ptrace enabled)
In other words it's a very enhanced version of pdb directly in your browser with nice features.
Install and run the server, and in your code add:
import wdb
wdb.set_trace()
According to the author, main differences with respect to pdb are:
For those who don’t know the project, wdb is a python debugger like pdb, but with a slick web front-end and a lot of additional features, such as:
Source syntax highlighting
Visual breakpoints
Interactive code completion using jedi
Persistent breakpoints
Deep objects inspection using mouse Multithreading / Multiprocessing support
Remote debugging
Watch expressions
In debugger code edition
Popular web servers integration to break on error
In exception breaking during trace (not post-mortem) in contrary to the werkzeug debugger for instance
Breaking in currently running programs through code injection (on supported systems)
It has a great browser-based user interface. A joy to use! :)
I use PyCharm and different debug tools. Also have a nice articles set about easy set up those things for novices. You may start here. It tells about PDB and GUI debugging in general with Django projects. Hope someone would benefit from them.
If using Aptana for django development, watch this: http://www.youtube.com/watch?v=qQh-UQFltJQ
If not, consider using it.
Most options are alredy mentioned.
To print template context, I've created a simple library for that.
See https://github.com/edoburu/django-debugtools
You can use it to print template context without any {% load %} construct:
{% print var %} prints variable
{% print %} prints all
It uses a customized pprint format to display the variables in a <pre> tag.
I find Visual Studio Code is awesome for debugging Django apps. The standard python launch.json parameters run python manage.py with the debugger attached, so you can set breakpoints and step through your code as you like.
For those that can accidentally add pdb into live commits, I can suggest this extension of #Koobz answer:
#register.filter
def pdb(element):
from django.conf import settings
if settings.DEBUG:
import pdb
pdb.set_trace()
return element
From my own experience , there are two way:
use ipdb,which is a enhanced debugger likes pdb.
import ipdb;ipdb.set_trace() or breakpoint() (from python3.7)
use django shell ,just use the command below. This is very helpfull when you are developing a new view.
python manage.py shell
i highly suggest to use PDB.
import pdb
pdb.set_trace()
You can inspect all the variables values, step in to the function and much more.
https://docs.python.org/2/library/pdb.html
for checking out the all kind of request,response and hits to database.i am using django-debug-toolbar
https://github.com/django-debug-toolbar/django-debug-toolbar
As mentioned in other posts here - setting breakpoints in your code and walking thru the code to see if it behaves as you expected is a great way to learn something like Django until you have a good sense of how it all behaves - and what your code is doing.
To do this I would recommend using WingIde. Just like other mentioned IDEs nice and easy to use, nice layout and also easy to set breakpoints evaluate / modify the stack etc. Perfect for visualizing what your code is doing as you step through it. I'm a big fan of it.
Also I use PyCharm - it has excellent static code analysis and can help sometimes spot problems before you realize they are there.
As mentioned already django-debug-toolbar is essential - https://github.com/django-debug-toolbar/django-debug-toolbar
And while not explicitly a debug or analysis tool - one of my favorites is SQL Printing Middleware available from Django Snippets at https://djangosnippets.org/snippets/290/
This will display the SQL queries that your view has generated. This will give you a good sense of what the ORM is doing and if your queries are efficient or you need to rework your code (or add caching).
I find it invaluable for keeping an eye on query performance while developing and debugging my application.
Just one other tip - I modified it slightly for my own use to only show the summary and not the SQL statement.... So I always use it while developing and testing. I also added that if the len(connection.queries) is greater than a pre-defined threshold it displays an extra warning.
Then if I spot something bad (from a performance or number of queries perspective) is happening I turn back on the full display of the SQL statements to see exactly what is going on. Very handy when you are working on a large Django project with multiple developers.
use pdb or ipdb. Diffrence between these two is ipdb supports auto complete.
for pdb
import pdb
pdb.set_trace()
for ipdb
import ipdb
ipdb.set_trace()
For executing new line hit n key, for continue hit c key.
check more options by using help(pdb)
An additional suggestion.
You can leverage nosetests and pdb together, rather injecting pdb.set_trace() in your views manually. The advantage is that you can observe error conditions when they first start, potentially in 3rd party code.
Here's an error for me today.
TypeError at /db/hcm91dmo/catalog/records/
render_option() argument after * must be a sequence, not int
....
Error during template rendering
In template /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/crispy_forms/templates/bootstrap3/field.html, error at line 28
render_option() argument after * must be a sequence, not int
18
19 {% if field|is_checkboxselectmultiple %}
20 {% include 'bootstrap3/layout/checkboxselectmultiple.html' %}
21 {% endif %}
22
23 {% if field|is_radioselect %}
24 {% include 'bootstrap3/layout/radioselect.html' %}
25 {% endif %}
26
27 {% if not field|is_checkboxselectmultiple and not field|is_radioselect %}
28
{% if field|is_checkbox and form_show_labels %}
Now, I know this means that I goofed the constructor for the form, and I even have good idea of which field is a problem. But, can I use pdb to see what crispy forms is complaining about, within a template?
Yes, I can. Using the --pdb option on nosetests:
tests$ nosetests test_urls_catalog.py --pdb
As soon as I hit any exception (including ones handled gracefully), pdb stops where it happens and I can look around.
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 537, in __str__
return self.as_widget()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 593, in as_widget
return force_text(widget.render(name, self.value(), attrs=attrs))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 513, in render
options = self.render_options(choices, [value])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 543, in render_options
output.append(self.render_option(selected_choices, *option))
TypeError: render_option() argument after * must be a sequence, not int
INFO lib.capture_middleware log write_to_index(http://localhost:8082/db/hcm91dmo/catalog/records.html)
INFO lib.capture_middleware log write_to_index:end
> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py(543)render_options()
-> output.append(self.render_option(selected_choices, *option))
(Pdb) import pprint
(Pdb) pprint.PrettyPrinter(indent=4).pprint(self)
<django.forms.widgets.Select object at 0x115fe7d10>
(Pdb) pprint.PrettyPrinter(indent=4).pprint(vars(self))
{ 'attrs': { 'class': 'select form-control'},
'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]],
'is_required': False}
(Pdb)
Now, it's clear that my choices argument to the crispy field constructor was as it was a list within a list, rather than a list/tuple of tuples.
'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]]
The neat thing is that this pdb is taking place within crispy's code, not mine and I didn't need to insert it manually.
During development, adding a quick
assert False, value
can help diagnose problems in views or anywhere else, without the need to use a debugger.
Behave is a great tool for behavior driven development in Python. However, in combination with PyDev I've got two Problems:
PyDev can't resolve behave's #given, #when and #then annotations. This probably happens because behave does some name magic the behave package.
behave suggests to name all methods step as every method has an annotation which defines the "real" name. PyDev complains about these "duplicate" methods.
Example:
from behave import given, when, then
#given('I navigate to Google')
def step(context):
# ...
#when('I enter coffee into the search field')
def step(context):
# ...
By now, I work around both Problems by including ##PydevCodeAnalysisIgnore, which turns off any PyDev analysis. I'd like to keep PyDev analysis for all other parts of the code.
Any suggestions how to solve this?
Just a guess. Have You tried with import redefinitions?
from behave import given as given_behaviour, when as when_behaviour, then as then_behaviour
#given_behaviour('I navigate to Google')
You can also check the eclipse settings: menu Window -> Preferences -> PyDev -> Interpreters -> Python Interpreters. Sometimes it might help to remove and re-add the python interpreter if new libraries has been added after that the interpreter was configured in order to include those in the System PYTHONPATH in eclipse.
Try adding ## followed by the message in camel case to the end of the method name to turn off the warning message in PyDev. Something like:
def step(context): ##DuplicatedSignature
I am working on a project using python 2.7.2, sqlalchemy 0.7, unittest, eclipse 3.7.2 and pydev 2.4. I am setting breakpoints in python files (unit test files), but they are completely ignored (before, at some point, they worked). By now i have upgraded all related software (see above), started new projects, played around with settings, hypnotized my screen, but nothing works.
The only idea i got from some post is that it has something to de with changing some .py file names to lower case.
Does anyone have any ideas?
added: I even installed the aptana version of eclipse and copied the .py files to it => same result; breakpoints are still ignored.
still no progress: I have changed some code that might be seen as unusual and replaced it with a more straightforward solution.
some more info: it probably has something to do with module unittest:
breakpoints in my files defining test suites work,
breakpoints in the standard unittest files themselves work
breakpoints in my tests methods in classes derived from unittest.TestCase do not work
breakpoints in my code being tested in the test cases do not work
at some point before i could define working breakpoints in test methods or the code being tested
some things i changed after that are: started using test suites, changed some filenames to lowercase, ...
this problem also occurs if my code works without exceptions or test failures.
what I already tried is:
remove .pyc files
define new project and copy only .py files to it
rebooted several times in between
upgraded to eclipse 3.7.2
installed latest pydev on eclipse 3.7.2
switch to aptana (and back)
removed code that 'manually' added classes to my module
fiddled with some configurations
what I can still do is:
start new project with my code, start removing/changing code until breakpoints work and sort of black box figure out if this has something to do with some part of my code
Does anyone have any idea what might cause these problems or how they might be solved?
Is there any other place i could look for a solution?
Do pydev developers look into the questions on stackoverflow?
Is there an older version of pydev that i might try?
I have been working with pydev/eclipse for a long time and it works well for me, but without debugging i'd forced to switch IDE.
In answer to Fabio's questions below:
The python version is 2.7.2,
The sys.gettrace gives None (but I have no idea what in my code could influence that)
This is the output of the debugger after changing the suggested parameters:
pydev debugger:
starting
('Executing file ', 'D:\\.eclipse\\org.eclipse.platform_3.7.0_248562372\\plugins\\org.python.pydev.debug_2.4.0.2012020116\\pysrc\\runfiles.py')
('arguments:', "['D:\\\\.eclipse\\\\org.eclipse.platform_3.7.0_248562372\\\\plugins\\\\org.python.pydev.debug_2.4.0.2012020116\\\\pysrc\\\\runfiles.py', 'D:\\\\Documents\\\\Code\\\\Eclipse\\\\workspace\\\\sqladata\\\\src\\\\unit_test.py', '--port', '49856', '--verbosity', '0']")
('Connecting to ', '127.0.0.1', ':', '49857')
('Connected.',)
('received command ', '501\t1\t1.1')
sending cmd: CMD_VERSION 501 1 1.1
sending cmd: CMD_THREAD_CREATE 103 2 <xml><thread name="pydevd.reader" id="-1"/></xml>
sending cmd: CMD_THREAD_CREATE 103 4 <xml><thread name="pydevd.writer" id="-1"/></xml>
('received command ', '111\t3\tD:\\Documents\\Code\\Eclipse\\workspace\\sqladata\\src\\testData.py\t85\t**FUNC**testAdjacency\tNone')
Added breakpoint:d:\documents\code\eclipse\workspace\sqladata\src\testdata.py - line:85 - func_name:testAdjacency
('received command ', '122\t5\t;;')
Exceptions to hook : []
('received command ', '124\t7\t')
('received command ', '101\t9\t')
Finding files... done.
Importing test modules ... testAtomic (testTypes.TypeTest) ... ok
testCyclic (testTypes.TypeTest) ...
The rest is output of the unit test.
Continuing from Fabio's answer part 2:
I have added the code at the start of the program and the debugger stops working at the last line of following the method in sqlalchemy\orm\attributes.py (it is a descriptor, but how or whther it interferes with the debugging is beyond my current knowledge):
class InstrumentedAttribute(QueryableAttribute):
"""Class bound instrumented attribute which adds descriptor methods."""
def __set__(self, instance, value):
self.impl.set(instance_state(instance),
instance_dict(instance), value, None)
def __delete__(self, instance):
self.impl.delete(instance_state(instance), instance_dict(instance))
def __get__(self, instance, owner):
if instance is None:
return self
dict_ = instance_dict(instance)
if self._supports_population and self.key in dict_:
return dict_[self.key]
else:
return self.impl.get(instance_state(instance),dict_) #<= last line of debugging
From there the debugger steps into the __getattr__ method of one of my own classes, derived from a declarative_base() class of sqlalchemy.
Probably solved (though not understood):
The problem seemed to be that the __getattr__ mentioned above, created something similar to infinite recursion, however the program/unittest/sqlalchemy recovered without reporting any error. I do not understand the sqlalchemy code sufficiently to understand why the __getattr__ method was called.
I changed the __getattr__ method to call super for the attribute name for which the recursion occurred (most likely not my final solution) and the breakpoint problem seems gone.
If i can formulate the problem in a consise manner, i will probably try to get some more info on the google sqlalchemy newsgroup, or at least check my solution for robustness.
Thank you Fabio for your support, the trace_func() function pinpointed the problem for me.
Seems really strange... I need some more info to better diagnose the issue:
Open \plugins\org.python.pydev.debug\pysrc\pydevd_constants.py and change
DEBUG_TRACE_LEVEL = 3
DEBUG_TRACE_BREAKPOINTS = 3
run your use-case with the problem and add the output to your question...
Also, it could be that for some reason the debugging facility is reset in some library you use or in your code, so, do the following: in the same place that you'd put the breakpoint do:
import sys
print 'current trace function', sys.gettrace()
(note: when running in the debugger, it'd be expected that the trace function is something as: <bound method PyDB.trace_dispatch of <__main__.PyDB instance at 0x01D44878>> )
Also, please post which Python version you're using.
Answer part 2:
The fact that sys.gettrace() returns None is probably the real issue... I know some external libraries which mess with it (i.e.:DecoratorTools -- read: http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html) and have even seen Python bugs and compiled extensions break it...
Still, the most common reason it breaks is probably because Python will silently disable the tracing (and thus the debugger) when a recursion throws a stack overflow error (i.e.: RuntimeError: maximum recursion depth exceeded).
You can probably put a breakpoint in the very beginning of your program and step in the debugger until it stops working.
Or maybe simpler is the following: Add the code below to the very beginning of your program and see how far it goes with the printing... The last thing printed is the code just before it broke (so, you could put a breakpoint at the last line printed knowing it should be the last line where it'd work) -- note that if it's a large program, printing may take a long time -- it may even be faster printing to a file instead of a console (such as cmd, bash or eclipse) and later opening that file (just redirect the print from the example to a file).
import sys
def trace_func(frame, event, arg):
print 'Context: ', frame.f_code.co_name, '\tFile:', frame.f_code.co_filename, '\tLine:', frame.f_lineno, '\tEvent:', event
return trace_func
sys.settrace(trace_func)
If you still can't figure it out, please post more information on the obtained results...
Note: a workaround until you don't find the actual place is using:
import pydevd;pydevd.settrace()
on the place where you'd put the breakpoint -- that way you'd have a breakpoint in code which should definitely work, as it'll force setting the tracing facility at that point (it's very similar to the remote debugging: http://pydev.org/manual_adv_remote_debugger.html except that as the debugger was already previously connected, you don't really have to start the remote debugger, just do the settrace to emulate a breakpoint)
Coming late into the conversation, but just in case it helps. I just run into a similar problem and I found that the debugger is very particular w.r.t. what lines it considers "executable" and available to break on.
If you are using line continuations, or multi-line expressions (e.g. inside a list), put the breakpoint in the last line of the statement.
I hope it helps.
Try removing the corresponding .pyc file (compiled) and then running.
Also I have sometimes realized I was running more than one instance of a program.. which confused pydev.
I've definitely seen this before too. Quite a few times.
Ran into a similar situation running a django app in Eclipse/pydev. what was happening was that the code that was running was the one installed in my virtualenv, not my source code. I removed my project from my virtual env site-packages, restarted the django up in the eclipse/pydev debugger and everything was fine.
I had similar-sounding symptoms. It turned out that my module import sequence was rexec'ing my entry-point python module because a binary (non-Python) library had to be dynamically loaded, i.e., the LD_LIBRARY_PATH was dynamically reset. I don't know why this causes the debugger to ignore subsequent breakpoints. Perhaps the rexec call is not specifying debug=true; it should specify debug=true/false based on the calling context state?
Try setting a breakpoint at your first import statement being cognizant of whether you are then s(tep)'ing into or n(ext)'ing over the imports. When I would "next" over the 3rdparty import that required the dynamic lib loading, the debug interpreter would just continue past all breakpoints.
I'm serving a Django app behind IIS 6. I'm wondering if I can restart IIS 6 within Python/Django and what one of the best ways to do would be.
Help would be great!
Besides what's already suggested, you can also use WMI via either the Win32_Service or the IIsWebService class, which inherits from it. There is a Python WMI wrapper available, which is based on pywin32.
UPDATE: A quick test of the following worked for me.
import wmi
c = wmi.WMI()
for service in c.Win32_Service(Name="W3SVC"):
result, = service.StopService()
I didn't test the next piece of code, but something like this should also work:
for service in c.IIsWebService():
result, = service.StopService()
You can see the documentation for the return values from the StopService and StartService methods.
The following post shows how to control Windows services from Python: http://fuzzytolerance.info/code/using-python-to-manage-windows-services/
You should be able that to restart the IIS web publishing service (known as 'w3svc')
I think that you can execute an iisreset via a commandline. I've never tried that with Django but it should work and be quite simple to implement.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
So, I started learning to code in Python and later Django. The first times it was hard looking at tracebacks and actually figure out what I did wrong and where the syntax error was. Some time has passed now and some way along the way, I guess I got a routine in debugging my Django code. As this was done early in my coding experience, I sat down and wondered if how I was doing this was ineffective and could be done faster. I usually manage to find and correct the bugs in my code, but I wonder if I should be doing it faster?
I usually just use the debug info Django gives when enabled. When things do end up as I thought it would, I break the code flow a lot with a syntax error, and look at the variables at that point in the flow to figure out, where the code does something other than what I wanted.
But can this be improved? Are there some good tools or better ways to debug your Django code?
There are a bunch of ways to do it, but the most straightforward is to simply
use the Python debugger. Just add following line in to a Django view function:
import pdb; pdb.set_trace()
or
breakpoint() #from Python3.7
If you try to load that page in your browser, the browser will hang and you get a prompt to carry on debugging on actual executing code.
However there are other options (I am not recommending them):
* return HttpResponse({variable to inspect})
* print {variable to inspect}
* raise Exception({variable to inspect})
But the Python Debugger (pdb) is highly recommended for all types of Python code. If you are already into pdb, you'd also want to have a look at IPDB that uses ipython for debugging.
Some more useful extension to pdb are
pdb++, suggested by Antash.
pudb, suggested by PatDuJour.
Using the Python debugger in Django, suggested by Seafangs.
I really like Werkzeug's interactive debugger. It's similar to Django's debug page, except that you get an interactive shell on every level of the traceback. If you use the django-extensions, you get a runserver_plus managment command which starts the development server and gives you Werkzeug's debugger on exceptions.
Of course, you should only run this locally, as it gives anyone with a browser the rights to execute arbitrary python code in the context of the server.
A little quickie for template tags:
#register.filter
def pdb(element):
import pdb; pdb.set_trace()
return element
Now, inside a template you can do {{ template_var|pdb }} and enter a pdb session (given you're running the local devel server) where you can inspect element to your heart's content.
It's a very nice way to see what's happened to your object when it arrives at the template.
There are a few tools that cooperate well and can make your debugging task easier.
Most important is the Django debug toolbar.
Then you need good logging using the Python logging facility. You can send logging output to a log file, but an easier option is sending log output to firepython. To use this you need to use the Firefox browser with the firebug extension. Firepython includes a firebug plugin that will display any server-side logging in a Firebug tab.
Firebug itself is also critical for debugging the Javascript side of any app you develop. (Assuming you have some JS code of course).
I also liked django-viewtools for debugging views interactively using pdb, but I don't use it that much.
There are more useful tools like dozer for tracking down memory leaks (there are also other good suggestions given in answers here on SO for memory tracking).
I use PyCharm (same pydev engine as eclipse). Really helps me to visually be able to step through my code and see what is happening.
Almost everything has been mentioned so far, so I'll only add that instead of pdb.set_trace() one can use ipdb.set_trace() which uses iPython and therefore is more powerful (autocomplete and other goodies). This requires ipdb package, so you only need to pip install ipdb
I've pushed django-pdb to PyPI.
It's a simple app that means you don't need to edit your source code every time you want to break into pdb.
Installation is just...
pip install django-pdb
Add 'django_pdb' to your INSTALLED_APPS
You can now run: manage.py runserver --pdb to break into pdb at the start of every view...
bash: manage.py runserver --pdb
Validating models...
0 errors found
Django version 1.3, using settings 'testproject.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
GET /
function "myview" in testapp/views.py:6
args: ()
kwargs: {}
> /Users/tom/github/django-pdb/testproject/testapp/views.py(7)myview()
-> a = 1
(Pdb)
And run: manage.py test --pdb to break into pdb on test failures/errors...
bash: manage.py test testapp --pdb
Creating test database for alias 'default'...
E
======================================================================
>>> test_error (testapp.tests.SimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../django-pdb/testproject/testapp/tests.py", line 16, in test_error
one_plus_one = four
NameError: global name 'four' is not defined
======================================================================
> /Users/tom/github/django-pdb/testproject/testapp/tests.py(16)test_error()
-> one_plus_one = four
(Pdb)
The project's hosted on GitHub, contributions are welcome of course.
The easiest way to debug python - especially for programmers that are used to Visual Studio - is using PTVS (Python Tools for Visual Studio).
The steps are simple:
Download and install it from https://microsoft.github.io/PTVS/
Set breakpoints and press F5.
Your breakpoint is hit, you can view/change the variables as easy as debugging C#/C++ programs.
That's all :)
If you want to debug Django using PTVS, you need to do the following:
In Project settings - General tab, set "Startup File" to "manage.py", the entry point of the Django program.
In Project settings - Debug tab, set "Script Arguments" to "runserver --noreload". The key point is the "--noreload" here. If you don't set it, your breakpoints won't be hit.
Enjoy it.
I use pyDev with Eclipse really good, set break points, step into code, view values on any objects and variables, try it.
I use PyCharm and stand by it all the way. It cost me a little but I have to say the advantage that I get out of it is priceless. I tried debugging from console and I do give people a lot of credit who can do that, but for me being able to visually debug my application(s) is great.
I have to say though, PyCharm does take a lot of memory. But then again, nothing good is free in life. They just came with their latest version 3. It also plays very well with Django, Flask and Google AppEngine. So, all in all, I'd say it's a great handy tool to have for any developer.
If you are not using it yet, I'd recommend to get the trial version for 30 days to take a look at the power of PyCharm. I'm sure there are other tools also available, such as Aptana. But I guess I just also like the way PyCharm looks. I feel very comfortable debugging my apps there.
From my perspective, we could break down common code debugging tasks into three distinct usage patterns:
Something has raised an exception: runserver_plus' Werkzeug debugger to the rescue. The ability to run custom code at all the trace levels is a killer. And if you're completely stuck, you can create a Gist to share with just a click.
Page is rendered, but the result is wrong: again, Werkzeug rocks. To make a breakpoint in code, just type assert False in the place you want to stop at.
Code works wrong, but the quick look doesn't help. Most probably, an algorithmic problem. Sigh. Then I usually fire up a console debugger PuDB: import pudb; pudb.set_trace(). The main advantage over [i]pdb is that PuDB (while looking as you're in 80's) makes setting custom watch expressions a breeze. And debugging a bunch of nested loops is much simpler with a GUI.
Ah, yes, the templates' woes. The most common (to me and my colleagues) problem is a wrong context: either you don't have a variable, or your variable doesn't have some attribute. If you're using debug toolbar, just inspect the context at the "Templates" section, or, if it's not sufficient, set a break in your views' code just after your context is filled up.
So it goes.
Add import pdb; pdb.set_trace() or breakpoint() (form python3.7) at the corresponding line in the Python code and execute it. The execution will stop with an interactive shell. In the shell you can execute Python code (i.e. print variables) or use commands such as:
c continue execution
n step to the next line within the same function
s step to the next line in this function or a called function
q quit the debugger/execution
Also see: https://poweruser.blog/setting-a-breakpoint-in-python-438e23fe6b28
Sometimes when I wan to explore around in a particular method and summoning pdb is just too cumbersome, I would add:
import IPython; IPython.embed()
IPython.embed() starts an IPython shell which have access to the local variables from the point where you call it.
I just found wdb (http://www.rkblog.rk.edu.pl/w/p/debugging-python-code-browser-wdb-debugger/?goback=%2Egde_25827_member_255996401). It has a pretty nice user interface / GUI with all the bells and whistles. Author says this about wdb -
"There are IDEs like PyCharm that have their own debuggers. They offer similar or equal set of features ... However to use them you have to use those specific IDEs (and some of then are non-free or may not be available for all platforms). Pick the right tool for your needs."
Thought i'd just pass it on.
Also a very helpful article about python debuggers:
https://zapier.com/engineering/debugging-python-boss/
Finally, if you'd like to see a nice graphical printout of your call stack in Django, checkout:
https://github.com/joerick/pyinstrument. Just add pyinstrument.middleware.ProfilerMiddleware to MIDDLEWARE_CLASSES, then add ?profile to the end of the request URL to activate the profiler.
Can also run pyinstrument from command line or by importing as a module.
I highly recommend epdb (Extended Python Debugger).
https://bitbucket.org/dugan/epdb
One thing I love about epdb for debugging Django or other Python webservers is the epdb.serve() command. This sets a trace and serves this on a local port that you can connect to. Typical use case:
I have a view that I want to go through step-by-step. I'll insert the following at the point I want to set the trace.
import epdb; epdb.serve()
Once this code gets executed, I open a Python interpreter and connect to the serving instance. I can analyze all the values and step through the code using the standard pdb commands like n, s, etc.
In [2]: import epdb; epdb.connect()
(Epdb) request
<WSGIRequest
path:/foo,
GET:<QueryDict: {}>,
POST:<QuestDict: {}>,
...
>
(Epdb) request.session.session_key
'i31kq7lljj3up5v7hbw9cff0rga2vlq5'
(Epdb) list
85 raise some_error.CustomError()
86
87 # Example login view
88 def login(request, username, password):
89 import epdb; epdb.serve()
90 -> return my_login_method(username, password)
91
92 # Example view to show session key
93 def get_session_key(request):
94 return request.session.session_key
95
And tons more that you can learn about typing epdb help at any time.
If you want to serve or connect to multiple epdb instances at the same time, you can specify the port to listen on (default is 8080). I.e.
import epdb; epdb.serve(4242)
>> import epdb; epdb.connect(host='192.168.3.2', port=4242)
host defaults to 'localhost' if not specified. I threw it in here to demonstrate how you can use this to debug something other than a local instance, like a development server on your local LAN. Obviously, if you do this be careful that the set trace never makes it onto your production server!
As a quick note, you can still do the same thing as the accepted answer with epdb (import epdb; epdb.set_trace()) but I wanted to highlight the serve functionality since I've found it so useful.
One of your best option to debug Django code is via wdb:
https://github.com/Kozea/wdb
wdb works with python 2 (2.6, 2.7), python 3 (3.2, 3.3, 3.4, 3.5) and pypy. Even better, it is possible to debug a python 2 program with a wdb server running on python 3 and vice-versa or debug a program running on a computer with a debugging server running on another computer inside a web page on a third computer!
Even betterer, it is now possible to pause a currently running python process/thread using code injection from the web interface. (This requires gdb and ptrace enabled)
In other words it's a very enhanced version of pdb directly in your browser with nice features.
Install and run the server, and in your code add:
import wdb
wdb.set_trace()
According to the author, main differences with respect to pdb are:
For those who don’t know the project, wdb is a python debugger like pdb, but with a slick web front-end and a lot of additional features, such as:
Source syntax highlighting
Visual breakpoints
Interactive code completion using jedi
Persistent breakpoints
Deep objects inspection using mouse Multithreading / Multiprocessing support
Remote debugging
Watch expressions
In debugger code edition
Popular web servers integration to break on error
In exception breaking during trace (not post-mortem) in contrary to the werkzeug debugger for instance
Breaking in currently running programs through code injection (on supported systems)
It has a great browser-based user interface. A joy to use! :)
I use PyCharm and different debug tools. Also have a nice articles set about easy set up those things for novices. You may start here. It tells about PDB and GUI debugging in general with Django projects. Hope someone would benefit from them.
If using Aptana for django development, watch this: http://www.youtube.com/watch?v=qQh-UQFltJQ
If not, consider using it.
Most options are alredy mentioned.
To print template context, I've created a simple library for that.
See https://github.com/edoburu/django-debugtools
You can use it to print template context without any {% load %} construct:
{% print var %} prints variable
{% print %} prints all
It uses a customized pprint format to display the variables in a <pre> tag.
I find Visual Studio Code is awesome for debugging Django apps. The standard python launch.json parameters run python manage.py with the debugger attached, so you can set breakpoints and step through your code as you like.
For those that can accidentally add pdb into live commits, I can suggest this extension of #Koobz answer:
#register.filter
def pdb(element):
from django.conf import settings
if settings.DEBUG:
import pdb
pdb.set_trace()
return element
From my own experience , there are two way:
use ipdb,which is a enhanced debugger likes pdb.
import ipdb;ipdb.set_trace() or breakpoint() (from python3.7)
use django shell ,just use the command below. This is very helpfull when you are developing a new view.
python manage.py shell
i highly suggest to use PDB.
import pdb
pdb.set_trace()
You can inspect all the variables values, step in to the function and much more.
https://docs.python.org/2/library/pdb.html
for checking out the all kind of request,response and hits to database.i am using django-debug-toolbar
https://github.com/django-debug-toolbar/django-debug-toolbar
As mentioned in other posts here - setting breakpoints in your code and walking thru the code to see if it behaves as you expected is a great way to learn something like Django until you have a good sense of how it all behaves - and what your code is doing.
To do this I would recommend using WingIde. Just like other mentioned IDEs nice and easy to use, nice layout and also easy to set breakpoints evaluate / modify the stack etc. Perfect for visualizing what your code is doing as you step through it. I'm a big fan of it.
Also I use PyCharm - it has excellent static code analysis and can help sometimes spot problems before you realize they are there.
As mentioned already django-debug-toolbar is essential - https://github.com/django-debug-toolbar/django-debug-toolbar
And while not explicitly a debug or analysis tool - one of my favorites is SQL Printing Middleware available from Django Snippets at https://djangosnippets.org/snippets/290/
This will display the SQL queries that your view has generated. This will give you a good sense of what the ORM is doing and if your queries are efficient or you need to rework your code (or add caching).
I find it invaluable for keeping an eye on query performance while developing and debugging my application.
Just one other tip - I modified it slightly for my own use to only show the summary and not the SQL statement.... So I always use it while developing and testing. I also added that if the len(connection.queries) is greater than a pre-defined threshold it displays an extra warning.
Then if I spot something bad (from a performance or number of queries perspective) is happening I turn back on the full display of the SQL statements to see exactly what is going on. Very handy when you are working on a large Django project with multiple developers.
use pdb or ipdb. Diffrence between these two is ipdb supports auto complete.
for pdb
import pdb
pdb.set_trace()
for ipdb
import ipdb
ipdb.set_trace()
For executing new line hit n key, for continue hit c key.
check more options by using help(pdb)
An additional suggestion.
You can leverage nosetests and pdb together, rather injecting pdb.set_trace() in your views manually. The advantage is that you can observe error conditions when they first start, potentially in 3rd party code.
Here's an error for me today.
TypeError at /db/hcm91dmo/catalog/records/
render_option() argument after * must be a sequence, not int
....
Error during template rendering
In template /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/crispy_forms/templates/bootstrap3/field.html, error at line 28
render_option() argument after * must be a sequence, not int
18
19 {% if field|is_checkboxselectmultiple %}
20 {% include 'bootstrap3/layout/checkboxselectmultiple.html' %}
21 {% endif %}
22
23 {% if field|is_radioselect %}
24 {% include 'bootstrap3/layout/radioselect.html' %}
25 {% endif %}
26
27 {% if not field|is_checkboxselectmultiple and not field|is_radioselect %}
28
{% if field|is_checkbox and form_show_labels %}
Now, I know this means that I goofed the constructor for the form, and I even have good idea of which field is a problem. But, can I use pdb to see what crispy forms is complaining about, within a template?
Yes, I can. Using the --pdb option on nosetests:
tests$ nosetests test_urls_catalog.py --pdb
As soon as I hit any exception (including ones handled gracefully), pdb stops where it happens and I can look around.
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 537, in __str__
return self.as_widget()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/forms.py", line 593, in as_widget
return force_text(widget.render(name, self.value(), attrs=attrs))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 513, in render
options = self.render_options(choices, [value])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py", line 543, in render_options
output.append(self.render_option(selected_choices, *option))
TypeError: render_option() argument after * must be a sequence, not int
INFO lib.capture_middleware log write_to_index(http://localhost:8082/db/hcm91dmo/catalog/records.html)
INFO lib.capture_middleware log write_to_index:end
> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/forms/widgets.py(543)render_options()
-> output.append(self.render_option(selected_choices, *option))
(Pdb) import pprint
(Pdb) pprint.PrettyPrinter(indent=4).pprint(self)
<django.forms.widgets.Select object at 0x115fe7d10>
(Pdb) pprint.PrettyPrinter(indent=4).pprint(vars(self))
{ 'attrs': { 'class': 'select form-control'},
'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]],
'is_required': False}
(Pdb)
Now, it's clear that my choices argument to the crispy field constructor was as it was a list within a list, rather than a list/tuple of tuples.
'choices': [[('_', 'any type'), (7, (7, 'type 7', 'RECTYPE_TABLE'))]]
The neat thing is that this pdb is taking place within crispy's code, not mine and I didn't need to insert it manually.
During development, adding a quick
assert False, value
can help diagnose problems in views or anywhere else, without the need to use a debugger.