Develop Reference

eventlet.monkey_patch() breaks ipdb in docker container

So I am using the micro-servicing python package nameko which runs a service using eventlet and calls eventlet.monkey_patch() on import.
I have deciphered that it is this piece of code that is blocking any debug attempts via ipdb. The ipdb console shows in the terminal but I cannot type anything and have to close the entire terminal session in order to quit the process.
The stuck console looks like:
How can I use ipdb with this function?
EDIT: This issue only seems to happen when within a docker container.

Sorry, no convenient solution, for now your best option is to skip docker when using ipdb (you can extract filesystem image from docker and run it in another virtualisation, such as qemu, Virtualbox, systemd-nspawn). See https://github.com/larsks/undocker for help.
Other things to try (may not work, please share results):
update eventlet to github master
pip install https://github.com/eventlet/eventlet/archive/master.zip
This issue is cross posted here https://github.com/eventlet/eventlet/issues/361

pycharm and flask autoreload and breakpoints not working

I'm using Pycharm 4, with flask 0.10.1, python 3.4
It seems that when running a flask application from inside pycharm, if I run it with:
app.run(debug=True)
My breakpoints are ignored. After some googling, I've found that in order to make PyCharm stop on breakpoints, I should run flask with:
app.run(debug=True, use_reloader=False)
Now PyCharm correctly stops on breakpoints, but I miss the autoreloading feature.
Is there any way to make both work together?
Using python 2.7 both things work
I reported this to PyCharm: https://youtrack.jetbrains.com/issue/PY-13976

I'm going to start with the short answer: No, what you want cannot be done with any releases of PyCharm up to 4.0.1.
The problem is that when you use the reloader the Flask application runs in a child process, so the PyCharm debugger is attached to the master process and has no control over the child.
The best way to solve this problem, in my opinion, is to ask Jetbrains to build a "restart on change" feature in their IDE. Then you don't need to use Werkzeug's reloader at all and you get the same functionality direct from PyCharm.
Until Jetbrains decides to implement this, I can share my workaround, which is not terribly bad.
In the "Edit Configurations", set the configuration you are going to use to "Single Instance only" (check box in the top right of the dialog box)
Make sure the configuration is the active one.
Configure your Flask app to not use the Werkzeug reloader.
Press Ctrl-D to start debugging (on Mac, others may have a different shortcut)
Breakpoints should work just fine because the reloader isn't active.
Make any code changes you need.
When you are ready to restart, hit Ctrl-D again. The first time you do it you will get a confirmation prompt, something like "stop and restart?". Say yes, and check the "do not show again" checkbox.
Now you can hit Ctrl-D to quickly restart the debugger whenever you need to.
I agree it is not perfect, but once the Ctrl-D gets into your muscle memory you will not even think about it.
Good luck!

I found that in PyCharm 2018.1.2 there is FLASK_DEBUG checbox in run configuration:
With this after making some changes, saving file triggers reload action.

In my setup, I'm debugging the flask app by running a main.py file which sets some configuration and calls app.run(). My python interpreter is set up in a Docker container.
My issue was that I needed to check Run with Python console.

The problem is because with use_reloader=True the werkzeug application is started in a seperate (child) thread of main application and PyCharm fails to correctly handle breakpoints because they are lost when the thread starts.
You can try to follow this thread: http://forum.jetbrains.com/thread/PyCharm-776 but it seams there was not too much progress on that.
I'd suggest using something Python-ish like pdb, i.e.:
#app.route('/<string:page>')
def main(page):
import pdb; pdb.set_trace() # This line actually stops application execution
# and starts Python debug shell in the console
# where you can examine current scope and continue
# normal code execution at any time.
# You can inject *any* code here.
# For example, if you type `print page` during pause,
# it will output content of "page" variable.
return render_template('index.html')

Try configuring this python running configuration in "Edit Configurations". After that, run in debug mode.

You need to unlock the console.
you start the app in debug mode
then you make something that causes an error.
at the end of the error message from flask is this
Here you enter the PIN flask prints in the console at the start
copy paste this pin into the console and click confirm PIN
now the breakpoints will work

from pycharm 2017 using python 2.7 (in my case with virtual env, but I suppose not necessary) I do:
run...
leave scripts and scripts parameters blank
I put in interpreter options: -m flask run
set the env variables FLASK_APP
than run attach to local process, and finally choose the running process
my use case is to connect from postman to flask rest services endpoints and interrupt on my breakpoints

How to debug Django app running on Heroku using a remote pdb connection?

To debug a bug I'm seeing on Heroku but not on my local machine, I'm trying to do step-through debugging.
The typical import pdb; pdb.set_trace() approach doesn't work with Heroku since you don't have access to a console connected to your app, but apparently you can use rpdb, a "remote" version of pdb.
So I've installed rpdb, added import rpdb; rpdb.set_trace() at the appropriate spot. When I make a request that hits the rpdb line, the app hangs as expected and I see the following in my heroku log:
pdb is running on 3d0c9fdd-c18a-4cc2-8466-da6671a72cbc:4444
Ok, so how to connect to the pdb that is running? I've tried heroku run nc 3d0c9fdd-c18a-4cc2-8466-da6671a72cbc 4444 to try to connect to the named host from within heroku's system, but that just immediately exits with status 1 and no error message.
So my specific question is: how do I now connect to this remote pdb?
The general related question is: is this even the right way for this sort of interactive debugging of an app running on Heroku? Is there a better way?
NOTE RE CELERY: Note, I've now also tried a similar approach with Celery, to no avail. The default host celery's rdb (remote pdb wrapper) uses is localhost, which you can't get to when it's Heroku. I've tried using the CELERY_RDB_HOST environment variable to the domain of the website that is being hosted on Heroku, but that gives a "Cannot assign requested address" error. So it's the same basic issue -- how to connect to the remote pdb instance that's running on Heroku?

In answer to your second question, I do it differently depending on the type of error (browser-side, backend, or view). For backend and view testing (unittests), will something like this work for you?
$ heroku run --app=your-app "python manage.py shell --settings=settings.production"
Then debug-away within ipython:
>>> %run -d script_to_run_unittests.py
Even if you aren't running a django app you could just run the debugger as a command line option to ipython so that any python errors will drop you to the debugger:
$ heroku run --app=your-app "ipython --pdb"
Front-end testing is a whole different ballgame where you should look into tools like selenium. I think there's also a "salad" test suite module that makes front end tests easier to write. Writing a test that breaks is the first step in debugging (or so I'm told ;).
If the bug looks simple, you can always do the old "print and run" with something like
import logging
logger = logging.getLogger(__file__)
logger.warn('here be bugs')`
and review your log files with getsentry.com or an equivalent monitoring tool or just:
heroku logs --tail

pydev remote debug path

I want to use eclipse, pydev to remote debug my python script. Python script is on a remote Ubuntu server, and Eclispe/pydev is running on my Windows 7 machine.
I followed every step according to this one.
http://pydev.org/manual_adv_remote_debugger.html
The problem is in the last step of configuring path in pydevd_file_utils.py on server, it does not recognize the change. This is what I changed:
PATHS_FROM_ECLIPSE_TO_PYTHON = [(r'c:\EZ_Green\plugins', r'/home/jiechao/EZ_Green/plugins')]
When I run the script, it gives me such error.
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/jiechao/EZ_Green/plugins/D:/EZ Green/backend/getData.py (will have no effect)
Seems the change does not apply, has anyone done this before or have any ideas?
Thanks a lot
-----------------update 1--------------
So I solve the previous problem and now here is the new problem.
This is the output of program, and it seems the path configuration is correct.
Debug Server at port: 5678
pydev debugger: replacing to server: D:\EZ Green\Product\EZ_Green\plugins\test.py
pydev debugger: sent to server: /home/jiechao/EZ_Green/plugins\test.py
pydev debugger: replacing to client: /home/jiechao/EZ_Green/plugins/test.py
pydev debugger: sent to client: D:\EZ Green\Product\EZ_Green\plugins/test.py
But eclipse does not stop at the breakpoint, not even at pydevd.settrace()
I have no idea why it does not stop.
When I use remote debug on local machine, it works pretty well. When I want to debug on a remote server machine, it does not work. I don't know what's the problem.
------------------update 2---------------------
Problem solved. The script on my client and server turns out to be a little different. So I did not see the breakpoint it stopped.
I am so stupid!
Thanks anyway.

Even though it is possibly not the exact approach you may expect,
one option is to start the Unittest from the command line and attach the debugger by RemoteDebugServer via 'pydevd.py'.
This is now a fully automated option of ePyUnit which includes the automation of remote debugging with PyDev and Eclipse by 'pydevd.py'. This works seamlessly for 'subprocesses' as well as independently started command line processes.
The hostame and the port number could be varied as required, default is
localhost:5678.
See:
https://pypi.python.org/pypi/epyunit
https://pythonhosted.org/epyunit/
For basics of remote debugging:
http://www.pydev.org/manual_adv_remote_debugger.html
Also enhanced unittest integration into PyUnit.
Comments and fixes are welcome.
Have fun.

Print statements on server give IOError: failed to write data

I am running Pylons on my local machine with paster, and on a Debian server using WSGI. I want to add some print statements to debug a problem: am not a Pylons or Python expert.
On my local machine this works fine: print statements go to the terminal. On the server, the statements don't print to the log files: instead the log file says "IOError: failed to write data" whenever a print statement is called.
Until I can fix this, I can't debug anything on the server.
Could someone advise how to get printing running on the server? Thanks!

It's wrong for a WSGI application to use sys.stdout or sys.stderr. If you want to spit debug to a server error log, use environ['wsgi.errors'].write().

Don't use print statements, use the logging module. We can't help you without knowing the setup of the server.