Develop Reference

Attaching Python debugger

I'd like to attach a Python debugger to a running process. Following this comment, I tried pdb-clone but have gotten confused. Here's the script I'm attaching to:
import os
import time
from pdb_clone import pdbhandler
pdbhandler.register()
def loop(my_pid):
print("Entering loop")
while True:
x = 'frog'
time.sleep(0.5)
print("Out of Loop")
if __name__ == '__main__':
my_pid = os.getpid()
print("pid = ", my_pid)
loop(my_pid)
If I run python3 target_code_1.py in one terminal and see PID = 95439, then in a second terminal try
sudo pdb-attach --kill --pid 95439
I get an error message (which I include below).
However, suppose I simultaneously run python3 target_code_1.py in a third terminal. I can now run sudo pdb-attach --kill --pid 95439 without error, but when I print my_pid, the value is 95440. On the other hand, if I run sudo pdb-attach --kill --pid 95440 and print my_pid, the value is 95439. (In other words, it looks like pdb-attach has swapped which thread it is attaching to.) This behavior appears to be repeatable. What is going on?
For the record, the initial error message is as follows:
sudo pdb-attach --kill --pid 95440
Traceback (most recent call last):
File "/usr/local/bin/pdb-attach", line 4, in <module>
attach.main()
File "/usr/local/lib/python3.7/site-packages/pdb_clone/attach.py", line 646, in main
attach(address)
File "/usr/local/lib/python3.7/site-packages/pdb_clone/attach.py", line 596, in attach
for count in asock.connect_retry(address, verbose):
File "/usr/local/lib/python3.7/site-packages/pdb_clone/attach.py", line 115, in connect_retry
self.connect(address)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncore.py", line 342, in connect
raise OSError(err, errorcode[err])
OSError: [Errno 22] EINVAL
FWIW, I'm running on macOS Mojave 10.14.2, Python 3.7.0, Clang 9.1.0.
(If I am solving this problem the wrong way, e.g., if there is a better Python module to use that can attach to live process, I'd be happy to use it instead.)

Error on Python serial import

When I try to import the serial I get the following error:
Traceback (most recent call last):
File "C:\Documents and Settings\eduardo.pereira\workspace\thgspeak\tst.py", line 7, in <module>
import serial
File "C:\Python27\lib\site-packages\serial\__init__.py", line 27, in <module>
from serial.serialwin32 import Serial
File "C:\Python27\lib\site-packages\serial\serialwin32.py", line 15, in <module>
from serial import win32
File "C:\Python27\lib\site-packages\serial\win32.py", line 182, in <module>
CancelIoEx = _stdcall_libraries['kernel32'].CancelIoEx
File "C:\Python27\lib\ctypes\__init__.py", line 375, in __getattr__
func = self.__getitem__(name)
File "C:\Python27\lib\ctypes\__init__.py", line 380, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'CancelIoEx' not found
I have installed the latest version of pySerial, Python 2.7 runing on a WinXP laptop. Tried everywhere and found no similar problem. Is there any solution for that?
Thanks in advance...

The version of pySerial that you're using is trying to call a function that's only available in Windows Vista, whereas you're running Windows XP.
It might be worth experimenting with using an older version of pySerial.
The code in question was added to pySerial on 3 May 2016, so a version just prior to that might be a good start.

Older versions seem unavailable. But, this worked for me (assuming nanpy version 3.1.1):
open file \lib\site-packages\serial\serialwin32.py
delete methods _cancel_overlapped_io(), cancel_read(), cancel_write() in lines 436-455 nearly at the botton of the file
change method _close() als follows:
(Python)
def _close(self):
"""internal close port helper"""
if self._port_handle is not None:
# Restore original timeout values:
win32.SetCommTimeouts(self._port_handle, self._orgTimeouts)
# Close COM-Port:
if self._overlapped_read is not None:
win32.CloseHandle(self._overlapped_read.hEvent)
self._overlapped_read = None
if self._overlapped_write is not None:
win32.CloseHandle(self._overlapped_write.hEvent)
self._overlapped_write = None
win32.CloseHandle(self._port_handle)
self._port_handle = None
Additionally, create a non-default serial connection when starting the communication, otherwise you'll be bound to some linux device:
a = ArduinoApi(SerialManager("COM5:"))
for i in range(10):
a.pinMode(13, a.OUTPUT)
a.digitalWrite(13, a.HIGH)
# etc.

And in serial\win32.py comments
#CancelIoEx = _stdcall_libraries['kernel32'].CancelIoEx
#CancelIoEx.restype = BOOL
#CancelIoEx.argtypes = [HANDLE, LPOVERLAPPED]

There is no obvious reason why CancelIO was replaced with the cross-thread version CancelIOex, which allows code in one thread to cancel IO in another thread. Certainly cpython 2.x is single threaded.
To get pySerial to run on python 2.7 on Win2K, I just changed CancelIOex in serial back to CancelIO.

uTorrent Automation using pywinauto

I am trying out an utorrent automation using pywinauto lib. I want to add a torrent with URL. This option is under the file menu. I can get as far as opening uTorrent and then nothing happens. I used Swapy for generating this code. The box below opens only when I run the code in swapy. But when I save it into a file and run with cmd, only utorrent opens and a traceback occurs in the cmd.
from pywinauto.application import Application
app = Application().Start(cmd_line=u'"C:\\Users\\User\\AppData\\Roaming\\uTorrent\\u Torrent.exe" ')
torrentdfb = app[u'\xb5Torrent4823DF041B09']
torrentdfb.Wait('ready')
menu_item = torrentdfb.MenuItem(u'&File->Add Torrent from &URL...\tCtrl+U')
menu_item.Click()
app.Kill_()
Traceback:
Traceback (most recent call last):
File "AddTorrent.py", line 5, in <module>
torrentdfb.Wait('ready')
File "C:\Python27\lib\site-packages\pywinauto\application.py", line 380, in Wait
WaitUntil(timeout, retry_interval, lambda: self.__check_all_conditions(check_method_names))
File "C:\Python27\lib\site-packages\pywinauto\timings.py", line 308, in WaitUntil
raise err
pywinauto.timings.TimeoutError: timed out
I am new to python coding and I am not an expert. It would be helpful if you provide the explanation to solve my problem or the code. Thanks!!

uTorrent is spawning another process, this is how I got it:
>>> app.windows_()
[]
>>> app.process
6096
>>> app.connect(title_re=u'^μTorrent.*(build \d+).*')
<pywinauto.application.Application object at 0x000000000405C240>
>>> app.process
4044L
This is a final code working for me (with 32-bit uTorrent and 32-bit Python 2.7):
import pywinauto
app = pywinauto.Application().start(r'uTorrent.exe')
time.sleep(5) # because method connect() has no timeout param yet (planned for 0.6.0)
app.connect(title_re=u'^\u03bcTorrent.*(build \d+).*')
main_window = app.window_(title_re=u'^\u03bcTorrent.*(build \d+).*')
main_window.MenuSelect(u'&File->Add Torrent from &URL...\tCtrl+U')
app.AddTorrentFromURL.Edit.SetText('some URL')
app.AddTorrentFromURL.OK.Click()
Bitness is important. 32-bit uTorrent crashes if I use 64-bit Python.

winpdb not working with python 3.3

I can't get rpdb2 to run with python 3.3, while that should be possible according to several sources.
$ rpdb2 -d myscript.py
A password should be set to secure debugger client-server communication.
Please type a password:x
Password has been set.
Traceback (most recent call last):
File "/usr/local/bin/rpdb2", line 31, in <module>
rpdb2.main()
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 14470, in main
StartServer(_rpdb2_args, fchdir, _rpdb2_pwd, fAllowUnencrypted, fAllowRemote, secret)
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 14212, in StartServer
g_module_main = -1
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 14212, in StartServer
g_module_main = -1
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 7324, in trace_dispatch_init
self.__set_signal_handler()
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 7286, in __set_signal_handler
handler = signal.getsignal(value)
File "/usr/local/lib/python3.3/dist-packages/rpdb2.py", line 13682, in __getsignal
handler = g_signal_handlers.get(signum, g_signal_getsignal(signum))
ValueError: signal number out of range
The version of rpdb2 is RPDB 2.4.8 - Tychod.
I installed it by running pip-3.3 install winpdb.
Any clues?

Got the same problem today here is what i've done for it to work.
Still i'm not too sure if this is correct to do it this way.
From:
def __getsignal(signum):
handler = g_signal_handlers.get(signum, g_signal_getsignal(signum))
return handler
To:
def __getsignal(signum):
try:
# The problems come from the signum which was 0.
g_signal_getsignal(signum)
except ValueError:
return None
handler = g_signal_handlers.get(signum, g_signal_getsignal(signum))
return handler
This function shall be at line 13681 or something like.

The reason of a problem is the extended list of attributes in a signal module that was used by rpdb2 to list all signals. New python versions added attributes like SIG_BLOCK, SIG_UNBLOCK, SIG_SETMASK
So the filtering should be extended too (patch changes just one line):
--- rpdb2.py
+++ rpdb2.py
## -7278,11 +7278,11 ##
def __set_signal_handler(self):
"""
Set rpdb2 to wrap all signal handlers.
"""
for key, value in list(vars(signal).items()):
- if not key.startswith('SIG') or key in ['SIGRTMIN', 'SIGRTMAX'] or key.startswith('SIG_'):
+ if not key.startswith('SIG') or key in ['SIG_IGN', 'SIG_DFL', 'SIGRTMIN', 'SIGRTMAX']:
continue
handler = signal.getsignal(value)
if handler in [signal.SIG_IGN, signal.SIG_DFL]:
continue
Unfortunately there is no current official development or fork of winpdb, so by now this patch would be just stored on SO.

Debugging GAE in Python Tools for Visual Studio

I'm able to run my Google App Engine webapp2 app using Python Tools for Visual Studio 2012 without issues after following this tutorial, and even step through the server initialization code, but I can't get it to break at get or post methods when the website is loaded, similar to what is shown in this video with the main() method. When I pause the debugger, it always ends up in the following infinite loop in wsgi_server.py:
def _loop_forever(self):
while True:
self._select()
def _select(self):
with self._lock:
fds = self._file_descriptors
fd_to_callback = self._file_descriptor_to_callback
if fds:
if _HAS_POLL:
# With 100 file descriptors, it is approximately 5x slower to
# recreate and reinitialize the Poll object on every call to _select
# rather reuse one. But the absolute cost of contruction,
# initialization and calling poll(0) is ~25us so code simplicity
# wins.
poll = select.poll()
for fd in fds:
poll.register(fd, select.POLLIN)
ready_file_descriptors = [fd for fd, _ in poll.poll(1)]
else:
ready_file_descriptors, _, _ = select.select(fds, [], [], 1)
for fd in ready_file_descriptors:
fd_to_callback[fd]()
else:
# select([], [], [], 1) is not supported on Windows.
time.sleep(1)
Is it possible to set breakpoints in a Google App Engine webapp2 app in PTVS, which are triggered when the page is loaded from localhost?
Edit: using cprcrack's settings, I was able to successfully run GAE, but when loading the main page I get the error
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 3003, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 2862, in _Dispatch
base_env_dict=env_dict)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 719, in Dispatch
base_env_dict=base_env_dict)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 1797, in Dispatch
self._module_dict)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 1648, in ExecuteCGI
app_log_handler = app_logging.AppLogsHandler()
File "C:\Python\lib\logging\__init__.py", line 660, in __init__
_addHandlerRef(self)
File "C:\Python\lib\logging\__init__.py", line 639, in _addHandlerRef
_releaseLock()
File "C:\Python\lib\logging\__init__.py", line 224, in _releaseLock
_lock.release()
File "C:\Python\lib\threading.py", line 138, in release
self.__count = count = self.__count - 1
File "C:\Python\lib\threading.py", line 138, in release
self.__count = count = self.__count - 1
File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\visualstudio_py_debugger.py", line 557, in trace_func
return self._events[event](frame, arg)
File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\visualstudio_py_debugger.py", line 650, in handle_line
if filename == frame.f_code.co_filename or (not bound and filename_is_same(filename, frame.f_code.co_filename)):
File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\visualstudio_py_debugger.py", line 341, in filename_is_same
import ntpath
File "C:\Python\lib\ntpath.py", line 8, in <module>
import os
File "C:\Python\lib\os.py", line 120, in <module>
from os.path import (curdir, pardir, sep, pathsep, defpath, extsep, altsep,
ImportError: cannot import name curdir
Is this error occurring because I need to roll back to Python 2.5 to use the old dev_appserver?

UPDATE#2
gcloud preview deprecated
it's back to original method
UPDATE#1
gcloud preview (it's newer and simpler),
replace this:
General->Startup File:
C:\Program Files\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\gcloud\gcloud.py
Debug->Script Arguments:
preview app run app.yaml --python-startup-script "pydevd_startup.py" --max-module-instances="default:1"
all rest is the same as the original answer below:
ORIGINAL ANSWER:
A.) Create A File to Inject remote debugger
make a new python file "pydevd_startup.py"
insert this:
import json
import sys
if ':' not in config.version_id:
# The default server version_id does not contain ':'
sys.path.append("lib")
import ptvsd #ptvsd.settrace() equivalent
ptvsd.enable_attach(secret = 'joshua')
ptvsd.wait_for_attach()
Save it in your working directory of your app
for more info look at the pytool remote debuging docu I mentioned above
B.) Edit Project Settings in VS 2013
Now open your Project Settings in VS and enter this:
General->Startup File: C:\Cloud SDK\google-cloud-sdk\bin\dev_appserver.py
General->Working Directory: .
Debug->Search Paths: C:\Cloud SDK\google-cloud-sdk\lib
Debug->Script Arguments: --python_startup_script=".\pydevd_startup.py" --automatic_restart=no --max_module_instances="default:1" ".\app.yaml"
You could probably also use . instead of <path-to-your-app> but I wanted to be safe.
C.) Run Debugger
With Ctrl+F5 you run the debugger, without debugging. This sound weird, but we are actually not debugging right now, just running the dev server which than starts our script to inject the debugger code and wait for our remote debugger to connect, which will happen in the next step
D.) Start Remote Debugger
DEBUG->Attach to Process <Ctrl+Alt+P>
Qualifier: tcp://joshua#localhost:5678 <ENTER>
joshua is your secret key. If you want to change it (and you should), you also have to change it in the pydevd_startup.py. See pytool reference for more info.
F.) Be really happy!
You now can remote debug your application locally (erm, weird). To test this you probably should use a breakpoint at the start of your own script.
If you have any question, please ask. In the end it seems really simple, but to get this going was rough. Especially because pytools said, they don't support it...
G.) Start Debugging for real!
Open http://localhost:8080 in a browser (or any other address you configure your app to use). Now it should invoke the breaking point. If you are done and reload the site, it starts all over again. If you really want to end debugging or change some code, you have to restart the server and attach again. Don't forget to close the terminal window with the server open (use <Crtl+C> )

This is a known issue with Google App Engine for Python: currently, debugging does not work on any debugger. See here, here and here.

There's a workaround, but I don't know about getting this working for python tools for vs. In theory it should be possible.
https://groups.google.com/forum/#!topicsearchin/google-appengine/Boa/google-appengine/-m00Qz4Vc7U
You'd probably need this guide to get it working:
https://docs.google.com/document/d/1CCSaRiIWCLgbD3OwmuKsRoHHDfBffbROWyVWWL0ZXN4/edit#heading=h.fj44xnkhr0gr

I'm using the old dev_appserver for debugging and it's working for me in an scenario similar to yours. I also got a bunch of exceptions but I was able to just skip all of them following the instructions on this link (I also had to add "exceptions" for some ValueError exceptions).
These are my project properties:
General tab:
Startup File: C:\Program Files (x86)\Google\google_appengine\old_dev_appserver.py
Working Directory: ./
Windows Application: (unchecked)
Interpreter: Python 2.7
Debug tab:
Search Paths: C:\Program Files (x86)\Google\google_appengine
Script Arguments: --use_sqlite ./
Interpreter Arguments: (blank)
Interpreter Path: C:\Python27\python.exe
When there is no need for breakpoints I run the project with DEBUG > Execute Project in Python Interactive. This way you don't get the unneeded console window.