I know how to drop into an interpreter with pdb and IPython, but this requires me knowing beforehand exactly where I want to stop. However, I often run number crunching scripts that take minutes to hours, and I would like to know exactly what it's progress is. One solution is to simply put lots of logging statements everywhere, but then I either inundate myself with too much information or fail to log exactly what I want to know.
Is there a way to initialize a listener loop that under some key combination will drop me into the code wherever it currently is? Think CTRL+Z but leaving me in Python rather than Bash.
You can use the signal module to setup a handler that will launch the debugger when you hit control-C or control-Z or whatever.. SIGINTR, SIGSUSP.
For example, define a module instant_debug.py that overrides SIGQUIT,
import signal
import pdb
def handler(signum, frame):
pdb.set_trace()
signal.signal(signal.SIGQUIT, handler)
Then make a script
import instant_debug
import time
for i in xrange(1000000):
print i
time.sleep(0.1)
At any point during execution, you can jump into the code by typing CTRL+\, examine the stack with u and d as in normal pdb, then continue with c as if nothing ever happened. Note that you will only jump in at the end of the next "atomic" operation -- that means no stopping in the middle of a giant C module.
You could do this
def main():
i = 1000
while True:
print "Count Down %s" % i
time.sleep(1)
i -= 1
try:
main()
except KeyboardInterrupt:
pass # Swallow ctrl-c
finally:
code.interact("Dropped into interpreter", local=globals())
Related
I need to break from time.sleep() using ctrl c.
While 1:
time.sleep(60)
In the above code when the control enters time.sleep function an entire 60 seconds needs to elapsed for python to handled the CTRL C
Is there any elegant way to do it. such that I can interrupt even when the control is in time.sleep function
edit
I was testing it on a legacy implementation which uses python 2.2 on windows 2000 which caused all the trouble . If I had used a higher version of python CTRL C would have interrupted the sleep() . I did a quick hack by calling sleep(1) inside a for loop . which temporarily fixed my issue
The correct answer is to use python stdlib's threading.Event
Sure you can tune down your sleep interval so you sleep for very short periods, but what if you actually want to run your loop once every 60s? Then you need to do more work to determine if it's time to run or just keep sleeping. Furthermore, you're still technically blocking but for only a short period of time. Contrast to threading.Event:
from threading import Event
exit = Event()
def main():
while not exit.is_set():
do_my_thing()
exit.wait(60)
print("All done!")
# perform any cleanup here
def quit(signo, _frame):
print("Interrupted by %d, shutting down" % signo)
exit.set()
if __name__ == '__main__':
import signal
for sig in ('TERM', 'HUP', 'INT'):
signal.signal(getattr(signal, 'SIG'+sig), quit);
main()
When the signal handler calls exit.set(), the main thread's wait() call will immediately be interrupted.
Now, you could use an Event to signal that there's more work to do, etc. But in this case it does double duty as a convenient indicator that we want to quit (e.g. the while not exit.is_set() part.)
You also have the option to put any cleanup code after your while loop.
Not sure what the sense of this code is - but if necessary use a shorter sleep() interval and put a for loop around it:
for i in range(60):
sleep(1)
Catching the KeyboardInterrupt exception using try..except is straight-forward
The KeyboardInterrupt exception is raised when a user hits the interrupt key, Ctrl-C. In python this is translated from a SIGINT signal. That means, you can get handle it however you want using the signal module:
import signal
def handler(signum, frame):
print("do whatever, like call thread.interrupt_main()")
signal.signal(signal.SIGINT, handler)
print("Waiting for SIGINT...")
signal.pause()
That way, you can do whatever you want at the receipt of a keyboard interrupt.
The most elegant solution is certainly threading.Event, though if you only need a quick hack, this code works well :
import time
def main():
print("It’s time !")
if __name__ == "__main__":
print("press ctrl-c to stop")
loop_forever = True
while loop_forever:
main()
try:
time.sleep(60)
except KeyboardInterrupt:
loop_forever = False
I tried your code with python versions 2.5, 2.6, 3 under Linux and all throw "KeyboardInterrupt" exception when hitting CTRL-C.
Maybe some exception handling catches the Interrupt or your problem is like this:
Why is KeyboardInterrupt not working in python?
Based on #Andreas Jung answer
for i in range(360):
try:
sleep(1)
except KeyboardInterrupt:
sys.exit(0)
Figured I'd throw this in.
import time
def sleep(seconds):
for i in range(seconds):
try:
time.sleep(1)
except KeyboardInterrupt:
print("Oh! You have sent a Keyboard Interrupt to me.\nBye, Bye")
break
sleep(60)
In my case, I am handling databases throughout the entire runtime of my program, and so I need to keep my 'cursor' open for the entire program. Is there anyway I can implement a termination protocol, so that when I terminate its execution or an error arises, I am able to run this quick piece of code that simply closes the cursor (I am using python sockets btw).
I would suspect I could do something like this:
if __name__ == "__main__":
Menu()
cursor.close()
However, the only reason that this doesn't work in my case is that Menu is simply starting up threads, and so its execution continues on, returning me back to cursor.close() whilst my program continues to run.
I'm not sure if there is a way to get around this problem.
Yes, you could use the signal library in python to achieve some of this functionality, in particular, capturing program termination as well interrupts to the program like ctrl + c. Example:
# a function to register the signal handlers
# once the program terminates or is halted by an interrupt like ctrl + c it executes the quit_gracefully function
def register_signal_handler():
signal.signal(signal.SIGINT, quit_gracefully)
signal.signal(signal.SIGTERM, quit_gracefully)
return
def quit_gracefully():
# close connections etc.
in case of a different error you could use a try-except block which handles the error and runs the quit_gracefully function in the except.
try:
# some code
except:
quit_gracefully()
EDIT:
this is a good post on signal. How do I capture SIGINT in Python?
You can also use the atexit module: https://docs.python.org/3/library/atexit.html.
Something like this:
import atexit
#atexit.register
def close_cursor():
print("Closing cursor before exiting.")
cursor.close()
I have a python thread that runs every 20 seconds. The code is:
import threading
def work():
Try:
#code here
except (SystemExit, KeyboardInterrupt):
raise
except Exception, e:
logger.error('error somewhere',exc_info=True)
threading.Timer(20, work).start ();
It usually runs completely fine. Once in a while, it'll return an error that doesnt make much sense. The errors are the same two errors. The first one might be legitimate, but the errors after that definitely aren't. Then after that, it returns that same error every time it runs the thread. If I kill the process and start over, then it runs cleanly. I have absolutely no idea what going on here. Help please.
As currently defined in your question, you are most likely exceeding your maximum recursion depth. I can't be certain because you have omitted any opportunities for flow control that may be evident in your try block. Furthermore, everytime your code fails to execute, the general catch for exceptions will log the exception and then bump you into a new timer with a new logger (assume you are declaring that in the try block). I think you probably meant to do the following:
import threading
import time
def work():
try:
#code here
pass
except (SystemExit, KeyboardInterrupt):
raise
except Exception, e:
logger.error('error somewhere',exc_info=True)
t = threading.Timer(20, work)
t.start()
i = 0
while True:
time.sleep(1)
i+=1
if i >1000:
break
t.cancel()
If this is in fact the case, the reason your code was not working is that when you call your work function the first time, it processes and then right at the end, starts another work function in a new timer. This happens add infinitum until the stack fills up, python coughs, and gets angry that you have recursed (called a function from within itself) too many times.
My code fix pulls the timer outside of the function so we create a single timer, which calls the work function once every 20 seconds.
Because threading.timers run in separate threads, we also need to wait around in the main thread. To do this, I added a simple while loop that will run for 1000 seconds and then close the timer and exit. If we didn't wait around in the main loop, it would call your timer and then close out immediately causing python to clean up the timer before it executed even once.
I've tried posting this in the reverse-engineering stack-exchange, but I thought I'd cross-post it here for more visibility.
I'm having trouble switching from debugging one thread to another in pydbg. I don't have much experience with multithreading, so I'm hoping that I'm just missing something obvious.
Basically, I want to suspend all threads, then start single stepping in one thread. In my case, there are two threads.
First, I suspend all threads. Then, I set a breakpoint on the location where EIP will be when thread 2 is resumed. (This location is confirmed by using IDA). Then, I enable single-stepping as I would in any other context, and resume Thread 2.
However, pydbg doesn't seem to catch the breakpoint exception! Thread 2 seems to resume and even though it MUST hit that address, there is no indication that pydbg is catching the breakpoint exception. I included a "print "HIT BREAKPOINT" inside pydbg's internal breakpoint handler, and that never seems to be called after resuming Thread 2.
I'm not too sure about where to go next, so any suggestions are appreciated!
dbg.suspend_all_threads()
print dbg.enumerate_threads()[0]
oldcontext = dbg.get_thread_context(thread_id=dbg.enumerate_threads()[0])
if (dbg.disasm(oldcontext.Eip) == "ret"):
print disasm_at(dbg,oldcontext.Eip)
print "Thread EIP at a ret"
addrstr = int("0x"+(dbg.read(oldcontext.Esp + 4,4))[::-1].encode("hex"),16)
print hex(addrstr)
dbg.bp_set(0x7C90D21A,handler=Thread_Start_bp_Handler)
print dbg.read(0x7C90D21A,1).encode("hex")
dbg.bp_set(oldcontext.Eip + dbg.instruction.length,handler=Thread_Start_bp_Handler)
dbg.set_thread_context(oldcontext,thread_id=dbg.enumerate_threads()[0])
dbg.context = oldcontext
dbg.resume_thread(dbg.enumerate_threads()[0])
dbg.single_step(enable=True)
return DBG_CONTINUE
Sorry about the "magic numbers", but they are correct as far as I can tell.
One of your problems is that you try to single step through Thread2 and you only refer to Thread1 in your code:
dbg.enumerate_threads()[0] # <--- Return handle to the first thread.
In addition, the code the you posted is not reflective of the complete structure of your script, which makes it hard to judge wether you have other errors or not. You also try to set breakpoint within the sub-brach that disassembles your instructions, which does not make a lot of sense to me logically. Let me try to explain what I know, and lay it out in an organized manner. That way you might look back at your code, re-think it and correct it.
Let's start with basic framework of debugging an application with pydbg:
Create debugger instance
Attache to the process
Set breakpoints
Run it
Breakpoint gets hit - handle it.
This is how it could look like:
from pydbg import *
from pydbg.defines import *
# This is maximum number of instructions we will log
MAX_INSTRUCTIONS = 20
# Address of the breakpoint
func_address = "0x7C90D21A"
# Create debugger instance
dbg = pydbg()
# PID to attach to
pid = int(raw_input("Enter PID: "))
# Attach to the process with debugger instance created earlier.
# Attaching the debugger will pause the process.
dbg.attach(pid)
# Let's set the breakpoint and handler as thread_step_setter,
# which we will define a little later...
dbg.bp_set(func_address, handler=thread_step_setter)
# Let's set our "personalized" handler for Single Step Exception
# It will get triggered if execution of a thread goes into single step mode.
dbg.set_callback(EXCEPTION_SINGLE_STEP, single_step_handler)
# Setup is done. Let's run it...
dbg.run()
Now having the basic structure, let's define our personalized handlers for breakpoint and single stepping. The code snippet below defines our "custom" handlers. What will happen is when breakpoint hits we will iterate through threads and set them to single step mode. It will in turn trigger single step exception, which we will handle and disassemble MAX_INSTRUCTIONS amount of instructions:
def thread_step_setter(dbg):
dbg.suspend_all_threads()
for thread_id in dbg.enumerate_threads():
print "Single step for thread: 0x%08x" % thread_id
h_thread = dbg.open_thread(thread_id)
dbg.single_step(True, h_thread)
dbg.close_handle(h_thread)
# Resume execution, which will pass control to step handler
dbg.resume_all_threads()
return DBG_CONTINUE
def single_step_handler(dbg):
global total_instructions
if instructions == MAX_INSTRUCTION:
dbg.single_step(False)
return DBG_CONTINUE
else:
# Disassemble the instruction
current_instruction = dbg.disasm(dbg.context,Eip)
print "#%d\t0x%08x : %s" % (total_instructions, dbg.context.Eip, current_instruction)
total_instructions += 1
dbg.single_step(True)
return DBG_CONTINUE
Discloser: I do not guarantee that the code above will work if copied and pasted. I typed it out and haven't tested it. However, if basic understanding is acquired, the small syntactical error could be easily fixed. I apologize in advanced if I have any. I don't currently have means or time to test it.
I really hope it helps you out.
I have a problem with my script. i want do do an event listender for the whole script from beginning to the end.
if someone presses ctrl-c or something it should ignore, or end the scipt with an print.
I can show you my code if needed.
I tried signal and sys but I didnt know how to use it right
A simple approach for ending the script with a print could be to wrap the whole script in a try/except block:
import sys
try:
while 1:
print 'To infinity and beyond!'
# etc etc...
except KeyboardInterrupt:
print 'Handling the keyboard interrupt...'
The interrupt still kills the script, but I would think this is desired behaviour for any user; personally I wouldn't recommend trying to override it.