I'm running this simple code:
import threading, time
class reqthread(threading.Thread):
def run(self):
for i in range(0, 10):
time.sleep(1)
print('.')
try:
thread = reqthread()
thread.start()
except (KeyboardInterrupt, SystemExit):
print('\n! Received keyboard interrupt, quitting threads.\n')
But when I run it, it prints
$ python prova.py
.
.
^C.
.
.
.
.
.
.
.
Exception KeyboardInterrupt in <module 'threading' from '/usr/lib/python2.6/threading.pyc'> ignored
In fact python thread ignore my Ctrl+C keyboard interrupt and doesn't print Received Keyboard Interrupt. Why? What is wrong with this code?
Try
try:
thread=reqthread()
thread.daemon=True
thread.start()
while True: time.sleep(100)
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'
Without the call to time.sleep, the main process is jumping out of the try...except block too early, so the KeyboardInterrupt is not caught. My first thought was to use thread.join, but that seems to block the main process (ignoring KeyboardInterrupt) until the thread is finished.
thread.daemon=True causes the thread to terminate when the main process ends.
To summarize the changes recommended in the comments, the following works well for me:
try:
thread = reqthread()
thread.start()
while thread.isAlive():
thread.join(1) # not sure if there is an appreciable cost to this.
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'
sys.exit()
Slight modification of ubuntu's solution.
Removing tread.daemon = True as suggested by Eric and replacing the sleeping loop by signal.pause():
import signal
try:
thread=reqthread()
thread.start()
signal.pause() # instead of: while True: time.sleep(100)
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'
My (hacky) solution is to monkey-patch Thread.join() like this:
def initThreadJoinHack():
import threading, thread
mainThread = threading.currentThread()
assert isinstance(mainThread, threading._MainThread)
mainThreadId = thread.get_ident()
join_orig = threading.Thread.join
def join_hacked(threadObj, timeout=None):
"""
:type threadObj: threading.Thread
:type timeout: float|None
"""
if timeout is None and thread.get_ident() == mainThreadId:
# This is a HACK for Thread.join() if we are in the main thread.
# In that case, a Thread.join(timeout=None) would hang and even not respond to signals
# because signals will get delivered to other threads and Python would forward
# them for delayed handling to the main thread which hangs.
# See CPython signalmodule.c.
# Currently the best solution I can think of:
while threadObj.isAlive():
join_orig(threadObj, timeout=0.1)
else:
# In all other cases, we can use the original.
join_orig(threadObj, timeout=timeout)
threading.Thread.join = join_hacked
Putting the try ... except in each thread and also a signal.pause() in true main() works for me.
Watch out for import lock though. I am guessing this is why Python doesn't solve ctrl-C by default.
Related
Was given a script I would reuse more or less, I need to be able to to both :
end the execution by itself
capture ctrl-c to exit on user action
I saw many clues to the second part on other answers/question of stackoverflow similar to :
try:
while True:
time.sleep(1)
except (KeyboardInterrupt, SystemExit):
pass
In my point of view I should run the execution functions (main) in
while mycondition:
try:
mainfunction()
except KeyboardInterrupt:
personalised_exit()
Why not (if I undersoud well as I am still a python noob), but why not a more declarative code with usage of signal modules ?
something might look then like
import signal
import sys
def signal_handler(sig, frame):
[...]
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
main()
signal.pause()
I am running a python script in the background using the command python script.py &. The script might look like this.
import time
def loop():
while True:
time.sleep(1)
if __name__=='__main__':
try:
loop()
except KeyboardInterrupt:
print("Terminated properly")
When it comes to terminating the script, I would like to do some cleanup before it is stopped (such as printing "Terminated properly"). If I run as a current process, this would be handled by the except statement after a keyboard interrupt.
Using the kill PID command means the cleanup is never executed. How can I stop a background process and execute some lines of code before it is terminated?
You can use signal module to catch any signals sent to your script via kill.
You setup a signal handler to catch the signal in question that would perform the cleanup.
import signal
import time
running = 0
def loop ():
global running
running = 1
while running:
try: time.sleep(0.25)
except KeyboardInterrupt: break
print "Ended nicely!"
def cleanup (signumber, stackframe):
global running
running = 0
signal.signal(signal.SIGABRT, cleanup)
signal.signal(signal.SIGTERM, cleanup)
signal.signal(signal.SIGQUIT, cleanup)
loop()
Use finally clause:
def loop():
while True:
time.sleep(1)
if __name__=='__main__':
try:
loop()
except KeyboardInterrupt:
print("Terminated properly")
finally:
print('executes always')
In a multi-threaded design, I want to do some clean steps when the program exits abnormally. The running thread should clean up the current task and then quit, rather than be killed immediately and leave some dirty data. I found that using threading module could not catch KeyInterrupt exception.
Here is my test code:
#!/usr/bin/env python3
from time import sleep
def do_sth():
print("I'm doing something...")
sleep(10)
if __name__ == "__main__":
do_sth()
Python will raise KeyInterrupt exception when I press CTRL-c
$ Python3 test.py
I'm doing something ...
^C
Traceback (most recent call last):
File "test.py", line 10, in <module>
do_sth ()
File "test.py", line 7, in do_sth
sleep (10)
KeyboardInterrupt
So I can catch this exception.
def do_sth ():
try:
print ("I'm doing something ...")
sleep (10)
except (KeyboardInterrupt, SystemExit):
print ("I'm doing some clean steps and exit.")
But when I use threading module, this exception is not raised at all.
#!/usr/bin/env python3
from time import sleep
import threading
def do_sth():
print("I'm doing something...")
sleep(10)
if __name__ == '__main__':
t = threading.Thread(target=do_sth)
t.start()
t.join()
result:
$ python3 test.py
I'm doing something...
^C
The running thread has been killed directly and no exception is raised.
How do I deal with this?
One way is to handle KeyboardInterrupt exceptions.
Another thing to do in such scenarios is to manage the state of your application across all threads.
One of the solutions is to add support for Signals in your code. It allows graceful handling of the shutting down of your process.
Here's one simple setup for that:
class SignalHandler:
continue_running = True
def __init__(self):
signal.signal(signal.SIGUSR2, self.signal_handler)
signal.signal(signal.SIGTERM, self.signal_handler)
signal.signal(signal.SIGINT, self.signal_handler)
logging.info("SignalHandler::Init")
def signal_handler(self, num, stack):
logging.warning('Received signal %d in %s' % (num, threading.currentThread()))
SignalHandler.continue_running = False
logging.warning("Time to SHUT DOWN ALL MODULES")
All threads would try and utilise the status from SignalHandler.continue_running to ensure that they all know when to stop.
If somebody tried to kill this python process by calling kill -2 [PID] for example - all threads will come to know about the need to shut down.
I have some threads running, and one of those threads contains an object that will be spawning subprocesses. I want one such subprocess to be able to kill the entire application. The aforementioned object will need to save some state when it receives this signal. Unfortunately I can't get the signal to be handled in the thread that causes the kill.
Here is some example code that attempts to replicate the situation.
parent.py: starts a thread. that thread runs some subprocesses, one of which will try to kill the parent process.
#!/usr/local/bin/python3
import subprocess, time, threading, random
def killer_func():
possible_cmds = [['echo', 'hello'],
['echo', 'world'],
['/work/turbulencetoo/tmp/killer.py']
]
random.shuffle(possible_cmds)
for cmd in possible_cmds:
try:
time.sleep(2)
subprocess.check_call(cmd)
time.sleep(2)
except KeyboardInterrupt:
print("Kill -2 caught properly!!")
print("Here I could properly save my state")
break
except Exception as e:
print("Unhandled Exception: {}".format(e))
else:
print("No Exception")
killer_thread = threading.Thread(target=killer_func)
killer_thread.start()
try:
while True:
killer_thread.join(4)
if not killer_thread.is_alive():
print("The killer thread has died")
break
else:
print("Killer thread still alive, try to join again.")
except KeyboardInterrupt:
print("Caught the kill -2 in the main thread :(")
print("Main program shutting down")
killer.py, a simple program that tries to kill its parent process with SIGINT:
#!/usr/local/bin/python3
import time, os, subprocess, sys
ppid = os.getppid()
# -2 specifies SIGINT, python handles this as a KeyboardInterrupt exception
cmd = ["kill", "-2", "{}".format(ppid)]
subprocess.check_call(cmd)
time.sleep(3)
sys.exit(0)
Here is some sample output from running the parent program:
$ ./parent.py
hello
Killer thread still alive, try to join again.
No Exception
Killer thread still alive, try to join again.
Caught the kill -2 in the main thread :(
Main program shutting down
No Exception
world
No Exception
I've tried using signal.signal() inside killer_func, but it doesn't work in a sub thread.
Is there a way to force the signal or exception to be handled by the function without the main thread being aware?
The main thread of your program will always be the one that receives the signal. The signal module documentation states this:
Some care must be taken if both signals and threads are used in the
same program. The fundamental thing to remember in using signals and
threads simultaneously is: always perform signal() operations in the
main thread of execution. Any thread can perform an alarm(),
getsignal(), pause(), setitimer() or getitimer(); only the main thread
can set a new signal handler, and the main thread will be the only one
to receive signals (this is enforced by the Python signal module, even
if the underlying thread implementation supports sending signals to
individual threads). This means that signals can’t be used as a means
of inter-thread communication. Use locks instead.
You'll need to refactor your program such that the main thread receiving the signal doesn't prevent you from saving state. The easiest way is use something like threading.Event() to tell the background thread that the program has been aborted, and let it clean up when it sees the event has been set:
import subprocess
import threading
import random
def killer_func(event):
possible_cmds = [['echo', 'hello'],
['echo', 'world'],
['/home/cycdev/killer.py']
]
random.shuffle(possible_cmds)
for cmd in possible_cmds:
subprocess.check_call(cmd)
event.wait(4)
if event.is_set():
print("Main thread got a signal. Time to clean up")
# save state here.
return
event = threading.Event()
killer_thread = threading.Thread(target=killer_func, args=(event,))
killer_thread.start()
try:
killer_thread.join()
except KeyboardInterrupt:
print("Caught the kill -2 in the main thread :)")
event.set()
killer_thread.join()
print("Main program shutting down")
Signals are always handled in the main thread. When you receive a signal, you don't know where it comes from. You can't say "handle it in the thread that spawned the signal-sending-process" because you don't know what signal-sending-process is.
The way to solve this is to use Condition Variables to notify all threads that a signal was received and that they have to shut down.
import threading
got_interrupt = False # global variable
def killer_func(cv):
...
with cv:
cv.wait(2)
interupted = got_interrupt # Read got_interrupt while holding the lock
if interrupted:
cleanup()
...
lock = threading.Lock()
notifier_cv = threading.Condition(lock)
killer_thread = threading.Thread(target=killer_func, args=(notifier_cv,))
killer_thread.start()
try:
...
except KeyboardInterrupt:
with cv:
got_interrupt = True
cv.notify_all()
I have my main application thread that spawns 2 threads and I catch SIGINT in my main thread to quit them nicely.
On linux, I'm using signal.pause() and it works perfectly.
What is the best way to implement signal.pause() on Windows?
My ugly solution is:
my_queue.get(True, averylongtime)
And put something in my_queue in my signal handler. Note that if I don't specify a timeout, SIGINT is not caught. But I wonder if there's a better solution.
Thank you
I use this:
#another:
while not self.quit:
# your code
# main
try:
# your code
except KeyboardInterrupt:
another.quit = True
time.sleep(5) # or wait for threading.enumerate() or similar
If I want it more robust, say, exit in presence of bugs too:
except KeyboardInterrupt:
another.quit = True
signal.alarm(5)
time.sleep(6)
A side effect to this is that every block where you except: or except Exception, e: (which is not something you should do anyway/much) you have to prepend except KeyboardInterrupt: raise so that the exception is not "eaten".
I use this for catching a ctrl-c on windows. In case I'm writing to a pipe or file or what have you.. I want to exit gracefully. Below is a toy example
import signal
import sys
def signal_handler(signal, frame):
print('Process Interrupted!\n\a')
sys.exit(0)
signal.signal(signal.SIGINT,signal_handler)
#Rest of your code