Develop Reference

Python Paramiko Ctrl Z is not working for threading [duplicate]

I am testing Python threading with the following script:
import threading
class FirstThread (threading.Thread):
def run (self):
while True:
print 'first'
class SecondThread (threading.Thread):
def run (self):
while True:
print 'second'
FirstThread().start()
SecondThread().start()
This is running in Python 2.7 on Kubuntu 11.10. Ctrl+C will not kill it. I also tried adding a handler for system signals, but that did not help:
import signal
import sys
def signal_handler(signal, frame):
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
To kill the process I am killing it by PID after sending the program to the background with Ctrl+Z, which isn't being ignored. Why is Ctrl+C being ignored so persistently? How can I resolve this?

Ctrl+C terminates the main thread, but because your threads aren't in daemon mode, they keep running, and that keeps the process alive. We can make them daemons:
f = FirstThread()
f.daemon = True
f.start()
s = SecondThread()
s.daemon = True
s.start()
But then there's another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Now it will keep print 'first' and 'second' until you hit Ctrl+C.
Edit: as commenters have pointed out, the daemon threads may not get a chance to clean up things like temporary files. If you need that, then catch the KeyboardInterrupt on the main thread and have it co-ordinate cleanup and shutdown. But in many cases, letting daemon threads die suddenly is probably good enough.

KeyboardInterrupt and signals are only seen by the process (ie the main thread)... Have a look at Ctrl-c i.e. KeyboardInterrupt to kill threads in python

I think it's best to call join() on your threads when you expect them to die. I've taken the liberty to make the change your loops to end (you can add whatever cleanup needs are required to there as well). The variable die is checked on each pass and when it's True, the program exits.
import threading
import time
class MyThread (threading.Thread):
die = False
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run (self):
while not self.die:
time.sleep(1)
print (self.name)
def join(self):
self.die = True
super().join()
if __name__ == '__main__':
f = MyThread('first')
f.start()
s = MyThread('second')
s.start()
try:
while True:
time.sleep(2)
except KeyboardInterrupt:
f.join()
s.join()

An improved version of #Thomas K's answer:
Defining an assistant function is_any_thread_alive() according to this gist, which can terminates the main() automatically.
Example codes:
import threading
def job1():
...
def job2():
...
def is_any_thread_alive(threads):
return True in [t.is_alive() for t in threads]
if __name__ == "__main__":
...
t1 = threading.Thread(target=job1,daemon=True)
t2 = threading.Thread(target=job2,daemon=True)
t1.start()
t2.start()
while is_any_thread_alive([t1,t2]):
time.sleep(0)

One simple 'gotcha' to beware of, are you sure CAPS LOCK isn't on?
I was running a Python script in the Thonny IDE on a Pi4. With CAPS LOCK on, Ctrl+Shift+C is passed to the keyboard buffer, not Ctrl+C.

How to catch Exceptions from thread

I've got this piece of code that starts a thread. Then it waits for a few seconds and then it checks if it got a Event. If it does have an event, the thread is 'canceled'. Else a Exception is thrown.
I want to know how to catch this Exception because I've searched for a long time and did not find a clear answer.
import threading
import time
def thread(args1, stop_event):
print "starting thread"
stop_event.wait(10)
if not stop_event.is_set():
raise Exception("signal!")
pass
try:
t_stop = threading.Event()
t = threading.Thread(target=thread, args=(1, t_stop))
t.start()
time.sleep(11)
#normally this should not be executed
print "stopping thread!"
t_stop.set()
except Exception as e:
print "action took to long, bye!"
First I've tried this concept with a Python signal but when performing more than 1 signal.alarm, it just got stuck for no reason at all (prob. a bug).
EDIT:
I don't want to extend the already existing class, I want to work with the native defined class.
Also I do not want to loop continuously to check if exceptions occured. I want the thread to pass the exception to its parent method. Therefore the time.sleep(11) action in my code

Created from my comments on your question.
Try something like this.
import sys
import threading
import time
import Queue
def thread(args1, stop_event, queue_obj):
print "starting thread"
stop_event.wait(10)
if not stop_event.is_set():
try:
raise Exception("signal!")
except Exception:
queue_obj.put(sys.exc_info())
pass
try:
queue_obj = Queue.Queue()
t_stop = threading.Event()
t = threading.Thread(target=thread, args=(1, t_stop, queue_obj))
t.start()
time.sleep(11)
# normally this should not be executed
print "stopping thread!"
t_stop.set()
try:
exc = queue_obj.get(block=False)
except Queue.Empty:
pass
else:
exc_type, exc_obj, exc_trace = exc
print exc_obj
except Exception as e:
print "action took to long, bye!"
When run, the Exception "signal!" is raised, and is printed by print exc_obj.

Python: Catch Ctrl-C command. Prompt "really want to quit (y/n)", resume execution if no

I have a program that may have a lengthy execution. In the main module I have the following:
import signal
def run_program()
...time consuming execution...
def Exit_gracefully(signal, frame):
... log exiting information ...
... close any open files ...
sys.exit(0)
if __name__ == '__main__':
signal.signal(signal.SIGINT, Exit_gracefully)
run_program()
This works fine, but I'd like the possibility to pause execution upon catching SIGINT, prompting the user if they would really like to quit, and resuming where I left off in run_program() if they decide they don't want to quit.
The only way I can think of doing this is running the program in a separate thread, keeping the main thread waiting on it and ready to catch SIGINT. If the user wants to quit the main thread can do cleanup and kill the child thread.
Is there a simpler way?

The python signal handlers do not seem to be real signal handlers; that is they happen after the fact, in the normal flow and after the C handler has already returned. Thus you'd try to put your quit logic within the signal handler. As the signal handler runs in the main thread, it will block execution there too.
Something like this seems to work nicely.
import signal
import time
import sys
def run_program():
while True:
time.sleep(1)
print("a")
def exit_gracefully(signum, frame):
# restore the original signal handler as otherwise evil things will happen
# in raw_input when CTRL+C is pressed, and our signal handler is not re-entrant
signal.signal(signal.SIGINT, original_sigint)
try:
if raw_input("\nReally quit? (y/n)> ").lower().startswith('y'):
sys.exit(1)
except KeyboardInterrupt:
print("Ok ok, quitting")
sys.exit(1)
# restore the exit gracefully handler here
signal.signal(signal.SIGINT, exit_gracefully)
if __name__ == '__main__':
# store the original SIGINT handler
original_sigint = signal.getsignal(signal.SIGINT)
signal.signal(signal.SIGINT, exit_gracefully)
run_program()
The code restores the original signal handler for the duration of raw_input; raw_input itself is not re-entrable, and re-entering it
will lead to RuntimeError: can't re-enter readline being raised from time.sleep which is something we don't want as it is harder to catch than KeyboardInterrupt. Rather, we let 2 consecutive Ctrl-C's to raise KeyboardInterrupt.

from https://gist.github.com/rtfpessoa/e3b1fe0bbfcd8ac853bf
#!/usr/bin/env python
import signal
import sys
def signal_handler(signal, frame):
# your code here
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
Bye!

when procedure end then do something
suppose you just want to the procedure will do something after the task end
import time
class TestTask:
def __init__(self, msg: str):
self.msg = msg
def __enter__(self):
print(f'Task Start!:{self.msg}')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print('Task End!')
#staticmethod
def do_something():
try:
time.sleep(5)
except:
pass
with TestTask('Hello World') as task:
task.do_something()
when the process leaves with that will run __exit__ even with KeyboardInterrupt happen that are same.
if you don't like to see the error, add try ... except ...
#staticmethod
def do_something():
try:
time.sleep(5)
except:
pass
pause, continue, reset, and etc.
I don't have a perfect solution, but it may be useful to you.
It's means divided your process to many subprocesses and save it that finished.it will not be executed again since you find it already done.
import time
from enum import Enum
class Action(Enum):
EXIT = 0
CONTINUE = 1
RESET = 2
class TestTask:
def __init__(self, msg: str):
self.msg = msg
def __enter__(self):
print(f'Task Start!:{self.msg}')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print('Task End!')
def do_something(self):
tuple_job = (self._foo, self._bar) # implement by yourself
list_job_state = [0] * len(tuple_job)
dict_keep = {} # If there is a need to communicate between jobs, and you don’t want to use class members, you can use this method.
while 1:
try:
for idx, cur_process in enumerate(tuple_job):
if not list_job_state[idx]:
cur_process(dict_keep)
list_job_state[idx] = True
if all(list_job_state):
print('100%')
break
except KeyboardInterrupt:
print('KeyboardInterrupt. input action:')
msg = '\n\t'.join([f"{action + ':':<10}{str(act_number)}" for act_number, action in
enumerate([name for name in vars(Action) if not name.startswith('_')])
])
case = Action(int(input(f'\t{msg}\n:')))
if case == Action.EXIT:
break
if case == Action.RESET:
list_job_state = [0] * len(tuple_job)
#staticmethod
def _foo(keep_dict: dict) -> bool: # implement by yourself
time.sleep(2)
print('1%')
print('2%')
print('...')
print('60%')
keep_dict['status_1'] = 'status_1'
return True
#staticmethod
def _bar(keep_dict: dict) -> bool: # implement by yourself
time.sleep(2)
print('61%')
print(keep_dict.get('status_1'))
print('...')
print('99%')
return True
with TestTask('Hello World') as task:
task.do_something()
console
input action number:2
Task Start!:Hello World
1%
2%
...
60%
KeyboardInterrupt. input action:
EXIT: 0
CONTINUE: 1
RESET: 2
:1
61%
status_1
...
99%
100%
Task End!

threading ignores KeyboardInterrupt exception

I'm running this simple code:
import threading, time
class reqthread(threading.Thread):
def run(self):
for i in range(0, 10):
time.sleep(1)
print('.')
try:
thread = reqthread()
thread.start()
except (KeyboardInterrupt, SystemExit):
print('\n! Received keyboard interrupt, quitting threads.\n')
But when I run it, it prints
$ python prova.py
.
.
^C.
.
.
.
.
.
.
.
Exception KeyboardInterrupt in <module 'threading' from '/usr/lib/python2.6/threading.pyc'> ignored
In fact python thread ignore my Ctrl+C keyboard interrupt and doesn't print Received Keyboard Interrupt. Why? What is wrong with this code?

Try
try:
thread=reqthread()
thread.daemon=True
thread.start()
while True: time.sleep(100)
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'
Without the call to time.sleep, the main process is jumping out of the try...except block too early, so the KeyboardInterrupt is not caught. My first thought was to use thread.join, but that seems to block the main process (ignoring KeyboardInterrupt) until the thread is finished.
thread.daemon=True causes the thread to terminate when the main process ends.

To summarize the changes recommended in the comments, the following works well for me:
try:
thread = reqthread()
thread.start()
while thread.isAlive():
thread.join(1) # not sure if there is an appreciable cost to this.
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'
sys.exit()

Slight modification of ubuntu's solution.
Removing tread.daemon = True as suggested by Eric and replacing the sleeping loop by signal.pause():
import signal
try:
thread=reqthread()
thread.start()
signal.pause() # instead of: while True: time.sleep(100)
except (KeyboardInterrupt, SystemExit):
print '\n! Received keyboard interrupt, quitting threads.\n'

My (hacky) solution is to monkey-patch Thread.join() like this:
def initThreadJoinHack():
import threading, thread
mainThread = threading.currentThread()
assert isinstance(mainThread, threading._MainThread)
mainThreadId = thread.get_ident()
join_orig = threading.Thread.join
def join_hacked(threadObj, timeout=None):
"""
:type threadObj: threading.Thread
:type timeout: float|None
"""
if timeout is None and thread.get_ident() == mainThreadId:
# This is a HACK for Thread.join() if we are in the main thread.
# In that case, a Thread.join(timeout=None) would hang and even not respond to signals
# because signals will get delivered to other threads and Python would forward
# them for delayed handling to the main thread which hangs.
# See CPython signalmodule.c.
# Currently the best solution I can think of:
while threadObj.isAlive():
join_orig(threadObj, timeout=0.1)
else:
# In all other cases, we can use the original.
join_orig(threadObj, timeout=timeout)
threading.Thread.join = join_hacked

Putting the try ... except in each thread and also a signal.pause() in true main() works for me.
Watch out for import lock though. I am guessing this is why Python doesn't solve ctrl-C by default.

How do I capture SIGINT in Python?

I'm working on a python script that starts several processes and database connections. Every now and then I want to kill the script with a Ctrl+C signal, and I'd like to do some cleanup.
In Perl I'd do this:
$SIG{'INT'} = 'exit_gracefully';
sub exit_gracefully {
print "Caught ^C \n";
exit (0);
}
How do I do the analogue of this in Python?

Register your handler with signal.signal like this:
#!/usr/bin/env python
import signal
import sys
def signal_handler(sig, frame):
print('You pressed Ctrl+C!')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Press Ctrl+C')
signal.pause()
Code adapted from here.
More documentation on signal can be found here.
 

You can treat it like an exception (KeyboardInterrupt), like any other. Make a new file and run it from your shell with the following contents to see what I mean:
import time, sys
x = 1
while True:
try:
print x
time.sleep(.3)
x += 1
except KeyboardInterrupt:
print "Bye"
sys.exit()

And as a context manager:
import signal
class GracefulInterruptHandler(object):
def __init__(self, sig=signal.SIGINT):
self.sig = sig
def __enter__(self):
self.interrupted = False
self.released = False
self.original_handler = signal.getsignal(self.sig)
def handler(signum, frame):
self.release()
self.interrupted = True
signal.signal(self.sig, handler)
return self
def __exit__(self, type, value, tb):
self.release()
def release(self):
if self.released:
return False
signal.signal(self.sig, self.original_handler)
self.released = True
return True
To use:
with GracefulInterruptHandler() as h:
for i in xrange(1000):
print "..."
time.sleep(1)
if h.interrupted:
print "interrupted!"
time.sleep(2)
break
Nested handlers:
with GracefulInterruptHandler() as h1:
while True:
print "(1)..."
time.sleep(1)
with GracefulInterruptHandler() as h2:
while True:
print "\t(2)..."
time.sleep(1)
if h2.interrupted:
print "\t(2) interrupted!"
time.sleep(2)
break
if h1.interrupted:
print "(1) interrupted!"
time.sleep(2)
break
From here: https://gist.github.com/2907502

You can handle CTRL+C by catching the KeyboardInterrupt exception. You can implement any clean-up code in the exception handler.

From Python's documentation:
import signal
import time
def handler(signum, frame):
print 'Here you go'
signal.signal(signal.SIGINT, handler)
time.sleep(10) # Press Ctrl+c here

Yet Another Snippet
Referred main as the main function and exit_gracefully as the Ctrl+C handler
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
pass
finally:
exit_gracefully()

I adapted the code from #udi to support multiple signals (nothing fancy) :
class GracefulInterruptHandler(object):
def __init__(self, signals=(signal.SIGINT, signal.SIGTERM)):
self.signals = signals
self.original_handlers = {}
def __enter__(self):
self.interrupted = False
self.released = False
for sig in self.signals:
self.original_handlers[sig] = signal.getsignal(sig)
signal.signal(sig, self.handler)
return self
def handler(self, signum, frame):
self.release()
self.interrupted = True
def __exit__(self, type, value, tb):
self.release()
def release(self):
if self.released:
return False
for sig in self.signals:
signal.signal(sig, self.original_handlers[sig])
self.released = True
return True
This code support the keyboard interrupt call (SIGINT) and the SIGTERM (kill <process>)

In contrast to Matt J his answer, I use a simple object. This gives me the possibily to parse this handler to all the threads that needs to be stopped securlery.
class SIGINT_handler():
def __init__(self):
self.SIGINT = False
def signal_handler(self, signal, frame):
print('You pressed Ctrl+C!')
self.SIGINT = True
handler = SIGINT_handler()
signal.signal(signal.SIGINT, handler.signal_handler)
Elsewhere
while True:
# task
if handler.SIGINT:
break

If you want to ensure that your cleanup process finishes I would add on to Matt J's answer by using a SIG_IGN so that further SIGINT are ignored which will prevent your cleanup from being interrupted.
import signal
import sys
def signal_handler(signum, frame):
signal.signal(signum, signal.SIG_IGN) # ignore additional signals
cleanup() # give your process a chance to clean up
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler) # register the signal with the signal handler first
do_stuff()

You can use the functions in Python's built-in signal module to set up signal handlers in python. Specifically the signal.signal(signalnum, handler) function is used to register the handler function for signal signalnum.

thanks for existing answers, but added signal.getsignal()
import signal
# store default handler of signal.SIGINT
default_handler = signal.getsignal(signal.SIGINT)
catch_count = 0
def handler(signum, frame):
global default_handler, catch_count
catch_count += 1
print ('wait:', catch_count)
if catch_count > 3:
# recover handler for signal.SIGINT
signal.signal(signal.SIGINT, default_handler)
print('expecting KeyboardInterrupt')
signal.signal(signal.SIGINT, handler)
print('Press Ctrl+c here')
while True:
pass

Personally, I couldn't use try/except KeyboardInterrupt because I was using standard socket (IPC) mode which is blocking. So the SIGINT was cueued, but came only after receiving data on the socket.
Setting a signal handler behaves the same.
On the other hand, this only works for an actual terminal. Other starting environments might not accept Ctrl+C, or pre-handle the signal.
Also, there are "Exceptions" and "BaseExceptions" in Python, which differ in the sense that interpreter needs to exit cleanly itself, so some exceptions have a higher priority than others (Exceptions is derived from BaseException)