Develop Reference

Stop Python script with ctrl + c

I have a script which uses threads, but it is unable to catch Ctrl + C.
Here it is the sample code to reproduce this error:
import threading
import time
import signal
class DummyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._running = True
signal.signal(signal.SIGINT, self.stop)
signal.signal(signal.SIGTERM, self.stop)
def stop(self, signum=None, frame=None):
self._running = False
def run(self):
while self._running:
time.sleep(1)
print("Running")
if __name__ == "__main__":
try:
t = DummyThread()
t.start()
while True:
print("Main thread running")
time.sleep(0.5)
except KeyboardInterrupt:
print("This never gets printed")
t.stop()
finally:
print("Exit")
When I run python3 script.py it starts running, but it does not catch ctrl+c. I have googled it but I have not found a solution. I have to kill the script with SIGTERM, but I want DummyThread to stop gracefully.

class DummyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._running = True
signal.signal(signal.SIGINT, self.stop)
signal.signal(signal.SIGTERM, self.stop)
The program actually does not work as expected because of those last two lines and would work without them.
The reason is that, if you press Ctrl-C, the SIGINT signal is handled by the signal handler that is set up by signal.signal and self.stop is called. So the thread should actually stop.
But in the main thread, the while True loop is still running. Since the signal has already been handled, there will be no KeyboardInterrupt exception raised by the Python runtime. Therefore you never get to the except part.
if __name__ == "__main__":
try:
t = DummyThread()
t.start()
while True: # you are stuck in this loop
print("Main thread running")
time.sleep(0.5)
except KeyboardInterrupt: # this never happens
print("This never gets printed")
t.stop()
Only one signal handler should be set up to call the stop method. So there are two options to solve the problem:
Handle the signal implicitly by catching the KeyboardInterrupt exception. This is achieved by simply removing the two signal.signal(...) lines.
Set up an explicit signal handler (as you did by using signal.signal in DummyThread.__init__), but remove the while True: loop from the main thread and do not try to handle KeyboardInterrupt. Instead, just wait for the DummyThread to finish on its own by using its join method:
if __name__ == "__main__":
t = DummyThread()
t.start()
t.join()
print("Exit")

The main point is that you can't work with signals in any other Thread except the Main Thread. The Main Thread is the only one which can receive signals and handle them. I can offer the following solution, it is based on Event sync primitive.
According to Python documantation:
Signals and threads
Python signal handlers are always executed in the main Python thread, even if the signal was received in another thread. This means that signals can’t be used as a means of inter-thread communication. You can use the synchronization primitives from the threading module instead.
Besides, only the main thread is allowed to set a new signal handler.
from threading import Thread, Event
import time
class DummyThread(Thread):
def __init__(self, event: Event):
Thread.__init__(self)
self.stop_event = event
def run(self):
# we are monitoring the event in the Main Thread
while not self.stop_event.is_set():
time.sleep(1)
print("Running")
# only Main Thread can make the point reachable
print("I am done !")
if __name__ == "__main__":
try:
e = Event()
t = DummyThread(e)
t.start()
while True:
print("Main thread running")
time.sleep(0.5)
except KeyboardInterrupt:
e.set()
finally:
print("Exit")
Another possible choice is to use daemon Thread for such tasks like in your code example (when you just printing smth in the screen every second, rather than e.g. close database connection or some similar task). If main thread is stoped the daemon Thread will stop too.

As shown in your code, you used KeyboardInterrupt to call stop() function. See how Listener does the same task and stops the execution which was unable to catch from Ctrl + C. You dont have to kill the script with SIGTERM anymore
import threading
import time
import signal
import os
from pynput.keyboard import Key, Listener
class DummyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._running = True
signal.signal(signal.SIGINT, self.stop)
signal.signal(signal.SIGTERM, self.stop)
def stop(self, signum=None, frame=None):
self._running = False
print ("Bye Bye . .")
os._exit(1)
def run(self):
while self._running:
time.sleep(1)
print("Running")
if __name__ == "__main__":
t = DummyThread()
def func2():
try:
t.start()
while True:
print("Main thread running")
time.sleep(0.5)
except KeyboardInterrupt:
print("No need for this")
t.stop()
finally:
print("Exit")
def func1():
with Listener(on_press = t.stop) as listener :
listener.join()
threading.Thread(target=func1).start()
threading.Thread(target=func2).start()

Simple multithreaded Server Client Program

I have a multithreaded server and client programs for a simple game. Whenever a client force exits the game, I try to catch the exception with a try catch "except BrokenPipeError" and inform the other players.
I also want to end the exited client thread; however, I take input such as this:
while True:
client = serverSocket.accept()
t = ServerThread(client)
t.start()
I tried to use a threading event with stop() function; however, I believe I can not use a .join statement to exit the thread because of the way I take input. How should I end the force exited client. I know that multiprocessing library has a terminate function but I am also required to use the threading library. I tried os_exit(1) but I believe this command kills the entire process. What is the standard exit process for programs such as this?

First of all join() do nothing else but waits for thread to stop.
Thread stops when it reaches end of threaded subroutine. For example
class ServerThread(threading.Thread):
def __init__(self,client,name):
super().__init__(self)
self.client = client
self.name = name
def inform(self,msg):
print("{}: got message {}".format( self.name, msg ))
self.client[0].send(msg)
def run(self):
while True:
try:
self.client[0].recv(1024)
except BrokenPipeError: #client exits
# do stuff
break # -> ends loop
return # -> thread exits, join returns
If you want to inform other clients that someone leaves, i would make another monitoring thread
class Monitoring(threading.Thread):
def __init__(self):
super().__init__(self,daemon=True) # daemon means thread stops when main thread do
self.clients=[]
def add_client(self,client):
self.clients.append(client)
def inform_client_leaves(self,client_leaved):
for client in self.clients:
if client.is_alive():
client.inform("Client '{}' leaves".format(client_leaved.name))
def run(self):
while True:
for thread in list(self.threads):
if not thread.is_alive(): # client exited
self.threads.remove(thread)
self.inform_client_exits(thread)
time.sleep(1)
So the initial code would looks like
mon = Monitoring()
mon.start()
while True:
client = serverSocket.accept()
t = ServerThread(client,"Client1")
t.start()
mon.add_client(t)

Python terminated does not return to command line

UPDATE: I found the bug. The serial port openned in listner should be closed when stopping thread. I was not doing that.
I have a simple python program where in its main, it starts 2 threads by calling their start() function. When the user presses a specific key, the program goes and clears the condition,self.is_running=false, that those 2 threads check everytime, so they would terminate when it's false:
def run(self):
...
while self.is_running:
...
return
Problem is I see threads terminate, but the program never returns to command prompt, and there is no combination of keys, ctrl+x etc. that would make it return to command prompt. I have to manually close the console window and restart a new one if I want to use the terminal console. I even used sys.exit() after threads are terminated but still doesn't make a difference.
Do you know where am I going wrong?
Here is the minimal code:
main.py:
def main():
listner_thread = None
try:
listner_thread = listner.listner()
listner_thread.start();
except:
Message = "Listener Thread Failed To Start"
print (Message)
return
ui_thread=None
try:
ui_thread = ui.ui(listner_thread)
ui_thread.start();
except:
Message = "UI Thread Failed To Start"
print (Message)
return
return
if __name__ == '__main__':
main()
listener.py and ui.py threads:
class listner(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.port=serial.serial('COM11',115200)
def run(self):
...
while self.is_running:
self.port.read(1)
return
Where threads are instructed to be terminated are in ui thread:
ui.py
class ui(threading.Thread):
def __init__(self,listner):
threading.Thread.__init__(self)
self.listner=listner
def run(self):
Message = ('\n Starting Thread (' + self.name + ')\n')
print Message
self.running = True
while self.running:
user = raw_input('\n Q- Quit\n')
if user.isalpha() and user.upper() == 'Q':
self.listner.is_running= False
self.running = False
return

You set running to False on the UI thread instance, but never mutate running on the listener thread instance. Your listener thread will thus never exit.
Try lifting the exit condition to a global, like should_exit = False. The UI thread can then set this True, and, since the global scope is shared across threads, both should see the change.

In listener I had a serial port open. It was not being closed (port.close()) upon thread terminatinon with setting 'is_running=False' , when I added port.close() it exited the thread properly.

How to create global error handler in a multi-threaded python applcation

I am developing a multi-threaded application in python. I have following scenario.
There are 2-3 producer threads which communicate with DB and get some data in large chunks and fill them up in a queue
There is an intermediate worker which breaks large chunks fetched by producer threads into smaller ones and fill them up in another queue.
There are 5 consumer threads which consume queue created by intermediate worker thread.
objects of data sources are accessed by producer threads through their API. these data sources are completely separate. So these producer understands only presence or absence of data which is supposed to be given out by data source object.
I create threads of these three types and i make main thread wait for completion of these threads by calling join() on them.
Now for such a setup I want a common error handler which senses failure of any thread, any exception and decides what to do. For e.g if I press ctrl+c after I start my application, main thread dies but producer, consumer threads continue to run. I would like that once ctrl+c is pressed entire application should shut down. Similarly if some DB error occurs in data source module, then producer thread should get notified of that.
This is what I have done so far:
I have created a class ThreadManager, it's object is passed to all threads. I have written an error handler method and passed it to sys.excepthook. This handler should catch exceptions, error and then it should call methods of ThreadManager class to control the running threads. Here is snippet:
class Producer(threading.Thread):
....
def produce():
data = dataSource.getData()
class DataSource:
....
def getData():
raise Exception("critical")
def customHandler(exceptionType, value, stackTrace):
print "In custom handler"
sys.excepthook = customHandler
Now when a thread of producer class calls getData() of DataSource class, exception is thrown. But this exception is never caught by my customHandler method.
What am I missing? Also in such scenario what other strategy can I apply? Please help. Thank you for having enough patience to read all this :)

What you need is a decorator. In essence you are modifying your original function and putting in inside a try-except:
def exception_decorator(func):
def _function(*args):
try:
result = func(*args)
except:
print('*** ESC default handler ***')
os._exit(1)
return result
return _function
If your thread function is called myfunc, then you add the following line above your function definition
#exception_decorator
def myfunc():
pass;

Can't you just catch "KeyboardInterrupt" when pressing Ctrl+C and do:
for thread in threading.enumerate():
thread._Thread__stop()
thread._Thread__delete()
while len(threading.enumerate()) > 1:
time.sleep(1)
os._exit(0)
and have a flag in each threaded class which is self.alive
you could theoretically call thread.alive = False and have it stop gracefully?
for thread in threading.enumerate():
thread.alive = False
time.sleep(5) # Grace period
thread._Thread__stop()
thread._Thread__delete()
while len(threading.enumerate()) > 1:
time.sleep(1)
os._exit(0)
example:
import os
from threading import *
from time import sleep
class worker(Thread):
def __init__(self):
self.alive = True
Thread.__init__(self)
self.start()
def run(self):
while self.alive:
sleep(0.1)
runner = worker()
try:
raw_input('Press ctrl+c!')
except:
pass
for thread in enumerate():
thread.alive = False
sleep(1)
try:
thread._Thread__stop()
thread._Thread__delete()
except:
pass
# There will always be 1 thread alive and that's the __main__ thread.
while len(enumerate()) > 1:
sleep(1)
os._exit(0)
Try going about it by changing the internal system exception handler?
import sys
origExcepthook = sys.excepthook
def uberexcept(exctype, value, traceback):
if exctype == KeyboardInterrupt:
print "Gracefully shutting down all the threads"
# enumerate() thingie here.
else:
origExcepthook(exctype, value, traceback)
sys.excepthook = uberexcept

How to properly handle and retain system shutdown (and SIGTERM) in order to finish its job in Python?

Basic need : I've a Python daemon that's calling another program through os.system. My wish is to be able to properly to handle system shutdown or SIGTERM in order to let the called program return and then exiting.
What I've already tried: I've tried an approach using signal :
import signal, time
def handler(signum = None, frame = None):
print 'Signal handler called with signal', signum
time.sleep(3)
#here check if process is done
print 'Wait done'
signal.signal(signal.SIGTERM , handler)
while True:
time.sleep(6)
The usage of time.sleep doesn't seems to work and the second print is never called.
I've read few words about atexit.register(handler) instead of signal.signal(signal.SIGTERM, handler) but nothing is called on kill.

Your code does almost work, except you forgot to exit after cleaning up.
We often need to catch various other signals such as INT, HUP and QUIT, but not so much with daemons.
import sys, signal, time
def handler(signum = None, frame = None):
print 'Signal handler called with signal', signum
time.sleep(1) #here check if process is done
print 'Wait done'
sys.exit(0)
for sig in [signal.SIGTERM, signal.SIGINT, signal.SIGHUP, signal.SIGQUIT]:
signal.signal(sig, handler)
while True:
time.sleep(6)
On many systems, ordinary processes don't have much time to clean up during shutdown. To be safe, you could write an init.d script to stop your daemon and wait for it.