If I have a thread in an infinite loop, is there a way to terminate it when the main program ends (for example, when I press Ctrl+C)?
If you make your worker threads daemon threads, they will die when all your non-daemon threads (e.g. the main thread) have exited.
http://docs.python.org/library/threading.html#threading.Thread.daemon
Check this question. The correct answer has great explanation on how to terminate threads the right way:
Is there any way to kill a Thread in Python?
To make the thread stop on Keyboard Interrupt signal (ctrl+c) you can catch the exception "KeyboardInterrupt" and cleanup before exiting. Like this:
try:
start_thread()
except (KeyboardInterrupt, SystemExit):
cleanup_stop_thread()
sys.exit()
This way you can control what to do whenever the program is abruptly terminated.
You can also use the built-in signal module that lets you setup signal handlers (in your specific case the SIGINT signal): http://docs.python.org/library/signal.html
Try to enable the sub-thread as daemon-thread.
Recommended:
from threading import Thread
t = Thread(target=desired_method)
t.daemon = True # Dies when main thread (only non-daemon thread) exits.
t.start()
Inline:
t = Thread(target=desired_method, daemon=True).start()
Old API:
t.setDaemon(True)
t.start()
When your main thread terminates (e.g. Ctrl+C keystrokes), other threads will also be killed by the instructions above.
Use the atexit module of Python's standard library to register "termination" functions that get called (on the main thread) on any reasonably "clean" termination of the main thread, including an uncaught exception such as KeyboardInterrupt. Such termination functions may (though inevitably in the main thread!) call any stop function you require; together with the possibility of setting a thread as daemon, that gives you the tools to properly design the system functionality you need.
If you spawn a Thread like so - myThread = Thread(target = function) - and then do myThread.start(); myThread.join(). When CTRL-C is initiated, the main thread doesn't exit because it is waiting on that blocking myThread.join() call. To fix this, simply put in a timeout on the .join() call. The timeout can be as long as you wish. If you want it to wait indefinitely, just put in a really long timeout, like 99999. It's also good practice to do myThread.daemon = True so all the threads exit when the main thread(non-daemon) exits.
Daemon threads are killed ungracefully so any finalizer instructions are not executed.
A possible solution is to check is main thread is alive instead of infinite loop.
E.g. for Python 3:
while threading.main_thread().isAlive():
do.you.subthread.thing()
gracefully.close.the.thread()
See Check if the Main Thread is still alive from another thread.
Related
So I have the main thread spawning worker threads that have infinite loops to deal with system monitoring. So it looks something like this
while True:
Check_server_status( host )
wait( *minutes* )
This worker thread should run forever because we need to constantly monitor the servers and each thread is currently monitoring one machine but I may scale it so each thread has a list of servers to check on. Also, each thread is writing to a csv file the information that it is finding.
The main thread just calls this thread for each host that it finds in a list.
hosts = [a,b]
threads = []
for host in hosts:
t = worker( host )
t.daemon = True
t.start()
threads.append( t )
I am trying to make this script exit cleanly on ctrl-c. So I want to make sure that the files are closed and that the threads exits. Is there any good way to handle this?
Thanks in advance for the help!
Well, for starters, daemon threads terminated by the main thread ending aren't cleaned up properly, so avoid that.
And KeyboardInterrupt isn't necessarily delivered to the main thread, or to any specific thread, so it's probably not best to rely on Python's default handler for SIGINT here, but instead write your own handler to replace it.
Probably the simplest approach is to have all the threads loop on a shared threading.Event object, looping based on a .wait(*seconds*) call (so your per loop sleep is folded into the while condition, and you can still exit immediately, anytime during the wait).
So you might do:
import threading
shouldexit = threading.Event()
Then your worker functions would be of the form:
def worker(host):
while not shouldexit.wait(minutes * 60):
Check_server_status( host )
They'd all be launched without setting them as daemons, and you'd register a SIGINT handler like this one:
import signal
def ctrlchandler(signum, frame):
print('User triggered quit')
shouldexit.set()
# Set the signal handlers for the Ctrl-C related events
if hasattr(signal, 'CTRL_C_EVENT'):
# Only on Windows
signal.signal(signal.CTRL_C_EVENT, ctrlchandler)
signal.signal(signal.SIGINT, ctrlchandler)
Ideally, there would be no living main thread to kill off in this case (all threads would be looping on the shared Event, with the main thread exiting after launching the workers); if that's not the case, you'd need to figure out some way of terminating the main thread.
One approach might be to store off the default handler for SIGINT when you register your replacement handler, have your handler join all the worker threads after setting the Event, then explicitly invoke the original SIGINT handler so KeyboardInterrupt fires in a surviving thread as normal and cleans up the now workerless main thread.
this is done in python 2.7.12
serialHelper is a class module arround python serial and this code does work nicely
#!/usr/bin/env python
import threading
from time import sleep
import serialHelper
sh = serialHelper.SerialHelper()
def serialGetter():
h = 0
while True:
h = h + 1
s_resp = sh.getResponse()
print ('response ' + s_resp)
sleep(3)
if __name__ == '__main__':
try:
t = threading.Thread(target=sh.serialReader)
t.setDaemon(True)
t.start()
serialGetter()
#tSR = threading.Thread(target=serialGetter)
#tSR.setDaemon(True)
#tSR.start()
except Exception as e:
print (e)
however the attemp to run serialGetter as thread as remarked it just dies.
Any reason why that function can not run as thread ?
Quoting from the Python documentation:
The entire Python program exits when no alive non-daemon threads are left.
So if you setDaemon(True) every new thread and then exit the main thread (by falling off the end of the script), the whole program will exit immediately. This kills all of the threads. Either don't use setDaemon(True), or don't exit the main thread without first calling join() on all of the threads you want to wait for.
Stepping back for a moment, it may help to think about the intended use case of a daemon thread. In Unix, a daemon is a process that runs in the background and (typically) serves requests or performs operations, either on behalf of remote clients over the network or local processes. The same basic idea applies to daemon threads:
You launch the daemon thread with some kind of work queue.
When you need some work done on the thread, you hand it a work object.
When you want the result of that work, you use an event or a future to wait for it to complete.
After requesting some work, you always eventually wait for it to complete, or perhaps cancel it (if your worker protocol supports cancellation).
You don't have to clean up the daemon thread at program termination. It just quietly goes away when there are no other threads left.
The problem is step (4). If you forget about some work object, and exit the app without waiting for it to complete, the work may get interrupted. Daemon threads don't gracefully shut down, so you could leave the outside world in an inconsistent state (e.g. an incomplete database transaction, a file that never got closed, etc.). It's often better to use a regular thread, and replace step (5) with an explicit "Finish up your work and shut down" work object that the main thread hands to the worker thread before exiting. The worker thread then recognizes this object, stops waiting on the work queue, and terminates itself once it's no longer doing anything else. This is slightly more up-front work, but is much safer in the event that a work object is inadvertently abandoned.
Because of all of the above, I recommend not using daemon threads unless you have a strong reason for them.
I would like to know why doesn't python2.7 drop blocking operations when ctrl+c is pressed, I am unable to kill my threaded application, there are several socket waits, semaphore waits and so on. In python3 ctrl+c dropped every blocking operation and garbage-collected everything, released all the sockets and whatsoever ... Is there (I am convinced there is, I just yet don't know how) a way to acomplish this? Signal handle? Thanks guys
I guess you are launching the threads and then the main thread is waiting to join them on termination.
You should catch the exception generated by Ctrl-C in the main thread, in order to signal the spawned threads to terminate (changing a flag in each thread, for instance). In this manner, all the children thread will terminate and the main thread will complete the join call, reaching the bottom of your main.
Is there a way to "pause" the main python thread of an application perminantly?
I have some code that fires off two threads
class start():
def __init__(self):
Thread1= functions.threads.Thread1()
Thread1.setDaemon(True)
Thread1.start()
Thread2= functions.threads.Thread2()
Thread2.setDaemon(True)
Thread2.start()
#Stop thread here
At the moment, when the program gets to the end of that function it exits (There is nothing else for the main thread to do after that), killing the threads which run infinately (Looping). How do I stop the main process from exiting? I can do it with a while True: None loop but that uses a lot of CPU and there's probably a better way.
If you don't do setDaemon(True) on the threads, the process will keep running as long as the threads run for.
The daemon flag indicates that the interpreter needn't wait for a thread. It will exit when only daemon threads are left.
Use join:
Thread1.join()
Thread2.join()
Also note that setDaemon is the old API.
Thread1.daemon = True
is the preferred way now.
The whole point of daemon threads is to not prevent the application from being terminated if any of them is still running. You obviously want your threads to keep the application process alive, so don't make them daemons.
I'm a little confused about what setting a thread to be a daemon means.
The documentation says this:
A thread can be flagged as a “daemon
thread”. The significance of this flag
is that the entire Python program
exits when only daemon threads are
left. The initial value is inherited
from the creating thread. The flag can
be set through the daemon property.
I'm not sure what makes this different from a normal thread.
Is this saying that this program won't ever finish?
def threadfunc():
while True:
time.sleep(1)
threading.Thread(target=threadfunc).start()
Even though the main thread finishes it's execution. While will finish immediately?
def threadfunc():
while True:
time.sleep(1)
th = threading.Thread(target=threadfunc)
th.daemon = True
th.start()
I ask because I have a situation where in my main thread I'm calling sys.exit(), and the process just hangs and my other threads are running as I can see the log.
Does this have anything to do with sys.exit() being called with threads alive?
Is this saying that this program won't ever finish?
Yes, that program won't finish, just try it out.
I ask because I have a situation where
in my main thread I'm calling
sys.exit(), and the process just hangs
and my other threads are running as I
can see the log. Does this have
anything to do with sys.exit() being
called with threads alive?
Yes, even exit won't stop other threads, it simply raises SystemExit in the main thread. So while the main thread will stop (just like it does on any other unhandled Exception), all other non-daemonic threads will continue to work.
Setting thread.daemon = True will allow the main program to exit. Apps normally wait till all child threads are finished before completing.
th.daemon = True #set this thread as a Daemon Thread
You can think in a Daemon thread as a service this means that it will be running in the background of your computer doing differents task, like indexing files, parsing xml, retrieving news etc, anything that is a long running process.
Your Main thread will finish and your daemon will still be running in the background, that is the reason why your program aka Main thread finish, if you want just put an infinite loop and you will see your thread still running.
An example for a daemon thread is the garbage collection.