This question already has answers here:
Daemon Threads Explanation
(9 answers)
Closed 3 years ago.
I have a python script that starts a process using multiprocessing Process class. Inside that process I start a thread with an infinite loop, and I simulate an exception on the main thread by doing exit(1). I was expecting exit to kill the process, but it doesn't, and the main thread reports it as alive.
I have tried with both raising exception and calling exit, but none work.
I have tried waiting for the process to finish by waiting for is_alive, and by calling join. But both have the same behavior.
from multiprocessing import Process
from threading import Thread
from time import sleep
def infinite_loop():
while True:
print("It's sleeping man")
sleep(1)
def start_infinite_thread():
run_thread = Thread(target=infinite_loop)
run_thread.start()
exit(1)
def test_multi():
x = Process(target=start_infinite_thread)
x.start()
sleep(5)
assert not x.is_alive()
I am expecting the process not to be alive, but the assert fails.
Thanks #Imperishable Night for the reply, by setting the thread as daemon the test passes.
Code that works:
from multiprocessing import Process
from threading import Thread
from time import sleep
def infinite_loop():
while True:
print("It's sleeping man")
sleep(1)
def start_infinite_thread():
run_thread = Thread(target=infinite_loop)
run_thread.daemon = True
run_thread.start()
exit(1)
def test_multi():
x = Process(target=start_infinite_thread)
x.start()
sleep(1)
assert not x.is_alive()
Related
I have a loop which makes a get request to a webservice to fetch data and do some stuff, but I want to 'manually' terminate the thread/event, which I achieved with the following example:
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
exit.wait(10) #wait 10 seconds
With that, the only thing that terminates it's the sleep time between loops. How can I also kill the loop so it doesn't keep running until it gets to the last iteration?
nvm i've solved it like this
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
if exit.wait(10):
break
the condition returns true when killed and also sleeps the 10 seconds, so it works
you have 2 options ,
kill the thread or process entirely
or making the loop's boolean false. going that way
you could use a global variable in this way: [Python 3.7] , run it to see
from threading import Thread
from time import sleep
global glob
glob=True
def threaded_function():
while glob:
print("\n [Thread] this thread is running until main function halts this")
sleep(0.8)
if __name__ == "__main__":
thread = Thread(target = threaded_function, args = ())
thread.start()
for i in range(4,0,-1):
print("\n [Main] thread will be terminated in "+str(i)+" seconds")
sleep(1)
glob=False
while True:
print("[Main] program is over")
sleep(1)
This question already has answers here:
Is there any way to kill a Thread?
(31 answers)
Closed 3 years ago.
To simplify the situation I'm having: I'm trying to terminate a thread while it is still running in Python 2.7, and I'm not sure how to do it.
Take this simple code:
import time
import threading
def thread1():
print "Starting thread 1"
while True:
time.sleep(0.5)
print "Working"
thread1 = threading.Thread(target=thread1, args=())
thread1.start()
time.sleep(2)
print "Killing thread 1"
thread2.stop()
print "Checking if it worked:"
print "Thread is: " + str(thread1.isAlive())
Thread 1 keeps on 'working' and I'm trying to kill it in the main thread. Any idea on how to do it? I've tried:
threat1.terminate
threat1.stop
threat1.quit
threat1.end
This all seems to point that there is no way to really stop it with a simple line of code. What could you suggest?
To terminate an Thread controlled, using a threadsafe threading.Event():
import threading, time
def Thread_Function(running):
while running.is_set():
print('running')
time.sleep(1)
if __name__ == '__main__':
running = threading.Event()
running.set()
thread = threading.Thread(target=Thread_Function, args=(running,))
thread.start()
time.sleep(1)
print('Event running.clear()')
running.clear()
print('Wait until Thread is terminating')
thread.join()
print("EXIT __main__")
Output:
running
running
Event running.clear()
Wait until Thread is terminating
EXIT __main__
Tested with Python:3.4.2
Online Demo: reply.it
Usually, in this cases, I use some kind of signal:
import time
import threading
class thread1(threading.Thread):
def run(self):
self.kill = False
print "Starting thread 1"
while not self.kill:
time.sleep(0.5)
print "Working"
thread_obj = thread1()
thread_obj.start()
time.sleep(2)
print "Killing thread 1"
thread_obj.kill = True
print "Checking if it worked:"
time.sleep(1)
print "Thread is: " + str(thread_obj.isAlive())
EDIT
After reading the answer suggested in one of the comment... I realized that this is just a simplified version of what is described there. I hope this will be useful anyway.
Indeed!
threads cannot be destroyed, stopped, suspended, resumed, or interrupted
(So say the docs in a paragraph below the link.)
Make your threads listen to signals you may send, via a queue (best), a shared variable (worse), or any other means. Be careful and don't let them run unchecked loops, as in your example code.
I have the following toy example for Python threading module
from __future__ import print_function
import threading
import time
import signal
import sys
import os
import time
class ThreadShutdown(Exception):
# Custom exception to allow clean thread exit
pass
def thread_shutdown(signum, frame):
print(" o Signal {} caught and raising ThreadShutdown exception".format(signum))
raise ThreadShutdown
def main():
"""
Register the signal handlers needed to stop
cleanly the child accessing thread
"""
signal.signal(signal.SIGTERM, thread_shutdown)
signal.signal(signal.SIGINT, thread_shutdown)
test_run_seconds = 120
try:
thread = ChildThread()
thread.start()
time.sleep(1)
while test_run_seconds > 0:
test_run_seconds -= 1
print(" o [{}] remaining time is {} seconds".format(time.asctime( time.localtime(time.time()) ), test_run_seconds))
time.sleep(1)
except ThreadShutdown:
thread.shutdown_flag.set()
thread.join()
print(" o ThreadShutdown procedure complete")
return
proc.terminate()
thread.shutdown_flag.set()
thread.join()
print(" o Test terminated")
class ChildThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.shutdown_flag = threading.Event()
def run(self):
while not self.shutdown_flag.is_set():
print(" o [{}] is the current time in child, sleep for 10s".format(time.asctime( time.localtime(time.time()))))
time.sleep(10)
return
if __name__ == "__main__":
sys.exit(main())
which behaves as expected (the main thread counts every second while the spawned thread prints only every 10 seconds.
I was trying to understand the behaviour of the same code snippet in presence of blocking waits in kernel mode in the spawned thread. For example, assume that the spawned thread now goes into a killable wait in an ioctl with a timeout of 10s, I would still expect to have the main thread counting every second. For some reason, it instead counts every 10s, as if it was blocked as well in the wait of the spawned thread. What is the reason?
I want to kill a thread in python. This thread can run in a blocking operation and join can't terminate it.
Simular to this:
from threading import Thread
import time
def block():
while True:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block)
thread.start()
#kill thread
#do other stuff
My problem is that the real blocking operation is in another module that is not from me so there is no place where I can break with a running variable.
The thread will be killed when exiting the main process if you set it up as a daemon:
from threading import Thread
import time
def block():
while True:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block, daemon = True)
thread.start()
sys.exit(0)
Otherwise just set a flag, I'm using a bad example (you should use some synchronization not just a plain variable):
from threading import Thread
import time
RUNNING = True
def block():
global RUNNING
while RUNNING:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block, daemon = True)
thread.start()
RUNNING = False # thread will stop, not killed until next loop iteration
.... continue your stuff here
Use a running variable:
from threading import Thread
import time
running = True
def block():
global running
while running:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block)
thread.start()
running = False
# do other stuff
I would prefer to wrap it all in a class, but this should work (untested though).
EDIT
There is a way to asynchronously raise an exception in a separate thread which could be caught by a try: except: block, but it's a dirty dirty hack: https://gist.github.com/liuw/2407154
Original post
"I want to kill a thread in python." you can't. Threads are only killed when they're daemons when there are no more non-daemonic threads running from the parent process. Any thread can be asked nicely to terminate itself using standard inter-thread communication methods, but you state that you don't have any chance to interrupt the function you want to kill. This leaves processes.
Processes have more overhead, and are more difficult to pass data to and from, but they do support being killed by sending SIGTERM or SIGKILL.
from multiprocessing import Process, Queue
from time import sleep
def workfunction(*args, **kwargs): #any arguments you send to a child process must be picklable by python's pickle module
sleep(args[0]) #really long computation you might want to kill
return 'results' #anything you want to get back from a child process must be picklable by python's pickle module
class daemon_worker(Process):
def __init__(self, target_func, *args, **kwargs):
self.return_queue = Queue()
self.target_func = target_func
self.args = args
self.kwargs = kwargs
super().__init__(daemon=True)
self.start()
def run(self): #called by self.start()
self.return_queue.put(self.target_func(*self.args, **self.kwargs))
def get_result(self): #raises queue.Empty if no result is ready
return self.return_queue.get()
if __name__=='__main__':
#start some work that takes 1 sec:
worker1 = daemon_worker(workfunction, 1)
worker1.join(3) #wait up to 3 sec for the worker to complete
if not worker1.is_alive(): #if we didn't hit 3 sec timeout
print('worker1 got: {}'.format(worker1.get_result()))
else:
print('worker1 still running')
worker1.terminate()
print('killing worker1')
sleep(.1) #calling worker.is_alive() immediately might incur a race condition where it may or may not have shut down yet.
print('worker1 is alive: {}'.format(worker1.is_alive()))
#start some work that takes 100 sec:
worker2 = daemon_worker(workfunction, 100)
worker2.join(3) #wait up to 3 sec for the worker to complete
if not worker2.is_alive(): #if we didn't hit 3 sec timeout
print('worker2 got: {}'.format(worker2.get_result()))
else:
print('worker2 still running')
worker2.terminate()
print('killing worker2')
sleep(.1) #calling worker.is_alive() immediately might incur a race condition where it may or may not have shut down yet.
print('worker2 is alive: {}'.format(worker2.is_alive())
I am testing Python threading with the following script:
import threading
class FirstThread (threading.Thread):
def run (self):
while True:
print 'first'
class SecondThread (threading.Thread):
def run (self):
while True:
print 'second'
FirstThread().start()
SecondThread().start()
This is running in Python 2.7 on Kubuntu 11.10. Ctrl+C will not kill it. I also tried adding a handler for system signals, but that did not help:
import signal
import sys
def signal_handler(signal, frame):
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
To kill the process I am killing it by PID after sending the program to the background with Ctrl+Z, which isn't being ignored. Why is Ctrl+C being ignored so persistently? How can I resolve this?
Ctrl+C terminates the main thread, but because your threads aren't in daemon mode, they keep running, and that keeps the process alive. We can make them daemons:
f = FirstThread()
f.daemon = True
f.start()
s = SecondThread()
s.daemon = True
s.start()
But then there's another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Now it will keep print 'first' and 'second' until you hit Ctrl+C.
Edit: as commenters have pointed out, the daemon threads may not get a chance to clean up things like temporary files. If you need that, then catch the KeyboardInterrupt on the main thread and have it co-ordinate cleanup and shutdown. But in many cases, letting daemon threads die suddenly is probably good enough.
KeyboardInterrupt and signals are only seen by the process (ie the main thread)... Have a look at Ctrl-c i.e. KeyboardInterrupt to kill threads in python
I think it's best to call join() on your threads when you expect them to die. I've taken the liberty to make the change your loops to end (you can add whatever cleanup needs are required to there as well). The variable die is checked on each pass and when it's True, the program exits.
import threading
import time
class MyThread (threading.Thread):
die = False
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run (self):
while not self.die:
time.sleep(1)
print (self.name)
def join(self):
self.die = True
super().join()
if __name__ == '__main__':
f = MyThread('first')
f.start()
s = MyThread('second')
s.start()
try:
while True:
time.sleep(2)
except KeyboardInterrupt:
f.join()
s.join()
An improved version of #Thomas K's answer:
Defining an assistant function is_any_thread_alive() according to this gist, which can terminates the main() automatically.
Example codes:
import threading
def job1():
...
def job2():
...
def is_any_thread_alive(threads):
return True in [t.is_alive() for t in threads]
if __name__ == "__main__":
...
t1 = threading.Thread(target=job1,daemon=True)
t2 = threading.Thread(target=job2,daemon=True)
t1.start()
t2.start()
while is_any_thread_alive([t1,t2]):
time.sleep(0)
One simple 'gotcha' to beware of, are you sure CAPS LOCK isn't on?
I was running a Python script in the Thonny IDE on a Pi4. With CAPS LOCK on, Ctrl+Shift+C is passed to the keyboard buffer, not Ctrl+C.