I have a loop which makes a get request to a webservice to fetch data and do some stuff, but I want to 'manually' terminate the thread/event, which I achieved with the following example:
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
exit.wait(10) #wait 10 seconds
With that, the only thing that terminates it's the sleep time between loops. How can I also kill the loop so it doesn't keep running until it gets to the last iteration?
nvm i've solved it like this
from threading import Event
exit = Event()
if external_condition():
exit.set()
for _ in range(mins):
fetch_data_and_do_stuff()
if exit.wait(10):
break
the condition returns true when killed and also sleeps the 10 seconds, so it works
you have 2 options ,
kill the thread or process entirely
or making the loop's boolean false. going that way
you could use a global variable in this way: [Python 3.7] , run it to see
from threading import Thread
from time import sleep
global glob
glob=True
def threaded_function():
while glob:
print("\n [Thread] this thread is running until main function halts this")
sleep(0.8)
if __name__ == "__main__":
thread = Thread(target = threaded_function, args = ())
thread.start()
for i in range(4,0,-1):
print("\n [Main] thread will be terminated in "+str(i)+" seconds")
sleep(1)
glob=False
while True:
print("[Main] program is over")
sleep(1)
Related
I want to kill a thread in python. This thread can run in a blocking operation and join can't terminate it.
Simular to this:
from threading import Thread
import time
def block():
while True:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block)
thread.start()
#kill thread
#do other stuff
My problem is that the real blocking operation is in another module that is not from me so there is no place where I can break with a running variable.
The thread will be killed when exiting the main process if you set it up as a daemon:
from threading import Thread
import time
def block():
while True:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block, daemon = True)
thread.start()
sys.exit(0)
Otherwise just set a flag, I'm using a bad example (you should use some synchronization not just a plain variable):
from threading import Thread
import time
RUNNING = True
def block():
global RUNNING
while RUNNING:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block, daemon = True)
thread.start()
RUNNING = False # thread will stop, not killed until next loop iteration
.... continue your stuff here
Use a running variable:
from threading import Thread
import time
running = True
def block():
global running
while running:
print("running")
time.sleep(1)
if __name__ == "__main__":
thread = Thread(target = block)
thread.start()
running = False
# do other stuff
I would prefer to wrap it all in a class, but this should work (untested though).
EDIT
There is a way to asynchronously raise an exception in a separate thread which could be caught by a try: except: block, but it's a dirty dirty hack: https://gist.github.com/liuw/2407154
Original post
"I want to kill a thread in python." you can't. Threads are only killed when they're daemons when there are no more non-daemonic threads running from the parent process. Any thread can be asked nicely to terminate itself using standard inter-thread communication methods, but you state that you don't have any chance to interrupt the function you want to kill. This leaves processes.
Processes have more overhead, and are more difficult to pass data to and from, but they do support being killed by sending SIGTERM or SIGKILL.
from multiprocessing import Process, Queue
from time import sleep
def workfunction(*args, **kwargs): #any arguments you send to a child process must be picklable by python's pickle module
sleep(args[0]) #really long computation you might want to kill
return 'results' #anything you want to get back from a child process must be picklable by python's pickle module
class daemon_worker(Process):
def __init__(self, target_func, *args, **kwargs):
self.return_queue = Queue()
self.target_func = target_func
self.args = args
self.kwargs = kwargs
super().__init__(daemon=True)
self.start()
def run(self): #called by self.start()
self.return_queue.put(self.target_func(*self.args, **self.kwargs))
def get_result(self): #raises queue.Empty if no result is ready
return self.return_queue.get()
if __name__=='__main__':
#start some work that takes 1 sec:
worker1 = daemon_worker(workfunction, 1)
worker1.join(3) #wait up to 3 sec for the worker to complete
if not worker1.is_alive(): #if we didn't hit 3 sec timeout
print('worker1 got: {}'.format(worker1.get_result()))
else:
print('worker1 still running')
worker1.terminate()
print('killing worker1')
sleep(.1) #calling worker.is_alive() immediately might incur a race condition where it may or may not have shut down yet.
print('worker1 is alive: {}'.format(worker1.is_alive()))
#start some work that takes 100 sec:
worker2 = daemon_worker(workfunction, 100)
worker2.join(3) #wait up to 3 sec for the worker to complete
if not worker2.is_alive(): #if we didn't hit 3 sec timeout
print('worker2 got: {}'.format(worker2.get_result()))
else:
print('worker2 still running')
worker2.terminate()
print('killing worker2')
sleep(.1) #calling worker.is_alive() immediately might incur a race condition where it may or may not have shut down yet.
print('worker2 is alive: {}'.format(worker2.is_alive())
The following code:
import threading
import time
from functools import partial
from itertools import count
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.is_set():
break
time.sleep(sleep_interval)
print('Slept %s' % sleep_interval)
print('Prod terminating')
if __name__ == '__main__':
stop_event = threading.Event() #https://stackoverflow.com/a/41139707/281545
target = partial(daemon_loop, sleep_interval=2, stop_event=stop_event)
prod_thread = threading.Thread(target=target,
# daemon=True
)
try:
prod_thread.start()
while True:
time.sleep(10)
except KeyboardInterrupt:
print('Terminating...')
stop_event.set()
prints on a keyboard interrupt:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
Slept 2
2
Prod terminating
Uncommenting the # daemon=True line results in the prod_thread being ended immediately:
C:\Users\MrD\.PyCharm2018.2\config\scratches>c:\_\Python363-64\python.exe thread_daemon.py
0
Slept 2
1
Terminating...
My question is what is the preferred/more pythonic way to deal with thread termination - should I drop the Event machinery and just mark the thread as daemon or is there some edge case I miss?
See:
Daemon Threads Explanation
How to stop daemon thread?
I haven't done enough Python to give you a "Pythonic" answer, but I can answer in more general programming terms.
Firstly, I'm not a fan of terminating threads. There are cases where it is safe and OK, such as your example here - but terminating in the middle of print writing its output would feel a little dirty.
Secondly, if you want to continue using sleep (which I'm also not a fan of) you could repeat your if stop_event.is_set(): and break after the sleep. (Don't move the code, copy it.) The main problem with sleep in this case is that it will wait the full sleep_interval even if the event is set during that time.
Thirdly - and my preference - instead of using sleep, do a wait on the event with a timeout. If the event is not set during the wait, wait returns false after waiting the timeout period. If the event is set before or during the wait, wait returns true immediately (that is, it aborts the timeout, giving you fast, clean shutdown of the thread.)
So your code would look something like this:
def daemon_loop(sleep_interval, stop_event):
for j in count():
print(j)
if stop_event.wait(sleep_interval):
break
print('Slept %s' % sleep_interval)
print('Prod terminating')
I'm not too familiar with threading, and probably not using it correctly, but I have a script that runs a speedtest a few times and prints the average. I'm trying to use threading to call a function which displays something while the tests are running.
Everything works fine unless I try to put input() at the end of the script to keep the console window open. It causes the thread to run continuously.
I'm looking for some direction in terminating a thread correctly. Also open to any better ways to do this.
import speedtest, time, sys, datetime
from threading import Thread
s = speedtest.Speedtest()
best = s.get_best_server()
def downloadTest(tries):
x=0
downloadList = []
for x in range(tries):
downSpeed = (s.download()/1000000)
downloadList.append(downSpeed)
x+=1
results_dict = s.results.dict()
global download_avg, isp
download_avg = (sum(downloadList)/len(downloadList))
download_avg = round(download_avg,1)
isp = (results_dict['client']['isp'])
print("")
print(isp)
print(download_avg)
def progress():
while True:
print('~ ',end='', flush=True)
time.sleep(1)
def start():
now=(datetime.datetime.today().replace(microsecond=0))
print(now)
d = Thread(target= downloadTest, args=(3,))
d.start()
d1 = Thread(target = progress)
d1.daemon = True
d1.start()
d.join()
start()
input("Complete...") # this causes progress thread to keep running
There is no reason for your thread to exit, which is why it does not terminate. A daemon thread normally terminates when your programm (all other threads) terminate, which does not happen in this as the last input does not quit.
In general it is a good idea to make a thread stop by itself, rather than forcefully killing it, so you would generally kill this kind of thread with a flag. Try changing the segment at the end to:
killflag = False
start()
killflag = True
input("Complete...")
and update the progress method to:
def progress():
while not killflag:
print('~ ',end='', flush=True)
time.sleep(1)
I have thrown together a quick threading test:
import threading
def test():
print "it don't work"
while True:
threading.Timer(1, test).start()
It runs test, but it doesn't wait. What's wrong?
In each loop iteration, you start a new thread. Therefore you will reach the limit of allowed thread and you will get an exception : can't start new thread.
while True:
threading.Timer(1, test).start()
You can add global flag and wait until the function is executed - You should use time.sleep to avoid busy waiting.
a = False
def test():
global a
print("hallo")
a = True
threading.Timer(10, test).start()
while not a:
time.sleep(1)
print('done')
I have a timer function which I am calling it in another function like this
import time
import threading
def f():
while(True):
print "hello"
time.sleep(5)
def execute():
t = threading.Timer(5,f)
t.start()
command = ''
while command != 'exit':
command = raw_input()
if command == 'exit':
t.cancel()
Even if after entering "exit" command, the function is printing "hello"
I am not able to figure out Whats wrong with the code
class threading.Timer - cancel() - Doc-Link
Stop the timer, and cancel the execution of the timer’s action. This will only work if the timer is still in its waiting stage.
A very simple Version of what you are trying to accomplish could look like this.
import threading
_f_got_killed = threading.Event()
def f():
while(True):
print "hello"
_f_got_killed.wait(5)
if _f_got_killed.is_set():
break
def execute():
t = threading.Timer(5,f)
t.start()
command = ''
while command != 'exit':
command = raw_input()
if command == 'exit':
_f_got_killed.set()
t.cancel()
execute()
For forcefully killing a thread look at this:
Is there any way to kill a Thread in Python?
You are using cancel wrong. In http://docs.python.org/2/library/threading.html, it states: "Timers are started, as with threads, by calling their start() method. The timer can be stopped (before its action has begun) by calling the cancel() method. The interval the timer will wait before executing its action may not be exactly the same as the interval specified by the user."
In your code, if you try to use cancel after the timed thread has already begun its execution (it will in 5 seconds), cancel accomplishes nothing. The thread will remain in the while loop in f forever until you give it some sort of forced interrupt. So typing "exit" in the first 5 seconds after you run execute works. It will successfully stop the timer before the thread even begins. But after your timer stops and your thread starts executing the code in f, there will be no way to stop it through cancel.