I'm trying to create a thread, that does stuff in the background. I need to be able to effectively 'pause' it when I need to and 'resume' it again later. Also, if the thread is in the middle of doing something when I 'pause' it, it should make the calling thread wait until it finishes what it's doing.
I'm pretty new to Multithreading in Python, so I haven't gotten all that far.
What I have pretty much does everything except make the calling thread wait if pause is called while my thread is doing something.
Here's the outline of what I'm trying to achieve in code:
import threading, time
class Me(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
#flag to pause thread
self.paused = False
def run(self):
while True:
if not self.paused:
#thread should do the thing if
#not paused
print 'do the thing'
time.sleep(5)
def pause(self):
self.paused = True
#this is should make the calling thread wait if pause() is
#called while the thread is 'doing the thing', until it is
#finished 'doing the thing'
#should just resume the thread
def resume(self):
self.paused = False
I think I basically need a locking mechanism, but within the same thread?
Conditions can be used for this.
Here's an example filling in your skeleton:
class Me(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
#flag to pause thread
self.paused = False
# Explicitly using Lock over RLock since the use of self.paused
# break reentrancy anyway, and I believe using Lock could allow
# one thread to pause the worker, while another resumes; haven't
# checked if Condition imposes additional limitations that would
# prevent that. In Python 2, use of Lock instead of RLock also
# boosts performance.
self.pause_cond = threading.Condition(threading.Lock())
def run(self):
while True:
with self.pause_cond:
while self.paused:
self.pause_cond.wait()
#thread should do the thing if
#not paused
print 'do the thing'
time.sleep(5)
def pause(self):
self.paused = True
# If in sleep, we acquire immediately, otherwise we wait for thread
# to release condition. In race, worker will still see self.paused
# and begin waiting until it's set back to False
self.pause_cond.acquire()
#should just resume the thread
def resume(self):
self.paused = False
# Notify so thread will wake after lock released
self.pause_cond.notify()
# Now release the lock
self.pause_cond.release()
Hope that helps.
Use threading.Event instead of a boolean variable, and add another event for busy state:
def __init__(self):
...
self.can_run = threading.Event()
self.thing_done = threading.Event()
self.thing_done.set()
self.can_run.set()
def run(self):
while True:
self.can_run.wait()
try:
self.thing_done.clear()
print 'do the thing'
finally:
self.thing_done.set()
def pause(self):
self.can_run.clear()
self.thing_done.wait()
def resume(self):
self.can_run.set()
edit: previous answer was wrong, I fixed it and changed variable names to be clear
Related
I'm writing a timer in python. When the timer reaches 0, I want the thread I made to automatically exit.
class Rollgame:
timer = 0
def timerf(self, timer):
self.timer = timer
while self.timer > 0:
time.sleep(0.1)
self.timer -= 0.1
sys.exit(0)
Is this a valid way to exit a thread? It seems to be working in the context of the program im building, however I'm not sure if it's a good way to do it.
If I ever choose to implement this in something like a flask/django app, will this still be valid?
Sorry if the question seems stupid or too simple, I've never worked with threading in python before.
In general, killing threads abruptly is considered a bad programming practice. Killing a thread abruptly might leave a critical resource that must be closed properly, open. But you might want to kill a thread once some specific time period has passed or some interrupt has been generated. There are the various methods by which you can kill a thread in python.
Set/Reset stop flag :
In order to kill a threads, we can declare a stop flag and this flag will be check occasionally by the thread. For Example:
# Python program showing
# how to kill threads
# using set/reset stop
# flag
import threading
import time
def run():
while True:
print('thread running')
global stop_threads
if stop_threads:
break
stop_threads = False
t1 = threading.Thread(target = run)
t1.start()
time.sleep(1)
stop_threads = True
t1.join()
print('thread killed')
In the above code, as soon as the global variable stop_threads is set, the target function run() ends and the thread t1 can be killed by using t1.join(). But one may refrain from using global variable due to certain reasons. For those situations, function objects can be passed to provide a similar functionality as shown below:
# Python program killing
# threads using stop
# flag
import threading
import time
def run(stop):
while True:
print('thread running')
if stop():
break
def main():
stop_threads = False
t1 = threading.Thread(target = run, args =(lambda : stop_threads, ))
t1.start()
time.sleep(1)
stop_threads = True
t1.join()
print('thread killed')
main()
Using traces to kill threads :
This methods works by installing traces in each thread. Each trace terminates itself on the detection of some stimulus or flag, thus instantly killing the associated thread. For Example:
# Python program using
# traces to kill threads
import sys
import trace
import threading
import time
class thread_with_trace(threading.Thread):
def __init__(self, *args, **keywords):
threading.Thread.__init__(self, *args, **keywords)
self.killed = False
def start(self):
self.__run_backup = self.run
self.run = self.__run
threading.Thread.start(self)
def __run(self):
sys.settrace(self.globaltrace)
self.__run_backup()
self.run = self.__run_backup
def globaltrace(self, frame, event, arg):
if event == 'call':
return self.localtrace
else:
return None
def localtrace(self, frame, event, arg):
if self.killed:
if event == 'line':
raise SystemExit()
return self.localtrace
def kill(self):
self.killed = True
def func():
while True:
print('thread running')
t1 = thread_with_trace(target = func)
t1.start()
time.sleep(2)
t1.kill()
t1.join()
if not t1.isAlive():
print('thread killed')
In this code, start() is slightly modified to set the system trace function using settrace(). The local trace function is defined such that, whenever the kill flag (killed) of the respective thread is set, a SystemExit exception is raised upon the excution of the next line of code, which end the execution of the target function func. Now the thread can be killed with join().
Finally, Using the multiprocessing module to kill threads :
The multiprocessing module of Python allows you to spawn processes in the similar way you spawn threads using the threading module. The interface of the multithreading module is similar to that of the threading module. For Example, in a given code we created three threads(processes) which count from 1 to 9. Now, suppose we wanted to terminate all of the threads. You could use multiprocessing to do that.
# Python program killing
# a thread using multiprocessing
# module
import multiprocessing
import time
def func(number):
for i in range(1, 10):
time.sleep(0.01)
print('Processing ' + str(number) + ': prints ' + str(number*i))
# list of all processes, so that they can be killed afterwards
all_processes = []
for i in range(0, 3):
process = multiprocessing.Process(target=func, args=(i,))
process.start()
all_processes.append(process)
# kill all processes after 0.03s
time.sleep(0.03)
for process in all_processes:
process.terminate()
To sum it up, there are many ways to terminate threads, but I peronally wouldn't use sys.exit().
In my application I have two threads. Main one and "thread". therad generates some data and stores it in a python list. The main thread periodically copies the content of the list generated by "thread". Both threads have an infinite while loop. My goal is stopping both threads when I press any key+enter. To achieve this goal, the program must wait for a keyboard input while the threads are running. I thought I need another thread (lets say manager) which is only waiting for a keyboard input during execution. Here is what I tried first:
class managerThread(threading.Thread):
is_running = True
def __init__(self):
threading.Thread.__init__(self)
signal.signal(signal.SIGINT, self.kill_all)
signal.signal(signal.SIGTERM, self.kill_all)
def run(self):
input("Press any key+enter to stop: ")
self.is_running = False
def kill_all(self,signum, frame):
print("Process ended with keyboard interrupt")
self.is_running = False
sys.exit(-1)
class thread(threading.Thread):
mgr = managerThread()
mgr.start()
def __init__(self):
threading.Thread.__init__(self)
def run(self):
while (self.mgr.is_running):
print("this is acquiring data")
sleep(2.5)
self.mgr.join()
print("manager stopped")
if __name__ == "__main__":
thread = thread()
thread.start()
while (thread.mgr.is_running):
print("this is copying data")
sleep(3)
thread.join()
print("thread is stopped")
sys.exit(0)
Above code is doing exactly what I want to do. Bu this does not seem correct. The manager manages all the others but it is created in one of the slave threads. Another problem is one may try to create multiple managers in different threads. This is something must be strictly avoided. Then I thought the manager must be inherited by the managed classes. Here is what I tried:
class managerThread(threading.Thread):
is_running = True
def __init__(self):
threading.Thread.__init__(self)
signal.signal(signal.SIGINT, self.kill_all)
signal.signal(signal.SIGTERM, self.kill_all)
self.start()
def run(self):
input("Press any key+enter to stop: ")
self.is_running = False
def kill_all(self,signum, frame):
print("Process ended with keyboard interrupt")
self.is_running = False
sys.exit(-1)
class thread(managerThread):
def __init__(self):
super().__init__()
threading.Thread.__init__(self)
def run(self):
while (self.is_running):
print("this is acquiring data")
sleep(2.5)
print("manager stopped")
if __name__ == "__main__":
thread = thread()
thread.start()
while (thread.is_running):
print("this is copying data")
sleep(3)
thread.join()
print("thread is stopped")
sys.exit(0)
As seen in the second code the major part is the same. I tried to make thread as a child of managerThread. However this is not working. The manager never executes "run" method. So I cannot stop the other threads. Another crucial problem is I do not how to stop super() with join(). I am sure I am doing a mistake about class inheritance but I could not resolve the problem since I do not have too much experience with OOP and threads doubled my confusion.
Note: I do not care about synchronization of the threads.
My questions are:
- Is creating a manager thread correct to safely stop the slave threads? If not, what is the proper way?
- Why the second code is not working? What do I have to modify to get it work?
- Why the parent class is initializing but it is never running "run" method?
- I believe that the parent class is never starting but how can I stop it in the second code if it is actually starting?
Thank you.
Even I did not want to use a global variable to stop safely all the threads I could not find a solution without a global running flag. I also tried to pass a mutable variable to the manager thread but I was unsuccessful. Here is a working sketch that explains how I solved my problem. I hope this helps someone else. Meanwhile, I would be happy if someone propose a better solution :).
Note: I did not debug it.
import sys
import threading, queue
import signal
from time import sleep
import numpy as np
global_running = False
class managerThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
signal.signal(signal.SIGINT, self.kill_all)
signal.signal(signal.SIGTERM, self.kill_all)
def run(self):
global is_running
is_running = True
input("Press any key+enter to stop: ")
is_running = False
print("manager finished")
def kill_all(self,signum, frame):
global is_running
is_running = False
print("Process ended with keyboard interrupt")
sys.exit(-1)
class thread(threading.Thread):
__mgr = managerThread()
__mgr.start()
running = True
def __init__(self):
threading.Thread.__init__(self)
self.myvar = 0
self.queue = queue.Queue()
def currentVar(self):
var = np.empty(1)
while (var.size<=1):
var = self.queue.get()
self.queue.task_done()
return var
def run(self):
global is_running
while (is_running):
print("this is acquiring data")
sleep(2.5)
self.myvar = np.empty(5)
self.queue.put(self.myvar)
self.running = False
self.__mgr.join()
print("manager stopped")
if __name__ == "__main__":
thread = thread()
thread.start()
while (thread.running):
# var = thread.queue.get()
var = thread.currentVar()
print ("value is: ", var)
# thread.queue.task_done()
thread.join()
print("thread is stopped")
sys.exit(0)
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
self._end = False
self._running = False
def run(self):
self._running = True
while not self._finished:
time.sleep(0.05)
self._end = True
def stop(self):
if not self._running:
return
self._finished = True
while not self._end:
time.sleep(0.05)
I wish to have a thread on which I can call run() and stop(). The stop method should wait for run to complete in an orderly manner. I also want stop to return without any issues if run hasn't even be called. How should I do this?
I create this thread in a setup() method in my test environment and run stop on it in the teardown(). However, in some tests I dont call run().
UPDATE
Here's my second attempt. Is it correct now?
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
def run(self):
while not self._finished:
print("*")
time.sleep(1)
print("Finished Other")
def finish(self):
self._finished = True
self.join()
m = MyThread()
m.start()
print("After")
time.sleep(5)
m.finish()
print("Finished Main")
You do not need to and should not implement this yourself. What you are looking for already exists, at least in large parts. It is, however, not called "stop". The concept you are describing is usually called "join".
Have a look at the documentation for join: https://docs.python.org/3.4/library/threading.html#threading.Thread.join
You write
The stop method should wait for run to complete in an orderly manner.
Join's documentation says: "Wait until the thread terminates." check ✓
You write
I also want stop to return without any issues if run hasn't even be
called
Join's documentation says: "It is also an error to join() a thread before it has been started"
So, the only thing you need to make sure is that you call join() only after you have started the thread via the start() method. That should be easy for you.
I'm uisng the psutil library in a thread, that posts my CPU usage statistics periodically. Here's a snippet:
class InformationThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self)
def run(self):
while True:
cpu = psutil.cpu_percent(interval=600) #this is a blocking call
print cpu
I need to stop this thread but I can't seem to understand how. The method cpu_percent is a blocking function that will block for 600 seconds.
I've been digging around and all the examples I saw relied on a tight-loop that checked a flag to see whether the loop should be interrupted but in this case, I'm not sure how to kill the thread.
Set interval to 0.0 and implement a tighter inner loop in which you can check whether your thread should terminate. It shouldn't be difficult to time it so that the elapsed time between calls to cpu_percent() is roughly the same as 600.
You could add astop()method to yourInformationThreadclass that terminates itsrun()loop as shown the following. But note that it won't unblock acpu_percent()call already in progress.
class InformationThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self)
self.daemon = True # OK for main to exit even if instance still running
self.running = False
self.status_lock = threading.Lock()
def run(self):
with self.status_lock:
self.running = True
while True:
with self.status_lock:
if not self.running:
break
cpu = psutil.cpu_percent(interval=600) # this is a blocking call
print cpu
def stop(self):
with self.status_lock:
self.running = False
I want to force threads termination in python: I don't want to set an event and wait until the thread checks it and exits. I'm looking for a simple solution like kill -9. Is this possible to do that without dirty hacks like operating with private methods etc.?
If you do not mind your code running about ten times slower, you can use the Thread2 class implemented below. An example follows that shows how calling the new stop method should kill the thread on the next bytecode instruction.
import threading
import sys
class StopThread(StopIteration): pass
threading.SystemExit = SystemExit, StopThread
class Thread2(threading.Thread):
def stop(self):
self.__stop = True
def _bootstrap(self):
if threading._trace_hook is not None:
raise ValueError('Cannot run thread with tracing!')
self.__stop = False
sys.settrace(self.__trace)
super()._bootstrap()
def __trace(self, frame, event, arg):
if self.__stop:
raise StopThread()
return self.__trace
class Thread3(threading.Thread):
def _bootstrap(self, stop_thread=False):
def stop():
nonlocal stop_thread
stop_thread = True
self.stop = stop
def tracer(*_):
if stop_thread:
raise StopThread()
return tracer
sys.settrace(tracer)
super()._bootstrap()
################################################################################
import time
def main():
test = Thread2(target=printer)
test.start()
time.sleep(1)
test.stop()
test.join()
def printer():
while True:
print(time.time() % 1)
time.sleep(0.1)
if __name__ == '__main__':
main()
The Thread3 class appears to run code approximately 33% faster than the Thread2 class.
Threads end when they do.
You can signal a thread that you want it to terminate ASAP, but that assumes collaboration of the code running in a thread, and it offers no upper bound guarantee for when that happens.
A classic way is to use a variable like exit_immediately = False and have threads' main routines periodically check it and terminate if the value is True. To have the threads exit, you set exit_immediately = True and call .join() on all threads. Obviously, this works only when threads are able to check in periodically.
If what you want is to just be able to let the program terminate at its end without caring about what happens to some threads, what you want is daemon threads.
From the docs:
The entire Python program exits when no alive non-daemon threads are
left.
Example usage program:
import threading
import time
def test():
while True:
print "hey"
time.sleep(1)
t = threading.Thread(target=test)
t.daemon = True # <-- set to False and the program will not terminate
t.start()
time.sleep(10)
Trivia: daemon threads are referred to as background threads in .Net.