I'm uisng the psutil library in a thread, that posts my CPU usage statistics periodically. Here's a snippet:
class InformationThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self)
def run(self):
while True:
cpu = psutil.cpu_percent(interval=600) #this is a blocking call
print cpu
I need to stop this thread but I can't seem to understand how. The method cpu_percent is a blocking function that will block for 600 seconds.
I've been digging around and all the examples I saw relied on a tight-loop that checked a flag to see whether the loop should be interrupted but in this case, I'm not sure how to kill the thread.
Set interval to 0.0 and implement a tighter inner loop in which you can check whether your thread should terminate. It shouldn't be difficult to time it so that the elapsed time between calls to cpu_percent() is roughly the same as 600.
You could add astop()method to yourInformationThreadclass that terminates itsrun()loop as shown the following. But note that it won't unblock acpu_percent()call already in progress.
class InformationThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self)
self.daemon = True # OK for main to exit even if instance still running
self.running = False
self.status_lock = threading.Lock()
def run(self):
with self.status_lock:
self.running = True
while True:
with self.status_lock:
if not self.running:
break
cpu = psutil.cpu_percent(interval=600) # this is a blocking call
print cpu
def stop(self):
with self.status_lock:
self.running = False
Related
I want to have a main program that works like a console from where I can call other processes (infinite loops) and kill them selectively whenever certain commands are entered.
For that I created this class:
class RunInThread(threading.Thread):
def __init__(self, function):
self.function = function
self.kill_pill = threading.Event()
threading.Thread.__init__(self)
def start(self): # This is controversial.
self.__init__(self.function)
threading.Thread.start(self)
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
The documentation for thread.Thread says that only the __init__() and run() methods should be overridden.
Is there any clear issue with my code? It works the way I intended but since it's going to be running for long periods of time I need to make sure I'm not creating any memory problems.
EDIT:
What about this solution?:
class StoppableThread(threading.Thread):
# threading.Thread class but can be stopped with the stop() method.
def __init__(self, function):
threading.Thread.__init__(self)
self.function = function
self.kill_pill = threading.Event()
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
class RunInThread():
def __init__(self, function, prnt=False):
self.function = function
self.running = False
self.prnt = prnt
def start(self):
if not self.running:
self.thread = StoppableThread(self.function)
self.thread.start()
self.running = True
else:
if self.prnt:
print('Thread already running.')
def stop(self):
self.thread.stop()
self.running = False
If you want to find out what things that could break, I'd suggest looking into the implementation of Thread class.
Among other things, Thread.__init__() initialises an Event() object to detect thread startup and shutdown, manages cleanup hooks/callbacks, some internal lock objects, and registers the thread to a list so you can introspect running threads. By calling Thread.__init__(), these variables gets reinitialised and screws up the internal mechanisms of many of these functionalities.
What could go wrong? I didn't test any of these, but from skimming through threading.py, these are likely some of the things that I expect could go wrong:
your python process now will be running a OS threads that doesn't show up in enumerate_thread()
multiple OS thread will now return the same Thread object when it calls current_thread(), which will likely also break threadlocal and anything that depends on threadlocal
Thread.join() depends on some internal locks, which likely would now become thread unsafe to call
Unhandled reception can go to the wrong exception hook handler
register_at_fork and shutdown handler likely will get confused
In other words, don't try to be sneaky. Create a new Thread object for each thread you want to start.
There's a good reason that the Thread class spent efforts trying to prevent you from accidentally calling start() twice. Don't try to subvert this.
I'm trying to create a thread, that does stuff in the background. I need to be able to effectively 'pause' it when I need to and 'resume' it again later. Also, if the thread is in the middle of doing something when I 'pause' it, it should make the calling thread wait until it finishes what it's doing.
I'm pretty new to Multithreading in Python, so I haven't gotten all that far.
What I have pretty much does everything except make the calling thread wait if pause is called while my thread is doing something.
Here's the outline of what I'm trying to achieve in code:
import threading, time
class Me(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
#flag to pause thread
self.paused = False
def run(self):
while True:
if not self.paused:
#thread should do the thing if
#not paused
print 'do the thing'
time.sleep(5)
def pause(self):
self.paused = True
#this is should make the calling thread wait if pause() is
#called while the thread is 'doing the thing', until it is
#finished 'doing the thing'
#should just resume the thread
def resume(self):
self.paused = False
I think I basically need a locking mechanism, but within the same thread?
Conditions can be used for this.
Here's an example filling in your skeleton:
class Me(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
#flag to pause thread
self.paused = False
# Explicitly using Lock over RLock since the use of self.paused
# break reentrancy anyway, and I believe using Lock could allow
# one thread to pause the worker, while another resumes; haven't
# checked if Condition imposes additional limitations that would
# prevent that. In Python 2, use of Lock instead of RLock also
# boosts performance.
self.pause_cond = threading.Condition(threading.Lock())
def run(self):
while True:
with self.pause_cond:
while self.paused:
self.pause_cond.wait()
#thread should do the thing if
#not paused
print 'do the thing'
time.sleep(5)
def pause(self):
self.paused = True
# If in sleep, we acquire immediately, otherwise we wait for thread
# to release condition. In race, worker will still see self.paused
# and begin waiting until it's set back to False
self.pause_cond.acquire()
#should just resume the thread
def resume(self):
self.paused = False
# Notify so thread will wake after lock released
self.pause_cond.notify()
# Now release the lock
self.pause_cond.release()
Hope that helps.
Use threading.Event instead of a boolean variable, and add another event for busy state:
def __init__(self):
...
self.can_run = threading.Event()
self.thing_done = threading.Event()
self.thing_done.set()
self.can_run.set()
def run(self):
while True:
self.can_run.wait()
try:
self.thing_done.clear()
print 'do the thing'
finally:
self.thing_done.set()
def pause(self):
self.can_run.clear()
self.thing_done.wait()
def resume(self):
self.can_run.set()
edit: previous answer was wrong, I fixed it and changed variable names to be clear
I have a few classes that look more or less like this:
import threading
import time
class Foo():
def __init__(self, interval, callbacks):
self.thread = threading.Thread(target=self.loop)
self.interval = interval
self.thread_stop = threading.Event()
self.callbacks = callbacks
def loop():
while not self.thread_stop.is_set():
#do some stuff...
for callback in self.callbacks():
callback()
time.sleep(self.interval)
def start(self):
self.thread.start()
def kill(self):
self.thread_stop.set()
Which I am using from my main thread like this:
interval = someinterval
callbacks = [some callbacks]
f = Foo(interval, callbacks)
try:
f.start()
except KeyboardInterrupt:
f.kill()
raise
I would like a KeyboardInterrupt to kill the thread after all the callbacks have been completed, but before the loop repeats. Currently they are ignored and I have to resort to killing the terminal process that the program is running in.
I saw the idea of using threading.Event from this post, but it appears like I'm doing it incorrectly, and it's making working on this project a pretty large hassle.
I don't know if it may be relevant, but the callbacks I'm passing access data from the Internet and make heavy use of the retrying decorator to deal with unreliable connections.
EDIT
After everyone's help, the loop now looks like this inside Foo:
def thread_loop(self):
while not self.thread_stop.is_set():
# do some stuff
# call the callbacks
self.thread_stop.wait(self.interval)
This is kind of a solution, although it isn't ideal. This code runs on PythonAnywhere and the price of the account is by CPU time. I'll have to see how much this uses over the course of a day with the constant waking and sleeping of threads, but it at least solves the main issue
I think your problem is that you have a try-except-block around f.start(), but that returns immediately, so you aren't going to catch KeyboardInterrupts after the thread was started.
You could try adding a while-loop at the bottom of your program like this:
f.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
f.kill()
raise
This isn't exactly the most elegant solution, but it should work.
Thanks to #shx2 and #jazzpi for putting together the two separate pieces of the puzzle.
so the final code is
import threading
import time
class Foo():
def __init__(self, interval, callbacks):
self.thread = threading.Thread(target=self.loop)
self.interval = interval
self.thread_stop = threading.Event()
self.callbacks = callbacks
def loop():
while not self.thread_stop.is_set():
#do some stuff...
for callback in self.callbacks():
callback()
self.thread_stop.wait(self.interval)
def start(self):
self.thread.start()
def kill(self):
self.thread_stop.set()
And then in main
interval = someinterval
callbacks = [some, callbacks]
f = Foo(interval, callbacks)
f.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
f.kill()
raise
#jazzpi's answer correctly addresses the issue you're having in the main thread.
As to the sleep in thread's loop, you can simply replace the call to sleep with a call to self.thread_stop.wait(self.interval).
This way, your thread wakes up as soon as the stop event is set, or after waiting (i.e. sleeping) for self.interval seconds. (Event docs)
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
self._end = False
self._running = False
def run(self):
self._running = True
while not self._finished:
time.sleep(0.05)
self._end = True
def stop(self):
if not self._running:
return
self._finished = True
while not self._end:
time.sleep(0.05)
I wish to have a thread on which I can call run() and stop(). The stop method should wait for run to complete in an orderly manner. I also want stop to return without any issues if run hasn't even be called. How should I do this?
I create this thread in a setup() method in my test environment and run stop on it in the teardown(). However, in some tests I dont call run().
UPDATE
Here's my second attempt. Is it correct now?
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
def run(self):
while not self._finished:
print("*")
time.sleep(1)
print("Finished Other")
def finish(self):
self._finished = True
self.join()
m = MyThread()
m.start()
print("After")
time.sleep(5)
m.finish()
print("Finished Main")
You do not need to and should not implement this yourself. What you are looking for already exists, at least in large parts. It is, however, not called "stop". The concept you are describing is usually called "join".
Have a look at the documentation for join: https://docs.python.org/3.4/library/threading.html#threading.Thread.join
You write
The stop method should wait for run to complete in an orderly manner.
Join's documentation says: "Wait until the thread terminates." check ✓
You write
I also want stop to return without any issues if run hasn't even be
called
Join's documentation says: "It is also an error to join() a thread before it has been started"
So, the only thing you need to make sure is that you call join() only after you have started the thread via the start() method. That should be easy for you.
I am new to python programming. I am trying to make a GUI with stoppable threads.
I borrowed some code from
https://stackoverflow.com/a/325528
class MyThread(threading.Thread):
"""Thread class with a stop() method. The thread itself has to check
regularly for the stopped() condition."""
def __init__(self, *args, **kwargs):
super(MyThread, self).__init__(*args, **kwargs)
self._stop = threading.Event()
def stop(self):
self._stop.set()
def stopped(self):
return self._stop.isSet()
I have function which creates a thread for another function in another class that runs an infinite loop.
class MyClass :
def clicked_practice(self):
self.practicethread = MyThread(target=self.infinite_loop_method)
self.practicethread.start()
def infinite_loop_method()
while True :
// Do something
#This doesn't seem to work and I am still stuck in the loop
def infinite_stop(self)
if self.practicethread.isAlive():
self.practicethread.stop()
I want to create a method to stop this thread .
What's happening here?
I think you missed the 'The thread itself has to check regularly for the stopped() condition' bit of that documentation.
Your thread needs to run like this:
while not self.stopped():
# do stuff
rather than while true. Note that it is still only going to exit at the 'start' of a loop, when it checks the condition. If whatever is in that loop is long-running, that may cause unexpected delays.
import threading
import time
class MultiThreading:
def __init__(self):
self.thread = None
self.started = True
def threaded_program(self):
while self.started:
print("running")
# time.sleep(10)
def run(self):
self.thread = threading.Thread(target=self.threaded_program, args=())
self.thread.start()
def stop(self):
self.started = False
self.thread.join()