I want to have a main program that works like a console from where I can call other processes (infinite loops) and kill them selectively whenever certain commands are entered.
For that I created this class:
class RunInThread(threading.Thread):
def __init__(self, function):
self.function = function
self.kill_pill = threading.Event()
threading.Thread.__init__(self)
def start(self): # This is controversial.
self.__init__(self.function)
threading.Thread.start(self)
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
The documentation for thread.Thread says that only the __init__() and run() methods should be overridden.
Is there any clear issue with my code? It works the way I intended but since it's going to be running for long periods of time I need to make sure I'm not creating any memory problems.
EDIT:
What about this solution?:
class StoppableThread(threading.Thread):
# threading.Thread class but can be stopped with the stop() method.
def __init__(self, function):
threading.Thread.__init__(self)
self.function = function
self.kill_pill = threading.Event()
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
class RunInThread():
def __init__(self, function, prnt=False):
self.function = function
self.running = False
self.prnt = prnt
def start(self):
if not self.running:
self.thread = StoppableThread(self.function)
self.thread.start()
self.running = True
else:
if self.prnt:
print('Thread already running.')
def stop(self):
self.thread.stop()
self.running = False
If you want to find out what things that could break, I'd suggest looking into the implementation of Thread class.
Among other things, Thread.__init__() initialises an Event() object to detect thread startup and shutdown, manages cleanup hooks/callbacks, some internal lock objects, and registers the thread to a list so you can introspect running threads. By calling Thread.__init__(), these variables gets reinitialised and screws up the internal mechanisms of many of these functionalities.
What could go wrong? I didn't test any of these, but from skimming through threading.py, these are likely some of the things that I expect could go wrong:
your python process now will be running a OS threads that doesn't show up in enumerate_thread()
multiple OS thread will now return the same Thread object when it calls current_thread(), which will likely also break threadlocal and anything that depends on threadlocal
Thread.join() depends on some internal locks, which likely would now become thread unsafe to call
Unhandled reception can go to the wrong exception hook handler
register_at_fork and shutdown handler likely will get confused
In other words, don't try to be sneaky. Create a new Thread object for each thread you want to start.
There's a good reason that the Thread class spent efforts trying to prevent you from accidentally calling start() twice. Don't try to subvert this.
I know this sounds a lot like this similarly-worded question, but there are differences, so bear with me.
I'm trying to create a reusable "Timer" class which calls a specified callback every N seconds, until you call stop. As inspiration, I used the link above, with a built-in event wrapped in a stop method. Here's how the basic class looks:
import time
import threading
from threading import Thread
from threading import Event
# Mostly inspired by https://stackoverflow.com/questions/12435211/python-threading-timer-repeat-function-every-n-seconds
class RepeatingTimer(Thread):
def __init__(self, interval_seconds, callback):
Thread.__init__(self)
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
self.setDaemon(True)
def start(self):
while not self.stop_event.wait(self.interval_seconds):
self.callback()
time.sleep(0) # doesn't seem to do anything
def stop(self):
self.stop_event.set()
Looks good, even includes time.sleep(0) based on this question.
It doesn't do what I thought; the call to start never seems to return or yield, ever. Consider this use-case:
def print_status(message):
print(message)
def print_r1():
print_status("R1")
def print_r2():
print_status("R2")
r1 = RepeatingTimer(1, print_r1)
r2 = RepeatingTimer(0.5, print_r2)
r1.start()
r2.start()
The call to r1.start never terminates. It continues on forever. The output on the console, after four seconds, is:
R1
R1
R1
R1
This prompted me to introduce the time.sleep(0) call, although that doesn't seem to do anything.
I also tried with and without self.setDaemon(True), but that also seems to have no effect.
I also tried converting this into two classes: one with just the event wrappers (a StoppableTimer class), and another that simply creates and recreates the StoppableTimer in the callback, but that doesn't work either. Here's what it looks like:
class StoppableTimer(Thread):
def __init__(self, interval_seconds, callback):
Thread.__init__(self)
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
self.setDaemon(True)
def start(self):
time.sleep(self.interval_seconds)
self.callback()
def stop(self):
self.stop_event.set()
class RepeatingTimer:
def __init__(self, interval_seconds, callback):
self.interval_seconds = interval_seconds
self.callback = callback
self.timer = StoppableTimer(interval_seconds, self.refresh_timer)
def start(self):
self.timer.start()
def stop(self):
self.timer.stop()
def refresh_timer(self):
self.stop()
self.callback()
self.timer = StoppableTimer(self.interval_seconds, self.refresh_timer)
self.timer.start()
I'm completely at a loss on how to make this work. I'm also mostly a beginner to Python, so please add sufficient explanation to your answer so I can grasp what the fundamental issue is.
I also read a bit about the Global Interpreter Lock on SO, but I don't understand how that could be an issue.
For reference, I'm running Python 3.6.3 on Ubuntu 17.10
Short answer :
Don't override start(). Override run() instead.
Long answer because you're asking for details :
With the class definition in your first snippet, you've created a class which inherits from Thread, however you've overriden the start() method supposed to start your thread by a new method which is looping until the stop_event is set, that is to say, the method supposed to actually start your thread doesn't do this anymore.
So, when you try to start your thread, you actually run the loop calling your callback function in your current and only thread. And since it's an infinite loop, your second "thread" is not started, and you have no way to "stop" it.
You mustn't override start (well not in this way). Instead, override the run method. This is the method that will be run by your thread when you start it.
Also, you should do super().__init__() instead of Thread.__init__(self). The first one is the proper way to call an inherited method in Python.
class RepeatingTimer(Thread):
def __init__(self, interval_seconds, callback):
super().__init__()
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
def run(self):
while not self.stop_event.wait(self.interval_seconds):
self.callback()
def stop(self):
self.stop_event.set()
And with the functions you've defined you can do :
r1 = RepeatingTimer(1, print_r1)
r2 = RepeatingTimer(0.5, print_r2)
r1.start()
r2.start()
time.sleep(4)
r1.stop()
r2.stop()
Here is the relevant documentation for Thread.
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
self._end = False
self._running = False
def run(self):
self._running = True
while not self._finished:
time.sleep(0.05)
self._end = True
def stop(self):
if not self._running:
return
self._finished = True
while not self._end:
time.sleep(0.05)
I wish to have a thread on which I can call run() and stop(). The stop method should wait for run to complete in an orderly manner. I also want stop to return without any issues if run hasn't even be called. How should I do this?
I create this thread in a setup() method in my test environment and run stop on it in the teardown(). However, in some tests I dont call run().
UPDATE
Here's my second attempt. Is it correct now?
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
def run(self):
while not self._finished:
print("*")
time.sleep(1)
print("Finished Other")
def finish(self):
self._finished = True
self.join()
m = MyThread()
m.start()
print("After")
time.sleep(5)
m.finish()
print("Finished Main")
You do not need to and should not implement this yourself. What you are looking for already exists, at least in large parts. It is, however, not called "stop". The concept you are describing is usually called "join".
Have a look at the documentation for join: https://docs.python.org/3.4/library/threading.html#threading.Thread.join
You write
The stop method should wait for run to complete in an orderly manner.
Join's documentation says: "Wait until the thread terminates." check ✓
You write
I also want stop to return without any issues if run hasn't even be
called
Join's documentation says: "It is also an error to join() a thread before it has been started"
So, the only thing you need to make sure is that you call join() only after you have started the thread via the start() method. That should be easy for you.
My program is designed in the following way:
First part of the program takes real time values from a sensor and plots it using Matplotlib. This has to be done for long durations. And also, it logs information into a database.
The second part is the IP Camera. I have to get the input from an IP Camera and display it. For displaying I am using OpenCV's imshow method. Also, I am storing the video from the IP Camera.
Question: I have the algorithms in place, the problem is I need to run both these in a while loops. The condition is that I cannot exit from any of them. Now threading is a good alternative for this but I have read about the GIL, so how do I go about running two infinite loops?
from multiprocessing import Process
def methodA():
while TRUE:
do something
def methodB():
while TRUE:
do something
p=Process(target=methodA())
p.start()
p1=Process(target=methodB())
p1.start()
Now when I start process p it starts executing, after that how do I start p1 to run simultaneously?
As far as I understood your question, you have two different tasks that you want them to perform continuously. Now regarding your questions:
how do I go about running two infinite loops?
You can create two different threads that will run these infinite loops for you. The first thread will perform your task1 and second one will perform task2.
Also, once I start executing a thread, how do I execute the other
thread when the first thread is running continuously/infinitely?
If you are using two different threads then you don't need to be worried about this issue. If the threads are not sharing any resource then you don't need to worry about this fact.
How ever if you want to stop/pause one thread from the other thread or vice versa then you can implement a mechanism using flags or locks. These questions will help in this case:
Is there any way to kill a Thread in Python?
Why does the python threading.Thread object has 'start', but not 'stop'?
making-a-program-munltithreaded
Sample example using threading:
from threading import Thread
class myClassA(Thread):
def __init__(self):
Thread.__init__(self)
self.daemon = True
self.start()
def run(self):
while True:
print 'A'
class myClassB(Thread):
def __init__(self):
Thread.__init__(self)
self.daemon = True
self.start()
def run(self):
while True:
print 'B'
myClassA()
myClassB()
while True:
pass
For shared resources?
Use Locks for them. Here are some examples. One, two and How to synchronize threads in python?
what if I don't want to run it using classes? How do I do this using only methods?
from threading import Thread
def runA():
while True:
print 'A\n'
def runB():
while True:
print 'B\n'
if __name__ == "__main__":
t1 = Thread(target = runA)
t2 = Thread(target = runB)
t1.setDaemon(True)
t2.setDaemon(True)
t1.start()
t2.start()
while True:
pass
from threading import Thread
import time
class PrintA(Thread):
def __init__(self):
Thread.__init__(self)
self.running = True
def run(self):
while self.running:
print('A')
time.sleep(1)
def stop(self):
self.running = False
class PrintB(Thread):
def __init__(self):
Thread.__init__(self)
self.running = True
def run(self):
while self.running:
print('B')
time.sleep(2)
def stop(self):
self.running = False
a = PrintA()
b = PrintB()
a.start()
b.start()
time.sleep(10)
a.stop()
time.sleep(10)
b.stop()
What's the proper way to tell a looping thread to stop looping?
I have a fairly simple program that pings a specified host in a separate threading.Thread class. In this class it sleeps 60 seconds, the runs again until the application quits.
I'd like to implement a 'Stop' button in my wx.Frame to ask the looping thread to stop. It doesn't need to end the thread right away, it can just stop looping once it wakes up.
Here is my threading class (note: I haven't implemented looping yet, but it would likely fall under the run method in PingAssets)
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
def run(self):
config = controller.getConfig()
fmt = config['timefmt']
start_time = datetime.now().strftime(fmt)
try:
if onlinecheck.check_status(self.asset):
status = "online"
else:
status = "offline"
except socket.gaierror:
status = "an invalid asset tag."
msg =("{}: {} is {}. \n".format(start_time, self.asset, status))
wx.CallAfter(self.window.Logger, msg)
And in my wxPyhton Frame I have this function called from a Start button:
def CheckAsset(self, asset):
self.count += 1
thread = PingAssets(self.count, asset, self)
self.threads.append(thread)
thread.start()
Threaded stoppable function
Instead of subclassing threading.Thread, one can modify the function to allow
stopping by a flag.
We need an object, accessible to running function, to which we set the flag to stop running.
We can use threading.currentThread() object.
import threading
import time
def doit(arg):
t = threading.currentThread()
while getattr(t, "do_run", True):
print ("working on %s" % arg)
time.sleep(1)
print("Stopping as you wish.")
def main():
t = threading.Thread(target=doit, args=("task",))
t.start()
time.sleep(5)
t.do_run = False
if __name__ == "__main__":
main()
The trick is, that the running thread can have attached additional properties. The solution builds
on assumptions:
the thread has a property "do_run" with default value True
driving parent process can assign to started thread the property "do_run" to False.
Running the code, we get following output:
$ python stopthread.py
working on task
working on task
working on task
working on task
working on task
Stopping as you wish.
Pill to kill - using Event
Other alternative is to use threading.Event as function argument. It is by
default False, but external process can "set it" (to True) and function can
learn about it using wait(timeout) function.
We can wait with zero timeout, but we can also use it as the sleeping timer (used below).
def doit(stop_event, arg):
while not stop_event.wait(1):
print ("working on %s" % arg)
print("Stopping as you wish.")
def main():
pill2kill = threading.Event()
t = threading.Thread(target=doit, args=(pill2kill, "task"))
t.start()
time.sleep(5)
pill2kill.set()
t.join()
Edit: I tried this in Python 3.6. stop_event.wait() blocks the event (and so the while loop) until release. It does not return a boolean value. Using stop_event.is_set() works instead.
Stopping multiple threads with one pill
Advantage of pill to kill is better seen, if we have to stop multiple threads
at once, as one pill will work for all.
The doit will not change at all, only the main handles the threads a bit differently.
def main():
pill2kill = threading.Event()
tasks = ["task ONE", "task TWO", "task THREE"]
def thread_gen(pill2kill, tasks):
for task in tasks:
t = threading.Thread(target=doit, args=(pill2kill, task))
yield t
threads = list(thread_gen(pill2kill, tasks))
for thread in threads:
thread.start()
time.sleep(5)
pill2kill.set()
for thread in threads:
thread.join()
This has been asked before on Stack. See the following links:
Is there any way to kill a Thread in Python?
Stopping a thread after a certain amount of time
Basically you just need to set up the thread with a stop function that sets a sentinel value that the thread will check. In your case, you'll have the something in your loop check the sentinel value to see if it's changed and if it has, the loop can break and the thread can die.
I read the other questions on Stack but I was still a little confused on communicating across classes. Here is how I approached it:
I use a list to hold all my threads in the __init__ method of my wxFrame class: self.threads = []
As recommended in How to stop a looping thread in Python? I use a signal in my thread class which is set to True when initializing the threading class.
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
self.signal = True
def run(self):
while self.signal:
do_stuff()
sleep()
and I can stop these threads by iterating over my threads:
def OnStop(self, e):
for t in self.threads:
t.signal = False
I had a different approach. I've sub-classed a Thread class and in the constructor I've created an Event object. Then I've written custom join() method, which first sets this event and then calls a parent's version of itself.
Here is my class, I'm using for serial port communication in wxPython app:
import wx, threading, serial, Events, Queue
class PumpThread(threading.Thread):
def __init__ (self, port, queue, parent):
super(PumpThread, self).__init__()
self.port = port
self.queue = queue
self.parent = parent
self.serial = serial.Serial()
self.serial.port = self.port
self.serial.timeout = 0.5
self.serial.baudrate = 9600
self.serial.parity = 'N'
self.stopRequest = threading.Event()
def run (self):
try:
self.serial.open()
except Exception, ex:
print ("[ERROR]\tUnable to open port {}".format(self.port))
print ("[ERROR]\t{}\n\n{}".format(ex.message, ex.traceback))
self.stopRequest.set()
else:
print ("[INFO]\tListening port {}".format(self.port))
self.serial.write("FLOW?\r")
while not self.stopRequest.isSet():
msg = ''
if not self.queue.empty():
try:
command = self.queue.get()
self.serial.write(command)
except Queue.Empty:
continue
while self.serial.inWaiting():
char = self.serial.read(1)
if '\r' in char and len(msg) > 1:
char = ''
#~ print('[DATA]\t{}'.format(msg))
event = Events.PumpDataEvent(Events.SERIALRX, wx.ID_ANY, msg)
wx.PostEvent(self.parent, event)
msg = ''
break
msg += char
self.serial.close()
def join (self, timeout=None):
self.stopRequest.set()
super(PumpThread, self).join(timeout)
def SetPort (self, serial):
self.serial = serial
def Write (self, msg):
if self.serial.is_open:
self.queue.put(msg)
else:
print("[ERROR]\tPort {} is not open!".format(self.port))
def Stop(self):
if self.isAlive():
self.join()
The Queue is used for sending messages to the port and main loop takes responses back. I've used no serial.readline() method, because of different end-line char, and I have found the usage of io classes to be too much fuss.
Depends on what you run in that thread.
If that's your code, then you can implement a stop condition (see other answers).
However, if what you want is to run someone else's code, then you should fork and start a process. Like this:
import multiprocessing
proc = multiprocessing.Process(target=your_proc_function, args=())
proc.start()
now, whenever you want to stop that process, send it a SIGTERM like this:
proc.terminate()
proc.join()
And it's not slow: fractions of a second.
Enjoy :)
My solution is:
import threading, time
def a():
t = threading.currentThread()
while getattr(t, "do_run", True):
print('Do something')
time.sleep(1)
def getThreadByName(name):
threads = threading.enumerate() #Threads list
for thread in threads:
if thread.name == name:
return thread
threading.Thread(target=a, name='228').start() #Init thread
t = getThreadByName('228') #Get thread by name
time.sleep(5)
t.do_run = False #Signal to stop thread
t.join()
I find it useful to have a class, derived from threading.Thread, to encapsulate my thread functionality. You simply provide your own main loop in an overridden version of run() in this class. Calling start() arranges for the object’s run() method to be invoked in a separate thread.
Inside the main loop, periodically check whether a threading.Event has been set. Such an event is thread-safe.
Inside this class, you have your own join() method that sets the stop event object before calling the join() method of the base class. It can optionally take a time value to pass to the base class's join() method to ensure your thread is terminated in a short amount of time.
import threading
import time
class MyThread(threading.Thread):
def __init__(self, sleep_time=0.1):
self._stop_event = threading.Event()
self._sleep_time = sleep_time
"""call base class constructor"""
super().__init__()
def run(self):
"""main control loop"""
while not self._stop_event.isSet():
#do work
print("hi")
self._stop_event.wait(self._sleep_time)
def join(self, timeout=None):
"""set stop event and join within a given time period"""
self._stop_event.set()
super().join(timeout)
if __name__ == "__main__":
t = MyThread()
t.start()
time.sleep(5)
t.join(1) #wait 1s max
Having a small sleep inside the main loop before checking the threading.Event is less CPU intensive than looping continuously. You can have a default sleep time (e.g. 0.1s), but you can also pass the value in the constructor.
Sometimes you don't have control over the running target. In those cases you can use signal.pthread_kill to send a stop signal.
from signal import pthread_kill, SIGTSTP
from threading import Thread
from itertools import count
from time import sleep
def target():
for num in count():
print(num)
sleep(1)
thread = Thread(target=target)
thread.start()
sleep(5)
pthread_kill(thread.ident, SIGTSTP)
result
0
1
2
3
4
[14]+ Stopped