I am really having a hard time wrapping my head around multithreading in python. My expectation of the following code is that appLoop() will run for 10 seconds and the cease to exist -- which it does when tracing through in PyCharm, but not, when I just run it. This results in an infinite loop.
import time
import multiprocessing
isRunning = True
runningSince = 0
def appLoop():
try:
global isRunning
while isRunning:
time.sleep(1)
global runningSince
runningSince = runningSince + 1
print(f'Looping since {runningSince} seconds.')
except KeyboardInterrupt:
print(f'appLoop stopped after {runningSince} seconds.')
class Process:
class __Process:
def __init__(self):
self.process = multiprocessing.Process(target=appLoop)
self.process.start()
instance = None
def __init__(self):
if not Process.instance:
Process.instance = Process.__Process()
def __del__(self):
print('Instance deleted.')
p = Process()
time.sleep(10)
isRunning = False
print(f'isRunning set to False.')
del p
This brings up (at least...) 2 questions for me:
why is process still running after del p -- am I creating a zombie process here?
why does my appLoop() keep running even after I set isRunning to false when I run the app (according to my observations this works when tracing through the code as mentioned above)?
My use case in the end is to be able to start / stop my appLoop() from a flask web interface -- which is why I am trying to implement a singleton here. Just in case you might wonder...
And: I do know that __del__ is not recommended as you never know when exactly garbage collection will call it -- in this case I just use it for (cave man) debugging.
isRunning = False changes the value of the variable in the parent process. The child process (the one that executes the while loop) has its own copy of isRunning that is not affected by the assignment.
For the same reason del p does not terminate the process: because it has its own copy of p. You should terminate the process explicitly in the destructor:
def __del__(self):
print('Instance deleted.')
Process.instance.process.terminate()
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
self._end = False
self._running = False
def run(self):
self._running = True
while not self._finished:
time.sleep(0.05)
self._end = True
def stop(self):
if not self._running:
return
self._finished = True
while not self._end:
time.sleep(0.05)
I wish to have a thread on which I can call run() and stop(). The stop method should wait for run to complete in an orderly manner. I also want stop to return without any issues if run hasn't even be called. How should I do this?
I create this thread in a setup() method in my test environment and run stop on it in the teardown(). However, in some tests I dont call run().
UPDATE
Here's my second attempt. Is it correct now?
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
def run(self):
while not self._finished:
print("*")
time.sleep(1)
print("Finished Other")
def finish(self):
self._finished = True
self.join()
m = MyThread()
m.start()
print("After")
time.sleep(5)
m.finish()
print("Finished Main")
You do not need to and should not implement this yourself. What you are looking for already exists, at least in large parts. It is, however, not called "stop". The concept you are describing is usually called "join".
Have a look at the documentation for join: https://docs.python.org/3.4/library/threading.html#threading.Thread.join
You write
The stop method should wait for run to complete in an orderly manner.
Join's documentation says: "Wait until the thread terminates." check ✓
You write
I also want stop to return without any issues if run hasn't even be
called
Join's documentation says: "It is also an error to join() a thread before it has been started"
So, the only thing you need to make sure is that you call join() only after you have started the thread via the start() method. That should be easy for you.
What's the proper way to tell a looping thread to stop looping?
I have a fairly simple program that pings a specified host in a separate threading.Thread class. In this class it sleeps 60 seconds, the runs again until the application quits.
I'd like to implement a 'Stop' button in my wx.Frame to ask the looping thread to stop. It doesn't need to end the thread right away, it can just stop looping once it wakes up.
Here is my threading class (note: I haven't implemented looping yet, but it would likely fall under the run method in PingAssets)
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
def run(self):
config = controller.getConfig()
fmt = config['timefmt']
start_time = datetime.now().strftime(fmt)
try:
if onlinecheck.check_status(self.asset):
status = "online"
else:
status = "offline"
except socket.gaierror:
status = "an invalid asset tag."
msg =("{}: {} is {}. \n".format(start_time, self.asset, status))
wx.CallAfter(self.window.Logger, msg)
And in my wxPyhton Frame I have this function called from a Start button:
def CheckAsset(self, asset):
self.count += 1
thread = PingAssets(self.count, asset, self)
self.threads.append(thread)
thread.start()
Threaded stoppable function
Instead of subclassing threading.Thread, one can modify the function to allow
stopping by a flag.
We need an object, accessible to running function, to which we set the flag to stop running.
We can use threading.currentThread() object.
import threading
import time
def doit(arg):
t = threading.currentThread()
while getattr(t, "do_run", True):
print ("working on %s" % arg)
time.sleep(1)
print("Stopping as you wish.")
def main():
t = threading.Thread(target=doit, args=("task",))
t.start()
time.sleep(5)
t.do_run = False
if __name__ == "__main__":
main()
The trick is, that the running thread can have attached additional properties. The solution builds
on assumptions:
the thread has a property "do_run" with default value True
driving parent process can assign to started thread the property "do_run" to False.
Running the code, we get following output:
$ python stopthread.py
working on task
working on task
working on task
working on task
working on task
Stopping as you wish.
Pill to kill - using Event
Other alternative is to use threading.Event as function argument. It is by
default False, but external process can "set it" (to True) and function can
learn about it using wait(timeout) function.
We can wait with zero timeout, but we can also use it as the sleeping timer (used below).
def doit(stop_event, arg):
while not stop_event.wait(1):
print ("working on %s" % arg)
print("Stopping as you wish.")
def main():
pill2kill = threading.Event()
t = threading.Thread(target=doit, args=(pill2kill, "task"))
t.start()
time.sleep(5)
pill2kill.set()
t.join()
Edit: I tried this in Python 3.6. stop_event.wait() blocks the event (and so the while loop) until release. It does not return a boolean value. Using stop_event.is_set() works instead.
Stopping multiple threads with one pill
Advantage of pill to kill is better seen, if we have to stop multiple threads
at once, as one pill will work for all.
The doit will not change at all, only the main handles the threads a bit differently.
def main():
pill2kill = threading.Event()
tasks = ["task ONE", "task TWO", "task THREE"]
def thread_gen(pill2kill, tasks):
for task in tasks:
t = threading.Thread(target=doit, args=(pill2kill, task))
yield t
threads = list(thread_gen(pill2kill, tasks))
for thread in threads:
thread.start()
time.sleep(5)
pill2kill.set()
for thread in threads:
thread.join()
This has been asked before on Stack. See the following links:
Is there any way to kill a Thread in Python?
Stopping a thread after a certain amount of time
Basically you just need to set up the thread with a stop function that sets a sentinel value that the thread will check. In your case, you'll have the something in your loop check the sentinel value to see if it's changed and if it has, the loop can break and the thread can die.
I read the other questions on Stack but I was still a little confused on communicating across classes. Here is how I approached it:
I use a list to hold all my threads in the __init__ method of my wxFrame class: self.threads = []
As recommended in How to stop a looping thread in Python? I use a signal in my thread class which is set to True when initializing the threading class.
class PingAssets(threading.Thread):
def __init__(self, threadNum, asset, window):
threading.Thread.__init__(self)
self.threadNum = threadNum
self.window = window
self.asset = asset
self.signal = True
def run(self):
while self.signal:
do_stuff()
sleep()
and I can stop these threads by iterating over my threads:
def OnStop(self, e):
for t in self.threads:
t.signal = False
I had a different approach. I've sub-classed a Thread class and in the constructor I've created an Event object. Then I've written custom join() method, which first sets this event and then calls a parent's version of itself.
Here is my class, I'm using for serial port communication in wxPython app:
import wx, threading, serial, Events, Queue
class PumpThread(threading.Thread):
def __init__ (self, port, queue, parent):
super(PumpThread, self).__init__()
self.port = port
self.queue = queue
self.parent = parent
self.serial = serial.Serial()
self.serial.port = self.port
self.serial.timeout = 0.5
self.serial.baudrate = 9600
self.serial.parity = 'N'
self.stopRequest = threading.Event()
def run (self):
try:
self.serial.open()
except Exception, ex:
print ("[ERROR]\tUnable to open port {}".format(self.port))
print ("[ERROR]\t{}\n\n{}".format(ex.message, ex.traceback))
self.stopRequest.set()
else:
print ("[INFO]\tListening port {}".format(self.port))
self.serial.write("FLOW?\r")
while not self.stopRequest.isSet():
msg = ''
if not self.queue.empty():
try:
command = self.queue.get()
self.serial.write(command)
except Queue.Empty:
continue
while self.serial.inWaiting():
char = self.serial.read(1)
if '\r' in char and len(msg) > 1:
char = ''
#~ print('[DATA]\t{}'.format(msg))
event = Events.PumpDataEvent(Events.SERIALRX, wx.ID_ANY, msg)
wx.PostEvent(self.parent, event)
msg = ''
break
msg += char
self.serial.close()
def join (self, timeout=None):
self.stopRequest.set()
super(PumpThread, self).join(timeout)
def SetPort (self, serial):
self.serial = serial
def Write (self, msg):
if self.serial.is_open:
self.queue.put(msg)
else:
print("[ERROR]\tPort {} is not open!".format(self.port))
def Stop(self):
if self.isAlive():
self.join()
The Queue is used for sending messages to the port and main loop takes responses back. I've used no serial.readline() method, because of different end-line char, and I have found the usage of io classes to be too much fuss.
Depends on what you run in that thread.
If that's your code, then you can implement a stop condition (see other answers).
However, if what you want is to run someone else's code, then you should fork and start a process. Like this:
import multiprocessing
proc = multiprocessing.Process(target=your_proc_function, args=())
proc.start()
now, whenever you want to stop that process, send it a SIGTERM like this:
proc.terminate()
proc.join()
And it's not slow: fractions of a second.
Enjoy :)
My solution is:
import threading, time
def a():
t = threading.currentThread()
while getattr(t, "do_run", True):
print('Do something')
time.sleep(1)
def getThreadByName(name):
threads = threading.enumerate() #Threads list
for thread in threads:
if thread.name == name:
return thread
threading.Thread(target=a, name='228').start() #Init thread
t = getThreadByName('228') #Get thread by name
time.sleep(5)
t.do_run = False #Signal to stop thread
t.join()
I find it useful to have a class, derived from threading.Thread, to encapsulate my thread functionality. You simply provide your own main loop in an overridden version of run() in this class. Calling start() arranges for the object’s run() method to be invoked in a separate thread.
Inside the main loop, periodically check whether a threading.Event has been set. Such an event is thread-safe.
Inside this class, you have your own join() method that sets the stop event object before calling the join() method of the base class. It can optionally take a time value to pass to the base class's join() method to ensure your thread is terminated in a short amount of time.
import threading
import time
class MyThread(threading.Thread):
def __init__(self, sleep_time=0.1):
self._stop_event = threading.Event()
self._sleep_time = sleep_time
"""call base class constructor"""
super().__init__()
def run(self):
"""main control loop"""
while not self._stop_event.isSet():
#do work
print("hi")
self._stop_event.wait(self._sleep_time)
def join(self, timeout=None):
"""set stop event and join within a given time period"""
self._stop_event.set()
super().join(timeout)
if __name__ == "__main__":
t = MyThread()
t.start()
time.sleep(5)
t.join(1) #wait 1s max
Having a small sleep inside the main loop before checking the threading.Event is less CPU intensive than looping continuously. You can have a default sleep time (e.g. 0.1s), but you can also pass the value in the constructor.
Sometimes you don't have control over the running target. In those cases you can use signal.pthread_kill to send a stop signal.
from signal import pthread_kill, SIGTSTP
from threading import Thread
from itertools import count
from time import sleep
def target():
for num in count():
print(num)
sleep(1)
thread = Thread(target=target)
thread.start()
sleep(5)
pthread_kill(thread.ident, SIGTSTP)
result
0
1
2
3
4
[14]+ Stopped
I have a function foo that only stops once a condition has been met. While foo is running, I need to ask for user input the whole time (keeps asking for user input). I want them to run separately without interfering with each other.
In my example below, foo keeps printing 'Hello' and getUserInput keeps looking for user input. I want foo to keep printing hello even if i do not enter anything for user input. It will keep asking for input as long as the user does not enter letter 'e'. I have my attempt below:
import threading
from time import sleep
class test:
def __init__(self):
self.running = True
def foo(self):
while(self.running):
print 'Hello\n'
sleep(2)
def getUserInput(self):
x = ''
while(x != 'e'):
x = raw_input('Enter value: ')
self.running = False
def go(self):
th1 = threading.Thread(target=self.foo)
th2 = threading.Thread(target=self.getUserInput)
th1.start()
th2.start()
t = test()
t.go()
My code prints out the first hello and asks for input but nothing after that. What am I doing wrong? Thanks for your help in advance.
Update: The opener was running his code on Windows in IDLE. Regarding I/O it behaves differently than a shell or the Windows command line. His code works on the Windows command line.
In principle, your code works for me. I am running Python 2.6.5.
Several comments here:
1) In your case it would be fine to only have two threads: the main thread and another one. However, it will also work with three. It's just that your main thread does nothing else than waiting for the other threads to finish.
2) You should to explicitly join() all threads you spawn. You do this in the main thread before terminating it. Keep record of the threads you spawn (e.g. in a list threads) and then join them at the end of your program (e.g. for t in threads: t.join()).
3) You share the variable self.running between threads. It is fine in this case, as one thread only reads it and another one only writes it. In general, you need to be very careful with shared variables and acquire a lock before changing it.
4) You should catch the KeyboardInterrupt exception in the main thread and find a way to communicate to your other threads to terminate :)
5) Use lowercase method names, so instead of getUserInput call it get_user_input. Use uppercase class names and inherit from object: class Test(object):
This is a running example:
import threading
from time import sleep
def main():
t = Test()
t.go()
try:
join_threads(t.threads)
except KeyboardInterrupt:
print "\nKeyboardInterrupt catched."
print "Terminate main thread."
print "If only daemonic threads are left, terminate whole program."
class Test(object):
def __init__(self):
self.running = True
self.threads = []
def foo(self):
while(self.running):
print '\nHello\n'
sleep(2)
def get_user_input(self):
while True:
x = raw_input("Enter 'e' for exit: ")
if x.lower() == 'e':
self.running = False
break
def go(self):
t1 = threading.Thread(target=self.foo)
t2 = threading.Thread(target=self.get_user_input)
# Make threads daemonic, i.e. terminate them when main thread
# terminates. From: http://stackoverflow.com/a/3788243/145400
t1.daemon = True
t2.daemon = True
t1.start()
t2.start()
self.threads.append(t1)
self.threads.append(t2)
def join_threads(threads):
"""
Join threads in interruptable fashion.
From http://stackoverflow.com/a/9790882/145400
"""
for t in threads:
while t.isAlive():
t.join(5)
if __name__ == "__main__":
main()
When typing e or E, the program ends after a short delay (as intended by you). When pressing ctrl+c, it immediately terminates. Making a program that uses threading responsive to exceptions is a bit trickier than expected. I have included important references in the source above.
This is how it looks like during runtime:
$ python supertest.py
Hello
Enter 'e' for exit:
Hello
Hello
Hello
e
$
I want to force threads termination in python: I don't want to set an event and wait until the thread checks it and exits. I'm looking for a simple solution like kill -9. Is this possible to do that without dirty hacks like operating with private methods etc.?
If you do not mind your code running about ten times slower, you can use the Thread2 class implemented below. An example follows that shows how calling the new stop method should kill the thread on the next bytecode instruction.
import threading
import sys
class StopThread(StopIteration): pass
threading.SystemExit = SystemExit, StopThread
class Thread2(threading.Thread):
def stop(self):
self.__stop = True
def _bootstrap(self):
if threading._trace_hook is not None:
raise ValueError('Cannot run thread with tracing!')
self.__stop = False
sys.settrace(self.__trace)
super()._bootstrap()
def __trace(self, frame, event, arg):
if self.__stop:
raise StopThread()
return self.__trace
class Thread3(threading.Thread):
def _bootstrap(self, stop_thread=False):
def stop():
nonlocal stop_thread
stop_thread = True
self.stop = stop
def tracer(*_):
if stop_thread:
raise StopThread()
return tracer
sys.settrace(tracer)
super()._bootstrap()
################################################################################
import time
def main():
test = Thread2(target=printer)
test.start()
time.sleep(1)
test.stop()
test.join()
def printer():
while True:
print(time.time() % 1)
time.sleep(0.1)
if __name__ == '__main__':
main()
The Thread3 class appears to run code approximately 33% faster than the Thread2 class.
Threads end when they do.
You can signal a thread that you want it to terminate ASAP, but that assumes collaboration of the code running in a thread, and it offers no upper bound guarantee for when that happens.
A classic way is to use a variable like exit_immediately = False and have threads' main routines periodically check it and terminate if the value is True. To have the threads exit, you set exit_immediately = True and call .join() on all threads. Obviously, this works only when threads are able to check in periodically.
If what you want is to just be able to let the program terminate at its end without caring about what happens to some threads, what you want is daemon threads.
From the docs:
The entire Python program exits when no alive non-daemon threads are
left.
Example usage program:
import threading
import time
def test():
while True:
print "hey"
time.sleep(1)
t = threading.Thread(target=test)
t.daemon = True # <-- set to False and the program will not terminate
t.start()
time.sleep(10)
Trivia: daemon threads are referred to as background threads in .Net.