I have the following code in python:
class gateWay:
def __init__(self):
self.var1 = []
self.var2 = {}
self.currentThread = None
def stateProcess(self, file):
# some irrelevant code
self.currentThread = saltGatWayThread(self, file).start()
return self.var1
def stopRunning(self):
self.currentThread.proc.stop()
In addition, here the source code of the saltGatWayThread:
class saltGatWayThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
# some irrelevant code
self.proc = src.proc.Process1()
In addition, I have the following code in src/proc/__init__.py:
class Process1:
def stop(self):
# code to stop operation
In the console, I notice that self.currentThread is null.
My purpose is to save the thread in local variable, when start it. If I get an abort request, I apply
stopRunning function. This function, would take the saved thread and will do "clean" exit (finish the process of the tread and exit).
Why can't I save the thread, and use the structure of it later on?
invoke currentThread = saltGatWayThread() and then call .start(). currentThread does not contains thread instance because starts() method always returns nothing according to the threading.py source code. See source of C:\Python27\Lib\threading.py
def start(self):
"""Start the thread's activity.
It must be called at most once per thread object. It arranges for the
object's run() method to be invoked in a separate thread of control.
This method will raise a RuntimeError if called more than once on the
same thread object.
"""
if not self.__initialized:
raise RuntimeError("thread.__init__() not called")
if self.__started.is_set():
raise RuntimeError("threads can only be started once")
if __debug__:
self._note("%s.start(): starting thread", self)
with _active_limbo_lock:
_limbo[self] = self
try:
_start_new_thread(self.__bootstrap, ())
except Exception:
with _active_limbo_lock:
del _limbo[self]
raise
self.__started.wait()
Related
I want to have a main program that works like a console from where I can call other processes (infinite loops) and kill them selectively whenever certain commands are entered.
For that I created this class:
class RunInThread(threading.Thread):
def __init__(self, function):
self.function = function
self.kill_pill = threading.Event()
threading.Thread.__init__(self)
def start(self): # This is controversial.
self.__init__(self.function)
threading.Thread.start(self)
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
The documentation for thread.Thread says that only the __init__() and run() methods should be overridden.
Is there any clear issue with my code? It works the way I intended but since it's going to be running for long periods of time I need to make sure I'm not creating any memory problems.
EDIT:
What about this solution?:
class StoppableThread(threading.Thread):
# threading.Thread class but can be stopped with the stop() method.
def __init__(self, function):
threading.Thread.__init__(self)
self.function = function
self.kill_pill = threading.Event()
def stop(self):
self.kill_pill.set()
def run(self):
while not self.kill_pill.is_set():
self.function()
class RunInThread():
def __init__(self, function, prnt=False):
self.function = function
self.running = False
self.prnt = prnt
def start(self):
if not self.running:
self.thread = StoppableThread(self.function)
self.thread.start()
self.running = True
else:
if self.prnt:
print('Thread already running.')
def stop(self):
self.thread.stop()
self.running = False
If you want to find out what things that could break, I'd suggest looking into the implementation of Thread class.
Among other things, Thread.__init__() initialises an Event() object to detect thread startup and shutdown, manages cleanup hooks/callbacks, some internal lock objects, and registers the thread to a list so you can introspect running threads. By calling Thread.__init__(), these variables gets reinitialised and screws up the internal mechanisms of many of these functionalities.
What could go wrong? I didn't test any of these, but from skimming through threading.py, these are likely some of the things that I expect could go wrong:
your python process now will be running a OS threads that doesn't show up in enumerate_thread()
multiple OS thread will now return the same Thread object when it calls current_thread(), which will likely also break threadlocal and anything that depends on threadlocal
Thread.join() depends on some internal locks, which likely would now become thread unsafe to call
Unhandled reception can go to the wrong exception hook handler
register_at_fork and shutdown handler likely will get confused
In other words, don't try to be sneaky. Create a new Thread object for each thread you want to start.
There's a good reason that the Thread class spent efforts trying to prevent you from accidentally calling start() twice. Don't try to subvert this.
I know this sounds a lot like this similarly-worded question, but there are differences, so bear with me.
I'm trying to create a reusable "Timer" class which calls a specified callback every N seconds, until you call stop. As inspiration, I used the link above, with a built-in event wrapped in a stop method. Here's how the basic class looks:
import time
import threading
from threading import Thread
from threading import Event
# Mostly inspired by https://stackoverflow.com/questions/12435211/python-threading-timer-repeat-function-every-n-seconds
class RepeatingTimer(Thread):
def __init__(self, interval_seconds, callback):
Thread.__init__(self)
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
self.setDaemon(True)
def start(self):
while not self.stop_event.wait(self.interval_seconds):
self.callback()
time.sleep(0) # doesn't seem to do anything
def stop(self):
self.stop_event.set()
Looks good, even includes time.sleep(0) based on this question.
It doesn't do what I thought; the call to start never seems to return or yield, ever. Consider this use-case:
def print_status(message):
print(message)
def print_r1():
print_status("R1")
def print_r2():
print_status("R2")
r1 = RepeatingTimer(1, print_r1)
r2 = RepeatingTimer(0.5, print_r2)
r1.start()
r2.start()
The call to r1.start never terminates. It continues on forever. The output on the console, after four seconds, is:
R1
R1
R1
R1
This prompted me to introduce the time.sleep(0) call, although that doesn't seem to do anything.
I also tried with and without self.setDaemon(True), but that also seems to have no effect.
I also tried converting this into two classes: one with just the event wrappers (a StoppableTimer class), and another that simply creates and recreates the StoppableTimer in the callback, but that doesn't work either. Here's what it looks like:
class StoppableTimer(Thread):
def __init__(self, interval_seconds, callback):
Thread.__init__(self)
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
self.setDaemon(True)
def start(self):
time.sleep(self.interval_seconds)
self.callback()
def stop(self):
self.stop_event.set()
class RepeatingTimer:
def __init__(self, interval_seconds, callback):
self.interval_seconds = interval_seconds
self.callback = callback
self.timer = StoppableTimer(interval_seconds, self.refresh_timer)
def start(self):
self.timer.start()
def stop(self):
self.timer.stop()
def refresh_timer(self):
self.stop()
self.callback()
self.timer = StoppableTimer(self.interval_seconds, self.refresh_timer)
self.timer.start()
I'm completely at a loss on how to make this work. I'm also mostly a beginner to Python, so please add sufficient explanation to your answer so I can grasp what the fundamental issue is.
I also read a bit about the Global Interpreter Lock on SO, but I don't understand how that could be an issue.
For reference, I'm running Python 3.6.3 on Ubuntu 17.10
Short answer :
Don't override start(). Override run() instead.
Long answer because you're asking for details :
With the class definition in your first snippet, you've created a class which inherits from Thread, however you've overriden the start() method supposed to start your thread by a new method which is looping until the stop_event is set, that is to say, the method supposed to actually start your thread doesn't do this anymore.
So, when you try to start your thread, you actually run the loop calling your callback function in your current and only thread. And since it's an infinite loop, your second "thread" is not started, and you have no way to "stop" it.
You mustn't override start (well not in this way). Instead, override the run method. This is the method that will be run by your thread when you start it.
Also, you should do super().__init__() instead of Thread.__init__(self). The first one is the proper way to call an inherited method in Python.
class RepeatingTimer(Thread):
def __init__(self, interval_seconds, callback):
super().__init__()
self.stop_event = Event()
self.interval_seconds = interval_seconds
self.callback = callback
def run(self):
while not self.stop_event.wait(self.interval_seconds):
self.callback()
def stop(self):
self.stop_event.set()
And with the functions you've defined you can do :
r1 = RepeatingTimer(1, print_r1)
r2 = RepeatingTimer(0.5, print_r2)
r1.start()
r2.start()
time.sleep(4)
r1.stop()
r2.stop()
Here is the relevant documentation for Thread.
I'm trying to create a subclass of threading.Thread whose methods are threaded. I'm using it for video, but I suspect that a working example will be generally useful for folks.
I realized here that I never instantiated a thread and never called the start() method, but I don't know where to call it from or how. I also want to save the thread handle so I can stop it if I receive a stop() signal.
import threading
class VideoThread(threading.Thread):
"""Thread class with a stop() method. The thread itself checks
regularly for the stopped() condition."""
def __init__(self, playlist=None):
super(VideoThread, self).__init__()
self._stop = threading.Event()
self._player_pgid_list = []
if playlist:
self.start_sequence(playlist)
def stop(self):
self._stop.set()
def stopped(self):
return self._stop.isSet()
def start_sequence(self, playlist):
if not isinstance(playlist, list):
raise ValueError("Expecting a list")
for video in playlist:
if not self.stopped():
self.__start_video__(video)
def __start_video__(self, video):
if not isinstance(video, dict):
raise ValueError("Expecting a dictionary of video data")
# start the video
# store the video pgid so we can kill it if we have to
# tight wait loop to check for stopped condition
# kill all video(s) if necessary using the stored pgids
The class works as far as it goes, but of course, none of the methods are actually threaded.
start_sequence() is public so I can start a threaded sequence of videos like this:
video = VideoThread()
video.start_sequence([films[1], films[3], films[2]])
Or when I instantiate the class like this:
video = VideoThread([films[1], films[3], films[2]])
Later, if I need to stop it, I can:
video.stop()
What am I missing?
You should rename the start_sequence method to run and delete the playlist parameter (use self.playlist instead). Also, delete those two last lines in __init__ method. I mean:
class VideoThread(threading.Thread):
def __init__(self, playlist=None):
super().__init__()
self._stop = threading.Event()
self._player_pgid_list = []
self.playlist = playlist
def run(self):
if not isinstance(self.playlist, list):
raise ValueError("Expecting a list")
for video in self.playlist:
if not self.stopped():
self.__start_video__(video)
...
Then, to use your class just do:
playlist = VideoThread(films)
playlist.start()
And you can stop it using:
playlist.stop()
Note that when you call .start, it invokes the run method in a separate thread of control, check the official documentation for more information.
I am using python 2.7 and new to Threading. I got a class file and run method. But I don't see the run method invoked when I create instances of thread. I am also planning to use subprocess.Popen inside the run method and get stdout of the process for each filename and print the output.
Please tell me what I am missing here for run method to be called.
class FileScanThread(threading.Thread):
def __init__(self, myFileName):
print("In File Scan Thread")
self.mapFile = myFileName
#myjar=myFileName
self.start()
def run(self):
print self.mapFile
x= FileScanThread("myfile.txt")
you're forgetting to call the mother class constructor to specify target. It's not java, and run has no particular meaning. By default, target is None and the thread does nothing.
import threading
class FileScanThread(threading.Thread):
def __init__(self, myFileName):
threading.Thread.__init__(self,target=self.run)
# another syntax uses "super", which is simpler in python 3
# super().__init__(target=self.run)
print("In File Scan Thread")
self.mapFile = myFileName
#myjar=myFileName
self.start()
def run(self):
print(self.mapFile)
x= FileScanThread("myfile.txt")
x.join() # when you're done
This will do what you want. You aren't calling __init__ from the class Thread.
class FileScanThread(threading.Thread):
def __init__(self, myFileName):
threading.Thread.__init__(self)
print("In File Scan Thread")
self.mapFile = myFileName
#myjar=myFileName
self.start()
def run(self):
print self.mapFile
x = FileScanThread("myfile.txt")
I don't think you have to pass target argument to it. At least that's not usually how I do it.
Output:
In File Scan Thread
myfile.txt
I have a Manager (main thread), that creates other Threads to handle various operations.
I would like my Manager to be notified when a Thread it created ends (when run() method execution is finished).
I know I could do it by checking the status of all my threads with the Thread.isActive() method, but polling sucks, so I wanted to have notifications.
I was thinking of giving a callback method to the Threads, and call this function at the end of the run() method:
class Manager():
...
MyThread(self.on_thread_finished).start() # How do I pass the callback
def on_thread_finished(self, data):
pass
...
class MyThread(Thread):
...
def run(self):
....
self.callback(data) # How do I call the callback?
...
Thanks!
The thread can't call the manager unless it has a reference to the manager. The easiest way for that to happen is for the manager to give it to the thread at instantiation.
class Manager(object):
def new_thread(self):
return MyThread(parent=self)
def on_thread_finished(self, thread, data):
print thread, data
class MyThread(Thread):
def __init__(self, parent=None):
self.parent = parent
super(MyThread, self).__init__()
def run(self):
# ...
self.parent and self.parent.on_thread_finished(self, 42)
mgr = Manager()
thread = mgr.new_thread()
thread.start()
If you want to be able to assign an arbitrary function or method as a callback, rather than storing a reference to the manager object, this becomes a bit problematic because of method wrappers and such. It's hard to design the callback so it gets a reference to both the manager and the thread, which is what you will want. I worked on that for a while and did not come up with anything I'd consider useful or elegant.
Anything wrong with doing it this way?
from threading import Thread
class Manager():
def Test(self):
MyThread(self.on_thread_finished).start()
def on_thread_finished(self, data):
print "on_thread_finished:", data
class MyThread(Thread):
def __init__(self, callback):
Thread.__init__(self)
self.callback = callback
def run(self):
data = "hello"
self.callback(data)
m = Manager()
m.Test() # prints "on_thread_finished: hello"
If you want the main thread to wait for children threads to finish execution, you are probably better off using some kind of synchronization mechanism. If simply being notified when one or more threads has finished executing, a Condition is enough:
import threading
class MyThread(threading.Thread):
def __init__(self, condition):
threading.Thread.__init__(self)
self.condition = condition
def run(self):
print "%s done" % threading.current_thread()
with self.condition:
self.condition.notify()
condition = threading.Condition()
condition.acquire()
thread = MyThread(condition)
thread.start()
condition.wait()
However, using a Queue is probably better, as it makes handling multiple worker threads a bit easier.