I have a main thread, which I want non-blocking, (it includes a PyQt5 GUI, I can not use join)
From this thread, I spawn a child thread performing an action.
I want to start an other action from the main thread when the child thread is finished.
I guess I have to generate a signal form the child when it is done and catch it in the main.
Is there a snippet showing the proper way to implement this?
There is already a built-in signal called finished, so you can connect it to your process method:
class NewThread(QtCore.QThread):
def run(self):
pass
class MainWidget(QtWidgets.QWidget):
def after_thread_done(self):
pass
def __init__(self):
self.thread = NewThread()
self.thread.finished.connect(self.after_thread_done)
If you want to emit signal by hand, you can define a new signal:
class NewThread(QtCore.QThread):
new_signal = QtCore.pyqtSignal()
def run(self):
...
self.new_signal.emit()
...
Then connect it to your process method through the same codes.
If you want to define signals in MainWidget, you can pass main_widget as thread's parent.
class NewThread(QtCore.QThread):
def run(self):
...
self.parent().signal.emit()
...
class MainWidget(QtWidgets.QWidget):
new_signal = QtCore.pyqtSignal()
def after_thread_done(self):
print("done")
def __init__(self):
self.new_signal.connect(self.after_thread_done)
self.thread = NewThread(self)
self.thread.start()
Related
I'm trying to understand which of the following approaches is better. I'll use PyQt5 to illustrate the example, as that's the system I'm working with.
We have a Class ControlWidget(QWidget) that provides UI components and spawns helper threads that are just workers executing some task. If one of the workers experiences a failure, we want to notify the parent so it can visually update the user and do some other things like cleanup/disable user input.
Approach 1 would be to send an event from the worker that encounters the error and catch it in the parent instance. The worker Class has an errorSignal that simply indicates the presence of an error.
# The worker class
Class Worker(QObject, Thread):
errorSignal = pyqtSignal(bool)
def __init__(self):
super().__init__()
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
except:
# notify parent that there was an error
errorSignal.emit(True)
self._running = False
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# instantiate the workers
self.worker1 = Worker()
self.worker2 = Worker()
# connect to the error signal
self.worker1.errorSignal.connect(self.handle_error)
self.worker2.errorSignal.connect(self.handle_error)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker1.join()
self.worker2.terminate()
self.worker2.join()
# notify user
self.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
...
Approach 2 would be to make an error_status property on the parent, and pass a reference to each child worker, so it can toggle the error bit directly.
# The worker class
Class Worker(QObject, Thread):
def __init__(self, parent):
super().__init__()
self._running = True
self.parent = parent
def run(self):
while self._running:
try:
# do some work here that might fail
except:
# notify parent that there was an error
self.parent.error_status = True
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# instantiate the workers
self.worker1 = Worker(parent=self)
self.worker2 = Worker(parent=self)
# no signals here :)
# start the workers
self.worker1.start()
self.worker2.start()
#property
def error_status(self):
return self._error_status
#error_status.setter
def error_status(self, value):
self._error_status = value
self.handle_error(value)
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker1.join()
self.worker2.terminate()
self.worker2.join()
# notify user
self.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
...
Events per second limits?
Functionally both approaches will work, however, I suppose I'm worried that if I have a lot of events in my application (I do) I'll overload the event queue and lock up or miss some. I read somewhere that Qt has a limit of about 2 million events per second (although now I can't find the reference). It may just depend on available memory. If I may conceivably reach this limit, isn't it better to try to offload some event handling where possible?
Is approach 2 a better way to handle this type of issue (presuming I've addressed the thread safety issue with my edit)? It does feel slightly gross to pass references to parent/child objects all over the place.
Perhaps this is 6 of one, a half dozen of the other? Any feedback would be appreciated.
Events cannot be missed, they are all processed and eventually discarded if not handled and/or ignored (which are two related but different things).
Both your approaches have flaws, though.
Signals
You should not use terminate() (the docs clearly say that it's dangerous and its use is discouraged).
QThread also doesn't have a join() method, it has wait() which does the same, but you should not use it in the main thread: Qt automatically calls the connected slot in the receiver's thread, and since the signal is connected to handle_error (which is a member of an object in the main thread) the result is that it will block the main thread until the thread has actually finished execution. To avoid that, wait() must be executed in the worker thread, so you can eventually consider a two-signal approach: first react to the error signal, then do something until the thread's finished() signal is finally emitted.
Flag setter
Doing this would only partially work, since the setter is called from the worker thread, so wait() will be correctly called from there. Unfortunately, you're also trying to access the UI in the same function, which is forbidden since UI elements are not thread safe. Best case scenario, you'll have some graphic artifacts and some widgets won't be properly updated; worst (and more likely, especially due to setStyleSheet()) case, the program will crash.
Signals are the only proper and safe way to communicate with the main thread, since they allow correct event handling and prevent freezing of the UI while waiting for the thread to quit.
If you need to change something in the UI when something else happens in a thread, create a function in an object that resides in the main thread and connect it to the appropriate signal that will be emitted from the other thread. If you need to do something that is potentially blocking while waiting for the thread, do it in that thread.
I'm posting an answer to my own here based on helpful comments from #ekhumoro and #musicamante, as well as some further digging I've done.
My main pushback is that it seems like the suggestions so far maintain that the only way to communicate data between threads is via signals. While this may be the best choice in many/most cases, and a good rule of thumb, there are other safe ways to do it. The reason I continue to beat this possibly dead horse is that in the Qt documentation it mentions a limit on the number of signals:
On an i586-500, you can emit around 2,000,000 signals per second connected to one receiver, or around 1,200,000 per second connected to two receivers.
Granted the hardware suggested here is dated and the current Qt version documentation makes no such reference. However, signals do impart additional overhead vs a normal callback (this is noted in both past and present versions of Qt docs). While this limit is unlikely to be reached in my case, I'm now more interested in understanding both approaches merits and drawbacks.
I think both approaches can be used in a thread-safe manner, although that wasn't properly addressed in the original question. Both approaches are posted here with full MWE.
Approach 1: Using Signals
# thread-signal.py
from PyQt5.QtCore import QObject, pyqtSignal
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QPushButton, QVBoxLayout
from threading import Thread
import time
# The worker class
class Worker(QObject, Thread):
errorSignal = pyqtSignal(bool)
def __init__(self, id=""):
super().__init__()
self.id = id
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
time.sleep(2)
raise ValueError(f"Worker Failure")
except:
# notify parent that there was an error
print(f"Worker {self.id} failed")
self.errorSignal.emit(True)
self._running = False
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# set up UI
vbox = QVBoxLayout()
self.button = QPushButton("Test")
vbox.addWidget(self.button)
self.setLayout(vbox)
# instantiate the workers
self.worker1 = Worker(id=1)
self.worker2 = Worker(id=2)
# connect to the error signal
self.worker1.errorSignal.connect(self.handle_error)
self.worker2.errorSignal.connect(self.handle_error)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker2.terminate()
# notify user
self.button.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
print(f"Worker 1 isAlive = {self.worker1.is_alive()}")
print(f"Worker 2 isAlive = {self.worker2.is_alive()}")
if __name__=='__main__':
app = QApplication([])
window = QMainWindow()
window.show()
ctrl = ControlWidget()
window.setCentralWidget(ctrl)
# window.setLayout(layout)
app.exec()
The result of running the example is that after 2 seconds, the workers fail (Worker 1 fails first because it was started first - although this may not always be the case depending on how the GIL is shared between threads). In the end, both workers are shutdown and the control widget is deactivated.
$ python thread-signal.py
Worker 1 failed
Worker 1 isAlive = False
Worker 2 isAlive = True
Worker 2 failed
Worker 1 isAlive = False
Worker 2 isAlive = False
Approach 2: Using parent References
# thread-reference.py
from PyQt5.QtCore import QObject, pyqtSignal
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QPushButton, QVBoxLayout
from threading import Thread, Lock
import time
# global thread lock
lock = Lock()
# The worker class
class Worker(Thread):
def __init__(self, id="", parent=None):
super().__init__()
self.id = id
self.parent = parent
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
time.sleep(2)
raise ValueError(f"Worker Failure")
except:
with lock:
# notify parent that there was an error
print(f"Worker {self.id} failed")
self._running = False
self.parent.error_status = True
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# set up UI
vbox = QVBoxLayout()
self.button = QPushButton("Test")
vbox.addWidget(self.button)
self.setLayout(vbox)
# instantiate the workers
self.worker1 = Worker(id=1, parent=self)
self.worker2 = Worker(id=2, parent=self)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker2.terminate()
# notify user
self.button.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
print(f"Worker 1 isAlive = {self.worker1.is_alive()}")
print(f"Worker 2 isAlive = {self.worker2.is_alive()}")
#property
def error_status(self):
return self._error_status
#error_status.setter
def error_status(self, value):
self._error_status = value
self.handle_error(value)
if __name__=='__main__':
app = QApplication([])
window = QMainWindow()
window.show()
ctrl = ControlWidget()
window.setCentralWidget(ctrl)
app.exec()
The end result is similar in this case, the workers are shut down and the control widget deactivated, but inspection of the output shows an interesting difference:
$ python thread-reference.py
Worker 1 failed
Worker 1 isAlive = True
Worker 2 isAlive = True
Worker 2 failed
Worker 1 isAlive = False
Worker 2 isAlive = True
The isAlive() value of the thread doesn't resolve to False until after the handle_error method is complete. I'm guessing this is just the way the bytecode execution comes out due to the GIL.
As I Mentioned in Title. How can i do something like this?
class Main(QWidget):
def __init__(self):
super().__init__()
def StartButtonEvent(self):
self.test = ExecuteThread()
self.test.start()
def MyEvent(self):
#MainThreadGUI
class ExecuteThread(QThread):
def run(self):
# A lot of work
# Signal to main thread about finishing of job = mainthread will perform MyEvent
I found some tutorials here pyqt4 emiting signals in threads to slots in main thread
and here Emit signal from pyQt Qthread
But it seems it does not working in PyQt5 :/
Just use the QThread.finished signal here. It will be executed automatically if you finish your thread.
Of course you can also define your own custom signal if you want.
from PyQt5.QtCore import pyqtSignal
class Main(QWidget):
def __init__(self):
super().__init__()
def StartButtonEvent(self):
self.test = ExecuteThread()
self.test.start()
self.test.finished.connect(thread_finished)
self.test.my_signal.connect(my_event)
def thread_finished(self):
# gets executed if thread finished
pass
def my_event(self):
# gets executed on my_signal
pass
class ExecuteThread(QThread):
my_signal = pyqtSignal()
def run(self):
# do something here
self.my_signal.emit()
pass
Following is the sample code structure I am intending to implementing a larger time consuming operation. For doing larger operation, I have used QThread and updating progressbar (from main class) using the emited signal. All works fine un till large time consuming operation is completed. However, I run in to problem when I call a function from main GUI class. Here is the code structure I am trying (read the comments):-
import time
from scripts.gui import Ui_Dialog
from PyQt4 import QtGui
from PyQt4 import QtCore
class AppGui(QtGui.QDialog, Ui_Dialog):
def __init__(self):
QtGui.QDialog.__init__(self)
# Main code here.
# This GUI pops up for user input and opens a main GUI.
def main_function(self):
# Doing some main coding here.
self.work_thread = WorkThread()
self.work_thread.update.connect(self.ui.progressBar.setValue)
self.work_thread.start()
# I wanted to continue more coding here after the thread is finished. But self.work_thread.wait() is blocking main gui.
# Therefore, I moved the continuation code to different function --> sub_function()
def sub_function(self):
# Do the remaining code left over from the main_function()
class WorkThread(QtCore.QThread):
update = QtCore.pyqtSignal(int)
def __init__(self):
QtCore.QThread.__init__(self)
def __del__(self):
self.wait()
def run(self):
self.thread = GenericThread(scripts.function, arg1, arg2) # This "scripts.function()" function does long process.
self.thread.start()
while self.thread.isRunning():
# Do some long process.
time.sleep(1)
self.update.emit(signal)
print "Distro extraction completed..."
if self.thread.isFinished():
self.main_class = AppGui()
self.main_class.sub_function() # <-- Problematic call to main AppGui function.
if self.isFinished():
return
class GenericThread(QtCore.QThread):
def __init__(self, function, *args, **kwargs):
QtCore.QThread.__init__(self)
self.function = function
self.args = args
self.kwargs = kwargs
def __del__(self):
self.wait()
def run(self):
self.function(*self.args, **self.kwargs)
return
This is what I got after running.
What I believe is that I am wrongly calling function of main AppGui() from WorkThread() class.
QPixmap: It is not safe to use pixmaps outside the GUI thread
Larger operation is complete...
QObject::installEventFilter(): Cannot filter events for objects in a different thread.
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python2.7: ../../src/xcb_io.c:179: dequeue_pending_request: Assertion `!xcb_xlib_unknown_req_in_deq' failed.
Any help to solve this issue is appreciated.
The reason is that worker thread emit a signal, this signal could not directly bind to an UI slot, but you need to bind it to a general slot, then you call the UI slot to upgrade. As I don't have all your code, so Ｉ write a similar file like this, it works fine
from PyQt4.QtCore import *
from PyQt4.QtGui import *
import time
class WorkerThread(QThread):
updateSignal = pyqtSignal(int)
def run(self):
count = 0
while True:
time.sleep(0.1)
self.updateSignal.emit(count)
count += 1
class ProgressBar(QProgressBar):
def __init__(self, parent=None):
super(ProgressBar, self).__init__(parent)
self.worker = WorkerThread()
self.worker.updateSignal.connect(self.updateProgress) # here should bind to a general slot
def startWorker(self):
self.worker.start()
def updateProgress(self, progress):
self.setValue(progress)
if __name__ == '__main__':
import sys
app = QApplication(sys.argv)
p = ProgressBar()
p.startWorker()
p.show()
app.exec_()
I'm trying to figure out how to implement the concept of having my main thread spawn a new thread that processes data concurrently as messages are passed to it.
From what I figured so far the simplest way of doing this would be something like:
from PySide.QtCore import QCoreApplication, QObject, Signal, QThread, QTimer
class Foo(QObject):
do_foo = Signal(str)
def __init__(self, parent=None):
super(Foo, self).__init__(parent)
self.do_foo.connect(self._do_foo)
def foo(self, string):
self.do_foo.emit(string)
def _do_foo(self, string):
# Process string
print "Process thread:", self.thread().currentThreadId()
class App(QCoreApplication):
def run(self):
print "Main thread:", self.thread().currentThreadId()
thread = QThread()
foo = Foo()
foo.moveToThread(thread)
thread.start()
# Obviously the following should be done with the event-loop
# or at doing processing of events.
running = True
while running:
try:
string = raw_input()
foo.foo(string)
except EOFError:
running = False
thread.exit()
thread.wait()
self.exit()
if __name__ == '__main__':
import sys
app = App(sys.argv)
QTimer.singleShot(0, app.run)
sys.exit(app.exec_())
But if this would be the way of doing it I can not see what the use of Slots would be.
Or you can use the design patter "Provider-Consumer". How it works? Well you have to implement a queue. The spwaned thread will get the data from this queue while your main thread will feed the queue with new data.
Your spawned threads blocks while the queue is empty. This way you can even process data in more that one thread, and you don't have to worry about two threads trying to read the same data.
Here is some seudo-code for consumer threads.
class MyThread:
def __init__(self, queue):
self.queue = queue
self.event = Event() # I generally use threading.Event for stopping threads. You don't need it here.
def run():
while not self.event.isSet():
data = self.queue.get() # This stop the thread until new data be available.
do_something_with_data(data)
Then in your main thread:
import Queue
queue = Queue.Queue()
mthread = MyThread(queue)
mthread.start()
# And now you can send data to threads by:
queue.put(data)
I have a Manager (main thread), that creates other Threads to handle various operations.
I would like my Manager to be notified when a Thread it created ends (when run() method execution is finished).
I know I could do it by checking the status of all my threads with the Thread.isActive() method, but polling sucks, so I wanted to have notifications.
I was thinking of giving a callback method to the Threads, and call this function at the end of the run() method:
class Manager():
...
MyThread(self.on_thread_finished).start() # How do I pass the callback
def on_thread_finished(self, data):
pass
...
class MyThread(Thread):
...
def run(self):
....
self.callback(data) # How do I call the callback?
...
Thanks!
The thread can't call the manager unless it has a reference to the manager. The easiest way for that to happen is for the manager to give it to the thread at instantiation.
class Manager(object):
def new_thread(self):
return MyThread(parent=self)
def on_thread_finished(self, thread, data):
print thread, data
class MyThread(Thread):
def __init__(self, parent=None):
self.parent = parent
super(MyThread, self).__init__()
def run(self):
# ...
self.parent and self.parent.on_thread_finished(self, 42)
mgr = Manager()
thread = mgr.new_thread()
thread.start()
If you want to be able to assign an arbitrary function or method as a callback, rather than storing a reference to the manager object, this becomes a bit problematic because of method wrappers and such. It's hard to design the callback so it gets a reference to both the manager and the thread, which is what you will want. I worked on that for a while and did not come up with anything I'd consider useful or elegant.
Anything wrong with doing it this way?
from threading import Thread
class Manager():
def Test(self):
MyThread(self.on_thread_finished).start()
def on_thread_finished(self, data):
print "on_thread_finished:", data
class MyThread(Thread):
def __init__(self, callback):
Thread.__init__(self)
self.callback = callback
def run(self):
data = "hello"
self.callback(data)
m = Manager()
m.Test() # prints "on_thread_finished: hello"
If you want the main thread to wait for children threads to finish execution, you are probably better off using some kind of synchronization mechanism. If simply being notified when one or more threads has finished executing, a Condition is enough:
import threading
class MyThread(threading.Thread):
def __init__(self, condition):
threading.Thread.__init__(self)
self.condition = condition
def run(self):
print "%s done" % threading.current_thread()
with self.condition:
self.condition.notify()
condition = threading.Condition()
condition.acquire()
thread = MyThread(condition)
thread.start()
condition.wait()
However, using a Queue is probably better, as it makes handling multiple worker threads a bit easier.