Develop Reference

When to use events versus pass a reference to a parent (PyQT)

I'm trying to understand which of the following approaches is better. I'll use PyQt5 to illustrate the example, as that's the system I'm working with.
We have a Class ControlWidget(QWidget) that provides UI components and spawns helper threads that are just workers executing some task. If one of the workers experiences a failure, we want to notify the parent so it can visually update the user and do some other things like cleanup/disable user input.
Approach 1 would be to send an event from the worker that encounters the error and catch it in the parent instance. The worker Class has an errorSignal that simply indicates the presence of an error.
# The worker class
Class Worker(QObject, Thread):
errorSignal = pyqtSignal(bool)
def __init__(self):
super().__init__()
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
except:
# notify parent that there was an error
errorSignal.emit(True)
self._running = False
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# instantiate the workers
self.worker1 = Worker()
self.worker2 = Worker()
# connect to the error signal
self.worker1.errorSignal.connect(self.handle_error)
self.worker2.errorSignal.connect(self.handle_error)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker1.join()
self.worker2.terminate()
self.worker2.join()
# notify user
self.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
...
Approach 2 would be to make an error_status property on the parent, and pass a reference to each child worker, so it can toggle the error bit directly.
# The worker class
Class Worker(QObject, Thread):
def __init__(self, parent):
super().__init__()
self._running = True
self.parent = parent
def run(self):
while self._running:
try:
# do some work here that might fail
except:
# notify parent that there was an error
self.parent.error_status = True
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# instantiate the workers
self.worker1 = Worker(parent=self)
self.worker2 = Worker(parent=self)
# no signals here :)
# start the workers
self.worker1.start()
self.worker2.start()
#property
def error_status(self):
return self._error_status
#error_status.setter
def error_status(self, value):
self._error_status = value
self.handle_error(value)
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker1.join()
self.worker2.terminate()
self.worker2.join()
# notify user
self.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
...
Events per second limits?
Functionally both approaches will work, however, I suppose I'm worried that if I have a lot of events in my application (I do) I'll overload the event queue and lock up or miss some. I read somewhere that Qt has a limit of about 2 million events per second (although now I can't find the reference). It may just depend on available memory. If I may conceivably reach this limit, isn't it better to try to offload some event handling where possible?
Is approach 2 a better way to handle this type of issue (presuming I've addressed the thread safety issue with my edit)? It does feel slightly gross to pass references to parent/child objects all over the place.
Perhaps this is 6 of one, a half dozen of the other? Any feedback would be appreciated.

Events cannot be missed, they are all processed and eventually discarded if not handled and/or ignored (which are two related but different things).
Both your approaches have flaws, though.
Signals
You should not use terminate() (the docs clearly say that it's dangerous and its use is discouraged).
QThread also doesn't have a join() method, it has wait() which does the same, but you should not use it in the main thread: Qt automatically calls the connected slot in the receiver's thread, and since the signal is connected to handle_error (which is a member of an object in the main thread) the result is that it will block the main thread until the thread has actually finished execution. To avoid that, wait() must be executed in the worker thread, so you can eventually consider a two-signal approach: first react to the error signal, then do something until the thread's finished() signal is finally emitted.
Flag setter
Doing this would only partially work, since the setter is called from the worker thread, so wait() will be correctly called from there. Unfortunately, you're also trying to access the UI in the same function, which is forbidden since UI elements are not thread safe. Best case scenario, you'll have some graphic artifacts and some widgets won't be properly updated; worst (and more likely, especially due to setStyleSheet()) case, the program will crash.
Signals are the only proper and safe way to communicate with the main thread, since they allow correct event handling and prevent freezing of the UI while waiting for the thread to quit.
If you need to change something in the UI when something else happens in a thread, create a function in an object that resides in the main thread and connect it to the appropriate signal that will be emitted from the other thread. If you need to do something that is potentially blocking while waiting for the thread, do it in that thread.

I'm posting an answer to my own here based on helpful comments from #ekhumoro and #musicamante, as well as some further digging I've done.
My main pushback is that it seems like the suggestions so far maintain that the only way to communicate data between threads is via signals. While this may be the best choice in many/most cases, and a good rule of thumb, there are other safe ways to do it. The reason I continue to beat this possibly dead horse is that in the Qt documentation it mentions a limit on the number of signals:
On an i586-500, you can emit around 2,000,000 signals per second connected to one receiver, or around 1,200,000 per second connected to two receivers.
Granted the hardware suggested here is dated and the current Qt version documentation makes no such reference. However, signals do impart additional overhead vs a normal callback (this is noted in both past and present versions of Qt docs). While this limit is unlikely to be reached in my case, I'm now more interested in understanding both approaches merits and drawbacks.
I think both approaches can be used in a thread-safe manner, although that wasn't properly addressed in the original question. Both approaches are posted here with full MWE.
Approach 1: Using Signals
# thread-signal.py
from PyQt5.QtCore import QObject, pyqtSignal
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QPushButton, QVBoxLayout
from threading import Thread
import time
# The worker class
class Worker(QObject, Thread):
errorSignal = pyqtSignal(bool)
def __init__(self, id=""):
super().__init__()
self.id = id
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
time.sleep(2)
raise ValueError(f"Worker Failure")
except:
# notify parent that there was an error
print(f"Worker {self.id} failed")
self.errorSignal.emit(True)
self._running = False
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# set up UI
vbox = QVBoxLayout()
self.button = QPushButton("Test")
vbox.addWidget(self.button)
self.setLayout(vbox)
# instantiate the workers
self.worker1 = Worker(id=1)
self.worker2 = Worker(id=2)
# connect to the error signal
self.worker1.errorSignal.connect(self.handle_error)
self.worker2.errorSignal.connect(self.handle_error)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker2.terminate()
# notify user
self.button.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
print(f"Worker 1 isAlive = {self.worker1.is_alive()}")
print(f"Worker 2 isAlive = {self.worker2.is_alive()}")
if __name__=='__main__':
app = QApplication([])
window = QMainWindow()
window.show()
ctrl = ControlWidget()
window.setCentralWidget(ctrl)
# window.setLayout(layout)
app.exec()
The result of running the example is that after 2 seconds, the workers fail (Worker 1 fails first because it was started first - although this may not always be the case depending on how the GIL is shared between threads). In the end, both workers are shutdown and the control widget is deactivated.
$ python thread-signal.py
Worker 1 failed
Worker 1 isAlive = False
Worker 2 isAlive = True
Worker 2 failed
Worker 1 isAlive = False
Worker 2 isAlive = False
Approach 2: Using parent References
# thread-reference.py
from PyQt5.QtCore import QObject, pyqtSignal
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QPushButton, QVBoxLayout
from threading import Thread, Lock
import time
# global thread lock
lock = Lock()
# The worker class
class Worker(Thread):
def __init__(self, id="", parent=None):
super().__init__()
self.id = id
self.parent = parent
self._running = True
def run(self):
while self._running:
try:
# do some work here that might fail
time.sleep(2)
raise ValueError(f"Worker Failure")
except:
with lock:
# notify parent that there was an error
print(f"Worker {self.id} failed")
self._running = False
self.parent.error_status = True
def terminate(self):
self._running = False
# The parent class
class ControlWidget(QWidget):
def __init__(self):
super().__init__()
# set up UI
vbox = QVBoxLayout()
self.button = QPushButton("Test")
vbox.addWidget(self.button)
self.setLayout(vbox)
# instantiate the workers
self.worker1 = Worker(id=1, parent=self)
self.worker2 = Worker(id=2, parent=self)
# start the workers
self.worker1.start()
self.worker2.start()
def handle_error(self, error):
if error:
# clean up worker threads
self.worker1.terminate()
self.worker2.terminate()
# notify user
self.button.setStyleSheet('background: red;')
# disable UI input etc
self.button.setEnabled(False)
print(f"Worker 1 isAlive = {self.worker1.is_alive()}")
print(f"Worker 2 isAlive = {self.worker2.is_alive()}")
#property
def error_status(self):
return self._error_status
#error_status.setter
def error_status(self, value):
self._error_status = value
self.handle_error(value)
if __name__=='__main__':
app = QApplication([])
window = QMainWindow()
window.show()
ctrl = ControlWidget()
window.setCentralWidget(ctrl)
app.exec()
The end result is similar in this case, the workers are shut down and the control widget deactivated, but inspection of the output shows an interesting difference:
$ python thread-reference.py
Worker 1 failed
Worker 1 isAlive = True
Worker 2 isAlive = True
Worker 2 failed
Worker 1 isAlive = False
Worker 2 isAlive = True
The isAlive() value of the thread doesn't resolve to False until after the handle_error method is complete. I'm guessing this is just the way the bytecode execution comes out due to the GIL.

QThread Windows not responding

I wrote a GUI program in PyQt on Windows. There's some expensive operations in my program. While these operations are running, the program shows "Not Responding" in the program bar.
I think it must be this operation block the main thread to update the UI, so I write multi-threading code by QThread to test it, it still not make sense.
I wrote a small program to test it, the operation did not run in new thread at all, here is my small test code:
from PyQt5.QtCore import QThread, QObject, QCoreApplication, qDebug, QTimer
class Worker(QObject):
def on_timeout(self):
qDebug('Worker.on_timeout get called from: %s' % hex(int(QThread.currentThreadId())))
if __name__ == '__main__':
import sys
app = QCoreApplication(sys.argv)
qDebug('From main thread: %s' % hex(int(QThread.currentThreadId())))
t = QThread()
qDebug(t)
worker = Worker()
timer = QTimer()
timer.timeout.connect(worker.on_timeout)
timer.start(1000)
timer.moveToThread(t)
worker.moveToThread(t)
t.start()
app.exec_()
Here is the output:
From main thread: 0x634
Worker.on_timeout get called from: 0x634

Your program has several errors and does not produce the output that you show.
Firstly, it is not possible to pass a thread object to qDebug - the argument must be a string. If you want to print objects, use qDebug(repr(obj)) - or even better, just use print(obj).
Secondly, you cannot start a timer outside of the thread that created it. Your example makes the signal connection in the main thread, and starts the timer in the main thread as well. So worker.on_timeout will be called in the main thread. But if you connect and start the timer after moving it to the worker thread, you will get this error:
QObject::startTimer: Timers can only be used with threads started with
QThread
I think using a timer is unnecessary, and confuses your example, so it is better to leave it out altogether. Instead, you should connect the started signal of the worker thread to a run method of the worker object. To simulate a long-running operation, you can use QThread.sleep():
from PyQt5.QtCore import QThread, QObject, QCoreApplication
class Worker(QObject):
def run(self):
print('Worker called from: %#x' % int(QThread.currentThreadId()))
QThread.sleep(2)
print('Finished')
QCoreApplication.quit()
if __name__ == '__main__':
import sys
app = QCoreApplication(sys.argv)
print('From main thread: %#x' % int(QThread.currentThreadId()))
t = QThread()
worker = Worker()
worker.moveToThread(t)
t.started.connect(worker.run)
t.start()
app.exec_()
Finally, note that you should always make signal connections after moving a worker object to a thread. The reasons for this are explained in this answer.

PyQt – Signal for object created in different thread doesn't work

I’m implementing multithreading in PyQt5 and Python 3.5 by running a Worker inside a QThread. In the following sample thread2_worker (runs inside a secondary thread and) creates thread3_worker, connects thread3_worker.finished signal to thread3_worker_finished() and runs it.
When thread3_worker is done it emits finished from within its thread but the connection doesn’t work. My guess is that it has to do with thread3_worker being created or connected not in the main thread but I'd welcome any clarification.
import time
from PyQt5.QtCore import QObject, QThread, pyqtSignal, pyqtSlot, QCoreApplication
class Worker(QObject):
# Generic worker.
finished = pyqtSignal()
def __init__(self, func):
super().__init__()
self.func = func
def run(self):
self.func()
self.finished.emit()
def thread_factory(func):
# Creates a Worker, a QThread and moves the Worker inside the QThread.
worker = Worker(func)
thread = QThread()
worker.moveToThread(thread)
thread.started.connect(worker.run)
# Provision graceful termination.
worker.finished.connect(thread.quit)
worker.finished.connect(worker.deleteLater)
thread.finished.connect(thread.deleteLater)
return worker, thread
def wait():
print("thread3:\t{}".format(QThread.currentThread()))
time.sleep(3)
# finished signal of thread3_worker is never received.
#pyqtSlot()
def thread3_worker_finished():
QCoreApplication.exit()
def create_thread3():
print("thread2:\t{}".format(QThread.currentThread()))
global thread3_worker, thread3
# thread3_worker runs inside thread3, and all it does is call wait().
thread3_worker, thread3 = thread_factory(wait)
thread3_worker.finished.connect(thread3_worker_finished) # FIXME Doesn't work.
thread3.start()
app = QCoreApplication([])
print("Main thread:\t{}".format(QThread.currentThread()))
thread3_worker, thread3 = None, None
# thread2_worker runs inside thread2, and creates and runs thread3_worker.
thread2_worker, thread2 = thread_factory(create_thread3)
thread2.start()
app.exec_()

Cross-thread signals require an event-loop. Your thread_factory function connects the finished signal of the worker to the quit slot of its thread. The quit slot asks the thread to exit its event-loop.
So after thread3 starts, worker2 finishes and thread2 quits. Then when the finished signal of worker3 is emitted, there is no longer an event-loop running that can process it. If you comment out the line worker.finished.connect(thread.quit), your example should work.

PyQt thread communication help? QThread and QObject

After read and searching I am trying to use the generate a QObject then use the movetoThread method to run an independent process and allow the QMainWindow to continue to respond. This has not worked when I have tried to implement the operation in a QThread.run() method. The following code is my attempt to make a simple example. While the code works in running thread independent of the MainWindow, it does not abort. The only way I can get a thread to stop is to set worker.end = True. Which I think should not be the way to do it.
"""
This is a program to test Threading with Objects in PyQt4.
"""
from time import sleep
import sys
from PyQt4.QtCore import QObject, pyqtSlot, pyqtSignal, QThread
from PyQt4.QtGui import QMainWindow, QApplication, QProgressBar
from PyQt4.QtGui import QPushButton, QVBoxLayout, QWidget
class workerObject(QObject):
bar_signal = pyqtSignal(int)
res_signal = pyqtSignal(str)
term_signal = pyqtSignal()
def __init__(self, maxIters):
super(workerObject, self).__init__()
self.maxIters = maxIters
def run(self):
self.bar_signal.emit(self.maxIters)
sleep(1)
self.end = False
for step in range(self.maxIters):
if self.end:
self.maxIters = step
break
self.bar_signal.emit(step)
sleep(2)
self.res_signal.emit("Got to {}".format(self.maxIters))
self.term_signal.emit()
#pyqtSlot()
def mystop(self):
print "stop signalled?"
self.end = True
class MCwindow(QMainWindow):
abort_signal = pyqtSignal(name='abort_signal')
def __init__(self):
super(MCwindow,self).__init__()
self.maxIters = 50
widget = QWidget()
layout = QVBoxLayout(widget)
self.go_btn = QPushButton()
self.go_btn.setText('Go')
layout.addWidget(self.go_btn)
self.abort_btn = QPushButton()
self.abort_btn.setText('Stop')
layout.addWidget(self.abort_btn)
self.simulation_bar = QProgressBar()
self.simulation_bar.setRange(0, self.maxIters)
self.simulation_bar.setFormat("%v")
layout.addWidget(self.simulation_bar)
self.setCentralWidget(widget)
self.go_btn.clicked.connect(self.run_mc)
# The button calls the windows method to stop --- it could
# be that is 'clicked' calls the worker.mystop
# self.abort_btn.clicked.connect(self.stop_mc)
# This allows for the abort button to do somethign in the MainWindow
# before the abort_signal is sent, this works
self.abort_btn.clicked.connect(self.stop_mc)
def run_mc(self):
self.thread = QThread()
self.worker = workerObject(self.maxIters)
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
# This is the simple stop method, but does not work
# self.abort_btn.clicked.connect(self.worker.mystop)
# This uses the signal in the MCwindow - this connection does NOT works
self.abort_signal.connect(self.worker.mystop)
# This does NOT stop the thread
# and would not allow for any clean up in the worker.
# self.abort_signal.connect(self.thread.terminate)
# This is a 'bad' way to stop the woker ... It does, however, work
# self.abort_signal.connect(self.stopper)
self.worker.bar_signal.connect(self.setBar)
self.worker.res_signal.connect(self.setData)
self.worker.term_signal.connect(self.thread.terminate)
self.thread.start()
def stop_mc(self):
print "Stopping?!"
# This signal is NEVER seen by the Worker.
self.abort_signal.emit()
def stopper(self):
print "I should stop?!"
# Should use signals to tell the worker to stop - and not setting a attribute
self.worker.end=True
#pyqtSlot(int)
def setBar(self, val):
self.simulation_bar.setValue(val)
#pyqtSlot(str)
def setData(self, txt):
print "Got done Sig!", txt
if __name__ == '__main__':
app = QApplication(sys.argv)
window = MCwindow()
window.show()
sys.exit(app.exec_())

The reason why the slot connected to abort_signal doesn't seem to get called, is because cross-thread signals are queued by default. This means the signal will be wrapped as an event and posted to the event queue of whichever thread the receiver is living in.
In your particular example, the receiver is a worker object which has been moved to a worker thread. Calling start() on the worker thread will start its event-loop, and that is where abort_signal will be queued. However, the run() method of the worker object starts a for loop, which will block the thread's event processing in exactly the same way it would if it was executed in the main gui thread!
You can more clearly see what's happening if you make a few adjustments to your example:
class MCwindow(QMainWindow):
abort_signal = pyqtSignal(name='abort_signal')
def __init__(self):
super(MCwindow,self).__init__()
# use a sane default
self.maxIters = 5
...
# DO NOT use QThread.terminate
self.worker.term_signal.connect(self.thread.quit)
Now run the example, and then click the Go button, click the Stop button, and wait for the worker to complete normally. This should produce output like this:
Stopping?!
Got done Sig! Got to 5
stop signalled?
Note that "stop signalled" is output last - i.e. after run() exits and control has returned to the thread's event-loop. In order to process in-coming signals while the worker is running, you will need to force immediate processing of the thread's pending events. This can be done like this:
for step in range(self.maxIters):
QApplication.processEvents()
...
With that in place, you should then see output like this:
Stopping?!
stop signalled?
Got done Sig! Got to 2
Which is presumably what you intended.

Typically a thread will close when it exits the run method. The other way to get a regular python thread to close is by calling it's join method.
For PyQt the join method should either be the quit or terminate method. You should probably still set your end variable to True.

How to properly terminate a QThread from a GUI application?

I tried using self.terminate() in the QThread class, and also self.thread.terminate() in the GUI class. I also tried putting self.wait() in both cases. However, there are two scenarios that happen:
1) The thread does not terminate at all, and the GUI freezes waiting for the thread to finish. Once the thread finished, the GUI unfreezes and everything is back to normal.
2) The thread indeed does terminate, but at the same time it freezes the entire application.
I also tried using self.thread.exit(). No joy.
To further clarify, I am trying to implement a user-abort button in GUI which would terminate the executing of the thread at any point in time.
Thanks in advance.
EDIT:
Here is the run() method:
def run(self):
if self.create:
print "calling create f"
self.emit(SIGNAL("disableCreate(bool)"))
self.create(self.password, self.email)
self.stop()
self.emit(SIGNAL("finished(bool)"), self.completed)
def stop(self):
#Tried the following, one by one (and all together too, I was desperate):
self.terminate()
self.quit()
self.exit()
self.stopped = True
self.terminated = True
#Neither works
And here is the GUI class' method for aborting the thread:
def on_abort_clicked(self):
self.thread = threadmodule.Thread()
#Tried the following, also one by one and altogether:
self.thread.exit()
self.thread.wait()
self.thread.quit()
self.thread.terminate()
#Again, none work

From the Qt documentation for QThread::terminate:
Warning: This function is dangerous and its use is discouraged. The
thread can be terminated at any point in its code path. Threads can be
terminated while modifying data. There is no chance for the thread to
clean up after itself, unlock any held mutexes, etc. In short, use
this function only if absolutely necessary.
It's probably a much better idea to re-think your threading strategy such that you can e.g. use QThread::quit() to signal the thread to quit cleanly, rather than trying to get the thread to terminate this way. Actually calling thread.exit() from within the thread should do that depending on how you have implemented run(). If you'd like to share the code for your thread run method that might hint as to why it doesn't work.

This is what I did:
class MainWindow(QtWidgets.QMainWindow, Ui_MainWindow):
stop_flag = 1
...
#=========500 ms class ===================
class Timer500msThread(QThread):
signal_500ms = pyqtSignal(str)
....
def timer500msProcess(self):
if MainWindow.stop_flag == 0 :
self.timer_500ms.stop()
#==========
#=========Main Window ===================
app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec()
MainWindow.stop_flag=0 #this sets the flag to 0 and when the next 500ms triggers the
#the thread ends
print("Program Ending")

I had a similar problem and solved it with use of pyqtSignals and pyqtSlots.
You could create a pyqtSignal in your MainWindow-class and use the aboutToQuit-function to your QApplication instance.
Then you connect this aboutToQuit-function with another which emit a signal to the slot in your separate thread.
Then you can define a stop() function in this Thread which runs if the signal is emitted.
In this case the thread would not terminate during its work.
MainWindow:
class mainSignals(QObject):
isClosed = pyqtSignal()
class mainwindow(QMainWindow, Ui_MainWindow):
def __init__(self):
super(mainwindow, self).__init__()
self.mainSignal = mainSignals()
...
...
if __name__ = "__main__":
# This function runs if the application is closed.
# (app.aboutToQuit.connect(windowClose))
def windowClose():
window.mainSignal.isClosed.emit() # Emit the signal to the slot in (sepThread)
app = QApplication(sys.argv)
app.aboutToQuit.connect(windowClose)
window = mainwindow()
window.show()
sys.exit(app.exec())
sepThread:
class sepThread(QRunnable):
def __init__(self, parent):
super(sepThread,self).__init__()
self._parent = parent
self.mainOpen = True
self._parent.mainSignal.isClosed.connect(self.stopThread)
# If the signal was emitted by the Mainapplication
# the stopThread function runs and set the mainOpen-attribute to False
def stopThread(self):
self.mainOpen = False
def run(self):
while self.mainOpen == True:
# Do something in a loop while mainOpen-attribute is True
...
...