I have a PySide2 application that runs mostly well. However, from time to time, it exits itself without any error (stdout, stderr are empty, even when run from terminal - tried everything) while a worker thread sends updates to the GUI thread via signals.
To emit these signals however, I'm doing it in a way such that my libraries can just take callbacks and not even know they are actually emitting signals, and I was wondering whether this could be a potential source of crashes:
Gui code:
from mylib import do_threaded_work
MyClass(QObject):
sig = Signal(int)
# ... proper initialization and connection of self.sig to a QProgressBar setValue slot
def on_some_button_pressed(self):
threading.Thread(target=do_threaded_work, args=(self.sig.emit,), daemon=True).start()
mylib.py dummy code:
do_threaded_work(on_progress):
i = 0
while (True):
# ... do some work
on_progress(i)
i = min(100, i + 1)
As you see, I'm directly passing the emit function of my signal instance to the library, that calls it and thus should emit the signal. Is it OK to do that?
Sometimes, I pass the signal object self.sig as argument instead of self.sig.emit, then call argument.emit(...) in the library code. I assume it has the same effect?
The reason I ask is because I didn't find any counter argument stating not to do this in PySide2, and the official documentation on signal and slots is quite light here.
You guys have any input on this? Thanks!
Related
I am having some trouble with a GUI that is freezing during a file save operation that is taking some time, and I'd love to understand why that is.
I've followed the instructions of Schollii's wonderful answer on a similar question, but there must be something I'm missing because I cannot get the GUI behaving as I expect.
The below example is not runnable, since it shows only the relevant parts, but hopefully it's enough to get a discussion going. Basically I have a main application class that generates some large data, and I need to save it to HDF5 format, but this takes some time. To leave the GUI responsive, the main class creates an object of the Saver class and a QThread to do the actual data saving (using moveToThread).
The output of this code is pretty much what I would expect (i.e. I see a message that the "saving thread" has a different thread id than the "main" thread) so I know that another thread is being created. The data is successfully saved, too, so that part is working correctly.
During the actual data saving however (which can take some minutes), the GUI freezes up and goes "Not responding" on Windows. Any clues as to what is going wrong?
Stdout during running:
outer thread "main" (#15108)
<__main__.Saver object at 0x0000027BEEFF3678> running SaveThread
Saving data from thread "saving_thread" (#13624)
Code sample:
from PyQt5 import QtCore, QtGui, QtWidgets
from PyQt5.QtCore import QThread, pyqtSignal, pyqtSlot, QObject
class MyApp(QtWidgets.QMainWindow, MyAppDesign.Ui_MainWindow):
def save_file(self):
self.save_name, _ = QtWidgets.\
QFileDialog.getSaveFileName(self)
QThread.currentThread().setObjectName('main')
outer_thread_name = QThread.currentThread().objectName()
outer_thread_id = int(QThread.currentThreadId())
# print debug info about main app thread:
print('outer thread "{}" (#{})'.format(outer_thread_name,
outer_thread_id))
# Create worker and thread to save the data
self.saver = Saver(self.data,
self.save_name,
self.compressionSlider.value())
self.save_thread = QThread()
self.save_thread.setObjectName('saving_thread')
self.saver.moveToThread(self.save_thread)
# Connect signals
self.saver.sig_done.connect(self.on_saver_done)
self.saver.sig_msg.connect(print)
self.save_thread.started.connect(self.saver.save_data)
self.save_thread.start())
#pyqtSlot(str)
def on_saver_done(self, filename):
print('Finished saving {}'.format(filename))
''' End Class '''
class Saver(QObject):
sig_done = pyqtSignal(str) # worker id: emitted at end of work()
sig_msg = pyqtSignal(str) # message to be shown to user
def __init__(self, data_to_save, filename, compression_level):
super().__init__()
self.data = data_to_save
self.filename = filename
self.compression_level = compression_level
#pyqtSlot()
def save_data(self):
thread_name = QThread.currentThread().objectName()
thread_id = int(QThread.currentThreadId())
self.sig_msg.emit('Saving data '
'from thread "{}" (#{})'.format(thread_name,
thread_id))
print(self, "running SaveThread")
h5f = h5py.File(self.filename, 'w')
h5f.create_dataset('data',
data=self.data,
compression='gzip',
compression_opts=self.compression_level)
h5f.close()
self.sig_done.emit(self.filename)
''' End Class '''
There are actually two issues here: (1) Qt's signals and slots mechanisms, and (2) h5py.
First, the signals/slots. These actually work by copying arguments passed to the signal, to avoid any race conditions. (This is just one of the reasons you see so many signals with pointer arguments in the Qt C++ code: copying a pointer is cheap.) Because you're generating the data in the main thread, it must be copied in the main thread's event loop. The data is obviously big enough for this to take some time, blocking the event loop from handling GUI events. If you instead (for testing purposes) generate the data inside the Saver.save_data() slot, the GUI remains responsive.
However, you'll now notice a small lag after the first "Saving data from thread..." message is printed, indicating that the main event loop is blocked during the actual save. This is where h5py comes in.
You're presumably importing h5py at the top of your file, which is the "correct" thing to do. I noticed that if you instead import h5py directly before you create the file, this goes away. My best guess is that the global interpreter lock is involved, as the h5py code is visible from both the main and saving threads. I would have expected that the main thread would be entirely inside Qt module code at this point, however, over which the GIL has no control. So, like I said, I'm not sure what causes the blocking here.
As far as solutions, to the extent you can do what I described here, that will alleviate the problem. Generating the data outside the main thread, if possible, is advisable. It may also be possible to pass some memoryview object, or a numpy.view object, to the saving thread, though you'll then have to deal with thread-synchronization yourself. Also, importing h5py inside the Saver.save_data() slot will help, but isn't feasible if you need the module elsewhere in the code.
Hope this helps!
Problem: I have a PySide application that already uses logging for console output, but its logging should be extended in a way that LogRecords are also displayed immediately in a widget like a QTextBrowser. I am aware that this would usually be done via a worker thread that signals a slot in the main/gui thread, however as the code base is fairly big, and logging is probably used in a few blocking core operations it would be nice if an immediate feedback in the GUI could be achieved anyways without a bigger refactoring.
Example: Here is some example code for demonstration. It shows:
a logger with two handlers:
a StreamHandler logging to the console
a QSignalHandler emitting a signal with a message connected to a slot that appends the message to a QTextBrowser.
a method long_running_core_operation_that_should_log_immediately_to_ui() that simulates logging from a blocking core operation.
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
import logging
import sys
from PySide import QtCore
from PySide import QtGui
class QSignaler(QtCore.QObject):
log_message = QtCore.Signal(unicode)
class SignalHandler(logging.Handler):
"""Logging handler to emit QtSignal with log record text."""
def __init__(self, *args, **kwargs):
super(SignalHandler, self).__init__(*args, **kwargs)
self.emitter = QSignaler()
def emit(self, logRecord):
msg = "{0}".format(logRecord.getMessage())
self.emitter.log_message.emit(msg)
# When the line below is enabled, logging is immediate/otherwise events
# on the queue will be processed when the slot has finished.
# QtGui.qApp.processEvents()
# configure logging
logging.basicConfig(level=logging.DEBUG) # adds StreamHandler
signal_handler = SignalHandler()
logger = logging.getLogger()
logger.addHandler(signal_handler)
class TestWidget(QtGui.QWidget):
def __init__(self, *args, **kwargs):
super(TestWidget, self).__init__(*args, **kwargs)
layout = QtGui.QVBoxLayout(self)
# text_browser
self.text_browser = QtGui.QTextBrowser()
layout.addWidget(self.text_browser)
# btn_start_operation
self.btn_start_operation = QtGui.QPushButton("Start operation")
self.btn_start_operation.clicked.connect(
self.long_running_core_operation_that_should_log_immediately_to_ui)
layout.addWidget(self.btn_start_operation)
# btn_clear
self.btn_clear = QtGui.QPushButton("Clear")
self.btn_clear.clicked.connect(self.text_browser.clear)
layout.addWidget(self.btn_clear)
def long_running_core_operation_that_should_log_immediately_to_ui(self):
for index in range(10000):
msg = "{0}".format(index)
logger.info(msg)
# test
if (__name__ == "__main__"):
app = QtGui.QApplication(sys.argv)
test_widget = TestWidget()
signal_handler.emitter.log_message.connect(test_widget.text_browser.append)
test_widget.show()
sys.exit(app.exec_())
Question: While the StreamHandler logging to stdout happens immediately, the QSignalHandler logging happens, when the PySide event loop processes events again, which happens after the for loop.
Is there a recommended way, to achieve immediate logging from the QSignalHandler without invoking a worker thread for the core operation?
Is it safe/recommended to just call QtGui.qApp.processEvents() after the QSignalHandler has emitted the logging signal? (When uncommented, logging to the GUI happens directly).
When reading the documentation for signal connection types, where it says Qt.DirectConnection: The slot is invoked immediately, when the signal is emitted. I would have kind of thought the QSignalHandler should have updated immediately just as the StreamHandler does, shouldn't it?
Is there a recommended way, to achieve immediate logging from the QSignalHandler without invoking a worker thread for the core operation?
I don't know of any other way to trigger a repaint of the log widget than processing events.
Note that calling repaint() on the log widget is misleading and does not have the desired effect, it only forces the paintEvent() method of the log widget to be called. repaint() does not do crucial things like copying the window surface to the windowing system.
Is it safe/recommended to just call QtGui.qApp.processEvents() after the QSignalHandler has emitted the logging signal? (When uncommented, logging to the GUI happens directly).
Using a separate thread or asynchronous operations is the recommended way. Calling processEvents() is the recommended way if you can't do that, like in your case.. Even Qt uses it for the same purpose inside QProgressDialog::setValue().
In general, manually processing events can be dangerous and should be done with care. After the call to processEvents(), the complete application state might be different. For example the log widget might no longer exist because the user closed the window! In your example code that is no problem, as the signal/slot connection will automatically disconnect, but imagine if you had tried to access the log widget after it has been deleted due to it being closed - you would have gotten a crash. So be careful.
When reading the documentation for signal connection types, where it says Qt.DirectConnection: The slot is invoked immediately, when the signal is emitted. I would have kind of thought the QSignalHandler should have updated immediately just as the StreamHandler does, shouldn't it?
The slot, in your case QTextBrowser::append(), is called immediately. However, QTextBrowser::append() does not immediately repaint. Instead, it schedules a repaint (via QWidget::update()), and the actual repainting happens when Qt gets around to process the events. That is either when you return to the event loop, or when you call processEvents() manually.
So slots are indeed called right away when emitting a signal, at least when using the default DirectConnection. However repainting does not happen immediately.
I have a Tkinter wrapper written over robocopy.exe, the code is organized as shown below :-
Tkinter Wrapper :
Spawns a new thread and pass the arguments, which includes source/destination and other parameters
(Note : Queue object is also passed to thread, since the thread will read the output from robocopy and will put in queue, main tkinter thread will keep on polling queue and will update Tkinter text widget with the output)
Code Snippet
... Code to poll queue and update tk widget ...
q = Queue.Queue()
t1 = threading.Thread(target=CopyFiles,args=(q,src,dst,), kwargs={"ignore":ignore_list})
t1.daemon = True
t1.start()
Thread : (In a separate file)
Below is the code snippet from thread
def CopyFiles(q,src,dst,ignore=None):
extra_args = ['/MT:15', '/E', '/LOG:./log.txt', '/tee', '/r:2', '/w:2']
if len(ignore) > 0:
extra_args.append('/xf')
extra_args.extend(ignore)
extra_args.append('/xd')
extra_args.extend(ignore)
command_to_pass = ["robocopy",src, dst]
command_to_pass.extend(extra_args)
proc = subprocess.Popen(command_to_pass,stdout=subprocess.PIPE)
while True:
line = proc.stdout.readline()
if line == '':
break
q.put(line.strip())
Code which is called, when tkinter application is closed :-
def onQuit(self):
global t1
if t1.isAlive():
pass
if tkMessageBox.askyesno("Title", "Do you really want to exit?"):
self.destroy()
self.master.destroy()
Problem
Whenever, I close the tkinter application when robocopy is running, python application closes but the robocopy.exe keeps on running.
I have tried setting the thread as daemon, but it has no effect. How can I stop robocopy.exe when onQuit method is called?
To simplify things let's ignore Tkinter and the fact that a separate thread is used.
The situation is that your app spawns a subprocess to execute an external program (robocopy.exe in this question), and you'd need to stop the spawned program from you're application on a certain event (when the Tkinter app is closing in this question).
This requires an inter process communication mechanism, so the spawned process would be notified of the event, and reacts accordingly. A common mechanism is to use signals provided by the OS.
You could send a signal (SIGTERM) to the external process and ask from it to quit. Assuming that the program reacts to signal as expected (most well written applications do) you'll get the desired behavior (the process will terminate).
Using the terminate method on the subprocess will send the proper signal of current platform to the subprocess.
You'd need a reference to the subprocess object proc in the onQuit function (from the provided code I see onQuit is a function and not an object method, so it could use a global variable to access proc), so you can call the terminate method of the process:
def onQuit(self):
global t1, proc
if t1.isAlive():
pass
# by the way I'm not sure about the logic, but I guess this
# below statement should be an elif instead of if
if tkMessageBox.askyesno("Title", "Do you really want to exit?"):
proc.terminate()
self.destroy()
self.master.destroy()
This code assumes you're storing the reference to the subprocess in the global scope, so you'd have to modify the CopyFiles as well.
I'm not sure how robocopy handles terminate signals and I'm guessing that's not something that we have any control over.
If you had more control on the external program (could modify the source), there might have been more options, for example sending messages using stdio, or using a shared memory, etc.
Im trying to send parameter to a Qthread that is not the main thread.
i have this object in my main thread and i want to send it to another thread:
q = Queue()
I want so send q to this thread:
class Sender(QtCore.QThread):
def __init__(self,q):
super(Sender,self).__init__()
self.q=q
def run(self):
while True:
try: line = q.get_nowait()
# or q.get(timeout=.1)
except Empty:
pass
else:
self.emit(QtCore.SIGNAL('tri()'))
Im trying this:
class Sender(QtCore.QThread):
def __init__(self,q):
super(Sender,self).__init__()
self.q=q
self.sender= Sender(q)
But im getting this error:
QObject::connect: Cannot queue arguments of type 'QTextCursor'
(Make sure 'QTextCursor' is registered using qRegisterMetaType().)
How can i do this?
Please help!
There is no problem with your QThread subclass, and how you are setting it up to pass a Queue object. Though I do also recommend passing and setting an optional parent as the second parameter.
What you are mostly likely running into is you are passing objects that perform QtGui operations (drawing related) into your thread. If that is the case, you cannot call any QtGui drawing related methods. They all must be performed in the main thread. Do any data processing in other threads and then emit signals for the main thread to do widget updates.
Look for what you are sending into the queue, and specifically what you are doing with it inside of your thread, as a cue to the source of the error. Somewhere, you are doing something with a QTextCursor, which is trying to trigger and queue up a paint event.
What is the correct way to make my PyQt application quit when killed from the console (Ctrl-C)?
Currently (I have done nothing special to handle unix signals), my PyQt application ignores SIGINT (Ctrl+C). I want it to behave nicely and quit when it is killed. How should I do that?
17.4. signal — Set handlers for asynchronous events
Although Python signal handlers are called asynchronously as far as the Python user is concerned, they can only occur between the “atomic” instructions of the Python interpreter. This means that signals arriving during long calculations implemented purely in C (such as regular expression matches on large bodies of text) may be delayed for an arbitrary amount of time.
That means Python cannot handle signals while the Qt event loop is running. Only when the Python interpreter run (when the QApplication quits, or when a Python function is called from Qt) the signal handler will be called.
A solution is to use a QTimer to let the interpreter run from time to time.
Note that, in the code below, if there are no open windows, the application will quit after the message box regardless of the user's choice because QApplication.quitOnLastWindowClosed() == True. This behaviour can be changed.
import signal
import sys
from PyQt4.QtCore import QTimer
from PyQt4.QtGui import QApplication, QMessageBox
# Your code here
def sigint_handler(*args):
"""Handler for the SIGINT signal."""
sys.stderr.write('\r')
if QMessageBox.question(None, '', "Are you sure you want to quit?",
QMessageBox.Yes | QMessageBox.No,
QMessageBox.No) == QMessageBox.Yes:
QApplication.quit()
if __name__ == "__main__":
signal.signal(signal.SIGINT, sigint_handler)
app = QApplication(sys.argv)
timer = QTimer()
timer.start(500) # You may change this if you wish.
timer.timeout.connect(lambda: None) # Let the interpreter run each 500 ms.
# Your code here.
sys.exit(app.exec_())
Another possible solution, as pointed by LinearOrbit, is signal.signal(signal.SIGINT, signal.SIG_DFL), but it doesn't allow custom handlers.
If you simply wish to have ctrl-c close the application - without being "nice"/graceful about it - then from http://www.mail-archive.com/pyqt#riverbankcomputing.com/msg13758.html, you can use this:
import signal
signal.signal(signal.SIGINT, signal.SIG_DFL)
import sys
from PyQt4.QtCore import QCoreApplication
app = QCoreApplication(sys.argv)
app.exec_()
Apparently this works on Linux, Windows and OSX - I have only tested this on Linux so far (and it works).
18.8.1.1. Execution of Python signal handlers
A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction). This has consequences:
[...]
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
The Qt event loop is implemented in C(++). That means, that while it runs and no Python code is called (eg. by a Qt signal connected to a Python slot), the signals are noted, but the Python signal handlers aren't called.
But, since Python 2.6 and in Python 3 you can cause Qt to run a Python function when a signal with a handler is received using signal.set_wakeup_fd().
This is possible, because, contrary to the documentation, the low-level signal handler doesn't only set a flag for the virtual machine, but it may also write a byte into the file descriptor set by set_wakeup_fd(). Python 2 writes a NUL byte, Python 3 writes the signal number.
So by subclassing a Qt class that takes a file descriptor and provides a readReady() signal, like e.g. QAbstractSocket, the event loop will execute a Python function every time a signal (with a handler) is received causing the signal handler to execute nearly instantaneous without need for timers:
import sys, signal, socket
from PyQt4 import QtCore, QtNetwork
class SignalWakeupHandler(QtNetwork.QAbstractSocket):
def __init__(self, parent=None):
super().__init__(QtNetwork.QAbstractSocket.UdpSocket, parent)
self.old_fd = None
# Create a socket pair
self.wsock, self.rsock = socket.socketpair(type=socket.SOCK_DGRAM)
# Let Qt listen on the one end
self.setSocketDescriptor(self.rsock.fileno())
# And let Python write on the other end
self.wsock.setblocking(False)
self.old_fd = signal.set_wakeup_fd(self.wsock.fileno())
# First Python code executed gets any exception from
# the signal handler, so add a dummy handler first
self.readyRead.connect(lambda : None)
# Second handler does the real handling
self.readyRead.connect(self._readSignal)
def __del__(self):
# Restore any old handler on deletion
if self.old_fd is not None and signal and signal.set_wakeup_fd:
signal.set_wakeup_fd(self.old_fd)
def _readSignal(self):
# Read the written byte.
# Note: readyRead is blocked from occuring again until readData()
# was called, so call it, even if you don't need the value.
data = self.readData(1)
# Emit a Qt signal for convenience
self.signalReceived.emit(data[0])
signalReceived = QtCore.pyqtSignal(int)
app = QApplication(sys.argv)
SignalWakeupHandler(app)
signal.signal(signal.SIGINT, lambda sig,_: app.quit())
sys.exit(app.exec_())
I found a way to do this. The idea is to force qt to process events often enough and in a python callabe to catch the SIGINT signal.
import signal, sys
from PyQt4.QtGui import QApplication, QWidget # also works with PySide
# You HAVE TO reimplement QApplication.event, otherwise it does not work.
# I believe that you need some python callable to catch the signal
# or KeyboardInterrupt exception.
class Application(QApplication):
def event(self, e):
return QApplication.event(self, e)
app = Application(sys.argv)
# Connect your cleanup function to signal.SIGINT
signal.signal(signal.SIGINT, lambda *a: app.quit())
# And start a timer to call Application.event repeatedly.
# You can change the timer parameter as you like.
app.startTimer(200)
w = QWidget()
w.show()
app.exec_()
The asynchronous approach by cg909 / Michael Herrmann is quite interesting to replace timers. Thus, here is a simplified version which also uses the default type for socket.socketpair (SOCK_STREAM).
class SignalWatchdog(QtNetwork.QAbstractSocket):
def __init__(self):
""" Propagates system signals from Python to QEventLoop """
super().__init__(QtNetwork.QAbstractSocket.SctpSocket, None)
self.writer, self.reader = socket.socketpair()
self.writer.setblocking(False)
signal.set_wakeup_fd(self.writer.fileno()) # Python hook
self.setSocketDescriptor(self.reader.fileno()) # Qt hook
self.readyRead.connect(lambda: None) # Dummy function call
The answer from Artur Gaspar worked for me when the terminal window was in focus, but would not work when the GUI was in focus. In order to get my GUI to close (which inherits from QWidget) I had to define the following function in the class:
def keyPressEvent(self,event):
if event.key() == 67 and (event.modifiers() & QtCore.Qt.ControlModifier):
sigint_handler()
Checking to make sure that the event key is 67 makes sure that 'c' was pressed. Then checking the event modifiers determines whether ctrl was being pressed when 'c' was released.
You can use the standard python unix signals handling mechanism:
import signal
import sys
def signal_handler(signal, frame):
print 'You pressed Ctrl+C!'
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print 'Press Ctrl+C'
while 1:
continue
where in signal_handler you can free all resources (close all db sessions etc) and gently close your appliction.
Code example taken from here
I think I have a simpler solution:
import signal
import PyQt4.QtGui
def handleIntSignal(signum, frame):
'''Ask app to close if Ctrl+C is pressed.'''
PyQt4.QtGui.qApp.closeAllWindows()
signal.signal(signal.SIGINT, handleIntSignal)
This just tells the application to try to close all windows if ctrl+c is pressed. If there is an unsaved document, your app should pop up a save or cancel dialog box as if it were exited.
You may also need to connect the QApplication signal lastWindowClosed() to the slot quit() to get the application to actually exit when the windows are closed.
You can piggyback on matplotlib's solution.
matplotlib has a function called _maybe_allow_interrupt hidden in matplotlib.backends.backend_qt
from PyQt5 import QtWidgets
from matplotlib.backends.backend_qt import _maybe_allow_interrupt
import sys
app = QtWidgets.QApplication(sys.argv)
mw = QtWidgets.QMainWindow()
mw.show()
with _maybe_allow_interrupt(app):
app.exec()
Of course since this is not a public function it might change or disappear in future versions of matplotlib, so this is more of a "quick and dirty" solution.