Develop Reference

How to display progress without multi-threading

Let's say I have a PyQt program that goes through a given directory, looks for *JPEG images, and does some processing every time it finds one. Depending on the size of the selected directory, this may take from some seconds to minutes.
I would like to keep my user updated with the status - preferably with something like "x files processed out of y files" . If not, a simple running pulse progress bar by setting progressbar.setRange(0,0) works too.
From my understanding, in order to prevent my GUI from freezing, I will need a seperate thread that process the images, and the original thread that updates the GUI every interval.
But I am wondering if there is any possible way for me to do both in the same thread?

Yes, you can easily do this using processEvents, which is provided for this exact purpose.
I have used this technique for implementing a simple find-in-files dialog box. All you need to do is launch the function that processes the files with a single-shot timer, and then periodically call processEvents in the loop. This is should be good enough to update a counter with the number of files processed, and also allow the user to cancel the process, if necessary.
The only real issue is deciding on how frequently to call processEvents. The more often you call it, the more responsive the GUI will be - but this comes at the cost of considerably slowing the processing of the files. So you may have to experiment a little bit in order to find an acceptable compromise.
UPDATE:
Here's a simple demo that shows how the code could be structured:
import sys, time
from PyQt5 import QtWidgets, QtCore
class Window(QtWidgets.QWidget):
def __init__(self):
super().__init__()
self.button = QtWidgets.QPushButton('Start')
self.progress = QtWidgets.QLabel('0')
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.button)
layout.addWidget(self.progress)
self.button.clicked.connect(self.test)
self._stop = False
self._stopped = True
def test(self):
if self._stopped:
self._stop = False
self.progress.setText('0')
self.button.setText('Stop')
QtCore.QTimer.singleShot(1, self.process)
else:
self._stop = True
def process(self):
self._stopped = False
for index in range(1, 1000):
time.sleep(0.01)
self.progress.setText(str(index))
if not index % 20:
QtWidgets.qApp.processEvents(
QtCore.QEventLoop.AllEvents, 50)
if self._stop:
break
self._stopped = True
self.button.setText('Start')
if __name__ == "__main__":
app = QtWidgets.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())

I could not achieve the thing you need without multi threading and this is not possible because gui can be only updated in main thread. Below is an algorithm how I did this with multithreading.
Let's say you have your application processing images. Then there are the following threads:
Main thread (that blocks by GUI/QApplication-derived classes.exec())
Timer with, for example, 1 second interval which updates a variable and calls a slot in GUI thread which updates a variable in user interface.
A thread which is processing images on your pc.
def process(self):
self._status = "processing image 1"
....
def _update(self):
self.status_label.setText(self._status)
def start_processing(self, image_path):
# create thread for process and run it
# create thread for updating by using QtCore.QTimer()
# connect qtimer triggered signal to and `self._update()` slot
# connect image processing thread (use connect signal to any slot, in this example I'll stop timer after processing thread finishes)
#pyqtSlot()
def _stop_timer():
self._qtimer.stop()
self._qtimer = None
_update_thread.finished.connect(_stop_timer)
In pyqt5 it is possible to assign a pyqtvariable from a one nested thread(first level). So you can make your variable a pyqtvariable with setter and getter and update gui in a setter or think how you can do this by yourself.

You could just use the python threading module and emit a signal in your threaded routine.
Here's a working example
from PyQt4 import QtGui, QtCore
import threading
import time
class MyWidget(QtGui.QWidget):
valueChanged = QtCore.pyqtSignal(int)
def __init__(self, parent=None):
super(MyWidget, self).__init__(parent)
self.computeButton = QtGui.QPushButton("Compute", self)
self.progressBar = QtGui.QProgressBar()
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.computeButton)
layout.addWidget(self.progressBar)
self.computeButton.clicked.connect(self.compute)
self.valueChanged.connect(self.progressBar.setValue)
def compute(self):
nbFiles = 10
self.progressBar.setRange(0, nbFiles)
def inner():
for i in range(1, nbFiles+1):
time.sleep(0.5) # Process Image
self.valueChanged.emit(i) # Notify progress
self.thread = threading.Thread(target = inner)
self.thread.start()
if __name__ == "__main__":
import sys
app = QtGui.QApplication(sys.argv)
widget = MyWidget()
widget.show()
sys.exit(app.exec_())

Gtk widget shows up with delay

Using python3 and gi.repository, I want to have a Gtk.HeaderBar with a Gtk.Button that is replaced by a Gtk.Spinner as soon as you click on it. After a calculation the button should appear again.
Here is an example of how I think it should work but the Gtk.Spinner only shows up after the calculation (in this example sleep) for a very short time. How can I achieve that the spinner shows up for the whole calculation (or sleep)?
from gi.repository import Gtk
import time
class window:
def __init__(self):
self.w = Gtk.Window()
self.button = Gtk.Button('x')
self.button.connect('clicked', self.on_button_clicked)
self.spinner = Gtk.Spinner()
self.hb = Gtk.HeaderBar()
self.hb.props.show_close_button = True
self.hb.pack_start(self.button)
self.w.set_titlebar(self.hb)
self.w.connect('delete-event', Gtk.main_quit)
self.w.show_all()
def on_button_clicked(self, widget):
self.button.hide()
self.hb.pack_start(self.spinner)
self.spinner.show()
self.spinner.start()
time.sleep(5)
self.spinner.stop()
self.hb.remove(self.spinner)
self.button.show()
if __name__ == '__main__':
w = window()
Gtk.main()

GTK+ is an event driven system where the mainloop should be left free to update the UI and everything that takes time (like reading from a file, making a network connection, long calculations) should happen asynchronously.
In your case this would look something like this:
def on_button_clicked(self, widget):
self.button.hide()
self.spinner.show()
self.spinner.start()
GLib.timeout_add_seconds (5, self.processing_finished)
def processing_finished(self):
self.spinner.stop()
self.spinner.hide()
self.button.show()
Note that I removed the pack and remove calls: do those in __init__(). You'll want from gi.repository import GLib in there as well.
This way the main loop is free to update the UI as often as it wants. If you really want to use a blocking call like sleep(), then you'll need to do that in another thread, but my suggestion is to use libraries that are asychronous like that timeout_add_seconds() call.

The problem is time.sleep(): it is a blocking function.
def on_button_clicked(self, widget):
self.button.hide()
self.hb.pack_start(self.spinner)
self.spinner.show()
self.spinner.start()
t = time.time()
while time.time() - t < 5:
Gtk.main_iteration()
self.spinner.stop()
self.hb.remove(self.spinner)
self.button.show()
I think that's what you expect.
Edit: You may put a time.sleep(.1) inside while loop, for cpu saving, but don't forget Gtk.main_iteration(): that is the function that exit from while loop to main loop (show spinner, progress bar and so on).

QDialog.show() method has a delayed reaction

I have a problem i can't quite figure out for some time. I have a main window application and a QDialog that should pop out after clicking one of the buttons, but the show() method on QDialog seems to be waiting for the funcion connected to the "clicked()" signal to end!
I want the dialog to show right after calling the QDialog.show() method, not after all the other code instructions in that function...
Of course in my code I am going to replace the sleep(5) part with much more complicated code, but this pictures the problem and the code I put there is irrelevant to the issue, i think (database connections and updates)
being more specific:
# -*- coding: utf-8 -*-
import sys
import PyQt4
from PyQt4 import QtCore, QtGui
from twython import Twython, TwythonError
from project import Ui_MainWindow
from time import sleep
import psycopg2, globalvals, updater
import updating, noconnection
class UpWindow(QtGui.QDialog):
def __init__(self, parent=None):
QtGui.QDialog.__init__(self, parent, QtCore.Qt.WindowStaysOnTopHint)
self.ui = updating.Ui_updating()
self.ui.setupUi(self)
class NoConnection(QtGui.QDialog):
def __init__(self, parent=None):
QtGui.QDialog.__init__(self, parent, QtCore.Qt.WindowStaysOnTopHint)
self.ui = noconnection.Ui_noConnection()
self.ui.setupUi(self)
QtCore.QObject.connect(self.ui.noConnectionClose, QtCore.SIGNAL("clicked()"), self.close)
class MyCounter(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
self.noConn = NoConnection(self)
self.upWin = UpWindow(self)
QtCore.QObject.connect(self.ui.refreshButton,QtCore.SIGNAL("clicked()"), self.refresh)
QtCore.QObject.connect(self.ui.manageButton,QtCore.SIGNAL("clicked()"), self.manage)
def refresh(self):
self.upWin.show()
self.upWin.show
self.upWin.setVisible(True)
self.setEnabled(False)
self.upWin.setEnabled(True)
#Thats the issue - the sleep instruction is being held
#BEFORE the showing of upWin QDialog
sleep(5)
def manage(self):
print 'ok'
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
myapp = MyCounter()
myapp.upcontent()
myapp.show()
sys.exit(app.exec_())

Think of the any Qt program as a cooperative-multitasking system. Graphics and events in general are handled by the main loop. You don't want to stay long in any slot, because the library won't process signals (say button clicks, repaints, but also other stuff) in the mean time.
If you want to do some heavy processing, or anything that needs to wait for resources while the rest of the program is chugging along, use a QThread.
Another option is to force the event processing with qApp.processEvents() (you can find qApp in QtGui), just before your sleep(5) (or whatever code you're going to put in place of it).
Edit: Now, keep in mind that forcing the event processing will just show the QDialog you're trying to popup. You can't do anything with it (remember, no event processing) without calling again qApp.processEvents() or returning from the slot.

If MyCounter represents a widget that does a long computation and updates a dialog during that time, then sleep(5) is not representative of it, because during those 5 seconds the GUI can't handle events. For a "long running" function you would move the blocking part to a QThread and either poll the thread or connect to a signal it emits as it progresses, either way you would not hold up the GUI event loop during that time (for example, the polling, which takes very little time, would occur in an idle callback). The simplest way to create your test would be to use a timed callback into your MyCounter:
def refresh(self):
... show stuff, then:
self.timer = QTimer()
self.timer.timeout.connect(self.updateDialog)
timer.start(100) # 10 times per sec
def updateDialog(self):
#get thread status
if self.thread.status != self.oldStatus:
self.upWin.updateStatus( self.thread.status )

How to use QTimer inside QThread which uses QWaitCondition? (pyside)

I'm using pyside but (I think) is a generic Qt question.
I know that QThread implementation calls ._exec() method so we should have an event loop on a started QThread. This way we can use QTimer on that thread (I've done this and it works perfectly). My problem is when QWaitCondition is also used, I'd like to have a "consumer" thread with a infinite loop waiting to be notify (from producers) on the QWaitCondition. The problem I have is that with this design I cannot use QTimer inside the consumer Thread.
This is a snippet of the scenario I'm trying to explain:
from PySide import QtGui
from PySide import QtCore
import sys
class MainWindow(QtGui.QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
self.button = QtGui.QPushButton(self)
self.button.setText("Periodical")
self.button.clicked.connect(self.periodical_call)
self.thread = QtCore.QThread()
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.loop)
self.thread.start()
def closeEvent(self, x):
self.worker.stop()
self.thread.quit()
self.thread.wait()
def periodical_call(self):
self.worker.do_stuff("main window") # this works
self.timer = QtCore.QTimer()
self.timer.timeout.connect(self.do_stuff) # this also works
self.timer.start(2000)
def do_stuff(self):
self.worker.do_stuff("timer main window")
class Worker(QtCore.QObject):
def do_stuff_timer(self):
do_stuff("timer worker")
def do_stuff(self, origin):
self.origin = origin
self.wait.wakeOne()
def stop(self):
self._exit = True
self.wait.wakeAll()
def loop(self):
self.wait = QtCore.QWaitCondition()
self.mutex = QtCore.QMutex()
self._exit = False
while not self._exit:
self.wait.wait(self.mutex)
print "loop from %s" % (self.origin,)
self.timer = QtCore.QTimer()
self.timer.setSingleShot(True)
self.timer.timeout.connect(self.do_stuff_timer)
self.timer.start(1000) # <---- this doesn't work
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
frame = MainWindow()
frame.show()
sys.exit(app.exec_())
Once you click the button we obtain an output like this:
loop from main window
loop from timer main window
loop from timer main window
loop from timer main window
...
This means that the QTimer created inside loop() method is never executed by the event loop.
If I change the design from QWaitCondition to Signals (which is better design imho) the QTimer works, but I'd like to know why they are not working when QWaitCondition is used.

To still process events in a long running task (aka a continuous loop) you need to call QCoreApplication::processEvents().
This will essentially go through all of the queued up slots for your thread.
Calling this function is also necessary for signals (if they are a QueuedConnection signal/slot connection) to make it out of the current thread and into another one.
For PySides, you will need to call PySide.QtCore.QCoreApplication.processEvents()

Your method loop completely occupies thread.
It doesn't return control to event loop. Timer sends its events to event loop which doesn't gain control.
IMO your wile loop is faulty.
One way to fix it is add QApplication.processEvents() in loop (bad approach).
I think you want something else, here are my corrections:
def loop(self):
self.timer = QtCore.QTimer()
self.timer.setSingleShot(False)
self.timer.timeout.connect(self.do_stuff_timer)
self.timer.start(1000)
def stop(self):
self.timer.stop()
this will invoke do_stuff_timer every second until you will call stop.

wxPython Pango error when using a while True loop in a thread

In this program I get an error when I use a while True loop in the thread. Without the loop I get no error. Of course in the real program I don't update a label continuously. Any idea what I'm doing wrong?
This is the program:
import wx
import thread
class Example(wx.Frame):
def __init__(self, parent):
wx.Frame.__init__(self,parent)
self.InitUI()
def InitUI(self):
self.SetSize((250, 200))
self.Show(True)
self.text = wx.StaticText(self, label='',pos=(20,30))
thread.start_new_thread(self.watch,(self,None))
def watch(self,dummy,e):
while True:
self.text.SetLabel('Closed')
def main():
ex = wx.App()
Example(None)
ex.MainLoop()
if __name__ == '__main__':
main()
And this is the error:
Pango:ERROR:/build/pango1.0-LVHqeM/pango1.0-1.30.0/./pango/pango- layout.c:3801:pango_layout_check_lines: assertion failed: (!layout->log_attrs) Aborted
Any suggestions as to what I'm doing wrong? I'm (obviously) new to threading.

I am not exactly sure if that is what causes you problem, but... You should not interact with the GUI from another thread. You should use wx.CallAfter(). I would consider adding sleep inside the loop also.
wx.CallAfter() documentation says:
Call the specified function after the current and pending event handlers have been completed. This is also good for making GUI method calls from non-GUI threads. Any extra positional or keyword args are passed on to the callable when it is called.
Updated code would than be:
import wx
import thread
import time
class Example(wx.Frame):
def __init__(self, parent):
wx.Frame.__init__(self,parent)
self.InitUI()
def InitUI(self):
self.SetSize((250, 200))
self.Show(True)
self.text = wx.StaticText(self, label='',pos=(20,30))
thread.start_new_thread(self.watch,(self,None))
def watch(self,dummy,e):
while True:
time.sleep(0.1)
wx.CallAfter(self.text.SetLabel, 'Closed')
def main():
ex = wx.App()
Example(None)
ex.MainLoop()
if __name__ == '__main__':
main()
Maybe you can also consider using wx.Timer.
BTW: Your code runs OK on my PC with Windows 7 and wxPython 2.8.

In addition to the no updates from background threads rule, I've found that in similar situations (high frequency update of UI objects) that it really helps to only update the value if it has changed from what is already displayed. That can greatly reduce the load on the application because if the value does not change then there will be no need for sending and processing paint events, moving pixels to the screen, etc. So in this example I would add a new method that is called via CallAfter that compares the current value in the widget with the requested value, and only calls SetLabel if they are different.