Develop Reference

How to display progress without multi-threading

Let's say I have a PyQt program that goes through a given directory, looks for *JPEG images, and does some processing every time it finds one. Depending on the size of the selected directory, this may take from some seconds to minutes.
I would like to keep my user updated with the status - preferably with something like "x files processed out of y files" . If not, a simple running pulse progress bar by setting progressbar.setRange(0,0) works too.
From my understanding, in order to prevent my GUI from freezing, I will need a seperate thread that process the images, and the original thread that updates the GUI every interval.
But I am wondering if there is any possible way for me to do both in the same thread?

Yes, you can easily do this using processEvents, which is provided for this exact purpose.
I have used this technique for implementing a simple find-in-files dialog box. All you need to do is launch the function that processes the files with a single-shot timer, and then periodically call processEvents in the loop. This is should be good enough to update a counter with the number of files processed, and also allow the user to cancel the process, if necessary.
The only real issue is deciding on how frequently to call processEvents. The more often you call it, the more responsive the GUI will be - but this comes at the cost of considerably slowing the processing of the files. So you may have to experiment a little bit in order to find an acceptable compromise.
UPDATE:
Here's a simple demo that shows how the code could be structured:
import sys, time
from PyQt5 import QtWidgets, QtCore
class Window(QtWidgets.QWidget):
def __init__(self):
super().__init__()
self.button = QtWidgets.QPushButton('Start')
self.progress = QtWidgets.QLabel('0')
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.button)
layout.addWidget(self.progress)
self.button.clicked.connect(self.test)
self._stop = False
self._stopped = True
def test(self):
if self._stopped:
self._stop = False
self.progress.setText('0')
self.button.setText('Stop')
QtCore.QTimer.singleShot(1, self.process)
else:
self._stop = True
def process(self):
self._stopped = False
for index in range(1, 1000):
time.sleep(0.01)
self.progress.setText(str(index))
if not index % 20:
QtWidgets.qApp.processEvents(
QtCore.QEventLoop.AllEvents, 50)
if self._stop:
break
self._stopped = True
self.button.setText('Start')
if __name__ == "__main__":
app = QtWidgets.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())

I could not achieve the thing you need without multi threading and this is not possible because gui can be only updated in main thread. Below is an algorithm how I did this with multithreading.
Let's say you have your application processing images. Then there are the following threads:
Main thread (that blocks by GUI/QApplication-derived classes.exec())
Timer with, for example, 1 second interval which updates a variable and calls a slot in GUI thread which updates a variable in user interface.
A thread which is processing images on your pc.
def process(self):
self._status = "processing image 1"
....
def _update(self):
self.status_label.setText(self._status)
def start_processing(self, image_path):
# create thread for process and run it
# create thread for updating by using QtCore.QTimer()
# connect qtimer triggered signal to and `self._update()` slot
# connect image processing thread (use connect signal to any slot, in this example I'll stop timer after processing thread finishes)
#pyqtSlot()
def _stop_timer():
self._qtimer.stop()
self._qtimer = None
_update_thread.finished.connect(_stop_timer)
In pyqt5 it is possible to assign a pyqtvariable from a one nested thread(first level). So you can make your variable a pyqtvariable with setter and getter and update gui in a setter or think how you can do this by yourself.

You could just use the python threading module and emit a signal in your threaded routine.
Here's a working example
from PyQt4 import QtGui, QtCore
import threading
import time
class MyWidget(QtGui.QWidget):
valueChanged = QtCore.pyqtSignal(int)
def __init__(self, parent=None):
super(MyWidget, self).__init__(parent)
self.computeButton = QtGui.QPushButton("Compute", self)
self.progressBar = QtGui.QProgressBar()
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.computeButton)
layout.addWidget(self.progressBar)
self.computeButton.clicked.connect(self.compute)
self.valueChanged.connect(self.progressBar.setValue)
def compute(self):
nbFiles = 10
self.progressBar.setRange(0, nbFiles)
def inner():
for i in range(1, nbFiles+1):
time.sleep(0.5) # Process Image
self.valueChanged.emit(i) # Notify progress
self.thread = threading.Thread(target = inner)
self.thread.start()
if __name__ == "__main__":
import sys
app = QtGui.QApplication(sys.argv)
widget = MyWidget()
widget.show()
sys.exit(app.exec_())

Dialog in thread freeze whole app despite Gdk.threads_enter/leave()

I've got an application which makes something in background. To inform the user it update some widgets with its progress. That works.
But somethings there is an error or something else in this background operation so it has to display a dialog. This freeze my whole application although I handle everything with the Threading-Lock. An example of code with exactly my problem is this:
import threading, time
from gi.repository import Gtk, Gdk
def background(label, parent):
for t in range(5):
label.set_text(str(t))
time.sleep(1)
Gdk.threads_enter()
dlg = Gtk.MessageDialog(
type=Gtk.MessageType.INFO,
buttons=Gtk.ButtonsType.OK,
message_format="Time is gone.",
title="Info")
dlg.run()
dlg.destroy()
Gdk.threads_leave()
def main():
window = Gtk.Window()
window.connect("delete-event", Gtk.main_quit)
label = Gtk.Label()
window.add(label)
window.show_all()
thread = threading.Thread(target=background, args=(label, window))
Gdk.threads_init()
Gdk.threads_enter()
thread.start()
Gtk.main()
Gdk.threads_leave()
if __name__=="__main__":
main()

In gtk3, all gtk functions like adding/removing/changing widgets must be executed by the gtk thread (the thread that's running Gtk.main()).
The fixed code:
import threading, time
from gi.repository import Gtk, Gdk, GLib # need GLib for GLib.PRIORITY_DEFAULT
# a short utility function that I like to use.
# makes the Gtk thread execute the given callback.
def add_mainloop_task(callback, *args):
def cb(args):
args[0](*args[1:])
return False
args= [callback]+list(args)
Gdk.threads_add_idle(GLib.PRIORITY_DEFAULT, cb, args)
def background(label, parent):
for t in range(5):
#~ label.set_text(str(t)) # let the gtk thread do this.
add_mainloop_task(label.set_text, str(t))
time.sleep(1)
#~ Gdk.threads_enter() # don't need this.
dlg = Gtk.MessageDialog(
type=Gtk.MessageType.INFO,
buttons=Gtk.ButtonsType.OK,
message_format="Time is gone.",
title="Info")
# put these two functions calls inside a little function, and let the gtk thread execute it.
def run_dialog(dlg):
dlg.run()
dlg.destroy()
add_mainloop_task(run_dialog, dlg)
#~ Gdk.threads_leave() # don't need this.
def main():
window = Gtk.Window()
window.connect("delete-event", Gtk.main_quit)
label = Gtk.Label()
window.add(label)
window.show_all()
thread = threading.Thread(target=background, args=(label, window))
Gdk.threads_init()
#~ Gdk.threads_enter() # don't need this.
thread.start()
Gtk.main()
#~ Gdk.threads_leave() # don't need this.
if __name__=="__main__":
main()

How to do background task in gtk3-python?

I have this main thread:
Gui.py
from gi.repository import Gtk, Gdk
import Process
import gobject
class gui():
def __init__(self):
self.window = Gtk.Window()
self.window.connect('delete-event', Gtk.main_quit)
self.box = Gtk.Box()
self.window.add(self.box)
self.label = Gtk.Label('idle')
self.box.pack_start(self.label, True, True, 0)
self.progressbar = Gtk.ProgressBar()
self.box.pack_start(self.progressbar, True, True, 0)
self.button = Gtk.Button(label='Start')
self.button.connect('clicked', self.on_button_clicked)
self.box.pack_start(self.button, True, True, 0)
self.window.show_all()
gobject.threads_init()
Gdk.threads_enter()
Gtk.main()
Gdk.threads_leave()
def working1():
self.label.set_text('working1')
t = Process.Heavy()
t.heavyworks1()
self.label.set_text('idle')
def on_button_clicked(self, widget):
Gdk.threads_enter()
working1()
Gdk.threads_leave()
if __name__ == '__main__':
gui = gui()
This code will generate this gui:
and I have second modul which will do the logic.
Process.py
import threading
class Heavy(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def heavyworks1(self):
#doing heavy works1
#return result
def heavyworks2(self, *param):
#doing heavy works2
#return result
When I execute this, the operation works, but the gui became freeze. How to do it well?
EDIT:
as user4815162342 said, I change my code to this:
from gi.repository import Gtk, Gdk, GLib
import Process
import gobject
import threading
class gui():
def __init__(self):
self.window = Gtk.Window()
self.window.connect('delete-event', Gtk.main_quit)
self.box = Gtk.Box()
self.window.add(self.box)
self.label = Gtk.Label('idle')
self.box.pack_start(self.label, True, True, 0)
self.progressbar = Gtk.ProgressBar()
self.box.pack_start(self.progressbar, True, True, 0)
self.button = Gtk.Button(label='Start')
self.button.connect('clicked', self.on_button_clicked)
self.box.pack_start(self.button, True, True, 0)
self.window.show_all()
gobject.threads_init()
GLib.threads_init()
Gdk.threads_init()
Gdk.threads_enter()
Gtk.main()
Gdk.threads_leave()
def init_progress(self, func, arg):
self.label.set_text('working1')
self.worker = threading.Thread(target=func, args=[arg])
self.running = True
gobject.timeout_add(200, self.update_progress)
self.worker.start()
def update_progress(self):
if self.running:
self.progressbar.pulse()
return self.running
def working(self, num):
Process.heavyworks2(num)
gobject.idle_add(self.stop_progress)
def stop_progress(self):
self.running = False
self.worker.join()
self.progressbar.set_fraction(0)
self.label.set_text('idle')
def on_button_clicked(self, widget):
self.init_progress(self.working, 100000)
if __name__ == '__main__':
gui = gui()
with that code, program sometimes working but sometimes getting this error.
1.
**
Gtk:ERROR:/build/buildd/gtk+3.0-3.4.2/./gtk/gtktextview.c:3726:gtk_text_view_validate_onscreen: assertion failed: (priv->onscreen_validated)
Aborted (core dumped)
2.
*** glibc detected *** python: free(): invalid next size (fast): 0x09c9f820 ***
3.
Segmentation fault (core dumped)

You didn't actually start the thread, you only instantiated an object that can be used to start it. A full solution requires a careful separation of responsibilities between your GUI thread and your worker thread(s). What you want to do is the following:
Do your heavy calculation in the separate thread, spawned and joined by the GUI code. The calculation should not spawn its own threads, nor does it need to be aware of threads (except for being thread-safe, of course).
When the thread is done, use gobject.idle_add() to tell the GUI that the progress indicator can be withdrawn. (gobject.idle_add is about the only GTK function that is safe to call from another thread.)
With such a setup, the GUI remains fully responsive and progress bar updated no matter what the calculation does, and the GUI thread is guaranteed to notice when the calculation finishes. Two additional points regarding your current code:
Instantiate threading.Thread instead of inheriting from it. That way you don't need to bother with implementing run(). In both cases you have to call thread.start(), though, to start off the thread.
Don't call threads_enter() and threads_leave(), unless you really know what you are doing. Just remember that as long as you call all your GTK functions from a single thread (the same thread in which you initialized GTK), you'll be fine.
Here is proof-of-concept code that implements the above suggestions:
def working1(self):
self.label.set_text('working1')
self.work_thread = threading.Thread(self.run_thread)
self.running = True
gobject.timeout_add(200, self.update_progress)
self.work_thread.start()
# the GUI thread now returns to the mainloop
# this will get periodically called in the GUI thread
def update_progress(self):
if self.running:
self.progressbar.pulse() # or set_fraction, etc.
return self.running
# this will get run in a separate thread
def run_thread(self):
Process.heavyworks1() # or however you're starting your calculation
gobject.idle_add(self.stop_progress)
# this will get run in the GUI thread when the worker thread is done
def stop_progress(self):
self.running = False
self.work_thread.join()
self.label.set_text('idle')

As you suggested you need to start another thread for this. Usually threading in python is pretty straightforward but it can get tricky with GUIs.
This should be of help: Python. Doing some work on background with Gtk GUI

python GTK: Dialog with information that app is closing

I have a problem
My application on close has to logout from web application. It's take some time. I want to inform user about it with " logging out" information
class Belt(gtk.Window):
def __init__(self):
super(Belt, self).__init__()
self.connect("destroy", self.destroy)
def destroy(self, widget, data=None):
if self.isLogged:
md = gtk.MessageDialog(None, gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT, ico, gtk.BUTTONS_NONE, txt)
md.showall()
self.send('users/logout.json', {}, False, False)
gtk.main_quit()
def main(self):
if self.iniError is False:
gtk.gdk.threads_init()
gtk.gdk.threads_enter()
gtk.main()
gtk.gdk.threads_leave()
if __name__ == "__main__":
app = Belt()
app.main()
When I try to show dialog in destroy method only window does appear, without icon and text.
I want to, that this dialog have no confirm button, just the information, and dialog have to be destroy with all app.
Any ideas?
Sorry for my poor English

Basically, GTK has to have the chance to work through the event queue all the time. If some other processing takes a long time and the event queue is not processed in the meantime, your application will become unresponsive. This is usually not what you want, because it may result in your windows not being updated, remaining grey, having strange artefacts, or other kinds of visible glitches. It may even cause your window system to grey the window out and offer to kill the presumably frozen application.
The solutution is to make sure the event queue is being processed. There are two primary ways to do this. If the part that takes long consists of many incremental steps, you can periodically process the queue yourself:
def this_takes_really_long():
for _ in range(10000):
do_some_more_work()
while gtk.events_pending():
gtk.main_iteration()
In the general case, you'll have to resort to some kind of asynchronous processing. The typical way is to put the blocking part into its own thread, and then signal back to the main thread (which sits in the main loop) via idle callbacks. In your code, it might look something like this:
from threading import Thread
import gtk, gobject
class Belt(gtk.Window):
def __init__(self):
super(Belt, self).__init__()
self.connect("destroy", self.destroy)
self.show_all()
self.isLogged = True
self.iniError = False
def destroy(self, widget, data=None):
if self.isLogged:
md = gtk.MessageDialog(None, gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT, 0, gtk.BUTTONS_NONE, "Text")
md.show_all()
Thread(target=self._this_takes_very_long).start()
def main(self):
if self.iniError is False:
gtk.gdk.threads_init()
gtk.gdk.threads_enter()
gtk.main()
gtk.gdk.threads_leave()
def _this_takes_very_long(self):
self.send('users/logout.json', {}, False, False)
gobject.idle_add(gtk.main_quit)
if __name__ == "__main__":
app = Belt()
app.main()

wxPython Pango error when using a while True loop in a thread

In this program I get an error when I use a while True loop in the thread. Without the loop I get no error. Of course in the real program I don't update a label continuously. Any idea what I'm doing wrong?
This is the program:
import wx
import thread
class Example(wx.Frame):
def __init__(self, parent):
wx.Frame.__init__(self,parent)
self.InitUI()
def InitUI(self):
self.SetSize((250, 200))
self.Show(True)
self.text = wx.StaticText(self, label='',pos=(20,30))
thread.start_new_thread(self.watch,(self,None))
def watch(self,dummy,e):
while True:
self.text.SetLabel('Closed')
def main():
ex = wx.App()
Example(None)
ex.MainLoop()
if __name__ == '__main__':
main()
And this is the error:
Pango:ERROR:/build/pango1.0-LVHqeM/pango1.0-1.30.0/./pango/pango- layout.c:3801:pango_layout_check_lines: assertion failed: (!layout->log_attrs) Aborted
Any suggestions as to what I'm doing wrong? I'm (obviously) new to threading.

I am not exactly sure if that is what causes you problem, but... You should not interact with the GUI from another thread. You should use wx.CallAfter(). I would consider adding sleep inside the loop also.
wx.CallAfter() documentation says:
Call the specified function after the current and pending event handlers have been completed. This is also good for making GUI method calls from non-GUI threads. Any extra positional or keyword args are passed on to the callable when it is called.
Updated code would than be:
import wx
import thread
import time
class Example(wx.Frame):
def __init__(self, parent):
wx.Frame.__init__(self,parent)
self.InitUI()
def InitUI(self):
self.SetSize((250, 200))
self.Show(True)
self.text = wx.StaticText(self, label='',pos=(20,30))
thread.start_new_thread(self.watch,(self,None))
def watch(self,dummy,e):
while True:
time.sleep(0.1)
wx.CallAfter(self.text.SetLabel, 'Closed')
def main():
ex = wx.App()
Example(None)
ex.MainLoop()
if __name__ == '__main__':
main()
Maybe you can also consider using wx.Timer.
BTW: Your code runs OK on my PC with Windows 7 and wxPython 2.8.

In addition to the no updates from background threads rule, I've found that in similar situations (high frequency update of UI objects) that it really helps to only update the value if it has changed from what is already displayed. That can greatly reduce the load on the application because if the value does not change then there will be no need for sending and processing paint events, moving pixels to the screen, etc. So in this example I would add a new method that is called via CallAfter that compares the current value in the widget with the requested value, and only calls SetLabel if they are different.