Develop Reference

Simplest way for PyQT Threading

I have a GUI in PyQt with a function addImage(image_path). Easy to imagine, it is called when a new image should be added into a QListWidget. For the detection of new images in a folder, I use a threading.Thread with watchdog to detect file changes in the folder, and this thread then calls addImage directly.
This yields the warning that QPixmap shouldn't be called outside the gui thread, for reasons of thread safety.
What is the best and most simple way to make this threadsafe? QThread? Signal / Slot? QMetaObject.invokeMethod? I only need to pass a string from the thread to addImage.

You should use the built in QThread provided by Qt. You can place your file monitoring code inside a worker class that inherits from QObject so that it can use the Qt Signal/Slot system to pass messages between threads.
class FileMonitor(QObject):
image_signal = QtCore.pyqtSignal(str)
#QtCore.pyqtSlot()
def monitor_images(self):
# I'm guessing this is an infinite while loop that monitors files
while True:
if file_has_changed:
self.image_signal.emit('/path/to/image/file.jpg')
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.file_monitor = FileMonitor()
self.thread = QtCore.QThread(self)
self.file_monitor.image_signal.connect(self.image_callback)
self.file_monitor.moveToThread(self.thread)
self.thread.started.connect(self.file_monitor.monitor_images)
self.thread.start()
#QtCore.pyqtSlot(str)
def image_callback(self, filepath):
pixmap = QtGui.QPixmap(filepath)
...

I believe the best approach is using the signal/slot mechanism. Here is an example. (Note: see the EDIT below that points out a possible weakness in my approach).
from PyQt4 import QtGui
from PyQt4 import QtCore
# Create the class 'Communicate'. The instance
# from this class shall be used later on for the
# signal/slot mechanism.
class Communicate(QtCore.QObject):
myGUI_signal = QtCore.pyqtSignal(str)
''' End class '''
# Define the function 'myThread'. This function is the so-called
# 'target function' when you create and start your new Thread.
# In other words, this is the function that will run in your new thread.
# 'myThread' expects one argument: the callback function name. That should
# be a function inside your GUI.
def myThread(callbackFunc):
# Setup the signal-slot mechanism.
mySrc = Communicate()
mySrc.myGUI_signal.connect(callbackFunc)
# Endless loop. You typically want the thread
# to run forever.
while(True):
# Do something useful here.
msgForGui = 'This is a message to send to the GUI'
mySrc.myGUI_signal.emit(msgForGui)
# So now the 'callbackFunc' is called, and is fed with 'msgForGui'
# as parameter. That is what you want. You just sent a message to
# your GUI application! - Note: I suppose here that 'callbackFunc'
# is one of the functions in your GUI.
# This procedure is thread safe.
''' End while '''
''' End myThread '''
In your GUI application code, you should create the new Thread, give it the right callback function, and make it run.
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
import os
# This is the main window from my GUI
class CustomMainWindow(QtGui.QMainWindow):
def __init__(self):
super(CustomMainWindow, self).__init__()
self.setGeometry(300, 300, 2500, 1500)
self.setWindowTitle("my first window")
# ...
self.startTheThread()
''''''
def theCallbackFunc(self, msg):
print('the thread has sent this message to the GUI:')
print(msg)
print('---------')
''''''
def startTheThread(self):
# Create the new thread. The target function is 'myThread'. The
# function we created in the beginning.
t = threading.Thread(name = 'myThread', target = myThread, args = (self.theCallbackFunc))
t.start()
''''''
''' End CustomMainWindow '''
# This is the startup code.
if __name__== '__main__':
app = QtGui.QApplication(sys.argv)
QtGui.QApplication.setStyle(QtGui.QStyleFactory.create('Plastique'))
myGUI = CustomMainWindow()
sys.exit(app.exec_())
''' End Main '''
EDIT
Mr. three_pineapples and Mr. Brendan Abel pointed out a weakness in my approach. Indeed, the approach works fine for this particular case, because you generate / emit the signal directly. When you deal with built-in Qt signals on buttons and widgets, you should take another approach (as specified in the answer of Mr. Brendan Abel).
Mr. three_pineapples adviced me to start a new topic in StackOverflow to make a comparison between the several approaches of thread-safe communication with a GUI. I will dig into the matter, and do that tomorrow :-)

Python lambda functions connected to Qt signals don't run when created in a different thread

When a QObject is created in a different thread and moved back to the main thread using QObject.moveToThread, lambda signals "disconnect" (they won't fire). My guess is that regular slots are linked to the QObject that is moved to the main thread, so they run in the main thread's event loop, but lambda functions aren't linked to the QObject, so there is no event loop for them to run.
This can be seen in the following short Python code:
if __name__ == "__main__":
from traits.etsconfig.api import ETSConfig
ETSConfig.toolkit = 'qt4'
import threading
from PySide import QtGui, QtCore
class MyObject(QtCore.QObject):
def __init__(self, button):
super(MyObject, self).__init__()
button.clicked.connect(self.mySlot)
button.clicked.connect(lambda: self.mySlot('lambda'))
#
def mySlot(self, printing='object'):
print printing
#
myObj = None
# global variable to keep it in memory
def myThread(mainThread, button):
global myObj
myObj = MyObject(button)
myObj.moveToThread(mainThread)
#
if __name__ == '__main__':
appQT = QtGui.QApplication([])
#
myWidget = QtGui.QWidget()
myButton = QtGui.QPushButton('Press to see output')
myLayout = QtGui.QVBoxLayout(myWidget)
myLayout.addWidget(myButton)
myWidget.show()
#
mainThread = QtCore.QThread.currentThread()
if True:
# run myThread in a new thread
# prints only the first slot (object slot)
threading.Thread(target=myThread, args=[mainThread, myButton]).start()
else:
# run myThread in this thread
# prints both slots (object and lambda slots)
myThread(mainThread, myButton)
#
appQT.exec_()
You can see how the results differ from expected by changing the conditional from True to False.
When it is set to True, the output after clicking the button is:
object
When it is set to False, the output after clicking the button is:
object
lambda
My question is, can someone explain more precisely why it behaves in this way, and is there an easy way to keep the lambda slots working when moving the QObject back to the main thread?

Okay, I figured out some details about what is going on. It's still a partial answer, but I think it's more suitable as an answer rather than update to the question.
It seems that I was correct in my original question that the slots are linked to their instance object's QObject event loop (thread), but only if that slot is a bound method (has an object instance).
If you look into the PySide source code on Github, you'll see that it defines the receiver (the QObject that receives the signal) based on the type of slot it receives. So if you pass the QtSignal.connect() function a bound object method, the receiver is defined as the slotMethod.__self__ (which is PyMethod_GET_SELF(callback)). If you pass it a general callable object (for example a lambda function) which is not bound (no __self__ property), the receiver is simply set to NULL. The receiver tells Qt which event loop to send the signal to, so if it's NULL, it doesn't know to send the signal to the main event loop.
Here's a snippet of the PySide source code:
static bool getReceiver(QObject *source, const char* signal, PyObject* callback, QObject** receiver, PyObject** self, QByteArray* callbackSig)
{
bool forceGlobalReceiver = false;
if (PyMethod_Check(callback)) {
*self = PyMethod_GET_SELF(callback);
if (%CHECKTYPE[QObject*](*self))
*receiver = %CONVERTTOCPP[QObject*](*self);
forceGlobalReceiver = isDecorator(callback, *self);
} else if (PyCFunction_Check(callback)) {
*self = PyCFunction_GET_SELF(callback);
if (*self && %CHECKTYPE[QObject*](*self))
*receiver = %CONVERTTOCPP[QObject*](*self);
} else if (PyCallable_Check(callback)) {
// Ok, just a callable object
*receiver = 0;
*self = 0;
}
...
...
}
Does this help us fix our problem with the lambda functions? Not really... If we bind the lambda functions using the following (with types.MethodType), the behavior does not change:
import types
class MyObject(QtCore.QObject):
def __init__(self, button):
super(MyObject, self).__init__()
button.clicked.connect(self.mySlot)
thisLambda = lambda self=self : self.mySlot('hello')
self.myLambda = types.MethodType( thisLambda, self )
button.clicked.connect(self.myLambda)
#
def mySlot(self, printing='object'):
print printing
Output:
object
This binding is definitely part of the problem since I have demonstrated below that the same behavior occurs with non-bound global methods, and by binding them using types.MethodType(), it fixes the problem:
import types
def abc(self):
print 'global1'
def xyz(self):
print 'global2'
class MyObject(QtCore.QObject):
def __init__(self, button):
super(MyObject, self).__init__()
button.clicked.connect(self.mySlot)
self.xyz = types.MethodType( xyz, self )
button.clicked.connect(abc)
button.clicked.connect(self.xyz)
#
def mySlot(self, printing='object'):
print printing
Output:
object
global2
Anyways, it seems the simplest solution is just to not create the QObject in a separate thread in the first place, but this answer is a step towards understanding why it doesn't work properly.

PyQt / PySide Echo signals of another object without subclassing

I'm managing some long running tasks using signals and slots to track progress and such. I had a Run class that starts like this:
from PySide import QtCore, QtGui
class Run(QtCore.QObject):
running = QtCore.Signal(bool)
finished = QtCore.Signal(object)
status = QtCore.Signal(str)
message = QtCore.Signal(object)
progress = QtCore.Signal(float)
time_remaining_update = QtCore.Signal(str)
initialized = QtCore.Signal()
failed = QtCore.Signal(object)
pitch_update = QtCore.Signal(float)
def __init__(self, parent=None):
super().__init__(parent=parent)
#... rest of the class ...
I recently refactored some things and wrote a Task class which includes a Run instance as an attribute. I would like to "echo" many of the Run's signals upstream. This is my current solution:
class Task(QtCore.QObject):
#Signals echoed from Run
running = QtCore.Signal(bool)
status = QtCore.Signal(str)
message = QtCore.Signal(object)
progress = QtCore.Signal(float)
time_remaining_update = QtCore.Signal(str)
initialized = QtCore.Signal()
def __init__(self, run, parent=None):
super().__init__(parent=parent)
self.run = run
#Echo signals from run
signals = ['running', 'status', 'message', 'progress', 'time_remaining_update', 'initialized']
for signal in signals:
#Re-emit the signal
getattr(self.run, signal).connect(lambda *args: getattr(self, signal).emit(*args))
#... rest of class ...
I confirmed that the lambda *args: func(*args) pattern will avoid passing anything to func if nothing is passed to the lambda, which is necessary to obey the different signals' signatures.
Is this the best way to handle "echoing" or "bouncing" signals from one QObject to another? I'm basically trying to preserve some of the Run API while changing/adding other aspects with my Task API.
I realize that I can get all of the Run API by subclassing and overriding things I want to change, but I didn't want to subclass since Task handles restarting the run after failure. I'd like to keep the run logic separate from the code that handles restarting the run. There are also Run signals and methods that I don't want propagated by Task.
The only more elegant solution I can think of is to automate the creation of the Task signals... maybe make a echo_signals decorator that takes a QObject and a list of signal names that then does these things?
Is there a Qt feature for implementing this more elegantly? Any suggestions?

In Qt you can connect signals to other signals, unless I'm missing something regarding your example.

pyqt QThread blocking main thread

I'm trying to create a simple threaded application whereby i have a method which does some long processing and a widget that displays a loading bar and cancel button.
My problem is that no matter how i implement the threading it doesn't actually thread - the UI is locked up once the thread kicks in. I've read every tutorial and post about this and i'm now resorting on asking the community to try and solve my problem as i'm at a loss!
Initially i tried subclassing QThread until the internet said this was wrong. I then attempted the moveToThread approach but it made zero difference.
Initialization code:
loadingThreadObject = LoadThread(arg1)
loadingThread = PythonThread()
loadingThreadObject.moveToThread(loadingThread)
loadingThread.started.connect(loadingThreadObject.load)
loadingThread.start()
PythonThread class (apparently QThreads are bugged in pyQt and don't start unless you do this):
class PythonThread (QtCore.QThread):
def __init__(self, parent=None):
QtCore.QThread.__init__(self, parent)
def start(self):
QtCore.QThread.start(self)
def run(self):
QtCore.QThread.run(self)
LoadThread class:
class LoadThread (QtCore.QObject):
results = QtCore.Signal(tuple)
def __init__ (self, arg):
# Init QObject
super(QtCore.QObject, self).__init__()
# Store the argument
self.arg = arg
def load (self):
#
# Some heavy lifting is done
#
loaded = True
errors = []
# Emits the results
self.results.emit((loaded, errors))
Any help is greatly appreciated!
Thanks.
Ben.

The problem was with the SQL library I was using (a custom in-house solution) which turned out not to be thread safe and thus performed blocking queries.
If you are having a similar problem, first try removing the SQL calls and seeing if it still blocks. If that solves the blocking issue, try reintroducing your queries using raw SQL via MySQLdb (or the equivalent for the type of DB you're using). This will diagnose whether or not the problem is with your choice of SQL library.

The function connected to the started signal will run the thread which it was connected, the main GUI thread. However, a QThread's start() function executes its run() method in the thread after the thread is initialized so a subclass of QThread should be created and its run method should run LoadThread.load, the function you want to execute. Don't inherit from PythonThread, there's no need for that. The QThread subclass's start() method should be used to start the thread.
PS: Since in this case the subclass of QThread's run() method only calls LoadThread.load(), the run() method could be simply set to LoadThread.load:
class MyThread(QtCore.QThread):
run = LoadThread.load # x = y in the class block sets the class's x variable to y
An example:
import time
from PyQt4 import QtCore, QtGui
import sys
application = QtGui.QApplication(sys.argv)
class LoadThread (QtCore.QObject):
results = QtCore.pyqtSignal(tuple)
def __init__ (self, arg):
# Init QObject
super(QtCore.QObject, self).__init__()
# Store the argument
self.arg = arg
def load(self):
#
# Some heavy lifting is done
#
time.sleep(5)
loaded = True
errors = []
# Emits the results
self.results.emit((loaded, errors))
l = LoadThread("test")
class MyThread(QtCore.QThread):
run = l.load
thread = MyThread()
button = QtGui.QPushButton("Do 5 virtual push-ups")
button.clicked.connect(thread.start)
button.show()
l.results.connect(lambda:button.setText("Phew! Push ups done"))
application.exec_()

libvlc and dbus interface

I'm trying a to create a basic media player using libvlc which will be controlled through dbus. I'm using the gtk and libvlc bindings for python. The code is based on the official example from the vlc website
The only thing I modified is to add the dbus interface to the vlc instance
# Create a single vlc.Instance() to be shared by (possible) multiple players.
instance = vlc.Instance()
print vlc.libvlc_add_intf(instance, "dbus"); // this is what i added. // returns 0 which is ok
All is well, the demo works and plays any video files. but for some reason the dbus control module doesn't work (I can't believe I just said the dreaded "doesn't work" words):
I already have the working client dbus code which binds to the MPRIS 2 interface. I can control a normal instance of a VLC media player - that works just fine, but with the above example nothing happens. The dbus control module is loaded properly, since libvlc_add_intf doesn't return an error and i can see the MPRIS 2 service in D-Feet (org.mpris.MediaPlayer2.vlc).
Even in D-Feet, trying to call any of the methods of the dbus vlc object returns no error but nothing happens.
Do I need to configure something else in order to make the dbus module control the libvlc player?
Thanks
UPDATE
It seems that creating the vlc Instance and setting a higher verbosity, shows that the DBus calls are received but they have no effect whatsoever on the player itself.
Also, adding the RC interface to the instance instead of DBus, has some problems too: When I run the example from the command line it drops me to the RC interface console where i can type the control commands, but it has the same behaviour as DBus - nothing happens, no error, nada, absolutely nothing. It ignores the commands completely.
Any thoughts?
UPDATE 2
Here is the code that uses libvlc to create a basic player:
from dbus.mainloop.glib import DBusGMainLoop
import gtk
import gobject
import sys
import vlc
from gettext import gettext as _
# Create a single vlc.Instance() to be shared by (possible) multiple players.
instance = vlc.Instance("--one-instance --verbose 2")
class VLCWidget(gtk.DrawingArea):
"""Simple VLC widget.
Its player can be controlled through the 'player' attribute, which
is a vlc.MediaPlayer() instance.
"""
def __init__(self, *p):
gtk.DrawingArea.__init__(self)
self.player = instance.media_player_new()
def handle_embed(*args):
if sys.platform == 'win32':
self.player.set_hwnd(self.window.handle)
else:
self.player.set_xwindow(self.window.xid)
return True
self.connect("map", handle_embed)
self.set_size_request(640, 480)
class VideoPlayer:
"""Example simple video player.
"""
def __init__(self):
self.vlc = VLCWidget()
def main(self, fname):
self.vlc.player.set_media(instance.media_new(fname))
w = gtk.Window()
w.add(self.vlc)
w.show_all()
w.connect("destroy", gtk.main_quit)
self.vlc.player.play()
DBusGMainLoop(set_as_default = True)
gtk.gdk.threads_init()
gobject.MainLoop().run()
if __name__ == '__main__':
if not sys.argv[1:]:
print "You must provide at least 1 movie filename"
sys.exit(1)
if len(sys.argv[1:]) == 1:
# Only 1 file. Simple interface
p=VideoPlayer()
p.main(sys.argv[1])
the script can be run from the command line like:
python example_vlc.py file.avi
The client code which connects to the vlc dbus object is too long to post so instead pretend that i'm using D-Feet to get the bus connection and post messages to it.
Once the example is running, i can see the players dbus interface in d-feet, but i am unable to control it. Is there anything else that i should add to the code above to make it work?

I can't see your implementation of your event loop, so it's hard to tell what might be causing commands to not be recognized or to be dropped. Is it possible your threads are losing the stacktrace information and are actually throwing exceptions?
You might get more responses if you added either a psuedo-code version of your event loop and DBus command parsing or a simplified version?

The working programs found on nullege.com use ctypes. One which acted as a server used rpyc. Ignoring that one.
The advantages of ctypes over dbus is a huge speed advantage (calling the C library code, not interacting using python) as well as not requiring the library to implement the dbus interface.
Didn't find any examples using gtk or dbus ;-(
Notable examples
PyNuvo vlc.py
Milonga Tango DJing program
Using dbus / gtk
dbus uses gobject mainloop, not gtk mainloop. Totally different beasts. Don't cross the streams! Some fixes:
Don't need this. Threads are evil.
gtk.gdk.threads_init()
gtk.main_quit() shouldn't work when using gobject Mainloop. gobject mainloop can't live within ur class.
if __name__ == '__main__':
loop = gobject.MainLoop()
loop.run()
Pass in loop into ur class. Then call to quit the app
loop.quit()
dbus (notify) / gtk working example
Not going to write ur vlc app for u. But here is a working example of using dbus / gtk. Just adapt to vlc. Assumed u took my advise on gtk above. As u know any instance of DesktopNotify must be called while using gobject.Mainloop . But u can place it anywhere within ur main class.
desktop_notify.py
from __future__ import print_function
import gobject
import time, dbus
from dbus.exceptions import DBusException
from dbus.mainloop.glib import DBusGMainLoop
class DesktopNotify(object):
""" Notify-OSD ubuntu's implementation has a 20 message limit. U've been warned. When queue is full, delete old message before adding new messages."""
#Static variables
dbus_loop = None
dbus_proxy = None
dbus_interface = None
loop = None
#property
def dbus_name(self):
return ("org.freedesktop.Notifications")
#property
def dbus_path(self):
return ("/org/freedesktop/Notifications")
#property
def dbus_interface(self):
return self.dbus_name
def __init__(self, strInit="initializing passive notification messaging")
strProxyInterface = "<class 'dbus.proxies.Interface'>"
""" Reinitializing dbus when making a 2nd class instance would be bad"""
if str(type(DesktopNotify.dbus_interface)) != strProxyInterface:
DesktopNotify.dbus_loop = DBusGMainLoop(set_as_default=True)
bus = dbus.SessionBus(mainloop=DesktopNotify.dbus_loop)
DesktopNotify.dbus_proxy = bus.get_object(self.dbus_name, self.dbus_path)
DesktopNotify.dbus_interface = dbus.Interface(DesktopNotify.dbus_proxy, self.dbus_interface )
DesktopNotify.dbus_proxy.connect_to_signal("NotificationClosed", self.handle_closed)
def handle_closed(self, *arg, **kwargs):
""" Notification closed by user or by code. Print message or not"""
lngNotificationId = int(arg[0])
lngReason = int(arg[1])
def pop(self, lngID):
""" ID stored in database, but i'm going to skip this and keep it simple"""
try:
DesktopNotify.dbus_interface.CloseNotification(lngID)
except DBusException as why:
print(self.__class__.__name__ + ".pop probably no message with id, lngID, why)
finally:
pass
def push(self, strMsgTitle, strMsg, dictField):
""" Create a new passive notification (took out retrying and handling full queues)"""
now = time.localtime( time.time() )
strMsgTime = strMsg + " " + time.asctime(now)
del now
strMsgTime = strMsgTime % dictField
app_name="[your app name]"
app_icon = ''
actions = ''
hint = ''
expire_timeout = 10000 #Use seconds * 1000
summary = strMsgTitle
body = strMsgTime
lngNotificationID = None
try:
lngNotificationID = DesktopNotify.dbus_interfacec.Notify(app_name, 0, app_icon, summary, body, actions, hint, expire_timeout)
except DBusException as why:
#Excellent spot to delete oldest notification and then retry
print(self.__class__.__name__ + ".push Being lazy. Posting passive notification was unsuccessful.", why)
finally:
#Excellent spot to add to database upon success
pass