I have a program that is structured like this:
The GUI is made with wxPython and is in the Main Thread. After launching the application, the GUI thread creates Thread1, which creates a static class Class1, which creates Thread2.
Thread1 talks with the GUI using wx.PostEvent, and everything works fine. I need Thread1 to communicate also with Thread2, so I decided to do that using pyPubSub. Thread2 needs to work in the background because it contains some operations that are executed periodically, but it contains also a function (let's say my_func()) that Thread1 needs to call on certain events. I decided to put my_func() in Thread2 because I don't want it to stop the execution of Thread1, but that's exactly what happens: in Thread1 after some events I use pub.sendMessage("events", message="Hello") to trigger my_func(); Thread2 is structured like this:
class Thread2(Thread)
def __init__(self, parent):
super().__init__()
self.parent = parent
pub.subscribe(self.my_func, "events")
def run(self):
while True:
# do stuff
def my_func(self, message):
# do other stuff using the message
# call parent functions
The parent of Thread2 is Class1, so when I create the thread in Class1 I do:
self.t2 = Thread2(self)
self.t2.start()
Why does Thread2 stops the execution of Thread1?
To put in different words what was said in the comments, pypubsub's sendMessage is actually calling your listener function and thus waits for it to return before moving on. If this listener function takes a significant amount of time to run, then the behavior you obtain is an apparent "blocking" of your thread.
Here you probably want to 'send' a lightweight message to trigger the computation and 'listen' to the result in another place, in a 'non-blocking' fashion.
Related
This class suppose to handle a long task in a separate thread.
Despite moving my object to a new thread, and using queued connection, the signal is handled at the main gui thread instead of self.thread. The main gui thread is the thread that did called startit of course..
This is the code:
class DoLongProcess(QObject):
def __init__(self,task):
QObject.__init__(self)
self._task=task
self._realtask=None
self.thread = QThread()
self.moveToThread(self.thread)
self.thread.started.connect(self.run,Qt.QueuedConnection)
#Slot()
def run(self):
self._realtask()
def startit(self,*params):
self._realtask= partial(self._task,*params)
self.thread.start()
I use pyside6. I tried to to split __init__ into two methods and moving the thread from outside. Also tried without the QueuedConnection attribute (which suppose to do this exactly). None of this had an effect.
Do you know why it doesn't work as expected?
This question already has answers here:
Updating GUI elements in MultiThreaded PyQT
(2 answers)
Closed 1 year ago.
I am creating a Server-Client GUI application in Python using PyQt5. Whenever a new clients is connected to server I need to display the client details at server side. I am using a separate thread for sockets. Whenever I call the client_connect function to add the widget at server side I get error due to which widget is not displayed. I think since the GUI and socket codes are in different threads due to this I am getting the error.
QObject::setParent: Cannot set parent, new parent is in a different thread
QObject::installEventFilter(): Cannot filter events for objects in a different thread.
QBasicTimer::start: QBasicTimer can only be used with threads started with QThread
QBasicTimer::start: QBasicTimer can only be used with threads started with QThread
QBasicTimer::start: QBasicTimer can only be used with threads started with QThread
QObject::startTimer: Timers can only be used with threads started with QThread
QBasicTimer::start: QBasicTimer can only be used with threads started with QThread
Main Function
if __name__ == "__main__":
thread1 = threading.Thread(target = ui.server_socket)
thread1.start()
client_connect Function - I have created a separate file for the widget and I am inserting the widget in tableWidget. It works if i directly call the function but if i call it from the socket code it gives me error.
def client_connect(self,clientid):
self.clientCount = self.clientCount + 1
self.clientList.append(clientid)
self.clientDict[clientid] = QtWidgets.QWidget()
self.clientDict[clientid].ui = clients()
self.clientDict[clientid].ui.setupUi(self.clientDict[clientid])
self.clientDict[clientid].ui.label_clientid.setText(str(clientid))
self.tableWidget_client.setRowCount(self.clientCount)
self.tableWidget_client.setCellWidget(self.clientCount-1,0,self.clientDict[clientid])
Socket programming
def start_socket(self):
self.ssock.listen(20)
while True:
conn, c_addr = self.ssock.accept()
print(conn,c_addr)
thread2 = threading.Thread(target=self.handle_client, args=(conn,c_addr))
thread2.start()
def handle_client(self, conn, c_addr):
try:
self.c = conn.recv(1024).decode(self.FORMAT)
thread3 = threading.Thread(target = self.client_connect, args = (self.c_id,))
thread3.start()
except socket.error as e:
print(e)
As you suspected, you shouldn't directly modify PyQt objects from different threads. You may read the docs about threads and objects.
Docs specifically mention that:
If you are calling a function on an QObject subclass that doesn't live in the current thread and the object might receive events, you must protect all access to your QObject subclass's internal data with a mutex; otherwise, you may experience crashes or other undesired behavior. [.......] On the other hand, you can safely emit signals from your QThread::run() implementation, because signal emission is thread-safe.
The way PyQt wants you to do it is to use Signals and Slots. Below is a small example of emitting a signal from worker's run function.
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
# Some Ui stuff is happening here
self.worker = Worker(self.startParm)
self.worker.client_connected.connect(self.display_client_info)
self.worker.start() # When you want to process a new client connection
def display_client_info(self, client_id):
# Code here should be similar to what you're trying to do in client_connect
class Worker(QThread):
def __init__(self):
super(Worker, self).__init__()
self.client_connected = pyqtSignal(int)
def run(self, clientid):
# The code you want to be executed in a thread
# For example whatever happens in handle_client, except in your design you're going
# through a middle thread and this is a basic example.
# Eventually emit client_connected signal with the client id:
self.client_connected.emit(client_id)
The main concepts to take from this:
Only modify PyQt objects from the main thread
Create signals and connect them to functions that will update UI for you in the main thread
Emit signals from within threads when you need to
Overriding QThread's run function is a more simple but can be limited way of using this. You may look into creating QObjects and using moveToThread function for more options.
I'm trying to run a function after my thread has completed but the function is not called. Code structure:
class():
def functiontocall() # uses data calculated in thread for plotting. only works when thread is complete
do something with self.A
def watchthread():
thread()
functiontocall()
# since this function depends on variable A, it throws an error.
# I tried: if thread.join == True: functiontocall but this did not call the function.
def thread():
def run():
pythoncom.CoInitialize()
--- do stuff --
for i in 1000:
thousands of calculations while updating state in GUI ---
A = result
self.A = A
thread = threading.Thread(target=run)
thread.start()
note: i removed 'self' for simplicity.
thread.join should tell me when the thread has finished but for some reason i still cant run the functiontocall.
Is this a bad way of organizing threads in general?
Edit: I can call the function after the thread is finished but I cannot access variables when the thread is running. e.g. 0-100% progress for a progress bar in my GUI. when I use:
def watchthread():
thread()
thread.join()
functiontocall()
I cannot update the status of the thread in my GUI. It just waits until the calculations are finished then runs functiontocall().
Because you're using threads, once the thread had started Python will move onto the next thing, it will not wait for the thread to finish unless you've asked it to.
With your code, if you want to wait for the thread function to finish before moving on then it doesn't sound like you need threading, a normal function would run, complete, and then Python will move onto running functiontocall()
If there's a reason you need to use threads which isn't coming across in the example then I would suggest using a thread.join()
threads = [] # list to hold threads if you have more than one
t = threading.Thread(target=run)
threads.append(t)
for thread in threads: # wait for all threads to finish
thread.join()
functiontocall() # will only run after all threads are done
Again, I'd suggest relooking at whether threads is what you need to use here as it doesn't seem apparent.
To update this answer based on the new information, this may be the way you want to have a variable be accessible. In this case the threads would all update your class variable A, your GUI update function also reads this periodically and updates your GUI.
class ThisClass():
def __init__(self):
self.A = 0
def function_to_call(self):
while self.A != 100: # assuming this is a progress bar to 100%
# update in GUI
def run(self):
# does calculations
with lock: # to prevent issues with threads accessing variable at the same time
self.A += calculations
def progress(self):
threads = [] # list to hold threads if you have more than one
t = threading.Thread(target=run)
threads.append(t)
f = threading.Thread(target=self.function_to_call)
threads.append(f)
for thread in threads:
thread.start()
for thread in threads: # wait for all threads to finish
thread.join()
I'm trying to let a button spin up a new thread that does nothing but sleep for 30 seconds. However, the main thread is blocked if the slot is a lambda function. Does anyone know why this is the case and not behaving what I was expecting it to be? Here's my code:
# ...
def setup(self):
# ...
self.pushButton_TestConnection.clicked.connect(self.process)
def process(self):
self.worker_thread = QtCore.QThread()
self.worker = Worker()
self.worker.moveToThread(self.worker_thread)
self.worker_thread.started.connect(lambda: self.worker.sleep(30))
self.worker_thread.start()
class Worker(QtCore.QObject):
def sleep(self, secs):
time.sleep(secs)
It works fine with the following
self.worker_thread.started.connect(self.worker.sleep)
self.worker_thread.start()
class Worker(QtCore.QObject):
def sleep(self):
time.sleep(30)
Thanks
In Qt the thread in which code is executed is determined by the thread affinity of the object receiving a signal. If you call a objects method directly from a different thread, it will be executed in the calling thread, no matter the thread affinity of the called object.
Lambdas and other python callables do not have a thread affinity (they're not QObjects after all, it's just a nice feature of PyQt allowing you to connect a signal to any python callable), so they will always be executed in the main (GUI) thread.
So in this case the lambda is executed in the GUI thread, so the worker.sleep call will also be executed there and will block it until the call returns.
To make this work, you need to connect the started signal directly to a slot of the Worker object, or communicate with the Worker using a signal you emit from the lambda.
There are a lot of tutorials etc. on Python and asynchronous coding techniques, but I am having difficulty filtering the through results to find what I need. I am new to Python, so that doesn't help.
Setup
I currently have two objects that look sort of like this (please excuse my python formatting):
class Alphabet(parent):
def init(self, item):
self.item = item
def style_alphabet(callback):
# this method presumably takes a very long time, and fills out some properties
# of the Alphabet object
callback()
class myobj(another_parent):
def init(self):
self.alphabets = []
refresh()
def foo(self):
for item in ['a', 'b', 'c']:
letters = new Alphabet(item)
self.alphabets.append(letters)
self.screen_refresh()
for item in self.alphabets
# this is the code that I want to run asynchronously. Typically, my efforts
# all involve passing item.style_alphabet to the async object / method
# and either calling start() here or in Alphabet
item.style_alphabet(self.screen_refresh)
def refresh(self):
foo()
# redraw screen, using the refreshed alphabets
redraw_screen()
def screen_refresh(self):
# a lighter version of refresh()
redraw_screen()
The idea is that the main thread initially draws the screen with incomplete Alphabet objects, fills out the Alphabet objects, updating the screen as they complete.
I've tried a number of implementations of threading.Tread, Queue.Queue, and even futures, and for some reason they either haven't worked, or they have blocked the main thread. so that the initial draw doesn't take place.
A few of the async methods I've attempted:
class Async (threading.Thread):
def __init__(self, f, cb):
threading.Thread.__init__(self)
self.f = f
self.cb = cb
def run(self):
self.f()
self.cb()
def run_as_thread(f):
# When I tried this method, I assigned the callback to a property of "Alphabet"
thr = threading.Thread(target=f)
thr.start()
def run_async(f, cb):
pool = Pool(processes=1)
result = pool.apply_async(func=f, args=args, callback=cb)
I ended up writing a thread pool to deal with this use pattern. Try creating a queue and handing a reference off to all the worker threads. Add task objects to the queue from the main thread. Worker threads pull objects from the queue and invoke the functions. Add an event to each task to be signaled on the worker thread at task completion. Keep a list of task objects on the main thread and use polling to see if the UI needs an update. One can get fancy and add a pointer to a callback function on the task objects if needed.
My solution was inspired by what I found on Google: http://code.activestate.com/recipes/577187-python-thread-pool/
I kept improving on that design to add features and give the threading, multiprocessing, and parallel python modules a consistent interface. My implementation is at:
https://github.com/nornir/nornir-pools
Docs:
http://nornir.github.io/packages/nornir_pools.html
If you are new to Python and not familiar with the GIL I suggest doing a search for Python threading and the global interpreter lock (GIL). It isn’t a happy story. Generally I find I need to use the multiprocessing module to get decent performance.
Hope some of this helps.