Im currently trying to learn Networking with Python asyncore and pyqt4.
I coded a small server, which basically listens on some port, and resends all messages it recieves to the sender.
Since both qts QApplication.exec_() and asyncore.loop() are functions which never return i could not start them both in one thread, so i stared asyncore.loop() in a seperate daemon thread.
Whenever my server class (derived from asyncore.dispatcher) establishes or drops a connection, or sends/recieves a message, it calls methods of my window class (derived from QtGui.QMainWindow), which displays the information in a QPlainTextEdit.
But the text is not visible, unless you mark the text with the mouse.
Python console displays following error msg:
QObject::connect: Cannot queue arguments of type 'QTextBlock'
(Make sure 'QTextBlock' is registered using qRegisterMetaType().)
QObject::connect: Cannot queue arguments of type 'QTextCursor'
(Make sure 'QTextCursor' is registered using qRegisterMetaType().)
I read on some forum, that this may be caused by calling qt-functions from another Thread, and that using signals & slots instead of plain function calling may fix the issue, but i have tried signals aswell, and i still get this error.
So, (if that is really the cause of my problems) whats the correct way to call methods of an qt object from another thread ?
EDIT More Info:
the asyncore.loop() call is located in the child thread, well its not really blocking, but only during the runtime of asyncore.loop() my Server class (asyncore.dispatcher) can do networking.
So, during the runtime of asyncore.loop() the methods of my Server class ARE called by asyncore.loop() (=child thread), and in these i
tried to emit signals to the window class running in the main thread
EDIT: Seems like i got it working now, i had some errors in my code, everything works as intended with signals now.
EDIT: small example: http://paste2.org/p/635612 (dead link)
It appears you're trying to access QtGui classes from a thread other than the main thread. Like in some other GUI toolkits (e.g. Java Swing), that's not allowed. From the Threads and QObjects web page:
Although QObject is reentrant, the GUI
classes, notably QWidget and all its
subclasses, are not reentrant. They
can only be used from the main thread.
A solution is to use signals and slots for communication between the main thread (where the GUI objects live) and your secondary thread(s). Basically, you emit signals in one thread that get delivered to the QObjects via the other thread. The page I linked to above has a good discussion of this. Actually, the whole section on Thread Support in Qt is a good read.
One potential issue you could run into is that, normally, to get full signals and slots support working across threads, you need to start an event loop in the child thread using QThread::exec() (or the PyQt equivalent) so that signals can be delivered to slots in the QObjects that live there. In your case, it sounds like you're making a blocking call to asyncore.loop(), which will prevent you from doing this. But, if you only need to emit signals in one direction (from the child thread to widgets in the main thread), I don't think you'll have a problem.
Related
I have some PyQt5 python code which downloads a set of files. I am using multiprocessing.dummy, as a wrapper for threading, using apply_async to obtain simultaneous downloads.
From my threaded function I want to update a progress bar in the main gui, but when I do this, QT selects a queued connection for the signal and the function in the main thread that plays with the downloaded files starts (the files having been downloaded), and then, all of a sudden, the progress bar updates reporting the files as downloading, even though they have downloaded some time before.
If I create the signal with Qt.DirectConnection, I get the desired behaviour but after the files have downloaded I get a segfault, and the main thread quits. I am sure there must be someway of dealing with this issue, since it is not an unusual task, but having tried mutexes, signals to signals, and a number of other attempts, I am almost tearing my hair out. I'm sure the answer is obvious.
I've read the other answers here, to similar problems but there is nothing particularly enlightening.
PyQt5 Signals and Threading
suggests to connect from inside the created thread, but I'm loathe to try this since the thread is created in a class that is composited into a custom object in the main thread, and I don't really want to pass a reference down the hierarchy. If I have to I suppose, but I tried instead connecting a signal inside the threaded function to call method inside the class that then called the signal, but with no luck, just the same results.
Many thanks in advance.
I eventually discovered that the segfault was being caused by my joining the threadpool from the main thread after calling the thread_pool.apply_async. I removed the close and join statements and the directconnection on the signal worked fine, as expected. I continued to get the warning:
QObject::setParent: Cannot set parent, new parent is in a different thread
But the program worked as expected.
Thanks!
i'm redesigning my software because the last one i did crashed due to a wrong access to a Qt interface by a process started with
serialThread = threading.Thread(target=serialCycle)
serialThread.setDaemon(True)
serialThread.start()
serialThread mainly wait for incoming serial data, decode them and place them in a list (probably i'll move to a numpy array). Informations about connection (serial port, speed, how many data have been received) have to be written in the qt ui. I mainly use global variables to exchange informations between main thread and the serial one; i can ensure that each variable is written only by one thread and read from the other to avoid problems.
The Qt updating is a bit hard to do sending informations to main thread so i looked for another solution. I found this thread ( Updating GUI elements in MultiThreaded PyQT ), but i didn't get the point. If i start a thread with slot and signal can't I have crashes due to multiple access to the same variable?
(Question1) started thread runs in parallel and so they shouldn't change the qt interface...
(Question2) Can't an entire .ui window be loaded "linked" to a different thread so this thread can update it (and obviously not the main thread)
(Question3) Which one is the simpliest way to have a gui that can be updated by different thread about the self status (and to let the user to change parameters)?
Thanks
I've made routine with Python's threading module to run a subprocess and wait for it to finish.
I do the threading with the following line :
t1=Thread(target=self.routineFunction, args=(self, "file1.txt", "file2.txt", self.nextFunction)).start()
Inside my function routineFunction(self,file1,file2,nextFunction) I call the next function to run once the subprocess has finished running.
Everything works fine until then.
But if I create new QObject items in my next function, I receive a lot of errors :
- "QPixmap: It is not safe to use pixmaps outside the GUI thread"
- "QObject::startTimer: QTimer can only be used with threads started with QThread"
My guess is that when I call nextFunction from the routine it is ran in the same thread as the routine, hence the errors.
Is there a way to call a function from the routine inside the "main" or "normal" thread ?.
Thank you for your help.
Generally, it is okay to call functions from other threads. But many GUI libraries (QT is among them) have some restrictions on this behavior.
For example, there are designated thread called 'GUI thread' which handles all graphical stuff, like dispatching messages from OS, redrawing windows, etc. And you also restricted to work with GUI withing this only thread. So, for example, you should not create QPixmap in other threads.
QTimer uses some QThread's internal data, so you should use such timers in threads only started with QThread, but not with plain Python thread module.
Returning to your question, if you want ot work with QT, you should spawn your threads using QThread and post events to GUI thread using postEvent() method. This will guarantee consistence of QT internal data structures.
So, you can ran your code in any QT thread, but if you want to work with GUI (your QObject uses QPixmap, so it is the case), you need to handle such calls only in GUI thread.
Ok so I think that werewindle solution is possible. Unfortunatly, I found it difficult to convert my current script from Thread to QThread (I am new to threading in general).
I found a workaround :
Instead of passing nextFunction as an argument, I pass a queue = Queue.Queue() and I use queue.put(True) if my subprocess in my new thread is a success. There is no more nextFunction, I simply wait for a value in the queue with queue.get().
By doing this, I can then continue in the GUI thread.
Background: I've written a simple RUMPS/PyObjC app that at various points makes HTTP requests and pops system alert dialogs. Most of the HTTP requests are threaded to avoid blocking the main execution.
Query: In PyObjC you're not supposed to create alert dialogs in background threads (only the main process is safe to do this) so I was wondering if there was a sensible pattern I could use to allow child threads to communicate back to their parent to notify it that an alert dialog should be created.
As an example:
Main program has a popAlert() method defined, which call PyObjC functions to create alert dialog.
Main program launches a thread that executes an HTTP request and wants to pop an alert when done.
Child thread, when HTTP request completes, somehow calls back to the main process to trigger popAlert().
At the moment I have the child thread creating the alert dialog which apparently isn't safe in PyObjC.
I'm fairly new to threading/multiprocessing and the inter-thread communication examples I've seen so far seem to mostly involve child threads receiving messages from a queue-like mechanism, with the parent optionally blocking until the queue is empty. I haven't yet seen anything that indicates if it's possible to have the parent respond to a message from a child thread in the way I describe.
If doing this requires re-architecting, that's fine too – I'm not wedded to any particular multiprocessing approach.
Any advice would be much appreciated. Please let me know if I can provide more detail. Many thanks for reading.
Take a look at pyobjc_performSelectorOnMainThread. It might be what you need. Here's a contrived usage example:
class MyClass (NSObject):
def divideByZero_(self, arg):
return arg/0
def doit(self):
try:
result = self.performSelectorOnMainThread_withObject_(
'divideByZero:', 55)
print result
except:
print "Division failed"
You can find it being used in a more concretely in this PyObjC example app. Check out WSTConnectionWindowControllerClass.py in particular.
You can also use a queue to transfer messages from children to parent. The problem here is, that you need a message loop in the main thread, that reads the message queue regularly to display the messages.
I am creating an application using Python.
I first designed an API, that is working fine.
I am now designing my GUI. The GUI starts a Thread that is used to perform tasks against the API.
Up to now, I used the Observer pattern to handle communication through the different layers.
Basically, communication can be of two types (mainly):
- The GUI asking the Thread (and the API subsequently) to START/STOP
- The API giving information back to the Thread, that propagates to the GUI.
Here is a simple schema of the current architecture I am talking about.
One arrow means "notify", basically.
My concern is that when the application Thread communicates, both the Gui and the API receive the message because they subscribed. Thing is, each message is only meant to be read by one of the two.
What I did to solve that is to send an message together with an ID. Each of the three elements have an id and they know whether the message is for them or now.
But I am not sure if this is the "correct" (understand nicest) way to do it. What if I have more parties in the future ?
I started thinking about some kind of manager handling communication, but It would then have to be at the top of the architecture, and I am not sure how to further organize it :s.
I am not asking for a complete solution, but mainly ideas or best practises by more experienced people ;)
I can keep handling multiple Observer pattern in this simple case.
But I was thinking about porting my code on a server. In this case, I am likely to have way more than one thread for the application, and handling API calls will become quite impossible.
Link to the code I am talking about :
GUI, ApplicationThread and Application API.
You want to look at notify and update methods.
Thx for any piece of advice !
One of the nice implementation of the observer pattern I've met is the signal/slot system in Qt. Objects have signals, and slots (which are actually methods) can be connected to signals. The connected slots are called when the signals are emitted.
It seems to me that some of your problems may stem from the fact you have single communication canal in each of your objects. This forces you to have some dispatch mechanism in every update method, and makes the code quite complex.
Taking inspiration from Qt, you could have different signals for each kind of message and recipient. The code for signal would look like :
class Signal:
def __init__(self):
self.subs = []
def subscribe(self, s):
self.subs.append(s)
def signal(self, *args, **kwargs):
for s in self.subs:
s(*args, **kwargs)
For example, the gui would have a signal stop_signal and the thread a method to handle it :
def handle_gui_stop(self):
self.console_logger.debug("Facemovie is going to stop")
self.my_logger.debug("Facemovie is going to stop")
self.stop_process = True
# ...
Somewhere in the initialization code, we would tie everything together :
gui.stop_signal.subscribe(thread.handle_gui_stop)
I recently created a GUI app with similar architecture (GUI thread + a separate work thread), and I end up creating an explicit protocol between threads, in a form of two queues (from Queue Python module). One queue is for requests made by GUI and is consumed by worker thread(s). The other queue is for answers produces by worker threads and consumed by GUI.
I find it much clearer when communication between threads is explicit, you have full control over when and where the updating is done (GUI methods can be called only from the GUI thread).
A natural extension of this model in a server environment is a message queue protocol like AMQP.
Application thread must be more explicit about communications since it's the communicator between GUI and Application API. This may be achieved by separating working sets (queues) coming from GUI and from Application API. Also, Application Thread must be able to handle delivery-type of pattern, which includes command senders and recipients. This includes managing communications between different queues (e.g. GUI queue has pending command which is awaiting for the command in the Application API queue. Once this one completes, then the Application Thread passes results back between the queues). And each queue is the observer itself.
In terms of extending the application, it seems to me that you want to add more GUIs in the future, which will be handled by request/response (or sender/receiver) pattern implementation described above (that will suffice).
If you plan to add more layers vertically instead of horisontally then you should not use the same Application Thread to communicate upper between new layers. Well, physically it can be the same, but virtually it must be different at least. This can be achieved by exactly implementing what I described above again (separate queues). By introducing dynamically adding queues you will open the possibility to add a new layer (new layer simply corresponds to new queue then).
Specially with GUIs, I recomend another pattern: MVC. It includes the Observer patterns in it and is more robust than the Observer alone.
It solves your concern because it separates concerns: each layer has a very specific role and you can change any of them, as long as you don't change the interface between them.