Why would a call to processEvents block doing nothing for up to 9 seconds?
I have an application with a PySide-based QT interface, where the UI code sits as a decoupled layer over the lower level actual application logic. When the user performs an action which executes lower level application logic that may run for a while, effectively what happens is:
Directly on the GUI thread a progress dialog is displayed.
Directly on the GUI thread, the lower level logic starts a worker thread.
Directly on the GUI thread, the lower level logic loops updating the progress dialog (indirectly/decoupled) and ticking the application event queue via QtGui.qApp.processEvents() (again indirectly/decoupled).
On the worker thread, QT functions are invoked (again indirectly/decoupled) in reaction to events, and these happen on the GUI thread via slots/signals, running when the GUI thread (as mentioned above) calls processEvents().
Directly on the GUI thread, before the loop exits, the last processEvents() call blocks for around 9 seconds. This is after all the logic on the worker thread is over and done with, and there are no more functions waiting to run on it via signal/slot calls. Nothing related to my application is happening in this call. What is it doing there? Why is it blocking? I've tried passing in a max processing time of 300 ms and seeing if it exits, but this makes no difference. The call locks up as long as it wants to.
The progress dialog closes and the user gets focus back.
This is all spread over a lot of files, implemented in a decoupled manner. I'll try and provide snippets to give a picture of the flow.
The decoupled lower level logic worker loop:
while not completed_event.wait(0.1) and not work_state.is_cancelled():
work_completeness, work_description = work_state.get_completeness(), work_state.get_description()
for client in self.clients:
if work_completeness != last_completeness or work_description != last_description:
client.event_prolonged_action_update(client is acting_client, work_description, step_count * work_completeness)
# THE LAST CALL TO THE NEXT LINE LOCKS UP FOR NO REASON
client.event_tick(client is acting_client)
last_completeness, last_description = work_completeness, work_description
The PySide/QT layer client event_tick function:
def event_tick(self, active_client):
# THIS IS WHERE THE LOCK UP HAPPENS
QtGui.qApp.processEvents()
Signal/slot usage in the PySide/QT layer to get worker thread calls happening on the GUI thread:
def event_pre_line_change(self, active_client, line0, line_count):
self.pre_line_change_signal.emit((line0, line_count))
def event_post_line_change(self, active_client, line0, line_count):
self.post_line_change_signal.emit((line0, line_count))
def event_uncertain_reference_modification(self, active_client, data_type_from, data_type_to, address, length):
self.uncertain_reference_modification_signal.emit((data_type_from, data_type_to, address, length))
The reason that I delegate the calls on the worker thread over to the GUI thread using signals/slots, is that this is what PySide/QT requires given they will update the UI.
Reproduction case if you want to dig deeper:
Download and get the code running according to the GitHub project readme text.
Download the test file "NEURO" (<200KB in size) from mega (apologies, it was easiest site to upload to).
Load NEURO in PeaSauce.
Go to offset 0x1A19E (CTRL-G)
Change the data type to code (Menu: Edit / Change address datatype / Code)
Observe progress dialog come and go.
Observe ~9 second lock up.
Related
I learned the hard way that tkinter is not thread-safe when starting independent threads with tkinter functionality from the main tkinter thread. I got error messages in a (for me) non-reproducible way, mostly "main thread is not in main loop" in connection to internal del calls after I stopped my application. Sometimes the kernel crashed during or after execution, often everything just ran smoothly.
These independent threads should run data acquisitions (DAQ) at a couple of instruments, with different GUIs depending on the type of instrument. Threading seems to be feasible as it is not known from start which instrument will be needed at some time, DAQ tasks should be queued up if an instrument is busy etc.
So, my idea now is to start the DAQ threads without any tkinter functionality from the main thread. The specific DAQ thread knows which specific GUI to use and puts this specific GUI class into a queue which is handled in the main GUI/tkinter thread. The instance of the GUI class will then be created in the GUI/tkinter thread.
Will this approach still violate thread-safety or is everything ok, as long as the GUI instances are created in the main tkinter thread?
As long as you only access tkinter widgets and functions from a single thread, it should work just fine. One exception, as far as I understand, is that it's safe to call the event_genereate method from other threads. You can push data on a queue and then generate an event, then the event can be handled in the main thread where data can be pulled off the queue and processed.
I have an OnClick function that is bound to a wx.Button object and when I click it, the function is run, it creates a thread and starts doing work behind the scenes. What should I do after this on the MainThread while the other thread finishes, to make the UI remain responsive? I'm currently doing this (in the main thread):
worker_thread = Thread(target = self.DoWork, name = "Solo Worker Thread")
worker_thread.start()
while worker_thread.is_alive():
self.Update() #self being the class that inherits wx.Panel
However the GUI is still unresponsive. Am I calling the wrong method?
Your example
while worker_thread.is_alive():
# do something
will block your GUI and eat up all available CPU cycles to ask the worker_tread if it is still working. wxPython seems to be not updateable if you block the main thread, even not if you force the update with wx.Yield or ….Update().
Therefore delete your while … statement. You can do the following to determine if the spun-off thread is still alive:
Signal the conclusion of the thread by calling back to the main-thread with the thread-save wx.CallAfter added to the end of the worker
Install a wx.Timer (e.g. every 1000 milliseconds) which polls the worker_thread if it is still alive
I suggest you to use wx.CallAfter(). You may find some useful examples here. Also you can use pubsub module to send messages to your GUI. Then your GUI won't block due to other threads.
Here is a nice blog that I read when I had a similar issue like yours.
You can also find some other questions on SO based that may assist you to understand this concept here, here, & here
You don't have to do anything. As long as the long running process is happening in a different thread, it shouldn't be blocking the GUI's main loop. See the following resources for examples:
http://wiki.wxpython.org/LongRunningTasks
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
If you take a look at those, you will see that you spin up a thread and the thread does not block the UI.
I'm using Python with wxPython for writing an app.
The method I'm considering to accomplish this may not be the best - if that's the case, let me know because I'm open to refactoring.
Right now, I have one GUI form. The main program start point instantiates an instance of the GUI form then runs wx.mainLoop(), which causes the app's main initial thread to block for the lifetime of the app.
We of course know that when events happen in the UI, the UI thread runs the code for them.
Now, I have another thread - a worker thread. This thread needs to sit idle, and then when something happens in the UI thread, e.g. a button is clicked, I want the worker thread to stop idling and do something else - run a function, say.
I can't envision this right now but I could see as the app gets more complex also having to signal the worker thread while it's actually busy doing something.
I have two questions about this setup:
How can I make my worker thread idle without using up CPU time? Doing something like while True: pass will suck CPU time, while something like while True: time.sleep(0.1) will not allow instantaneous reaction to events.
What's the best way to signal into the worker thread to do something? I don't want the UI thread to execute something, I want the worker thread to be signaled, by the UI thread, that it should change what it's doing. Ideally, I'd have some way for the worker thread to register a callback with the UI itself, so that when a button is clicked or any other UI Event happens, the worker thread is signalled to change what it's doing.
So, is this the best way to accomplish this? And what's the best way to do it?
Thanks!
First: Do you actually need a background thread to sit around idle in the first place?
On most platforms, starting a new thread is cheap. (Except on Windows and Linux, where it's supercheap.) So, why not just kick off a thread whenever you need it? (It's just as easy to keep around a list of threads as a single thread, right?)
Alternatively, why not just create a ThreadPoolExecutor, and just submit jobs to it, and let the executor worry about when they get run and on which thread. Any time you can just think in terms of "tasks that need to get run without blocking the main thread" instead of "worker threads that need to wait on work", you're making your life easier. Under the covers, there's still one or more worker threads waiting on a queue, or something equivalent, but that part's all been written (and debugged and optimized) for you. All you have to write are the tasks, which are just regular functions.
But, if you want to write explicit background threads, you can, so I'll explain that.
How can I make my worker thread idle without using up CPU time? … What's the best way to signal into the worker thread to do something?
The way to idle a thread until a value is ready is to wait on a synchronization object. On any modern OS, waiting on a synchronization object means the operating system stops giving you any CPU time until the object is ready for you.*
There are a variety of different options you can see in the Threading module docs, but the obvious one to use in most cases like this is a Condition. The way to signal the worker thread is then to notify the Condition.
However, often a Queue is a lot simpler. To wait on a Queue, just call its get method with block=True. To signal another thread to wake up, just put something on the Queue. (Under the covers, a Queue wraps up a list or deque or other collection, a Lock, and a Condition, so you just tell it what you want to do—check for a value, block until there's a value, add a value—instead of dealing with waiting and signaling and protecting the collection.)
See the answer to controlling UI elements in wxPython using threading for how to signal in both directions, from a worker thread to a UI thread and vice-versa.
I'd have some way for the worker thread to register a callback with the UI itself, so that when a button is clicked or any other UI Event happens, the worker thread is signalled to change what it's doing.
You can do it this way if you want. Just pass self.queue.put or def callback(value): self.value = value; self.condition.notify() or whatever as a callback, and the GUI thread doesn't even have to know that the callback is triggering another thread.
In fact, that's a pretty nice design that may make you very happy later, when you decide to move some code back and forth between inline and background-threaded, or move it off to a child process instead of a background thread, or whatever.
I can't envision this right now but I could see as the app gets more complex also having to signal the worker thread while it's actually busy doing something.
But what do you want to happen if it's busy?
If you just want to say "If you're idle, wake up and do this task; otherwise, hold onto it and do it whenever you're ready", that's exactly what a Queue, or an Executor, will do for you automatically.
If you want to say, "If you're idle, wake up, otherwise, don't worry about it", that's what a Condition or Event will do.
If you want to say, "If you're idle, wake up and do this, otherwise, cancel what you're doing and do this instead", that's a bit more complicated. You pretty much need to have the background thread periodically check an "interrupt_me" variable while it's busy (and put a Lock around it), and then you'll set that flag as well as notifying the Condition… although in some cases, you can merge the idle and busy cases into a single Condition or Event (by calling an infinite wait() when idle, and a quick-check wait(timeout=0) when busy).
* In some cases—e.g., a Linux futex or a Windows CriticalSection—it may actually spin off a little bit of CPU time in some cases, because that happens to be a good optimization. But the point is, you're not asking for any CPU time until you're ready to use it.
I've written a GUI program with PyQt4 that has to send a message string over a serial data link.
I have implemented a GUI interface with two button widgets. I need to send the data over the serial link continuously, once per second when the first button is clicked, and then stop when the second button is clicked.
My current program is able to send data only at the instant a button is clicked. This is the method I wrote to handle the button click signal:
def sendMessage(self):
while 1:
print "Hello........"
if checke == False:
break
Do I need to use threads to solve this problem?
It depends... if the send operation is fast, you can use the QTimer class. It integrates with the Qt event loop so you don't have to worry about threading issues. Serial communications can be slow, depending on how much data you are sending, so I can't say for sure if this is the right solution for you.
Yes. The key to GUI programming is never do any long operation on the main thread, because it'll block the whole program until that operation is complete.
If you want to continuously send data over network, do it in a background thread.
Some example code for you.
class MessageWorker(QtCore.QThread):
def __init__(self):
super(ParserWorker, self).__init__()
self.ok_to_send = False
self.terminated = True
def run(self):
while not self.terminated:
if self.ok_to_send:
self.send_message()
time.sleep(1)
def start_send():
self.ok_to_send = True
def pause_send():
self.ok_to_send = False
def terminated():
self.terminated = False
Then in the main program just call
worker = MessageWorker()
worker.start() # Start the background thread
worker.start_send() # Start sending message
worker.pause_send() # Pause sending message
worker.terminated() # Stop sending message permanently
Yes, you need to use threads. In any GUI-based program, any work that's going to take a non-trivial amount of time should always happen on a separate thread to avoid blocking the UI—whenever you see an "unresponsive" program, that's almost always due to the program failing to process window messages because its UI thread is blocked inside some long operation.
One easy way to startup a background thread is to use the threading module. Here's how you might use it to write data to the serial port once per second:
class MyClass:
# This method will run on a separate thread
def _serial_port_worker(self):
while self._run_worker:
self.send_data_to_serial_port()
time.sleep(1)
# Call this to start the worker thread
def start_worker_thread(self):
self._run_worker = True
worker_thread = threading.Thread(target=self._serial_port_worker,
args=(self,))
worker_thread.start()
# Call this to tell the worker thread to stop
def stop_worker_thread(self):
self._run_worker = False
Basically you have three options:
Use a second thread to do the serial comms. GUI toolkits aren't always thread-safe, so you should only make calls to them from the main thread. Additionally, there is a limitation to threading in Python; only one thread at a time can be executing Python bytecode.
Use the GUI toolkit's timeout function (might be called differently) to create an event every now and then. In the event handler do the serial comms. Make sure that you use non-blocking reads and writes (in pyserial, configure a timeout in the Serial object), otherwise your app might become unresponsive.
Do the serial communications from the second program using the multiprocessing module. Even if the second process blocks, it won't affect the GUI. You can use multiprocessing.Queue to communicate between the QUI and the other process.
I'm attempting to build a very simple wxPython GUI that monitors and displays external data. There is a button that turns the monitoring on/off. When monitoring is turned on, the GUI updates a couple of wx StaticLabels with real-time data. When monitoring is turned off, the GUI idles.
The way I tried to build it was with a fairly simple Python Thread layout. When the 'Start Monitoring' button is clicked, the program spawns a thread that updates the labels with real-time information. When the 'Stop Monitoring' button is clicked, thread.join() is called, and it should stop.
The start function works and the real-time data updating works great, but when I click 'Stop', the whole program freezes. I'm running this on Windows 7 64-bit, so I get the usual "This Program has Stopped Responding" Windows dialog.
Here is the relevant code:
class MonGUI(wx.Panel):
def __init__(self, parent):
wx.Panel.__init__(self, parent)
...
... other code for the GUI here ...
...
# Create the thread that will update the VFO information
self.monThread = Thread(None, target=self.monThreadWork)
self.monThread.daemon = True
self.runThread = False
def monThreadWork(self):
while self.runThread:
...
... Update the StaticLabels with info
... (This part working)
...
# Turn monitoring on/off when the button is pressed.
def OnClick(self, event):
if self.isMonitoring:
self.button.SetLabel("Start Monitoring")
self.isMonitoring = False
self.runThread = False
self.monThread.join()
else:
self.button.SetLabel("Stop Monitoring")
self.isMonitoring = True
# Start the monitor thread!
self.runThread = True
self.monThread.start()
I'm sure there is a better way to do this, but I'm fairly new to GUI programming and Python threads, and this was the first thing I came up with.
So, why does clicking the button to stop the thread make the whole thing freeze up?
In wxPython, GUI operations need to take place in the main thread. At places in your code you are calling the GUI from a different thread.
The easiest solution is to use wx.CallAfter(). A line of code would look like
wx.CallAfter(self.button.SetLabel, “Start Monitoring”)
which will then call self.button.SetLabel(“Start Monitoring”) from the main thread after the function completes.
There are other ways around this as well, such as using a Python threading Queue or wx.PostEvent, but start with CallAfter because it's easiest.
Other issues are also relevant, like you can't restart the same thread, but using CallAfter will stop the crashing.
It's likely hanging on join([timeout]), which blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception – or until the optional timeout occurs.
Do you have some inner loop in your thread, or a blocking call that waits for some source of data that may never come? When I wrote a basic serial program that grabbed COM port data, it would sometimes hang because a read function in my thread would block until it got something.
I would sprinkle in a few debugging print statements to see whats happening.
Edit:
I'd also use a threading.Event() instead of a Boolean flag, e.g.:
# in the init code...
self.runThread = threading.Event()
# when starting thread...
self.runThread.set()
self.monThread.start()
# in the thread...
while self.runThread.isSet():
pass # do stuff
# killing the thread...
self.runThread.clear()
self.monThread.join()
This shouldn't make it work differently, but it's a slightly safer way to do it.
tom10 has the right idea with avoiding UI updates from the monitor thread.
Also, it is probably not a good idea to have the blocking call self.monThread.join() in your UI thread. If you want the UI to give some feedback that the monitor thread has actually ended, have monThreadWorker issue a wx.CallAfter() or wx.PostEvent() just before it closes.
Avoid anything that blocks in your UI thread, and you will avoid deadlocking the UI