I have a Python application that uses wxPython and some additional threads. One thread uses PIL.Image.open. Under certain circumstances the application freezes so that you see an uncomplete GUI. I found out that it hangs at PIL.Image.open. When I put debug prints in the PIL module, I can see one time it hangs here, one time there ... -- which I can't understand. It seems totally unrelated.
Is there anything a thread can do in Python, that causes other threads to stop at actually unproblematic lines like import string? Or is wxPython able to give such influence?
Long running tasks will freeze a GUI, like wxPython or Tkinter. Putting the long running process into a thread usually takes care of the issue though. I am guessing that you are doing something in your thread that communicates with wxPython in a non-thread-safe manner. If you are not using wx.CallAfter, wx.CallLater or wx.PostEvent to communicate with wxPython from the thread, then that is the issue. You have to use one of those methods.
Otherwise we'll need a small runnable example to diagnose the issue.
Related
I have a flask app that uses a large integer optimization GoogleOrTools. The underlying package is writen in C but there is a python API that I use to call it. When I get a very long running task it seems as if it blocks other tasks and server becomes unresponsive. The obvious solution is naturally to put in a different service. Which I will impliment when I have the time.
But I'm curious if there is something I'm missing as I read Python threading after every x lines of byte code the thread changes, which should allow the other process to take over? But does that also happen in the case of Gevent? Or if it is acturaly allowing the other threads, but it might greedyly capture all the resources?
It would be really nice if someone with some inside could tell me what is and is not posible.
Anyone who has worked with Multithreaded PyQt4 apps? I was just wondering if the inbuilt signal/slot mechanism coupled with QtThread of PyQt4 framework has any benefit over the standard Python threads (which are designed in my code to handle the UI components in a thread safe way offcourse) using event driven async callback.
I am looking for any major speed or security concerns, any specific run-time exceptions or edge cases. (The UI is quite complex hence a re-write at a later stage would be very counter-productive).
Thanks.
Edit: I realize this might mean replicating some of the already present PyQt core functionality but it is ok if it allows more flexibility within the app.
There's no point really using Qt/PyQt if your're not using the signal-and-slot mechanism and rolling your own event loop. Basically, you'd be reimplementing the core of the framework itself. But I'm guessing this is not what you're asking about.
It would be nice if you could clarify your question a bit, (because of which I've had to make a few assumptions)
but here's the deal:
I think you're a bit confused about what the signal and slot mechanism does. (or maybe not, forgive me for reiterating some stuff that might probably be obvious to you).
The signals-and-slots do not implement threading for you (so the question of using signal/slots having any benefit over standard Python threads is moot)
You're probably assuming that the signal-slot mechanism is multithreaded, and that a slot when called by a signal, executes in a new thread. Well, this is not the case.
The signal and slot mechanism in Qt runs in a single event loop in Qt (implemented by QApplication), which itself runs in a single thread.
So signals and slots are not replacements for multi-threading in any way.
If there's a slot that blocks, then it will block your entire application.
So any blocking I/O or time intensive functions should ideally be in a separate thread from the UI, and your slots should start execution of these threads.
Now whether to use QThread or standard Python threads to implement your own threads is another issue, and it's been asked on StackOverflow before, but I tend to use QThreads for Qt apps.
So if you have a button, and you want to start a file download with the Requests library when its clicked, you'll connect the clicked signal of the QPushButton to a slot say for example downloadButtonClicked, and that slot would start a new QThread which would take care of downloading the file using Requests. You can further connect the finished() signal from the QThread to know when the download is complete and to update your UI
(I'll add a code example if this is indeed what you're asking about. So please clarify your question)
Based on your comment to another reply:
Sorry for the ambiguity, I was talking about QtThread Slot/Signal
Mechanism vs. callbacks using inbuilt Python Threads. I intend on
creating separate threads from UI on event arrival (clicks, etc) and
then use callbacks to the main UI thread from the new threads to
update the UI (all UI logic in the main thread with locks to keep it
thread safe.) I know this might mean replicating some of the already
present PyQt functionality but I feel this way I would have a lot more
control over my app. (The extra work isn't a concern if it allows more
flexibility in the app. Plus it isn't so much of work)
I would say that what you are after is to use QApplication.postEvent() from your threads. With a bit of extra code, you can use this to execute arbitrary methods in the main thread either synchronously or asynchronously.
I'm not sure there are really any advantages or disadvantages to either option (Qt or Python threads). As far as I'm aware, they both still hold the GIL, meaning your program is never truly multithreaded. QThreads come with an event loop, but as you say that isn't hard to write yourself in your own Python thread.
Have you considered using multiple processes instead of multiple threads? While slower to start, you get the advantage of actually having your code run in parallel.
At the end of the day, I think the answer to your question is simply personal preference. Mine would be to avoid using a QThread because it makes it easier to port your application to another widget toolkit in the future if PyQt\PySide\Qt ever die (not that it is very likely, but I've had a bad experience with PyGTK so now I'm wary)
EDIT: Please also look at this, as it has people far better answers than I've given: Threading in a PyQt application: Use Qt threads or Python threads?
I've been doing normal python scripts for years, but had to recently dive into GUIs (with Tkinter). Simply put, I have a basic Tk window set up with a Start/Stop button, which should call function Foo(). Function Foo() contains a while True though, which would cause the UI to lock up forever.
I am wondering if I can get some guidance from here. Should I use pythons multiprocessing tools? I was thinking about Twisted / Gevent / threads as well, but I don't think they fit the task (could be wrong).
Thanks!
I ended up using the multiprocessing library from python. I start Foo() in a new process and start. When the stop button is pressed, it terminates the sub process.
I have 3 Threads in a wxPython application. The application hangs at the following three methods:
PIL.Image.open
ElementTree.write
time.sleep
Do you know what could be the common cause of this?
I don't know what's going on exactly, but I'm guessing you're not using wxPython's thread-safe methods to communicate with the GUI. They're wx.CallAfter, wx.CallLater, and wx.PostEvent. Don't access the GUI directly.
If these are actually in separate threads, then they shouldn't cause the GUI to hang. I've done the time.sleep() one in a thread on my blog, so I know it doesn't:
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
We'll probably need a small, runnable sample app to figure this out: http://wiki.wxpython.org/MakingSampleApps
I have a PyQt4 application, which at some point packs a big file using the tarfile module. Since the tarfile module does not implement any callback strategy, it blocks and the Qt GUI gets unresponsive.
I want the GUI to keep updating during that time. The only possibility is a separate thread.
So, I start a QThread. What do I have to do in the QThread to make the GUI update itself?
As soon, as the tar process is finished, I want the thread to finish.
Thanks!
Nathan
QThread's are pretty much identical to normal Python threads so you can just use normal communication methods. However, QThreads also have a few signals available, so if you simply connect to those, than you're done.
In your GUI code do something like this and you're pretty much done:
thread = Thread()
thread.finished.connect(gui.do_update_thingy)
There is also a terminated and started signal available which you can use :)