I have a problem with threads in pygtk. My application consist of a program that downloads pictures off the internet and then displays it with pygtk. The problem is that in order to do this and keep the GUI responsive, I need to use threads.
So I got into a callback after the user clicked on the button "Download pictures" and I call the method to download the pictures that is within that same class.
thread.start_new_thread(self.images_download, (path,pages)
This won't work. The only way I get my program to get into the thread is by using
gtk.threads_init()
Before starting any thread. Now it downloads the pictures but the GUI remains unresponsive.
I googled this and I tried putting gtk.threads_enter and gtk.threads_leave around the threads but it just doesn't work.
Your question is a bit vague, and without a reference to your actual code it's hard to speculate what you're doing wrong.
So I'll give you some pointers to read, then speculate wildly based on experience.
First of all, you seem to think that you can only keep the GUI responsive by using threads. This is not true. You can also write your code asynchronously, and do everything in a single-threaded application. Twisted is built on this programming model. I recently made a blog post that explains how I created an asynchronous task interface, and example runners both for CLI and GTK+. You can look at those examples to see how tasks can be implemented asynchronously, and the UI still gets updated.
Second, if you prefer to use threads for some reason, you will need to understand the GTK+ threading model a little.
You should start by reading The PyGTK FAQ entry on the subject, and you might find this blog post easy to understand too.
Now, on to speculation. I am guessing that you are trying to update your GTK UI from the thread, and not handling the locking properly. If this is the case, you are better off for now deferring all your UI updates you want to do from threads to the main thread by using gobject.idle_add() This way, all UI calls will be made from the main thread. It is an easier mental model to follow in your programming.
Once you feel you really understand the threading and locking models, you could consider updating the UI from your threads, but it's easy to miss a threads_enter()/threads_leave()
You can use gtk.gdk.threads_init() in order to allow any thread modify the UI with the respecting gtk.gdk.threads_enter() and gtk.gdk.theads_leave() lock, but, the problem with this is that doesn't work well on windows. I have tested it on Linux and performs quite well, but I had no luck making this to work over win32.
=== Edit ===
I have been browsing about this, you could make use of the gobject.io_add_watch to check if there is something in your socket, grab it and then update the GUI. check my post about this:
Sockets (and some other files) and PyGTK without threads.
Related
It is easy to create highly integrated code in python, so the need to quit and restart an application when its code is changed is understandable.
However, it surely must exist some strategies and models to be able to isolate parts of the code so it can be updated on-the-fly without the need to quit and restart.
For the applications I am working on many features will be independant background tasks that the main application will talk to, present status information and also instruct to perform tasks based on the current state. In many ways these background tasks can be seen as independent programs, just that they share some of the codebase with the main application and other tools, tasks, etc built on-top it.
While it probably is hard to make the whole shebang live updateable, I'm sure there must be ways where it is possible to roll out updates and have the running code to notice and update itself as needed.
Since I'm also keen on taking advantage of multithreading and asyncio (in Python 3.5), as well as explore making things stateless, it seems logically possible to do some interesting stuff here that at least makes it possible to avoid some forced hard restarts when rolling out new code.
Would be very grateful for tips and pointers to information about how to make this working.
There's a built-in function reload() that will reload a module, but it's easy to mess up. You'd have to be very careful about what holds references to objects created by the old version of the module, and then be sure to replace those when you reload the module.
I use Django a lot, and when it's running its web server in debug mode, it reloads the entire web server process whenever a source file changes. It's a nice compromise between a manual restart and reloading a single module.
I haven't used it, but watchdog might be helpful to monitor the file system for changes to trigger a reload.
I have written a monitoring program for the control system at our plant. It is basically a GUI which lets the operator see the current status of the lock of the closed loop system and aware the operator in case the lock/loop breaks.
Now, the operation is heavily dependent on the responses of the GUI. My seniors told me that they prefer just the console prints instead of using TKinter based GUI as TKinter has lags while working in real time.
Can anyone please comment on this aspect?
Can this lag be checked and corrected?
Thanks in advance.
I would say that if your program is simply accessing data and not interacting with the data, then a GUI seems to be a bit of overkill. GUI's are guided user interfaces, as you know, and are made for guiding a user through an interface. If the interface is just a status, as you indicated, then I see nothing wrong with a console program.
If, however, your program also interacts with data in a way that would be difficult without a GUI, then the GUI is likely the right choice.
Have you considered a GUI in another programming language? Python is known to be a bit slow, even in console. In my experience, C++ is faster in terms of viewing data. Best of luck!
Python / tkinter in general
In a tkinter program, your code falls in one of four categories;
Initialization code that runs before the mainloop is started.
Callbacks that are run from within the mainloop.
Code running in other threads.
Code running in different processes.
In the first case, the time the code takes only influences the startup time, which for a long-running program is probably not all that relevant.
Concerning the second case, well-written callbacks should not take that long to run. In the order of tens of milliseconds, maybe up to 100 ms. If they take longer, they will render the GUI unresponsive. So unless you notice a sluggish GUI (without threads; see below) this should not be a problem.
One pitfall here are after callbacks, that is functions that will be scheduled to run after a certain time. If you launch them too often, this will also starve the GUI of time.
Another possible problem might be the manipulation of a Canvas with lots and lots of items in it.
As of Python 3.x, tkinter is thread-safe to the best of my understanding. However, in the reference implementation of Python, only one thread at a time can be executing Python bytecode. So doing heavy calculations in a second thread would slow down the GUI.
If you GUI uses multiprocessing to run calculations in another process, that should not influence the speed of your GUI much, unless you do things wrong when communicating with that other process.
Your monitoring program
What is too slow depends on the situation. In general Python is not considered a language suitable for hard real-time programs. To do hard real-time one also needs a suitable operating system.
So the question then becomes what is the acceptable lag in your system specification? Without knowing that it is impossible to precisely answer your question.
It seems that your GUI is just displaying some system status. That should not cause too much of a load, provided that you don't read/check the data too often. As described in the callbacks paragraph above it is possible to starve your GUI of CPU cycles with callbacks that run too often. From what you've written, I gather that the GUI's task is just to inform the human operator.
That leads me to believe that the task is not hugely time critical; a system that requires millisecond intervention time should not rely on a human operator.
So based on your information I would say that a competently written GUI should probably not be too slow.
For those of you who have written fairly complex PyQt applications, what tips and tricks would you offer for speeding up your applications? I have a few examples of where my program begins to slow down as it grows larger:
I have a 'dashboard' written that is destroyed and re-created when a user clicks on an item in a TreeWidget. What would be a better way to have a modular interface where clicking an item in the TreeWidget changes the dashboard, but doesn't require destroying a widget and recreating it.
Each dashboard also loads an image from a network location. This creates some slowdown as one navigates around the application, but after it's loaded into memory, 'going back to that same dash' is faster. Is there a good method or way to run a thread on program load that maybe pre-loads the images into memory? If so, how do you implement that?
When you have a large variety of dashboard items and data that gets loaded into them, do you guys normally thread the data load and load it back in which each thread completes? Is this viable when somebody is browsing around quickly? Would implementing a kill-switch for the threads such that when a user changes dashboards, the threads die work? Or would the constant creation and killing of threads cause some sort of, well, meltdown.
Sorry for the huge barrage of questions, but they seemed similar enough to warrant bundling them together.
I'm not sure if this is exactly the same thing you are doing, but it sounds similar to something I have in some apps, where there is some list of custom widget. And it does significantly slow down when you are creating and destroying tons of widgets.
If its an issue of lesser amounts of total widgets, but just being created and deleted a lot, you can just create the widgets once, and only change the data of those widgets as information needs to be updated... as opposed to creating new widgets each time the information changes. That way you can even change the data from threads without having to worry about creating widgets.
Another situation is where you are displaying a list with custom widgets and there are a TON of results. I notice it always slows down when you have 1000's of custom widgets in a list. A trick my co-worker came up with was to have a fake kind of list where it is using a static number of slots in the display. Say, it shows 10 slots in the view. The scrollbar doesn't really scroll down across MORE widget...what it does is scrolls the DATA through the 10 visible widgets. You can a crazy performance increase doing that. But only if it is an acceptable display style for your application.
Are you using QNetworkAccessManager to load you images? It has cache support. Also it loads everything in background with finishing callbacks.
I don't really understand what your dashboard is doing. Have you think about using QWebkit? Maybe your dashboard content is easy to be implemented in HTML?
PS. I don't like threads in Python and don't think they are good idea. Deferred jobs delegated to Qt core is better.
We are in the process of standing up a UI on top of a python system. This is all throw away code, so we want something quick, yet presentable.
We will have a couple of "interfaces" but they will be of two types. One will be control, it will basically be sitting on top of a python thread, and accepting requests from the user.
The other will be more of a display screen that will need to be able to display images, and some classic "grid views" of text to the user.
We pretty much know we could* do all of this in HTML but wasn't sure what would be the best way to interact with the core python code?
Anyone know of a good UI python presentation layer? Since we know we can do all of this in HTML/Jquery pretty quickly, we are also open to suggestions on how to integrate this with a web server..
Any suggestions? Really interested in finding out if there is any way to use python as the back end to a webserver.
Let me know if you all need more information.
I like wxPython. The demo application is excellent and lets you browse, tweak and re-run the code right in the demo.
We have found the DJango meets our needs. It is a pretty slick mvc style python web stack. Really is easy to use, and very quick to develop in. I will say that the ORM layer is a little young so it is hard to do some simple queries, but luckly since this is throw away code we can just use native sql.
Tkinter is probably going to be the solution you can use quickest. Its API is simple and straight-forward, and you probably already have it installed.
As the other 2 classic Python GUI options have been given already, I feel duty bound to suggest PyQt :)
Using QT Designer I've found it much simpler than TKInter to get some basic GUIs up and running. Build your GUI up in a WYSIWYG way, then hook it up to the back-end logic. I've also found that the large amount of C++ help on QT available on the interent usually translates more or less directly across to PyQt. The resources available for TKInter are IMO pretty obtuse, and simply stop as soon as you want to do anything more interesting than Hello World. YMMV.
The Rapid GUI Programming with Python and QT book is a fantastic resource. Had me programming real applications in no time.
I'm writing an application that sends files over network, I want to develop a custom protocol to not limit myself in term on feature richness (http wouldn't be appropriate, the nearest thing is the bittorrent protocol maybe).
I've tried with twisted, I've built a good app but there's a bug in twisted that makes my GUI blocking, so I've to switch to another framework/strategy.
What do you suggest? Using raw sockets and using gtk mainloop (there are select-like functions in the toolkit) is too much difficult?
It's viable running two mainloops in different threads?
Asking for suggestions
Disclaimer: I have little experience with network applications.
That being said, the raw sockets isn't terribly difficult to wrap your head around/use, especially if you're not too worried about optimization. That takes more thought, of course. But using GTK and raw sockets should be fairly straightforward. Especially since you've used the twisted framework, which IIRC, just abstracts some of the more nitty-gritty details of socket managing.
Two threads: one for the GUI, one for sending/receiving data. Tkinter would be a perfectly fine toolkit for this. You don't need twisted or any other external libraries or toolkits -- what comes out of the box is sufficient to get the job done.
If your application is somewhat similar to bittorrent, why not check the source code of Deluge http://deluge-torrent.org/ and build from it? It is written in Python, it does use the bittorrent protocol and it does have a GTK user interface.
As an alternative to twisted and whatever GUI library you seem to be using, how about trying PyQt? It provides a GUI and non-blocking sockets all in the same event loop. That way you don't have to worry about interoperability issues, which seem to be the issue you are facing.
Hope this helps!