I am writing a simple image viewer that lets the user flip very quickly through tens of thousands of images, about 100 at a time. The images are files on disk.
In order for the viewer to function, it must continuously preload images ahead of the user's current one (or the viewer would be unusably sluggish).
The basic recipe that I'm using to display the images in a grid of Tkinter labels, is the following (this has been tested and works):
def load_image(fn):
image = Image.open(fn)
print "Before photoimage"
img = ImageTk.PhotoImage(image)
print "After photoimage"
label.config(image=load_image("some_image.png")
I need the ImageTk.PhotoImage instance to display the image on a label. I have implemented two different approaches, each with an associated problem.
First approach:
Launch a separate thread which pre-loads the images:
def load_ahead():
for fn in images:
cache[fn] = load_image()
threading.Thread(target=load_ahead).start()
top.mainloop()
This works quite well on my Linux machine. However, on another machine (which happens to be running Windows, and compiled with pyinstaller), a deadlock seems to happen. "Before Photoimage" is printed, and then the program freezes, which suggests that the loader thread gets stuck at creating the ImageTk.PhotoImage object. Musst the creation of an ImageTk.PhotoImage object happen within the main (Tkinter mainloop's) thread? Is the creation of PhotoImage computationally expensive, or is negligible compared to actually loading the image from disk?
Second approach:
In order to circumvent this possible requirement of PhotoImage objects being created from within Tkiner's mainloop thread, I resorted to Tk.after:
def load_some_images():
#load only 10 images. function must return quickly to prevent freezing GUI
for i in xrange(10):
fn = get_next_image()
cache[fn] = load_image(fn)
top.after_idle(load_some_images)
top.after_idle(load_some_images)
The problem with this is that, appart from creating additional overhead (ie the image-loading procedure must be broken up into very small chunks since it is competing with the GUI) that it periodically freezes the GUI for the duration of the call, and it seems to consume any keyboard events that happened during its execution.
Third approach
Is there a way I can detect pending user events? How can I accomplish something like this?
def load_some_images():
while True:
try: top.pending_gui_events.get_nowait()
except: break
#user is still idle! continuing caching of images
fn = get_next_image()
cache[fn] = load_image(fn)
top.after_idle(load_some_images)
top.after(5,load_some_images)
Edit: I have tried using top.tk.call('after','info') to check pending keyboard events. This doesn't always reliably, and the interface is still sluggish/unresponsive.
Thanks in advance for any ideas
I recommend creating an load_one_image function rather than a load_some_images function. It will be less likely to interfere with the event loop.
Also, as a rule of thumb, a function called via after_idle shouldn't reschedule it self with after_idle. The reason is that after_idle will block until the idle event queue is drained. If you keep adding stuff on to the queue while the queue is being processed, it never gets completely drained. This could be the reason why your GUI seems to hang once in a while with your second approach.
Try after(5, ...) rather than after_idle(...). If your system can create an image in less than 5ms, you can process 100 images in about half a second, which is probably fast enough to give a pretty snappy interface. You can tweak the delay to see how it affects the overall feel of the app.
Related
I have a main process that does some stuff (e.g. analyzing data) and it runs alone just fine. I also want to make a simple GUI that displays the result of the main task using PyQT5. The idea is that the GUI should not interfere in the main process, that is, if I remove the GUI it shouldn't cause any issue to the main process.
The code of the main process is quite simple:
if __name__ == '__main__':
# initialize the object that performs the main task
tasker = Task()
# the graphical interface to visualize the result of tasker
gui = GUI(task) # GUI is a separate class that keeps a reference to tasker
# read the input data and do stuff on each new data instance
for f in listdir(inrepo):
data = read_new_data(f) # an utility function that reads new data from file
result = tasker.process(data) # tasker processes the new data and return some results
gui.update(data, result) # pass the data and result in the GUI to update it
The code of the GUI class is quite long so I only paste a few lines here, but the lines I skip are just to create widgets, nothing fancy (I didn't connect any event yet)
class GUI(QApplication):
def __init__(self, tasker):
"""Initialize the application"""
super().__init__([])
self.tasker = tasker
# define the main window
self.window = QWidget()
self.window.setWindowTitle('GUI')
... # layout and components etc.
# show stuff
self.window.show()
self.exec()
So I want the GUI to be completely independent from my main process. For example, if I don't need the GUI anymore, I can just comment out the 2 lines gui = GUI(task) and gui.update(data, result).
However, the problem is that starting the GUI blocks the entire process (I assume it's because of self.exec() in GUI.__init__, so my main process cannot proceed to loop over the data. Could you please show me how to make PyQT non-blocking? Is it even feasible?
Some options I have considered:
Threading: it seems more complicated than necessary for my use case and it may make referencing to the task instance difficult from a thread. All new updates to task should be reflected in the GUI. If I'm not mistaken, PyQT's application already runs on a thread. So multi-level threading may be troublesome.
Run the GUI from another Python process, communicated via shared folders: may incur high latency. Any new data and result from task should be immediately reflected in the GUI. Writing to file then reading from file then updating the GUI will cause some delay.
Perform the task in GUI: I could use some timeout event to read new data periodically and run task on them, but then everything depends heavily on the GUI, and I can't just comment it out if I don't need the GUI anymore.
Any suggestion is very much appreciated! Thank you very much!
Switching your program between GUI mode/console mode is often not as simple as commenting out some lines. PyQt in particular does not allow you to run a GUI loop from anything other than a main thread. Not all hope is lost, though - this simply means that you should decide as early as possible whether your program is going to run as a console application or as a GUI.
Instead of relying in commenting out code, you can create a "switch" of sorts inside your code that tells your code how to execute. One way to do this is to check the command-line arguments when you execute the code, e.g.:
import sys
if "--headless" in sys.argv[1:]: # checking the command-line arguments
run_code_without_gui()
else:
run_code_with_gui()
This way, executing your code as python mycode.py --headless will execute it without the GUI (through the run_code_without_gui function), while executing python mycode.py will run it as a GUI (through the run_code_with_gui function). (Although if you're actually going to parse command-line arguments, I recommend using the argparse library).
You can even keep the analysis code completely decoupled from the GUI code, so that you simply run a function from, say, analysis.py when executing without the GUI, and have the GUI call that exact same function when e.g. the user clicks on the "Analyze" button.
Another thing to note is that if your analysis code takes long to execute, it may inadvertently block the GUI. In this case, you should run the analysis code in a separate "worker" thread that spawns upon clicking the "Analyze" button, keeping the GUI responsive while it executes. This might be the way to go if you want the analysis to keep running alongside the GUI indefinitely - create the worker thread for the analysis at the same time that you display the GUI to the user.
I have a program which uses multiple cores to process images before drawing to a canvas for a main view (using multiprocessing). I would like to know the best way to tackle this problem.
Is it possible to have each core drawing to its own canvas which are layered on-top of each other in the same view? Is it possible to have this behavior?
No, it is not possible. GUI widgets cannot be shared between processes. On some platforms, it just isn't possible at all; on other platforms, it would be possible, but only by doing things very differently from the way Tk does; on others, it sort of works, but the event loops are all screwed up. So, the result may be that nothing shows up, that one or both processes freezes, that the GUI doesn't respond to events, that tkinter raises an exception in the child, that tkinter creates a whole separate independent GUI, or, if you're really unlucky, that things unpredictably work sometimes but do one of the other things other times.
However, that doesn't mean there's no way to do what you want, just that you can't do it directly.
The simplest solution is to marshal your Canvas commands and pass them over a Pipe or Queue for the main process to execute.
A fully general solution isn't that hard, but in your case, it should be even simpler: all you want the background process to do is process an image and then display it. So the only Canvas command you need is create_image.
And, in fact, you can probably do with tasks on a Pool, which just return the image when they're done, with the main process doing the create_image with the results.
Mixing waiting on multiprocessing async results with a tkinter event loop is a bit of a pain, but if you use concurrent.futures, you can just attach the create_image as a callback on the Future returned by the task.
A different option is to have the background processes create off-screen Canvas objects, draw to them, then capture the results as a BitmapImage or a postscript rendering, which you can then pass to the main process to blit onto a Canvas of its own. But this is a lot more complicated; I think the other solution will probably work a lot better for you.
I am writing an app in kivy which does cpu-heavy calculations at launch. I want the app to display what it's doing at the moment along with the progress, however, since the main loop is not reached yet, it just displays empty white screen until it finishes working. Can I force kivy to update the interface?
Basically I'm looking for kivy's equivalent of Tkinter's root.update()
I could create a workaround by defining a series of functions with each calling the next one through Clock.schedule_once(nextFunction, 1), but that would be very sloppy.
Thanks in advance.
Leaving aside the question of whether you should be using threading or something instead (which possibly you should), the answer is just that you should move your cpu calculations to somewhere else. Display something simple initially (i.e. returning a simple widget from your build method), then do the calculations after that, such as by clock scheduling them.
Your calculations will still block the gui in this case. You can work around this by doing them in a thread or by manually breaking them up into small pieces that can be sequentially scheduled.
It might be possible to update the gui by manually calling something like Clock.tick(), but I'm not sure if this will work right, and even if so it won't be able to display graphics before they have been initialised.
I have a program that I'm just adding graphics to, but I'm having trouble running my main code along with the graphics. Basically I have something like this:
def mainFunction():
while True:
run code in here
root = Tk()
board = Canvas(root, height=710, width=1000)
board_image = PhotoImage(file="/path/example.jpg")
photo = board.create_image(0,0, anchor=NW, image=board_image)
board.pack()
mainFunction
root.mainloop()
I can only run either the mainFunction or the graphics because whichever one I make run first in the code is the only that runs. It doesn't stop to allow the next code to run. There has to be a simple way to get graphics and code to run together side by side. Thanks!
Use Tk.after_idle() to register a function that will do a piece of the work required. Keep doing piece after piece until all the work is done.
Generally speaking, you cannot put an infinite loop inside a Tkinter application. Why? It's already running an infinite loop: the event loop. Your code is the equivalent of this:
while <there are more events to service>:
while True:
<run code in here>
<get the next event>
<service the event>
See the problem? You're preventing the code from ever servicing events, and events are the life blood of a GUI.
Instead, you need to take advantage of the already-running infinite loop by adding code to be run inside the loop. You do this with after (and after_idle). This will put one even on the queue. If, during the processing of that event you again call after_idle, you've effectively set up an infinite loop that works within the event loop.
For example:
def do_one_iteration(self):
<run code in here>
self.after(100, self.do_one_iteration)
Then, somewhere in your main logic, or in response to a button, you call do_one_iteration. It will do one iteration of your previously-infinite-loop. When it is done it instructs Tkinter to call itself again 100 milliseconds later. When that time period elapses your code is run, it schedules another iteration in 100 milliseconds, etc. etc. You can change the interval to whatever you want; the smaller the interval the faster your code runs, but the greater the chance that you starve the GUI for events.
Note that this only works if <run code in here> runs relatively fast. While it is running your GUI will freeze. If it can complete one iteration in a couple hundred milliseconds then the user will never know. If it takes a second or more it will be noticeable.
Note: this example assumes your main application is an object that inherits from a Tkinter widget. If that's not the case it will still work, you just have to remove the self parameter. An even better solution is to refactor your GUI to use objects -- it's a much more flexible way of implementing GUIs.
in my python program to upload a file to the internet, im using a GTK progress bar to show the upload progress. But the problems that im facing is that the progress bar does not show any activity until the upload is complete, and then it abruptly indicates upload complete. im using pycurl to make the http requests...my question is -
do i need to have a multi-threaded application to upload the file and simultaneously update the gui? or is there some other mistake that im making?
Thanks in advance!
I'm going to quote the PyGTK FAQ:
You have created a progress bar inside a window, then you start running a loop that does some work:
while work_left:
...do something...
progressbar.set_fraction(...)
You will notice that the window doesn't even show up, or if it does the progress bar stays frozen until the end of the task. The explanation is simple: gtk is event driven, and you are stealing control away from the gtk main loop, thus preventing it from processing normal GUI update events.
The simplest solution consists on temporarily giving control back to gtk every time the progress is changed:
while work_left:
...do something...
progressbar.set_fraction(...)
while gtk.events_pending():
gtk.main_iteration()
Notice that with this solution, the user cannot quit the application (gtk.main_quit would not work because of new loop [gtk.main_iteration()]) until your heavy_work is done.
Another solution consists on using gtk idle functions, which are called by the gtk main loop whenever it has nothing to do. Therefore, gtk is in control, and the idle function has to do a bit of work. It should return True if there's more work to be done, otherwise False.
The best solution (it has no drawbacks) was pointed out by James Henstridge. It is taking advantage of python's generators as idle functions, to make python automatically preserve the state for us. It goes like this:
def my_task(data):
...some work...
while heavy_work_needed:
...do heavy work here...
progress_label.set_text(data) # here we update parts of UI
# there's more work, return True
yield True
# no more work, return False
yield False
def on_start_my_task_button_click(data):
task = my_task(data)
gobject.idle_add(task.next)
The 'while' above is just an example. The only rules are that it should yield True after doing a bit of work and there's more work to do, and it must yield False when the task is done.
More than likely the issue is that in your progress callback, which is where I presume you're updating the progress bar, you're not making a call to manually update the display i.e. run through the GUI's event loop. This is just speculation though, if you can provide more code, it might be easier to narrow it down further.
The reason you need to manually update the display is because your main thread is also performing the upload, which is where it's blocking.
In python 2.x integer operands result in integer division. Try this:
#Callback function invoked when download/upload has progress
def progress(download_t, download_d, upload_t, upload_d):
print 'in fileupload progress'
mainwin.mainw.prog_bar.set_fraction(float(upload_d) / upload_t)
Yes, you probably need concurrency, and yes threads are one approach, but if you do use threads, please use an method like this one: http://unpythonic.blogspot.com/2007/08/using-threads-in-pygtk.html which will abstract away the pain, and allow you to focus on the important aspects.
(I have not repeated everything in that blog post through laziness, hence community wiki).
One option, if you are not married to pycurl, is to use GObject's IO watchers.
http://pygtk.org/pygtk2reference/gobject-functions.html#function-gobject--io-add-watch
Using this you can interleave the file upload with the normal PyGTK event loop, and even do the set_progress call in your IO watch callback. If you are offloading all the work for uploading onto pycurl this is not really feasible, but if you're just uploading a file over HTTP, io_add_watch will make using a socket for this much less painful as well.