I have a program which uses multiple cores to process images before drawing to a canvas for a main view (using multiprocessing). I would like to know the best way to tackle this problem.
Is it possible to have each core drawing to its own canvas which are layered on-top of each other in the same view? Is it possible to have this behavior?
No, it is not possible. GUI widgets cannot be shared between processes. On some platforms, it just isn't possible at all; on other platforms, it would be possible, but only by doing things very differently from the way Tk does; on others, it sort of works, but the event loops are all screwed up. So, the result may be that nothing shows up, that one or both processes freezes, that the GUI doesn't respond to events, that tkinter raises an exception in the child, that tkinter creates a whole separate independent GUI, or, if you're really unlucky, that things unpredictably work sometimes but do one of the other things other times.
However, that doesn't mean there's no way to do what you want, just that you can't do it directly.
The simplest solution is to marshal your Canvas commands and pass them over a Pipe or Queue for the main process to execute.
A fully general solution isn't that hard, but in your case, it should be even simpler: all you want the background process to do is process an image and then display it. So the only Canvas command you need is create_image.
And, in fact, you can probably do with tasks on a Pool, which just return the image when they're done, with the main process doing the create_image with the results.
Mixing waiting on multiprocessing async results with a tkinter event loop is a bit of a pain, but if you use concurrent.futures, you can just attach the create_image as a callback on the Future returned by the task.
A different option is to have the background processes create off-screen Canvas objects, draw to them, then capture the results as a BitmapImage or a postscript rendering, which you can then pass to the main process to blit onto a Canvas of its own. But this is a lot more complicated; I think the other solution will probably work a lot better for you.
Related
So after posting another question about this issue I realised the problem was only happening with my custom resize bind. When I resize the window by the default edges of the window the Issue does not happen and the contents of the canvas is drawn accurately - however when using my custom resize bind the contents of the canvas is laggy and lags behind the true position of the window.
import tkinter as tk
def resize(self, event=None):
y=root.winfo_height()
x1 = root.winfo_pointerx()
x0 = root.winfo_rootx()
root.geometry("%sx%s" % (x1-x0,y))
root=tk.Tk()
canvas = tk.Canvas(root,bg='red')
canvas.pack(padx=20,pady=20)
inside=tk.Frame(root)
inside.pack(fill='both')
for i in range(10):
x=tk.Label(inside,text='Lsdkfjhaskfjhskljdfhaskjfhksfhkjasdhf')
x.pack()
g=tk.Button(root,text='Drag to resize')
g.bind('<B1-Motion>',resize)
g.pack()
canvas.create_window(5,5,window=inside,anchor='nw')
Original content:
Screenshot while resizing window with manual bind via button - as you can see the content is not visible while resizing the window and lags behind where the canvas is.
The issue is fixed if I call root.update() at the start of my resize function however this then causes a recursion depth error to the thousands of calls being made to update in such a small period of time.
Finally to repeat. When resizing with the default window Resize nudges at the edge of the window the canvas resizes perfectly with perfect draw rate and the content stays visible all the time. Why is my binding not acting the same way?
Geometry calculations are quite complex in general (and Tk has some fairly sophisticated solvers behind the scenes for handling them) so they only run when the GUI is considered to be "idle". Idleness occurs when the event queue would otherwise block, waiting for the next message from the external world, i.e., when there's nothing else queued up for immediate attention. Tk also postpones redrawing (another actually really expensive operation) the same way; it makes the UI seem far faster than it really is. This was critical back 20 years ago and more, and yet is still really useful now.
But it breaks if there there is a total flood of events coming in, and drags can do that, especially when there's resizing involved. That's why you can do update idletasks (in Tcl/Tk notation) and update_idletasks() (in Tkinter notation). That that does is immediately process all the pending idle event callbacks without taking further real events, running the pending resize computations and redraws. It is far less likely to trigger reentrant event processing than a full update(), which is the problem you were hitting (and why update is considered harmful under normal circumstances).
Note that it is important to let the event loop itself have some time to process still, as parts of handling widget resizes are inevitably done via real events (as they impact upon the OS/GUI). By far the best way to do that is to just make sure to return normally from your event handlers as soon as practical.
Are you using the right bind event? Try ''
g.bind('<ButtonRelease-1>',resize)
I am writing an app in kivy which does cpu-heavy calculations at launch. I want the app to display what it's doing at the moment along with the progress, however, since the main loop is not reached yet, it just displays empty white screen until it finishes working. Can I force kivy to update the interface?
Basically I'm looking for kivy's equivalent of Tkinter's root.update()
I could create a workaround by defining a series of functions with each calling the next one through Clock.schedule_once(nextFunction, 1), but that would be very sloppy.
Thanks in advance.
Leaving aside the question of whether you should be using threading or something instead (which possibly you should), the answer is just that you should move your cpu calculations to somewhere else. Display something simple initially (i.e. returning a simple widget from your build method), then do the calculations after that, such as by clock scheduling them.
Your calculations will still block the gui in this case. You can work around this by doing them in a thread or by manually breaking them up into small pieces that can be sequentially scheduled.
It might be possible to update the gui by manually calling something like Clock.tick(), but I'm not sure if this will work right, and even if so it won't be able to display graphics before they have been initialised.
I am writing a simple image viewer that lets the user flip very quickly through tens of thousands of images, about 100 at a time. The images are files on disk.
In order for the viewer to function, it must continuously preload images ahead of the user's current one (or the viewer would be unusably sluggish).
The basic recipe that I'm using to display the images in a grid of Tkinter labels, is the following (this has been tested and works):
def load_image(fn):
image = Image.open(fn)
print "Before photoimage"
img = ImageTk.PhotoImage(image)
print "After photoimage"
label.config(image=load_image("some_image.png")
I need the ImageTk.PhotoImage instance to display the image on a label. I have implemented two different approaches, each with an associated problem.
First approach:
Launch a separate thread which pre-loads the images:
def load_ahead():
for fn in images:
cache[fn] = load_image()
threading.Thread(target=load_ahead).start()
top.mainloop()
This works quite well on my Linux machine. However, on another machine (which happens to be running Windows, and compiled with pyinstaller), a deadlock seems to happen. "Before Photoimage" is printed, and then the program freezes, which suggests that the loader thread gets stuck at creating the ImageTk.PhotoImage object. Musst the creation of an ImageTk.PhotoImage object happen within the main (Tkinter mainloop's) thread? Is the creation of PhotoImage computationally expensive, or is negligible compared to actually loading the image from disk?
Second approach:
In order to circumvent this possible requirement of PhotoImage objects being created from within Tkiner's mainloop thread, I resorted to Tk.after:
def load_some_images():
#load only 10 images. function must return quickly to prevent freezing GUI
for i in xrange(10):
fn = get_next_image()
cache[fn] = load_image(fn)
top.after_idle(load_some_images)
top.after_idle(load_some_images)
The problem with this is that, appart from creating additional overhead (ie the image-loading procedure must be broken up into very small chunks since it is competing with the GUI) that it periodically freezes the GUI for the duration of the call, and it seems to consume any keyboard events that happened during its execution.
Third approach
Is there a way I can detect pending user events? How can I accomplish something like this?
def load_some_images():
while True:
try: top.pending_gui_events.get_nowait()
except: break
#user is still idle! continuing caching of images
fn = get_next_image()
cache[fn] = load_image(fn)
top.after_idle(load_some_images)
top.after(5,load_some_images)
Edit: I have tried using top.tk.call('after','info') to check pending keyboard events. This doesn't always reliably, and the interface is still sluggish/unresponsive.
Thanks in advance for any ideas
I recommend creating an load_one_image function rather than a load_some_images function. It will be less likely to interfere with the event loop.
Also, as a rule of thumb, a function called via after_idle shouldn't reschedule it self with after_idle. The reason is that after_idle will block until the idle event queue is drained. If you keep adding stuff on to the queue while the queue is being processed, it never gets completely drained. This could be the reason why your GUI seems to hang once in a while with your second approach.
Try after(5, ...) rather than after_idle(...). If your system can create an image in less than 5ms, you can process 100 images in about half a second, which is probably fast enough to give a pretty snappy interface. You can tweak the delay to see how it affects the overall feel of the app.
I've seen a lot of stuff about running code in subprocesses or threads, and using the multiprocessing and threading modules it's been really easy. However, doing this in a GUI adds an extra layer of complication.
From what I understand, the GUI classes don't like it if you try and manipulate them from multiple threads (or processes). The workaround is to send the data from whatever thread you created it in to the thread responsible for the graphics and then render it there.
Unfortunately, for the scenario I have in mind this is not an option: The gui I've created allows users to write their own plotting code which is then executed. This means I have no control over how they plot exactly, nor do I want to have it. (Update: these plots are displayed in separate windows and don't need to be embedded anywhere in the main GUI. What I want is for them to exist separated from the main GUI, without sharing any of the underlying stack of graphics libraries.)
So what I'm wondering now is
Is there some clean(ish) way of executing a string of python code in a whole new interpreter instance with its own ties to the windowing system?
In response to the comments:
The current application is set up as follows: A simple python script loads a wxPython gui (a wx.App). Using this gui users can set up a simulation, part of which involves creating a script in plain python that runs the simulation and post-processes the results (which usually involves making plots and displaying them). At the moment I'm doing this by simply calling exec() on the script code. This works fine, but the gui freezes while the simulation is running. I've experimented with running the embedded script in a subprocess, which also works fine, right up until you try to display the created graphs (usually using matplotlib's show()). At this point some library deep down in the stack of wxPython, wx, gtk etc starts complaining because you cannot manipulate it from multiple threads.
The set-up I would like to have is roughly the same, but instead of the embedded script sharing a GUI with the main application, I would like it to show graphics in an environment of its own.
And just to clarify:
This is not a question about "how do I do multithreading/multiprocessing" or even "how do I do multithreading/multiprocessing within a single wxpython gui". The question is how I can start a script from a gui that loads an entirely new gui. How do I get the window manager to see this script as an entirely separate application?
The easiest way would be to generate it in a temporary folder somewhere and then make a non-blocking call to the python interpreter, but this makes communication more difficult and it'd be quite hard to know when I could delete the temp files again. I was hoping there was a cleaner, dynamical way of doing this.
Can you simply use subprocess to run 'python.exe' and pipe the script in?
Alternatively, the multiprocessing package should suffice if you want to move some (pickle-able) data over to the new process in which you run the script. Just create a function/callable that runs the script, and create a Process object with the callable as target. That way, you should be able to pass some data over, without having GUI issues.
Capturing text with either is easy, subprocess allows that and no more. With multiprocess, you can pass python objects back and forth more easily.
On Windows, you can create window with a parent window from another process, and draw to that.
See the hWndParent argument to CreateWindowEx.
If wxWindows supports getting/setting that explicitly, then you should be good to go.
Depending on your platform, something similar might be possible in any windows system.
So, just giving your users the ability to find the handle of your apps window should give them the option to plot away at views embedded in your app, while running in their own processes.
I don't no much about wx, I work with jython(python implemented in java and you can use java) and swing. Swing has its own worker thread, and if you do gui updates you wrap your code into a runnable and invoke it with swing.invokelater.
You could see if wx has something like that, if you however are only allowed to manipulate the gui from the thread in which you created it try something similar. create a proxy object for your gui, which forwards all your calls to your thread which forwards them to the gui.
But proxying like this gets messy. how about you let them define classes, with an 'updateGui' function, that they should hand back to you over a queue and that you will execute in your gui thread.
In wxPython land when you use threads, you have to use its thread-safe methods to communicate with the GUI: wx.CallAfter, wx.CallLater or wx.PostEvent. In your case, I would run any long running code in a separate thread/process and when it's done its processing, send the result to the GUI. The GUI can instantiate a new frame and use matplotlib or PyPlot to show the plot, depending on which way you want to go. I've heard you can draw the plot using FloatCanvas too.
Anyway, if you instantiate the new frame correctly, then you can instantiate N frames and show them and be fine. See the wxPython wiki for a few examples of using Threads with wx: http://wiki.wxpython.org/LongRunningTasks
How can I get a Progressbar to "pulse" while another function is run?
There is an example of how to do this here.
Push that another function into a separate thread. As long as your main thread runs any code, GUI is frozen. This is not a problem for short code pieces, but obviously a problem in your case.
Also read what PyGTK FAQ has to say about using threads in PyGTK program.
If your function runs in many iterations that don't take too long by themselves, then you don't necessarily need to mess around with separate threads. You can also cause the GUI to update itself during your long calculation:
def long_function(some_args):
while task_is_not_finished():
do_some_stuff_that_doesnt_take_too_long()
progress_bar.pulse()
while gtk.events_pending():
gtk.main_iteration()