I'm having an issue with my GUI freezing, and I don't know why. The run method is not releasing the lock.
Demo program
import time
import threading
import Tkinter as tk
import ttk
LOCK = threading.Lock()
class Video(threading.Thread):
def __init__(self):
super(Video, self).__init__()
self.daemon = True
self.frame = tk.DoubleVar(root, value=0)
self.frames = 1000
def run(self):
while True:
with LOCK:
position = self.frame.get()
if position < self.frames:
position += 1
else:
position = 0
self.frame.set(position)
time.sleep(0.01)
root = tk.Tk()
video = Video()
root.minsize(500, 50)
def cb_scale(_):
with LOCK:
print('HELLO')
scale = ttk.Scale(
root, from_=video.frame.get(), to=video.frames, variable=video.frame,
command=cb_scale)
scale.grid(row=0, column=0, sticky=tk.EW)
root.columnconfigure(0, weight=1)
if __name__ == '__main__':
video.start()
root.mainloop()
Problem
Spam-clicking the progress bar freezes the program.
Attempts at debugging
I used mttkinter by adding import mttkinter to the import statements and the problem persists. The issue is the lock not being released.
I inserted print statements to find out where exactly the program freezes.
Program with print statements:
from __future__ import print_function
import time
import threading
import Tkinter as tk
import ttk
def whichthread(say=''):
t = threading.current_thread()
print('{}{}'.format(say, t))
LOCK = threading.Lock()
class Video(threading.Thread):
def __init__(self):
super(Video, self).__init__()
self.daemon = True
self.frame = tk.DoubleVar(root, value=0)
self.frames = 1000
def run(self):
while True:
whichthread('run tries to acquire lock in thread: ')
with LOCK:
whichthread('run acquired lock in thread: ')
position = self.frame.get()
if position < self.frames:
position += 1
else:
position = 0
self.frame.set(position)
whichthread('run released lock in thread: ')
time.sleep(0.01)
root = tk.Tk()
video = Video()
root.minsize(500, 50)
def cb_scale(_):
whichthread('cb_scale tries to acquire lock in thread: ')
with LOCK:
whichthread('cb_scale acquired lock in thread: ')
print('HELLO')
whichthread('cb_scale released lock in thread: ')
scale = ttk.Scale(
root, from_=video.frame.get(), to=video.frames, variable=video.frame,
command=cb_scale)
scale.grid(row=0, column=0, sticky=tk.EW)
root.columnconfigure(0, weight=1)
if __name__ == '__main__':
video.start()
root.mainloop()
This produces the following output right before the program freezes:
...
run tries to acquire lock in thread: <Video(Thread-1, started daemon 140308329449216)>
run acquired lock in thread: <Video(Thread-1, started daemon 140308329449216)>
cb_scale tries to acquire lock in thread: <_MainThread(MainThread, started 140308415592256)>
This shows that for some reason, the run method does not release the lock.
I tried to comment out lines in order to narrow the problem down.
Removing any of the two with LOCK statements fixes the issue. Unfortunately, in my real program the run and cb_scale function do something meaningful that requires locking.
Commenting out both the calls to get and set in run fixes the issue.
... and this is where I am stuck! :)
EDIT
Thanks to Mike - SMT I was able to track the problem down further.
Using
class DummyDoubleVar(object):
def get(self):
return 500
def set(self, _):
pass
and
self.frame = DummyDoubleVar()
in Video.__init__ prevents the program from freezing.
(Remember that the original program reliably freezes even with mttkinter. I am stumped what's going on here!)
In this post, I will show the solution to the problem and what led me to discover it. It involves going over CPython _tkinter.c code, so if that's not something you are up for, you can just skip to the TL;DR section below. Now, let's dive down the rabbit hole.
Lead-Up
The problem occurs only when moving the sliding bar manually. The MainThread and the Video-thread are then in dead-lock with each other over the LOCK, which I will call the user-lock. Now, the run method never releases the user-lock after is has acquired it, which implies that is hanging because it is waiting for another lock or some operation to complete which cannot. Now, looking at the log output of your verbose example, it becomes clear that the program does not hang consistently: It takes a few tries.
By adding more prints to the run method, you may discover that the problem is not consistently caused by either get or set. When the problem is caused, get may have already finished, or it may not have. This implies that the problem is not caused by get or set specifically, rather by some more generic mechanism.
Variable.set and Variable.get
For this section, I considered only Python 2.7 code, even though the problem is also present in Python 3.6. From the Variable-class in the Tkinter.py file of CPython 2.7:
def set(self, value):
"""Set the variable to VALUE."""
return self._tk.globalsetvar(self._name, value)
def get(self):
"""Return value of variable."""
return self._tk.globalgetvar(self._name)
The self._tk attribute is the Tk-object defined in the C-code of Tkinter, and for the code of globalgetvar we must jump back to _tkinter.c:
static PyObject *
Tkapp_GlobalGetVar(PyObject *self, PyObject *args)
{
return var_invoke(GetVar, self, args, TCL_LEAVE_ERR_MSG | TCL_GLOBAL_ONLY);
}
Jumping to var_invoke:
static PyObject*
var_invoke(EventFunc func, PyObject *selfptr, PyObject *args, int flags)
{
#ifdef WITH_THREAD
// Between these brackets, Tkinter marshalls the call to the mainloop
#endif
return func(selfptr, args, flags);
}
Just to make sure: I compiled Python with thread support and the problem persists. The call is marshalled to the main thread, which I checked with a simple printf in that location. Now, is this done correctly? The function var_invoke will wait until the MainThread has resumed and has executed the requested call. What is the MainThread doing at this point? Well, it is executing its queue of events, in the sequence it got them. What sequence did it get them in? That depends on the timing. This is what causes the problem: In some cases, Tkinter will execute the call to the callback right before a get or set, but while the lock is held.
Regardless of whether mtTkinter is imported (as long as Python is compiled WITH_THREAD support), the call of get and set is marshalled off to the mainloop, but that mainloop may just be trying at that moment to call the callback, which also needs the lock... This is what causes the deadlock and your problem. So basically mtTkinter and plain Tkinter offer the same behaviour, though for mtTkinter this behaviour is caused in the Python code and for plain Tkinter it happes in C-code.
TL;DR; In short
The problem is caused by only the user-lock. Neither GIL nor the Tcl-interpreter lock is involved. The problem is caused by the get and set methods marshalling their actual invocation off to the MainThread and then waiting for completion of the call by this MainThread, while the MainThread tries to do the events in order and execute the callback first.
Is this intended behaviour? Maybe, I'm not sure. I sure can see that with all the ENTER_TCL and LEAVE_TCL macro's in the _tkinter.c file, a better solution might be possible than the current one. For now though, there is no real work-around for this problem (bug? feature?) that I can see, apart from using Tk.after(0, Variable.set), so that the Video-thread does not hold the lock while the MainThread might need it. My suggestion would be removing the DoubleVar.get and set invocations from the code where the lock is held. After all, if your program does something meaningful, it might not need to hold the lock while it sets the DoubleVar. Or, if that is not an option, you will have to find some other means of synchronizing the value, like a subclass of the DoubleVar. Which suits your needs best highly depends on your actual application.
I do not know 100% why your program locks up when clicking on the slider however I suspect that it is due to the tk.DoubleVar() as this is part of the main thread.
Consider using after() instead.
See below example and let me know if you have any questions.
import tkinter as tk
import tkinter.ttk as ttk
class Video(tk.Tk):
def __init__(self):
super().__init__()
self.minsize(500, 50)
self.daemon = True
self.frames = 1000
self.columnconfigure(0, weight=1)
self.vid_var = tk.DoubleVar(self, value=0)
scale = ttk.Scale(self, from_=self.vid_var.get(), to=self.frames, variable=self.vid_var, command=self.cb_scale)
scale.grid(row=0, column=0, sticky='ew')
self.run()
def cb_scale(self, var):
print('HELLO', var)
def run(self):
position = self.vid_var.get()
if position < self.frames:
position += 1
self.after(10, self.run)
else:
position = 0
self.vid_var.set(position)
if __name__ == '__main__':
Video().mainloop()
I had the same problem but i found the solution.
Just create your own sleep function.
like this -
from time import time
def sleep(timeforsleep):
global root
starttime = time()
while True:
if (starttime <= time()-timeforsleep):
break
else:
root.update()
and then use sleep(2)
and then remember to change import time to from time import time
Easy Peasy
Related
Part of my code is as follows:
def get_songs():
label6.configure(text='Wait')
os.system('/home/norman/my-startups/grabsongs')
label6.configure(text='Done')
The label is not updated at the first .configure() but is at the second one.
Except if I cause a deliberate error immediately after the first one at which point it is updated and then the program terminates.
The system call takes about 2 minutes to complete so it isn't as if there isn't time to display the first one.
I am using Python 2.7.6
Does anyone know why please?
I'm going to guess you're using Tkinter. If so, as #albert just suggested, you'll want to call label.update_idletasks() or label.update() to tell Tkinter to refresh the display.
As a very crude example to reproduce your problem, let's make a program that will:
Wait 1 second
Do something (sleep for 2 seconds) and update the text to "wait"
Display "done" afterwards
For example:
import Tkinter as tk
import time
root = tk.Tk()
label = tk.Label(root, text='Not waiting yet')
label.pack()
def do_stuff():
label.configure(text='Wait')
time.sleep(2)
label.configure(text='Done')
label.after(1000, do_stuff)
tk.mainloop()
Notice that "Wait" will never be displayed.
To fix that, let's call update_idletasks() after initially setting the text:
import Tkinter as tk
import time
root = tk.Tk()
label = tk.Label(root, text='Not waiting yet')
label.pack()
def do_stuff():
label.configure(text='Wait')
label.update_idletasks()
time.sleep(2)
label.configure(text='Done')
label.after(1000, do_stuff)
tk.mainloop()
As far as why this happens, it actually is because Tkinter doesn't have time to update the label.
Calling configure doesn't automatically force a refresh of the display, it just queues one the next time things are idle. Because you immediately call something that will halt execution of the mainloop (calling an executable and forcing python to halt until it finishes), Tkinter never gets a chance to process the changes to the label.
Notice that while the gui displays "Wait" (while your process/sleep is running) it won't respond to resizing, etc. Python has halted execution until the other process finishes running.
To get around this, consider using subprocess.Popen (or something similar) instead of os.system. You'll then need to perodically poll the returned pipe to see if the subprocess has finished.
As an example (I'm also moving this into a class to keep the scoping from getting excessively confusing):
import Tkinter as tk
import subprocess
class Application(object):
def __init__(self, parent):
self.parent = parent
self.label = tk.Label(parent, text='Not waiting yet')
self.label.pack()
self.parent.after(1000, self.do_stuff)
def do_stuff(self):
self.label.configure(text='Wait')
self._pipe = subprocess.Popen(['/bin/sleep', '2'])
self.poll()
def poll(self):
if self._pipe.poll() is None:
self.label.after(100, self.poll)
else:
self.label.configure(text='Done')
root = tk.Tk()
app = Application(root)
tk.mainloop()
The key difference here is that we can resize/move/interact with the window while we're waiting for the external process to finish. Also note that we never needed to call update_idletasks/update, as Tkinter now does have idle time to update the display.
I am new to Tkinter, so I apologize if this is easy, but I have search for a couple of hours and can't figure it out. What I want to do is after the mainloop is idle, I always want to call the function checkForGroupUpdates(). When I run the code below, it only runs once. I can't figure out to have it run every time the mainloop is idle. I appreciate the help.
from Tkinter import *
import random
class Network(Frame):
""" Implements a stop watch frame widget. """
def __init__(self, parent=None, **kw):
Frame.__init__(self, parent, kw)
self.makeWidgets()
def makeWidgets(self):
""" Make the time label. """
self._canvas = Canvas(self, width=600, height=400)
self._canvas.pack()
def checkForGroupUpdates(self):
print "checking"
h=0
this=10
while this>.0001:
this=random.random()
print h
h=h+1
print "checked"
def main():
root = Tk()
nw = Network(root)
nw.pack(side=TOP)
root.after_idle(nw.checkForGroupUpdates)
root.mainloop()
if __name__ == '__main__':
main()
#user1763510, notice that in Bryan Oakley's answer, he has checkForGroupUpdates call self.after again. This is because self.after only does a single call, so getting repeated calls requires having it call itself within the function that gets called by the first call. This way, it keeps repeatedly calling itself.
The same goes for the after_idle() function. You have to have checkForGroupUpdates call after_idle() again at the bottom.
Here is the documentation for after, after_idle, etc. There is even a little example in the after description, which makes it all clear.
Documentation: http://effbot.org/tkinterbook/widget.htm
Example from link above, under the afterdescription:
#Method 1
class App:
def __init__(self, master):
self.master = master
self.poll() # start polling
def poll(self):
... do something ...
self.master.after(100, self.poll)
To use after_idle instead, it would look like this:
#Method 2
class App:
def __init__(self, master):
self.master = master
self.poll() # start polling
def poll(self):
... do something ...
self.master.update_idletasks()
self.master.after_idle(self.poll)
Notice the addition of the self.master.update_idletasks() line. This draws the GUI and handles button presses and things. Otherwise, after_idle() will suck up all resources and not let the GUI self-update properly in the mainloop().
An alternative to using
self.master.update_idletasks()
self.master.after_idle(self.poll)
is to use:
#Method 3
self.master.update_idletasks()
self.master.after(0, self.poll)
Using self.master.after(0, self.poll) is my preferred technique, as it allows me to easily change the 0 to something else if I decide I don't need to run self.poll constantly. By increasing the delay time to at least 1 ms, you no longer need to call self.master.update_idletasks() at all. So, this works too:
#Method 4
self.master.after(1, self.poll)
Also notice that for all examples above, calling self.poll() in the __init__ function is what kicks it all off, and storing master into self.master is necessary simply so that inside poll you can call the after or after_idle function via self.master.after_idle, for example.
Q: Is this stable/does it work?
A: I ran a test code using Method 3 just above for ~21 hrs, and it ran stably the whole time, allowing the GUI to be usable and all.
Q: What is the speed comparison for each method above?
A:
Method 1: (I didn't speed test it)
Method 2: ~0.44 ms/iteration
Method 3: ~0.44 ms/iteration
Method 4: ~1.61 ms/iteration
Q: Which is my preferred method?
A: Method 3 or 4.
Instead of calling the function all the time when the app is idle, you should just call it once every fraction of a second. For example, if you want to check 10 times every second you would do something like this:
def checkForGroupUpdates(self):
<do whatever you want>
self.after(100, self.checkForGroupUpdates)
Once you call that function once, it will arrange for itself to be called again in 100ms. This will continue until the program exits. If the program goes "non-idle" (ie: while responding to a button click), this function will pause since tkinter is single-threaded. Once the program goes idle again, the check will continue.
I'm trying to learn how to use the thread module. I followed along with the instructions here: http://effbot.org/zone/tkinter-threads.htm
My hope is the test script will:
Print out the "count" every two seconds
Show a pop-up dialog window (also every 2 seconds)
The pop-ups should be allowed to accumulate (if I don't click "OK" for a while, there should be
multiple pop-ups)
However, when I run this script it will freeze the main window and after a while crash. I think I'm not implementing the thread module correctly.
Could someone please have a look and point out what I'm doing wrong?
Here is what I've tried so far:
from Tkinter import *
import thread
import Queue
import time
class TestApp:
def __init__(self, parent):
self.super_Parent = parent
self.main_container = Frame(parent)
self.main_container.pack()
self.top_frame = Frame(self.main_container)
self.top_frame.pack(side=TOP)
self.bottom_frame = Frame(self.main_container)
self.bottom_frame.pack(side=TOP)
self.text_box = Text(self.top_frame)
self.text_box.config(height=20, width=20)
self.text_box.pack()
self.queue = Queue.Queue()
self.update_me()
def show_popup(self):
self.my_popup = Toplevel(self.main_container)
self.my_popup.geometry('100x100')
self.popup_label = Label(self.my_popup, text="Hello!")
self.popup_label.pack(side=TOP)
self.pop_button = Button(self.my_popup, text="OK", command=self.my_popup.destroy)
self.pop_button.pack(side=TOP)
def write(self, line):
self.queue.put(line)
def update_me(self):
try:
while 1:
line = self.queue.get_nowait()
if line is None:
self.text_box.delete(1.0, END)
else:
self.text_box.insert(END, str(line))
self.text_box.see(END)
self.text_box.update_idletasks()
except Queue.Empty:
pass
self.text_box.after(100, self.update_me)
def pipeToWidget(input, widget):
widget.write(input)
def start_thread():
thread.start_new(start_test, (widget,))
def start_test(widget):
count = 0
while True:
pipeToWidget(str(count) + "\n", widget)
count += 1
time.sleep(2)
widget.show_popup()
root = Tk()
widget = TestApp(root)
start_button = Button(widget.bottom_frame, command=start_thread)
start_button.configure(text="Start Test")
start_button.pack(side=LEFT)
root.title("Testing Thread Module")
root.mainloop()
I can't reproduce your problem, but I can see why it would happen.
You're using the queue to pass messages from the background thread to the main thread for updating text_box, which is correct. But you're also calling widget.show_popup() from the background thread, which means it creates and displays a new Toplevel in the background thread. That's not correct.
All UI code must run in the same thread—not all UI code for each top-level window, all UI code period. On some platforms, you may get away with running each window in its own thread (or even free-threading everything), but that isn't supposed to work, and definitely will crash or do improper things on some platforms. (Also, that single UI thread has to be the initial thread on some platforms, but that isn't relevant here.)
So, to fix this, you need to do the same dance for creating the popups that you do for updating the textbox.
The obvious way to do that is to move the widget.show_popup() to the loop in update_me(). If you want it to happen 2 seconds after the textbox updates, just add self.top_frame.after(2000, self.show_popup) to the method.
But I'm guessing you're trying to teach yourself how to have multiple independent updating mechanisms, so telling you "just use a single update queue for everything" may not be a good answer. In that case, just create two queues, and a separate update method servicing each queue. Then, do your pipeToWidget, sleep 2 seconds, then pipeToPopup.
Another way around this is to use mtTkinter. It basically does exactly what you're doing, but makes it automatic, pushing each actual Tk GUI call onto a queue to be run later by the main loop. Of course your objects themselves have to be thread-safe, and this also means that you have to deal with the GUI calls from one thread getting interleaved with calls from another thread. But as long as neither of those is a problem (and they don't seem to be in your case), it's like magic.
If you want to know why this is freezing and/or crashing for you on Win7 and not for me on OS X 10.8… well, you really need to look into a mess of Tcl, C, and Python code, and also at how each thing is built. And, unless it's something simple (like your Tk build isn't free-threaded), it wouldn't tell you much anyway. The code isn't supposed to work, and if it seems to work for me… that probably just means it would work every time until the most important demo of my career, at which point it would fail.
How to run multiple processes in python without multithreading? For example consider the following problem:-
We have to make a Gui,which has a start button which starts a function(say, prints all integers) and there is a stop button, such that clicking it stops the function.
How to do this in Tkinter?
Then you need to bind the Button widget with the function which starts the working thread. For example:
import time
import threading
import Tkinter as tk
class App():
def __init__(self, root):
self.button = tk.Button(root)
self.button.pack()
self._resetbutton()
def _resetbutton(self):
self.running = False
self.button.config(text="Start", command=self.startthread)
def startthread(self):
self.running = True
newthread = threading.Thread(target=self.printints)
newthread.start()
self.button.config(text="Stop", command=self._resetbutton)
def printints(self):
x = 0
while self.running:
print(x)
x += 1
time.sleep(1) # Simulate harder task
With the self.running approach, you can end the thread gracefully only by changing its value. Note that the use of multiple threads serves to avoid blocking the GUI while printints is being executed.
I have read this previous question and I suppose why you explicitly asked here for a solution without multithreading. In Tkinter this solution can be used in a scenario where the other threads have to communicate with the GUI part. For example: filling a progressbar while some images are being rendered.
However, as it has been pointed in the comments, this approach is too complex for just printing numbers.
Here you can find a lot of information and more examples about Tkinter.
Edit:
Since your new question has been closed, I'll change the code here to clarify the last point.
Did you try to used multiprocessing module? Seems to be the one you're looking for.
I have a relatively large application written in Python and using PyQT as a GUI frontend. The entire application is in one class, in one file.
Here's an example code:
class Application(QMainWindow):
def __init__(self):
super(etc...)
self.connect(self.mainBtn, SIGNAL("clicked()"), self.do_stuff)
def do_stuff(self):
<checking some parameters>
else:
do_some_other_long_stuff()
def do_some_other_long_stuff(self):
500 lines of code of stuff doing
However, this is the problem: when I click the mainBtn, everything goes fine, except the GUI kind of freezes - I can't do anything else until the function is performed (and it's a web scraper so it takes quite a bit of time). When the function do_some_other_long_stuff ends, everything goes back to normal. This is really irritating.
Is there a way to somehow "background" the do_some_other_stuff process? I looked into QThreads and it seems it does just that, however that would require me to rewrite basically all of code, put half of my program in a different class, and therefore have to change all the variable names (when getting a variable from GUI class and putting it in working class)
Duplicate of Handling gui with different threads,
How to keep track of thread progress in Python without freezing the PyQt GUI?, etc.
Your do_stuff() function needs to start up the computing thread and then return. Multi-threading is the name given to running multiple activities in a single process - by definition if something is going on "in the background", it's running on a separate thread. But you don't need to split functions into a different classes to use threads, just be sure that the computing functions don't do anything with the GUI and the main thread doesn't call any of the functions used by the computing thread.
EDIT 10/23: Here's a silly example of running threads in a single class - nothing in the language or the threading library requires a different class for each thread. The examples probably use a separate class for processing to illustrate good modular programming.
from tkinter import *
import threading
class MyApp:
def __init__(self, root):
self.root = root
self.timer_evt = threading.Event()
cf = Frame(root, borderwidth=1, relief="raised")
cf.pack()
Button(cf, text="Run", command=self.Run).pack(fill=X)
Button(cf, text="Pause", command=self.Pause).pack(fill=X)
Button(cf, text="Kill", command=self.Kill).pack(fill=X)
def process_stuff(self): # processing threads
while self.go:
print("Spam... ")
self.timer_evt.wait()
self.timer_evt.clear()
def Run(self): # start another thread
self.go = 1
threading.Thread(target=self.process_stuff, name="_proc").start()
self.root.after(0, self.tick)
def Pause(self):
self.go = 0
def Kill(self): # wake threads up so they can die
self.go = 0
self.timer_evt.set()
def tick(self):
if self.go:
self.timer_evt.set() # unblock processing threads
self.root.after(1000, self.tick)
def main():
root = Tk()
root.title("ProcessingThread")
app = MyApp(root)
root.mainloop()
main()