I've seen numerous question on this site, where people ask how user-written code, say a function foo, can be executed in tkinter's mainloop, e.g. this or this. Two alternatives exist: Use the after method, or use threading. I'd like to know more about how the after method actually works.
More precisely, inspired by this excellent article, where a very high-level description of how the GIL in Python works is given, I'd like to know more how the after method works in terms of the Python interpreter processing foo inside tkinter's mainloop.
I'm particularly confused how the CPython interpreter steps through the code, when I insert code using after. How is foo ending up being executed by tkinter's mainloop?
What I found so far:
Bryan Oakley, quoted from the first link, says: "after does not create another thread of execution. Tkinter is single-threaded. after merely adds a function to a queue."
But inspecting the source code
def after(self, ms, func=None, *args):
"""Call function once after given time.
MS specifies the time in milliseconds. FUNC gives the
function which shall be called. Additional parameters
are given as parameters to the function call. Return
identifier to cancel scheduling with after_cancel."""
if not func:
# I'd rather use time.sleep(ms*0.001)
self.tk.call('after', ms)
else:
def callit():
try:
func(*args)
finally:
try:
self.deletecommand(name)
except TclError:
pass
callit.__name__ = func.__name__
name = self._register(callit)
return self.tk.call('after', ms, name)
doesn't really help me, as it doesn't reveal the answers to these questions, and I'm a novice programmer, so I don't really understand how to trace this further.
I'd like to know more about how the after method actually works.
mainloop is just an endless loop which scans some internal queues to see if there are any events to process. Think of it as if it was implemented something like this:
def mainloop():
while the_window_exists():
if len(after_queue) > 0:
event = after_queue.pop()
if event.time_to_execute >= time.time():
event.command(**event.args)
if len(event_queue) > 0:
...
It's not literally implemented that way -- it's a bit more efficient and there's a little more going on, but logically it's nearly identical.
When you call after, it simply puts something on the "after" queue. Nothing more, nothing less.
Using the same analogy, after might be implemented something like this:
def after(delay, code_to_run, *args):
event = Event()
event.code_to_run = code_to_run
event.args = args
event.time_to_execute = time.time() + delay
event_queue.append(event)
That's all there is to it. You're putting something on a queue when you call after, and mainloop pulls things off of that queue. It all happens in the same thread.
To mainloop, the function you added with after is no different than a function that gets added when you move the mouse or press a button -- it's just an event object on a queue.
Related
Using PyGubu (tool to create Tkinter interfaces) I obtained the following GUI:
Current situation:
When I click the Button "Create", a function is called. This function takes quite some time, and the graphical interfaces is just frozen. As I would like to keep the graphical interface and the functional part as much separated as possible, I don't want to update the Progress Bar or the interface in general from the function I call
Desired situation
The best case for me would be a solution without Threading: I would like, upon clicking "Create", that my function runs, while the Progress Bar updates itself (just to show a feedback to the user, and to tell him "look, I am doing something") and the interface remains responsive, so the user can actually interact with it while the function finish.
Current attempts
I tried to solve this using Threading:
#I have this code in my main.py:
from threading import Thread
from queue import Queue, Empty
my_queue=Queue()
#And this is a simplified version of the Command of the "Create" Button:
def create_command(self):
#Show the progress bar and start it
self.show_item(self.progress)
self.progress.start()
#Run the big function
thrd = Thread(target = my_big_function, args=(some_arguments, my_queue))
thrd.start()
do_retry = True
while do_retry: #Repeat until you have a result
try:
result = my_queue.get(False) #If you have a result, exit loop. Else, throw Empty
do_retry = False
except Empty: #Queue is still empty
self.progress_var.set(self.progress_var.get()+1)
sleep(0.05)
self.mainwindow.update() #Update the progress bar
q.task_done()
self.progress.stop()
Problem of the current attempt
As I am not used to work with threads, I am facing two problems:
In some runs (not all of them, just some) I have a RuntimeError stating
RuntimeError: main thread is not in main loop
I tried to overcome this looking at other question in StackOverflow, but now it just happens randomly, and I don't know how to avoid it. The module mtTinker is no more maintained for python 3.x (there is a vague attempt full of ToDoes and blood)
If there is some kind of exception in the big function, I don't know how to handle it. The program would just run forever waiting for a result that will never come back
So
How can I obtain my desired outcome? Thanks in advance
You can try adding root.update() inside the function you call, inside the main loop. Hope that's helpful!
Is there an easy way to execute time delay (like time.sleep(3)) between every statement of Python code without having to explicitly write between every statement?
Like in the below Python Script which performs certain action on SAP GUI window. Sometimes, the script continues to the next statement before the previous statement is complete. So, I had to add a time delay between every statement so that it executes correctly. It is working with time delay, but I end up adding time.sleep(3) between every line. Just wondering if there is a better way?
import win32com.client
import time
sapgui = win32com.client.GetObject("SAPGUI").GetScriptingEngine
session = sapgui.FindById("ses[0]")
def add_record(employee_num, start_date, comp_code):
try:
time.sleep(3)
session.findById("wnd[0]/tbar[0]/okcd").text = "/npa40"
time.sleep(3)
session.findById("wnd[0]").sendVKey(0)
time.sleep(3)
session.findById("wnd[0]/usr/ctxtRP50G-PERNR").text = employee_num
time.sleep(3)
session.findById("wnd[0]").sendVKey(0)
time.sleep(3)
session.findById("wnd[0]/usr/ctxtRP50G-EINDA").text = start_date
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-WERKS[1,0]").text = comp_code
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-PERSG[2,0]").text = "1"
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT/ctxtRP50G-PERSK[3,0]").text = "U1"
time.sleep(3)
session.findById("wnd[0]/usr/tblSAPMP50ATC_MENU_EVENT").getAbsoluteRow(0).selected = True
time.sleep(3)
return "Pass"
except:
return "failed"
The right way to do what you asked for is almost certainly to use the debugger, pdb.
The right way to do what you want is probably something completely different: find some signal that tells you that the step is done, and wait for that signal. With problems like this, almost any time you pick will be way, way too long 99% of the time, but still too short 1% of the time. That signal may be joining a thread, or waiting on a (threading or multiprocessing) Condition, or getting from a queue, or awaiting a coroutine or future, or setting the sync flag on an AppleEvent, or… It really depends on what you're doing.
But if you really want to do this, you can use settrace:
def sleeper(frame, event, arg):
if event == 'line':
time.sleep(2)
return sleeper
sys.settrace(sleeper)
One small problem is that the notion of line used by the interpreter may well not be what you want. Briefly, a 'line' trace event is triggered whenever the ceval loop jumps to a different lnotab entry (see lnotab_notes.txt in the source to understand what that means—and you'll probably need at least a passing understanding of how bytecode is interpreted, at least from reading over the dis docs, to understand that). So, for example, a multiline expression is a single line; the line of a with statement may appear twice, etc.1
And there's probably an even bigger problem.
Sometimes, the script continues to next step before the previous step is fully complete.
I don't know what those steps are, but if you put the whole thread to sleep for 2 seconds, there's a good chance the step you're waiting for won't make any progress, because the thread is asleep. (For example, you're not looping through any async or GUI event loops, because you're doing nothing at all.) If so, then after 2 seconds, it'll still be just as incomplete as it was before, and you'll have wasted 2 seconds for nothing.
1. If your notion of "line" is closer to what's described in the reference docs on lexing and parsing Python, you could create an import hook that walks the AST and adds an expression statement with a Call to time.sleep(2) after each list element in each body with a module, definition, or compound statement (and then compiles and execs the result as usual).
Anything you want to happen in a program has to be explicitly stated - this is the nature of programming. This is like asking if you can print hello world without calling print("hello world").
I think the best advice to give you here is: don't think in terms of "lines", but think in term of functions.
use debugging mode and watch each and every line executing line by line.
I am creating a little script which check the number of mail in my gmail account and print them in the
status bar. The function gmail() returns the number of new emails. I have few questions, but first this is the code I wrote so far (clearly I am a novice):
class MyApplicationAppDelegate(NSObject):
var = 1
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
global ngmail
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.setTitle_(ngmail2)
ngmail = ngmail2
time.sleep(6)
1) Why do I need the line "self.statusItem.setTitle_("loading")" ? without that line it wouldn't update itself. I really do not know why.
2) it runs as it should, but whenever I get close to the number in the status bar, the spinning wheel appear.
I guess the reason is because I am using while, and instead I should be using something like nsrunloop or something like that. Can anyone advice on this?
3) If I put my mac to sleep and I wake it up, the script stops working. Any solution? maybe this is related to question 2) above.
Thanks!
All of your problems come from the fact that you're blocking the main thread.
In Cocoa, or almost any other GUI framework, the main thread runs a loop that waits for the next event, calls an event handler, and repeats until quit.
Your event handler, applicationDidFinishLaunching_, never returns. This means Cocoa can never handle the next event. Eventually, the OS will notice that you're not responding and put up the beachball.
With Cocoa, sometimes it sneaks in some other events each time you give it a chance, like on the setTitle_ calls, and there are some things the OS can fake even if you're not responding, like keeping the window redrawing, so it isn't always obvious that your app is not responsive. But that doesn't mean you don't need to solve the problem.
There are a number ways to do this, but is the easiest is probably to use a background thread. Then, applicationDidFinishLaunching_ can kick off the background thread and then return immediately, allowing the main thread to get back to its job handling events.
The only tricky bit is that code running on background threads can't make calls to UI objects. So, what do you do about that?
That's what performSelectorOnMainThread_withObject_waitUntilDone_ is for.
Here's an example:
class MyApplicationAppDelegate(NSObject):
var = 1
def background_work(self):
global ngmail
while var == 1 :
ngmail2 = gmail();
if ngmail2 !=ngmail:
self.statusItem.setTitle_("loading")
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
time.sleep(6)
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
self.statusItem = NSStatusBar.systemStatusBar().statusItemWithLength_(NSVariableStatusItemLength)
self.background_worker = threading.Thread(target=self.background_work)
self.background_worker.start()
The only tricky bit is that you have to use the ObjC name for the selector (setTitle:), not the Python name (setTitle_).
However, your code has another subtle bug: var isn't actually synchronized, so it's possible for you to change its value in the main thread, without the background thread ever noticing.
On top of that, doing a sleep(6) means that it will take up to 6 seconds to quit your app, because the background thread won't get to the code that checks var until it finishes sleeping.
You can fix both of these by using a Condition.
class MyApplicationAppDelegate(NSObject):
var = 1
condition = threading.Condition()
def background_work(self):
global ngmail
with condition:
while var == 1:
ngmail2 = gmail();
if ngmail2 != ngmail:
self.statusItem.performSelectorOnMainThread_withObject_waitUntilDone_('setTitle:', ngmail2, False)
condition.wait(6)
#classmethod
def shutdown_background_threads(cls):
with condition:
var = 0
condition.notify_all()
(I assume you used a class attribute for var instead of an instance attribute on purpose, so I likewise made the condition a class attribute and the shutdown method a class method.)
I was reading Roger Stuckey's wxPython Multiprocessing code to try to make a similar program myself. Full code can be found here.
The code runs fine without any modification. However, I found a parameter self.update been pass around between the GUI class MyFrame to the processing class TaskSErverMP. I have been searched throughout the entire code snippet and couldn't figure out what it is doing in the code -- it has never been initialized and used anyhow.
In the class MyFrame:
def OnStart(self, event):
...
self.taskserver.processTasks(self.update)
...
def OnStop(self, event):
...
self.taskserver.processStop(self.update)
...
def update(self, output):
"""
Get and print the results from one completed task.
"""
self.output_tc.AppendText('%s [%d] calculate(%d) = %.2f\n'...
...
# Give the user an opportunity to interact
wx.YieldIfNeeded()
In the class TaskServerMP:
def run(self):
...
self.processTasks(self.update)
...
def processTasks(self, resfunc=None):
...
def processStop(self, resfunc=None):
...
def update(self, output):
"""
Get and print the results from one completed task.
"""
sys.stdout.write('%s [%d] calculate(%d) = %.2f' % ....
So I thought that is a dependency injection practice but nothing more. I then removed it from the code and the strangest thing happened -- the program doesn't work anymore! I have the GUI displayed and I was able to start the processing. However, the GUI just hanged and then later Windows reported that the program is not responding. I have end up kill all the pythonw.exe processes manually from the Windows Task Manager.
Then I start to think if there is anything to do with the signature of the functions processTasks and processStop in the class TaskServerMP. But I really have no idea how I can associate the parameter self.update to the optional argument resfunc.
I don't think there is anything wrong in Roger's code. But it bothers me if I cannot twisted around the source to test out my understanding of the code.
I use Python 2.7 in Windows 7.
MyFrame.update is a method. You can see its definition on line 365.
So self.update is a bound method, meaning it can be called as if it were a regular function.
You can see that processTasks takes a resfunc parameter; then, at least 165, if it got a function or method as that resfunc parameter, it calls it.
The idea here is that processTasks leaves it up to the caller to decide how to print out progress updates as each task completes. One class might do it by writing them to stdout. A different class might instead update a GUI progress bar.
This is a pretty typical way to pass callbacks around in Python code.
So, why does the program hang if you take out the self.update? Well, look what's inside it, at line 372:
# Give the user an opportunity to interact
wx.YieldIfNeeded()
In wx, as in most GUI frameworks, the main thread is running an "event loop", something which has to process each event (a mouse move, a keypress, whatever) as it comes in, and then wait for the next one. You write your code as a bunch of event handlers—when someone clicks this button, run that function; etc. Your event handlers have to return quickly. Otherwise, the event loop doesn't get to pick up and dispatch the next event, so your GUI isn't responding. In wx, the Yield family of functions make life easier. As long as you Yield often enough, you don't have to return quickly. But you still have to do one or the other—either return early, or Yield—or the GUI will hang.
Here's a very simple example showing how to use bound methods:
class Foo(object):
def __init__(self, name):
self.name = name
def print_name(self):
print(self.name)
def give_me_a_printer_function(self):
return self.print_name
spam = Foo('Spam')
my_function1 = spam.print_name
my_function2 = spam.give_me_a_printer_function()
my_function1()
my_function2()
This will print Spam twice.
Functions and methods are first class values in Python—you can pass them around just like you can pass around numbers, strings, lists, and class instances. You can even print them out (although you'll get something ugly like <bound method Foo.print_name of <__main__.Foo object at 0x104629190>>).
I have got stuck with a problem.
It goes like this,
A function returns a single result normally. What I want is it to return continuous streams of result for a certain time frame(optional).
Is it feasible for a function to repeatedly return results for a single function call?
While browsing through the net I did come across gevent and threading. Will it work if so any heads up how to solve it?
I just need to call the function carry out the work and return results immediately after every task is completed.
Why you need this is not specified in the question, so it is hard to know what you need, but I will give you a general idea, and code too.
You could return in that way: return var1, var2, var3 (but that's not what you need I think)
You have multiple options: either blocking or non-blocking. Blocking means your code will no longer execute while you are calling the function. Non-blocking means that it will run in parallel. You should also know that you will definitely need to modify the code calling that function.
That's if you want it in a thread (non-blocking):
def your_function(callback):
# This is a function defined inside of it, just for convenience, it can be any function.
def what_it_is_doing(callback):
import time
total = 0
while True:
time.sleep(1)
total += 1
# Here it is a callback function, but if you are using a
# GUI application (not only) for example (wx, Qt, GTK, ...) they usually have
# events/signals, you should be using this system.
callback(time_spent=total)
import thread
thread.start_new_thread(what_it_is_doing, tuple(callback))
# The way you would use it:
def what_I_want_to_do_with_each_bit_of_result(time_spent):
print "Time is:", time_spent
your_function(what_I_want_to_do_with_each_bit_of_result)
# Continue your code normally
The other option (blocking) involves a special kind of functions generators which are technically treated as iterators. So you define it as a function and acts as an iterator. That's an example, using the same dummy function than the other one:
def my_generator():
import time
total = 0
while True:
time.sleep(1)
total += 1
yield total
# And here's how you use it:
# You need it to be in a loop !!
for time_spent in my_generator():
print "Time spent is:", time_spent
# Or, you could use it that way, and call .next() manually:
my_gen = my_generator()
# When you need something from it:
time_spent = my_gen.next()
Note that in the second example, the code would make no sense because it is not really called at 1 second intervals, because there's the other code running each time it yields something or .next is called, and that may take time. But I hope you got the point.
Again, it depends on what you are doing, if the app you are using has an "event" framework or similar you would need to use that, if you need it blocking/non-blocking, if time is important, how your calling code should manipulate the result...
Your gevent and threading are on the right track, because a function does what it is programmed to do, either accepting 1 var at a time or taking a set and returning either a set or a var. The function has to be called to return either result, and the continuous stream of processing is probably taking place already or else you are asking about a loop over a kernel pointer or something similar, which you are not, so ...
So, your calling code which encapsulates your function is important, the function, any function, eg, even a true/false boolean function only executes until it is done with its vars, so there muse be a calling function which listens indefinitely in your case. If it doesn't exist you should write one ;)
Calling code which encapsulates is certainly very important.
Folks aren't going to have enough info to help much, except in the super generic sense that we can tell you that you are or should be within in some framework's event loop, or other code's loop of some form already- and that is what you want to be listening to/ preparing data for.
I like "functional programming's," "map function," for this sort of thing. I think. I can't comment at my rep level or I would restrict my speculation to that. :)
To get a better answer from another person post some example code and reveal your API if possible.