GLib.idle_add(function) has different policies for different functions - python

I have a GTK window. It turns out that if I schedule several redraw calls all at once, without any delay, from a separate thread, using idle_add(window.queue_draw), only one call will execute.
While if I do idle_add(custom_function), every single scheduled call to custom_function will run.
While it's clear that this is done for optimization, I can't see if/where this is mentioned in the documentation [1] and I also wonder if there are other such rules for idle_add.
[1] https://developer.gnome.org/pygobject/stable/glib-functions.html

It actually isn't idle_add that is making that behavior. The docs for widget-queue-draw-region, which gets called by queue-redraw, state that redrawing only gets done after the main loop is no longer busy.

Related

Orphan stacktraces in sys._current_frames()

Here's an esoteric pure-Python question.
I'm doing some statistical profiling using sys._current_frames(). i.e. I've got a background thread that runs sys._current_frames() once every second, dumps the results in a text file, and then later I've got some Python code that sorts the tracebacks from most common to least.
One curious phenomenon I've seen is tracebacks like these:
File "/opt/foo/bar.py", line 1437, in __iter__
yield key
This yield is a generator that I wrote. The curious thing is that there's just one frame on this traceback. How could this be? The other traceback have lots of frames, either from the top level of the process or the top level of the frame. What is the meaning of this single-frame stacktrace?
One theory that I had is that this is a generator's frozen state, after it's yielded a value and it's waiting to have next called on it again. But I think I disproved this theory with a separate experiment: I made a generator, ensured it was paused, called sys._current_frames() and I didn't see that kind of stacktrace.
As the sys._current_frames() documentation warns,
This is most useful for debugging deadlock: this function does not require the deadlocked threads’ cooperation, and such threads’ call stacks are frozen for as long as they remain deadlocked. The frame returned for a non-deadlocked thread may bear no relationship to that thread’s current activity by the time calling code examines the frame.
sys._current_frames() is naturally prone to race conditions in any situation where you cannot guarantee the threads of interest are paused.
As you suspected, you're seeing a stack trace for a suspended generator. When a generator suspends, its stack frame has no parent frames. Its f_back is set to null.
sys._current_frames() retrieves stack frames for currently running threads, but by the time you look at those frames, they may not be running any more. If a generator suspends between the time you call sys._current_frames() and the time you inspect the frame, this is what it will look like. You might also see it on top of a call stack that looks completely different from when you actually called sys._current_frames(), if it gets resumed somewhere else.
Your test didn't show the generator frame because you suspended the generator before calling sys._current_frames() instead of afterward. The generator's stack frame was not the active frame of any thread at that point.

What are all these deprecated "loop" parameters in asyncio?

A lot of the functions in asyncio have deprecated loop parameters, scheduled to be removed in Python 3.10. Examples include as_completed(), sleep(), and wait().
I'm looking for some historical context on these parameters and their removal.
What problems did loop solve? Why would one have used it in the first place?
What was wrong with loop? Why is it being removed en masse?
What replaces loop, now that it's gone?
What problems did loop solve? Why would one have used it in the first place?
Prior to Python 3.6, asyncio.get_event_loop() was not guaranteed to return the event loop currently running when called from an asyncio coroutine or callback. It would return whatever event loop was previously set using set_event_loop(some_loop), or the one automatically created by asyncio. But sync code could easily create a different loop with another_loop = asyncio.new_event_loop() and spin it up using another_loop.run_until_complete(some_coroutine()). In this scenario, get_event_loop() called inside some_coroutine and the coroutines it awaits would return some_loop rather than another_loop. This kind of thing wouldn't occur when using asyncio casually, but it had to be accounted for by async libraries which couldn't assume that they were running under the default event loop. (For example, in tests or in some usages involving threads, one might want to spin up an event loop without disturbing the global setting with set_event_loop.) The libraries would offer the explicit loop argument where you'd pass another_loop in the above case, and which you'd use whenever the running loop differed from the loop set up with asyncio.set_event_loop().
This issue would be fixed in Python 3.6 and 3.5.3, where get_event_loop() was modified to reliably return the running loop if called from inside one, returning another_loop in the above scenario. Python 3.7 would additionally introduced get_running_loop() which completely ignores the global setting and always returns the currently running loop, raising an exception if not inside one. See this thread for the original discussion.
Once get_event_loop() became reliable, another problem was that of performance. Since the event loop was needed for some very frequently used calls, most notably call_soon, it was simply more efficient to pass around and cache the loop object. Asyncio itself did that, and many libraries followed suit. Eventually get_event_loop() was accelerated in C and was no longer a bottleneck.
These two changes made the loop arguments redundant.
What was wrong with loop? Why is it being removed en masse?
As any other redundancy, it complicates the API and opens up possibilities for errors. Async code should almost never just randomly communicate with a different loop, and now that get_event_loop() is both correct and fast, there is no reason not to use it.
Also, passing the loop through all the layers of abstraction of a typical application is simply tedious. With async/await becoming mainstream in other languages, it has become clear that manually propagating a global object is not ergonomic and should not be required from programmers.
What replaces loop, now that it's gone?
Just use get_event_loop() to get the loop when you need it. Alternatively, you can use get_running_loop() to assert that a loop is running.
The need for accessing the event loop is somewhat reduced in Python 3.7, as some functions that were previously only available as methods on the loop, such as create_task, are now available as stand-alone functions.
The loop parameter was the way to pass the global event loop around. New implementations of the same functions no longer require you to pass the global event loop, they instead just request it where it's needed.
As the documentation suggests https://docs.python.org/3/library/asyncio-eventloop.html: "Application developers should typically use the high-level asyncio functions, such as asyncio.run(), and should rarely need to reference the loop object or call its methods."
Removing the need for you to pass it around to library functions aligns with that principle. The loop is not replaced, but its disappearance simply means you no longer have to deal with it 'manually'.

Internal working of Tkinter after method

I'm working Tkinter GUI for quite sometime. And I was wondering, how after method actually works on it. My questions are very simple.
How after method is implemented internally in Tkinter ?
Are after calls threaded ?
What are the cons (if any) of multiple after calls in Tkinter ?
How are after callbacks scheduled ?
I did go through a couple of links like this. But couldn't get these informations.
after() calls Tck/Tk's after command, which registers a callback to be called later by the Tcl/Tk event loop.
after() calls are not threaded.
One issue is that a blocking or long-running operation may cause something scheduled with after() to run significantly later than you meant. There are many more possible pros and cons which may be relevant, but you'll need to specify your concerns or use cases...
A callback is registered into Tcl/Tk event loop, which takes care of the scheduling. For details on how that works, see the documentation.
Explanation:
As you may already know, Tkinter uses Tcl/Tk internally.
Taking a look at the code for the after() method, it appears that Tkinter is just calling the Tcl/Tk after command.
The documentation for Tcl/Tk's after command doesn't mention anything about threads. This makes sense given Tcl/Tk's single-threaded event loop design.
So, my conclusion is that Tkinter's after() method does not use threads, rather it just registers a callback using the internal Tcl/Tk after command, which will be called at the most appropriate time possible by the Tcl/Tk event loop.
It's really quite simple: Tkinter maintains a queue of work to be done (the event queue), and mainloop (as well as update) processes that queue. after merely adds a timestamped event to the queue. No threads are involved.

stop tkinter window from freezing while program is sleeping

This is quite an essential part of my program and I need to have sorted out as soon as possible so anything would be a massive help.
My program consists of three modules which are imported to each other. One module consists of my user interface for which I am using tkinter. The user inputs data on a canvas which is sent to a second program to be processed and is then sent to the third module which contains the algorithm which I intend to step through with the user.
The "first" and "third" modules can interact with each other and during certain points in explaining the algorithm I change the appearance of the canvas and some text on the interface. The third module should then pause (for which I'm currently using a basic sleep method), and wait (ideally it will wait for the user to press the "Next Step" button on the user interface). It is during this step that my interface decides that it wants to freeze.
Is there any way I can stop this?
Many thanks in advance.
Edit: I've found a way to fix this. Thank you for all the suggestions!
Calling time.sleep() will stop your program doing anything until it finishes sleeping. You need Tkinter to keep processing events until it should run the next part of your code.
To do that, put the next part of your code in a separate function, and get Tkinter to call it when it's ready. Typically, you want this to happen when the user triggers it (e.g. by clicking a button), so you need to bind it to an event (docs). If you actually want it to happen after a fixed time, you can use the .after() method on any tkinter widget (docs).
GUI programming takes a bit of getting used to. You don't code as a series of things happening one after the other, you write separate bits of code which are triggered by what the user does.
Terminology note: if your Python files import each other, you have three modules, but it's still all one program. Talking about the "first program" will confuse people.
H.E.P -
The traditional way to do this does indeed involve using a separate thread and co-ordinating the work between the "worker" thread and the GUI thread using some sort of polling or eventing mechanism.
But, as Thomas K. points out, that can get very complex and tricky, especially regarding Python's use of the Global Interpreter Lock (GIL) etc. and having to also contend with Tkinter's processing loop.
(The only good reason to use a multi-threaded GUI is if you absolutely MUST ensure that the GUI remains responsive during a potentially long-running background task, which I don't believe is the issue in this case.)
What I would suggest instead is a generator-based "co-routine"-type architecture.
As noted in "The Python (2.7) Language Reference", Section 6.8, [the "yield" statement is used when defining a generator function and is only used in the body of the generator function. Using a yield statement in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.]
(This effectively forms the basis of a co-routine architecture. (ed.))
[When a generator function is called, it returns an iterator known as a generator iterator, or more commonly, a generator. The body of the generator function is executed by calling the generator’s next() method repeatedly until it raises an exception.
When a yield statement is executed, the state of the generator is frozen and the value of expression_list is returned to next()‘s caller. By “frozen” we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack: enough information is saved so that the next time next() is invoked, the function can proceed exactly as if the yield statement were just another external call.]
(Also see "PEP 0342 - Coroutines via Enhanced Generators " for additional background and general info.)
This should allow your GUI to call the next part of your algorithm specification generator, on demand, without it having to be put to sleep until the operator presses the "Next" button.
You would basically just be creating a little 'domain-specific language', (DSL), consisting of just the list of steps for your presentation of this particular algorithm, and the generator (iterator) would simply execute each next step when called (on demand).
Much simpler and easier to maintain.
A GUI program is always waiting for some action to occur. When actions do occur, the event code corresponding to that action is executed. Therefore, there is no need to call sleep(). All you need to do is set it up so that the third program is executed from the appropriate event.

Is using multiple Timers in Python dangerous?

I am working on a text-based game in Python 3.1 that would use timing as it's major source of game play. In order to do this effectively (rather than check the time every mainloop, my current method, which can be inaccurate, and slow if multiple people are playing the game at once) I was thinking about using the Threading.Timer class. Is it a bad thing to have multiple timers going at the same time? if so, how many timers is recommended?
For example, the user inputs to start the game. every second after the game starts it decides whether or not something happens, so there's a Timer(1) for every user playing at the same time. If something happens, the player has a certain time to react to it, so a timer must be set for that. If the user reacts quickly enough, that timer needs to end and it will set a new timer depending on what's going to happen next, etc
I think its a bad idea to use Timers in your case.
Using the delayed threads in python will result in more complex code, less accuracy, and quite possible worse performance. Basically, the rule is that if you think you need threads, you don't. Very few programs benefit from the use of threads.
I don't know what you are doing for input. You make reference to multiple players and I'm not sure whether thats on a single keyboard or perhaps networked. Regardless, your current strategy of a main loop may well be the best strategy. Although without seeing how your main loop operates its hard to say for certain.
It should be perfectly safe to have multiple timers going at the same time. Beware that it may not give much of a performance boost, as the CPython interpreter (the standard Python interpreter) uses a GIL (Global Interpreter Lock) which makes threading stuff a bit.... slow.

Categories

Resources