I'm trying to create a chess game using tkinter. I don't have a huge experience in python programming, but I kind of find weird the philosophy of tkinter : if my assumptions are correct, it seems to me that using tkinter means setting it as the base of the project, and everything has to work around it. And what I mean by that is that using whatever code that is not 'wrapped' in the tkinter framework is a pain to deal with (you have to use the event system, you have to use the after method if you want to perform an action after starting the main loop, etc.)
I have a rather different view on that, and in my chess project I simply consider the tkinter display as a part of my rendering system, and the event system provided by tkinter as a part of my input parser system. That being said, I want to be able to easily change the renderer or the input parser, which means that I could want to detect input from the terminal (for instance by writing D2 D3) instead of moving the objects on the screen. I could also want to print the chessboard on the terminal instead of having a GUI.
More to the point, because tkinter blocks the thread through the mainloop method instead of looping in another thread, I have to put my Tk object in a different thread, so that I can run the rest of my program in parallel. And I'm having a tough time doing it, because my Tk variable contained by my thread needs to be accessed by my program, to update it for instance.
After quite a bit of research, I found that queues in python were synchronized, which means that if I put my Tk object in a queue, I could access it without any problem from the main thread. I tried to see if the following code was working :
import threading, queue
class VariableContainer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.queue = queue.Queue()
def run(self):
self.queue.put("test")
container = VariableContainer()
container.start()
print(container.queue.get(False))
and it does ! The output is test.
However, if I replace my test string by a Tk object, like below :
import threading, queue
import tkinter
class VariableContainer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.queue = queue.Queue()
def run(self):
root = tkinter.Tk()
self.queue.put(root)
root.mainloop() # whether I call the mainloop or not doesn't change anything
container = VariableContainer()
container.start()
print(container.queue.get(False))
then the print throws an error, stating that the queue is empty.
(Note that the code above is not the code of my program, it is just an exemple since posting sample codes from my project might be less clear)
Why?
The answer to the trivial question you actually asked: you have a race condition because you call Queue.get(block=False). Tk taking a lot longer to initialize, the main thread almost always wins and finds the queue still empty.
The real question is “How do I isolate my logic from the structure of my interface library?”. (While I understand the desire for a simple branch point between “read from the keyboard” and “wait for a mouse event”, it is considered more composable, in the face of large numbers of event types, sources, and handlers, to have one event dispatcher provided by the implementation. It can sometimes be driven one event at a time, but that composes less well with other logic than one might think.)
The usual answer to that is to make your logic a state machine rather than an algorithm. Mechanically, this means replacing local variables with attributes on an object and dividing the code into methods on its class (e.g., one call per “read from keyboard” in a monolithic implementation). Sometimes language features like coroutines can be used to make this transformation mostly transparent/automatic, but they’re not always a good fit. For example:
def algorithm(n):
tot=0
for i in range(n):
s=int(input("#%s:"%i))
tot+=(i+1)*(n-i)*s
print(tot)
class FSM(object):
def __init__(self,n):
self.n=n
self.i=self.tot=0
def send(self,s):
self.tot+=(self.i+1)*(self.n-self.i)*s
self.i+=1
def count(self): return self.i
def done(self): return self.i>=self.n
def get(self):
return self.tot
def coroutine(n): # old-style, not "async def"
tot=0
for i in range(n):
s=(yield)
tot+=(i+1)*(n-i)*s
yield tot
Having done this, it’s trivial to layer the traditional stream-driven I/O back on top, or to connect it to an event-driven system (be it a GUI or asyncio). For example:
def fsmClient(n):
fsm=FSM(n)
while not fsm.done():
fsm.send(int(input("#%s:"%fsm.count())))
return fsm.get()
def coClient(n):
co=coroutine(n)
first=True
while True:
ret=co.send(None if first else
int(input("#%s:"%fsm.count())))
if ret is not None:
co.close()
return ret
first=False
These clients can work with any state machine/coroutine using the same interface; it should be obvious how to instead supply values from a Tkinter input box or so.
Related
The context for this is much, much too big for an SO question so the code below is a extremely simplified demonstration of the actual implementation.
Generally, I've written an extensive module for academic contexts that launches a subprocess at runtime to be used for event scheduling. When a script or program using this module closes on pre-El Capitan machines my efforts to join the child process fail, as do my last-ditch efforts to just kill the process; OS X gives a "Python unexpectedly quit" error and the the orphaned process persists. I am very much a nub to multiprocessing, without a CS background; diagnosing this is beyond me.
If I am just too ignorant, I'm more than willing to go RTFM; specific directions welcome.
I'm pretty sure this example is coherent & representative, but, know that the actual project works flawlessly on El Capitan, works during runtime on everything else, but consistently crashes as described when quitting. I've tested it with absurd time-out values (30 sec+); always the same result.
One last note: I started this with python's default multiprocessing libraries, then switched to billiard as a dev friend suggested it might run smoother. To date, I've not experienced any difference.
UPDATE:
Had omitted the function that gives the #threaded decorator purpose; now present in code.
Generally, we have:
shared_queue = billiard.Queue() # or multiprocessing, have used both
class MainInstanceParent(object):
def __init__(self):
# ..typically init stuff..
self.event_ob = EventClass(self) # gets a reference to parent
def quit():
try:
self.event_ob.send("kkbai")
started = time.time()
while time.time - started < 1: # or whatever
self.event_ob.recieve()
if self.event_ob.event_p.is_alive():
raise RuntimeError("Little bugger still kickin'")
except RuntimeError:
os.kill(self.event_on.event_p.pid, SIGKILL)
class EventClass(object):
def __init__(self, parent):
# moar init stuff
self.parent = parent
self.pipe, child = Pipe()
self.event_p = __event_process(child)
def receive():
self.pipe.poll()
t = self.pipe.recv()
if isinstance(t, Exception):
raise t
return t
def send(deets):
self.pipe.send(deets)
def threaded(func):
def threaded_func(*args, **kwargs):
p = billiard.Process(target=func, args=args, kwargs=kwargs)
p.start()
return p
return threaded_func
#threaded
def __event_process(pipe):
while True:
if pipe.poll():
inc = pipe.recv()
# do stuff conditionally on what comes through
if inc == "kkbai":
return
if inc == "meets complex condition to pass here":
shared_queue.put("stuff inferred from inc")
Before exiting the main program, call multiprocessing.active_children() to see how many child processes are still running. This will also join the processes that have already quit.
If you would need to signal the children that it's time to quit, create a multiprocessing.Event before starting the child processes. Give it a meaningful name like children_exit. The child processes should regularly call children_exit.is_set() to see if it is time for them to quit. In the main program you call children_exit.set() to signal the child processes.
Update:
Have a good look through the Programming guidelines in the multiprocessing documentation;
It is best to provide the abovementioned Event objects as argument to the target of the Process initializer for reasons mentioned in those guidelines.
If your code also needs to run on ms-windows, you have to jump through some extra hoop, since that OS doesn't do fork().
Update 2:
On your PyEval_SaveThread error; could you modify your question to show the complete trace or alternatively could you post it somewhere?
Since multiprocessing uses threads internally, this is probably the culprit, unless you are also using threads somewhere.
If you also use threads note that GUI toolkits in general and tkinter in particular are not thread safe. Tkinter calls should therefore only be made from one thread!
How much work would it be to port your code to Python 3? If it is a bug in Python 2.7, it might be already fixed in the current (as of now) Python 3.5.1.
I'll do my best to explain this issue in a clear way, it's come up as part of a much larger piece of software I'm developing for an A level project - a project that aims to create a simple version of a graphical programming system (think scratch made by monkeys with about 7 commands).
My trouble currently stems from the need to have an execution function running on a unique thread that is capable of interacting with a user interface that shows the results of executing the code blocks made by the user (written using the Tkinter libraries) on the main thread. This function is designed to go through a dynamic list that contains information on the user's "code" in a form that can be looped through and dealt with "line by line".
The issue occurs when the execution begins, and the threaded function attempts to call a function that is part of the user interface class. I have limited understanding of multi threading, so it's all too likely that I am breaking some important rules and doing things in ways that don't make sense, and help here would be great.
I have achieved close to the functionality I am after previously, but always with some errors coming up in different ways (mostly due to my original attempts opening a tkinter window in a second thread... a bad idea).
As far as I'm aware my current code works in terms of opening a second thread, opening the UI in the main thread, and beginning to run the execution function in the second thread. In order to explain this issue, I have created a small piece of code that works on the same basis, and produces the same "none type" error, I would use the original code, but it's bulky, and a lot more annoying to follow than below:
from tkinter import *
import threading
#Represents what would be my main code
class MainClass():
#attributes for instances of each of the other classes
outputUI = None
threadingObject = None
#attempt to open second thread and the output ui
def beginExecute(self):
self.threadingObject = ThreadingClass()
self.outputUI = OutputUI()
#called by function of the threaded class, attempts to refer to instance
#of "outputUI" created in the "begin execute" function
def execute(self):
return self.outputUI.functionThatReturns()
#class for the output ui - just a blank box
class OutputUI():
#constructor to make a window
def __init__(self):
root = Tk()
root.title = ("Window in main thread")
root.mainloop()
#function to return a string when called
def functionThatReturns(self):
return("I'm a real object, look I exist! Maybe")
#inherits from threading library, contains threading... (this is where my
#understanding gets more patchy)
class ThreadingClass(threading.Thread):
#constructor - create new thread, run the thread...
def __init__(self):
threading.Thread.__init__(self)
self.start()
#auto called by self.start() ^ (as far as I'm aware)
def run(self):
#attempt to run the main classes "execute" function
print(mainClass.execute())
#create instance of the main class, then attempt execution of some
#threading
mainClass = MainClass()
mainClass.beginExecute()
When this code is run, it produces the following result:
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Python34\lib\threading.py", line 920, in _bootstrap_inner
self.run()
File "H:/Programs/Python/more more threading tests.py", line 33, in run
print(mainClass.execute())
File "H:/Programs/Python/more more threading tests.py", line 14, in execute
return self.outputUI.functionThatReturns()
AttributeError: 'NoneType' object has no attribute 'functionThatReturns'
I guess it should be noted that the tkinter window opens as I hoped, and the threading class does what it's supposed to, but does not appear to be aware of the existence of the output UI. I assume this is due to some part of object orientation and threading of which I am woefully under-informed.
So, is there a way in which I can call the function in the output ui from the threaded function? Or is there a work around to something similar?
It should be noted that I didn't put the creation of the output window in the init function of the main class, as I need to be able to create the output window and start threading etc as a result of another input.
Sorry if this doesn't make sense, shout at me and I'll try and fix it, but help would be greatly appreciated, cheers.
The problem here is your order of operations. You create a ThreadingClass in MainClass.beginExecute(), which calls self.start(). That calls run which ultimately tries to call a function on main.outputUI. But main.outputUI hasn't been initialized yet (that's the next line in beginExecute()). You could probably make it work just by reordering those two lines. But you probably don't want to be calling self.start() in the thread's __init__. That seems like poor form.
I have a project that I'm writing in Python that will be sending hardware (Phidgets) commands. Because I'll be interfacing with more than one hardware component, I need to have more than one loop running concurrently.
I've researched the Python multiprocessing module, but it turns out that the hardware can only be controlled by one process at a time, so all my loops need to run in the same process.
As of right now, I've been able to accomplish my task with a Tk() loop, but without actually using any of the GUI tools. For example:
from Tk import tk
class hardwareCommand:
def __init__(self):
# Define Tk object
self.root = tk()
# open the hardware, set up self. variables, call the other functions
self.hardwareLoop()
self.UDPListenLoop()
self.eventListenLoop()
# start the Tk loop
self.root.mainloop()
def hardwareLoop(self):
# Timed processing with the hardware
setHardwareState(self.state)
self.root.after(100,self.hardwareLoop)
def UDPListenLoop(self):
# Listen for commands from UDP, call appropriate functions
self.state = updateState(self.state)
self.root.after(2000,self.UDPListenLoop)
def eventListenLoop(self,event):
if event == importantEvent:
self.state = updateState(self.event.state)
self.root.after(2000,self.eventListenLoop)
hardwareCommand()
So basically, the only reason for defining the Tk() loop is so that I can call the root.after() command within those functions that need to be concurrently looped.
This works, but is there a better / more pythonic way of doing it? I'm also wondering if this method causes unnecessary computational overhead (I'm not a computer science guy).
Thanks!
The multiprocessing module is geared towards having multiple separate processes. Although you can use Tk's event loop, that is unnecessary if you don't have a Tk based GUI, so if you just want multiple tasks to execute in the same process you can use the Thread module. With it you can create specific classes which encapsulate a separate thread of execution, so you can have many "loops" executing simultaneously in the background. Think of something like this:
from threading import Thread
class hardwareTasks(Thread):
def hardwareSpecificFunction(self):
"""
Example hardware specific task
"""
#do something useful
return
def run(self):
"""
Loop running hardware tasks
"""
while True:
#do something
hardwareSpecificTask()
class eventListen(Thread):
def eventHandlingSpecificFunction(self):
"""
Example event handling specific task
"""
#do something useful
return
def run(self):
"""
Loop treating events
"""
while True:
#do something
eventHandlingSpecificFunction()
if __name__ == '__main__':
# Instantiate specific classes
hw_tasks = hardwareTasks()
event_tasks = eventListen()
# This will start each specific loop in the background (the 'run' method)
hw_tasks.start()
event_tasks.start()
while True:
#do something (main loop)
You should check this article to get more familiar with the threading module. Its documentation is a good read too, so you can explore its full potential.
I have an application that has a GUI thread and many different worker threads. In this application, I have a functions.py module, which contains a lot of different "utility" functions that are used all over the application.
Yesterday the application has been released and some users (a minority, but still) has reported problems with the application crashing. I looked over my code and noticed a possible design flaw, and would like to check with the lovely people of SO and see if I am right and if this is indeed a flaw.
Suppose I have this defined in my functions.py module:
class Functions:
solveComputationSignal = Signal(str)
updateStatusSignal = Signal(int, str)
text = None
#classmethod
def setResultText(self, text):
self.text = text
#classmethod
def solveComputation(cls, platform, computation, param=None):
#Not the entirety of the method is listed here
result = urllib.urlopen(COMPUTATION_URL).read()
if param is None:
cls.solveComputationSignal.emit(result)
else:
cls.solveAlternateComputation(platform, computation)
while not self.text:
time.sleep(3)
return self.text if self.text else False
#classmethod
def updateCurrentStatus(cls, platform, statusText):
cls.updateStatusSignal.emit(platform, statusText)
I think these methods in themselves are fine. The two signals defined here are connected to in the GUI thread. The first signal pops-up a dialog in which the computation is presented. The GUI thread calls the setResultText() method and sets the resulting string as entered by the user (if anyone knows of a better way to wait until the user has inputted the text other than sleeping and waiting for self.text to become True, please let me know). The solveAlternateComputation is another method in the same class that solves the computation automatically, however, it too calls the setResultText() method that sets the resulting text.
The second signal updates the statusBar text of the main GUI as well.
What's worse is that I think the above design, while perhaps flawed, is not the problem.
The problem lies, I believe, in the way I call these methods, whihch is from the worker threads (note that I have multiple similar workers, all of which are different "platforms")
Assume I have this (and I do):
class WorkerPlatform1(QThread):
#Init and other methods are here
def run(self):
#Thread does its job here, but then when it needs to present the
#computation, instead of emitting a signal, this is what I do
self.f = functions.Functions
result = self.f.solveComputation(platform, computation)
if result:
#Go on with the task
else:
self.f.updateCurrentStatus(platform, "Error grabbing computation!")
In this case I think that my flaw is that the thread itself is not emitting any signals, but rather calling callables residing outside of that thread directly. Am I right in thinking that this could cause my application to crash? Although the faulty module is reported as QtGui4.dll
One more thing: both of these methods in the Functions class are accessed by many threads almost simultaneously. Is this even advisable - have methods residing outside of a thread be accessed by many threads all at the same time? Can it so happen that I "confuse" my program? The reason I am asking is because people who say that the application is not crashing report that, very often, the solveComputation() returns the incorrect text - not all the time, but very often. Since that COMPUTATION_URL's server can take some time to respond (even 10+ seconds), is it possible that, once a thread calls that method, while the urllib library is still waiting for server response, in that time another thread can call it, causing it to use a different COMPUTATION_URL, which will result in it returning an incorrect value on some cases?
Finally, I am thinking of solutions: for my first (crashing) problem, do you think the proper solution would be to directly emit a Signal from the thread itself, and then connect it in the GUI thread? Is that the right way to go about it?
Secondly, for the solveComputation returning incorrect values, would I solve it by moving that method (and accompanying methods) to every Worker class? then I could call them directly and hopefully have the correct response - or, dozens of different responses (since I have that many threads) - for every thread?
Thank you all and I apologize for the wall of text.
EDIT: I would like to add that when running in console with some users, this error appears QObject: Cannot create children for a parent that is in a different thread.
(Parent is QLabel(0x4795500), parent's thread is QThread(0x2d3fd90), current thread is WordpressCreator(0x49f0548)
Your design is flawed if you really are using your Functions class like this with classmethods storing results on class attributes, being shared amongst multiple workers. It should be using all instance methods, and each thread should be using an instance of this class:
class Functions(QObject):
solveComputationSignal = pyqtSignal(str)
updateStatusSignal = pyqtSignal(int, str)
def __init__(self, parent=None):
super(Functions, self).__init__(parent)
self.text = ""
def setResultText(self, text):
self.text = text
def solveComputation(self, platform, computation, param=None):
result = urllib.urlopen(COMPUTATION_URL).read()
if param is None:
self.solveComputationSignal.emit(result)
else:
self.solveAlternateComputation(platform, computation)
while not self.text:
time.sleep(3)
return self.text if self.text else False
def updateCurrentStatus(self, platform, statusText):
self.updateStatusSignal.emit(platform, statusText)
# worker_A
def run(self):
...
f = Functions()
# worker_B
def run(self):
...
f = Functions()
Also, for doing your urlopen, instead of doing sleeps to check for when it is ready, you can make use of the QNetworkAccessManager to make your requests and use signals to be notified when results are ready.
I need to communicate update events to all running instances of my python script, and i would like to keep the code as simple as possible. I have zero experience with communicating between running processes. Up until now, i have been reading/writing configuration files, which each instance will read and/or update.
Here is some pseudo code i have written (sort of a simple template) to wrap my head around how to solve this problem. Please try to help me fill in the blanks. And remember, i have no experience with sockets, threads, etc...
import process # imaginary module
class AppA():
def __init__(self):
# Every instance that opens will need to attach
# itself to the "Instance Manager". If no manager
# exists, then we need to spawn it. Only one manager
# will ever exist no matter how many instances are
# running.
try:
hm = process.get_handle(AppA_InstanceManager)
except NoSuchProgError:
hm.spawn_instance(AppA_InstanceManager)
finally:
hm.register(self)
self.instance_manager = hm
def state_update(self):
# This method won't exist in the real code, however,
# it emulates internal state changes for the sake of
# explaination.
#
# When any internal state changes happen, we will then
# propagate the changes outward by calling the
# appropriate method of "self.instance_manager".
self.instance_manager.propagate_state()
def cb_state_update(self):
# Called from the "Instance Manager" only!
#
# This may be as simple as reading a known
# config file. Or could simply pass data
# to this method.
class AppA_InstanceManager():
def __init__(self):
self.instances = []
def register_instance(self, instance):
self.instances.append(instance)
def unregister_instance(self, instance):
# nieve example for now.
self.instances.remove(instance)
def propagate_state(self):
for instance in self.instances:
instance.cb_state_update(data)
if __name__ == '__main__':
app = AppA()
Any Suggestions?
There are a few options for this kind of design.
You could use a message queue, its made for this kind of stuff, e.g. AMQP or some ZeroMQ or something like it.
Or you could use something like Redis or some other (in-memory) database for synchronization.
If you don't want to use something like that, you could use the multiprocessing modules synchronization stuff.
Or use a platform specific IPC system, e.g. shared memory via mmap, sysv sockets, etc.
If you want to do things the way you explained, you could have a look at Twisteds perspective broker.