I'm trying to implement multiprocessing to a program I made for my job, and I'm running into issues. Please see the simplified code below; when I launch a process, it launches another instance of the gui; when I close that instance, the cpu_intensive function is processed. It's my first time using Tkinter and threads and processes so there might be more than one problem in there ;-) Thanks for looking into this:
from Tkinter import *
from threading import Thread
from multiprocessing import Process, Queue
source_queue = Queue()
def thread_launch():
"""launches a thread to keep the gui responsive"""
thread1 = Thread(target=process_engine)
return thread1.start()
def process_engine():
"""spawns processes and populates the queue"""
p1 = Process(target=cpu_intensive, args=(source_queue,))
p1.start()
p2 = Process(target=cpu_intensive, args=(source_queue,))
p2.start()
for i in range(10):
source_queue.put(i)
print "Added item", i, "to queue"
source_queue.close()
source_queue.join_thread()
p1.join()
p2.join()
def cpu_intensive(item_toprocess):
"""function to be multiprocessed"""
print "Processed: item", item_toprocess.get()
class Application:
def __init__(self, master):
"""defines the gui"""
self.master = master
self.process_button = Button(self.master,
text="Process",
command=thread_launch)
self.process_button.pack(padx=100, pady=100)
def __main__():
"""launches the program"""
root = Tk()
app = Application(root)
root.mainloop()
if __name__ == __main__():
__main__()
Try:
if __name__ == "__main__":
instead. The previous version was calling main but checking the returned value (None in this case). The problem was (I suspect you are on Windows) that the child process was also calling main in the if statement.
Related
I am trying to run a script which runs Asynchronously using threadings. I have run into an issue on how to check periodically if a thread is still alive (under start_thread1). I don't want to use join() as that will freeze the GUI until the threads is finished.
If that is not possible, I am open-minded with any other ways of doing it.
Here is the code I am using - this is a part of part of the code just to outline the "issue" that I have:
from tkinter.constants import LEFT, RIGHT, S
import tkinter.messagebox
from matplotlib import pyplot as plt
import tkinter as tk, time, threading, random, queue
class GuiPart(object):
def __init__(self, master, queue, queue2, client_instance):
self.queue = queue
self.queue2 = queue2
self.x = []
self.y= []
# Set up the GUI
self.Button2 = tk.Button(master, text="Button2", padx=10,
pady=5, fg="white", bg="#263D42", command=client_instance.start_thread1)
self.Button2.pack(side = RIGHT)
def processIncoming(self):
""" Handle all messages currently in the queue, if any. """
while not self.queue.empty():
msg = self.queue.get_nowait()
self.x.append(msg)
print(msg)
while not self.queue2.empty():
msg2 = self.queue2.get_nowait()
self.y.append(msg2)
fig, ax = plt.subplots()
ax.plot(self.x, self.y)
plt.show()
class ThreadedClient(object):
"""
Launch the main part of the GUI and the worker thread. periodic_call()
and end_application() could reside in the GUI part, but putting them
here means that you have all the thread controls in a single place.
"""
def __init__(self, master):
"""
Start the GUI and the asynchronous threads. We are in the main
(original) thread of the application, which will later be used by
the GUI as well. We spawn a new thread for the worker (I/O).
"""
self.master = master
# Create the queue
self.queue = queue.Queue()
self.queue2 = queue.Queue()
self.running=True
# Set up the GUI part
self.gui = GuiPart(master, self.queue, self.queue2, self)
# Set up the thread to do asynchronous I/O
# More threads can also be created and used, if necessary
def start_thread1(self):
thread1=threading.Thread(target=self.worker_thread1)
thread1.start()
# how to check periodically if the thread is finished and when is finished run self.gui.processIncoming()
# if I run it straight away like this the self.gui.processIncoming() will run before the thread will finish and nothing will be plotted
if thread1.is_alive() == False:
self.gui.processIncoming()
def worker_thread1(self):
"""
This is where we handle the asynchronous I/O. For example, it may be
a 'select()'. One important thing to remember is that the thread has
to yield control pretty regularly, be it by select or otherwise.
"""
if self.running:
time.sleep(5) # I am using time.sleep(5) just to simulate a long-running process
for i in range(20):
msg = i
self.queue.put(msg)
for j in range(20):
msg2 = j
self.queue2.put(msg2)
root = tk.Tk()
root.title('Matplotlib threading')
client = ThreadedClient(root)
root.mainloop()
If I understand correctly, you have a tkinter program with a second thread that does some data collection, and you need to get data from that second thread back into the gui. You can't simply wait for the second thread to finish because that would block the tkinter main loop.
One solution is to create a callback function, pass it to the second thread, and call it as the very last step in the second thread. Most tkinter objects aren't threadsafe, so if you're going to update the GUI in the callback function, you have to run the callback in the main thread. To do this, base the callback on tkinter's after_idle function. This causes the callback to occur in tk's event loop, in the main thread, much like a tkinter event handler.
This program does that, and is similar to your program. I changed a few minor things to make my static type checker (pylint) happy. I don't use matplotlib so I took that code out.
The important stuff is in start_thread1. The function f is declared and passed as an argument to the thread. Note that f doesn't call processIncoming, but passes it to after_idle; that instructs the tk main loop to perform the actual call. The function that got passed to worker_thread1 is called as the last step in the thread.
The end result is that processIncoming() is fired into the main thread when the worker thread finishes.
from tkinter.constants import RIGHT
import tkinter as tk
import time
import threading
import queue
class GuiPart:
def __init__(self, master, queue1, queue2, client_instance):
self.queue1 = queue1
self.queue2 = queue2
self.x = []
self.y= []
# Set up the GUI
self.Button2 = tk.Button(master, text="Button2", padx=10,
pady=5, fg="white", bg="#263D42",
command=client_instance.start_thread1)
self.Button2.pack(side = RIGHT)
def processIncoming(self):
""" Handle all messages currently in the queue, if any. """
print(threading.current_thread())
while not self.queue1.empty():
msg = self.queue1.get_nowait()
self.x.append(msg)
print("X", msg)
while not self.queue2.empty():
msg2 = self.queue2.get_nowait()
self.y.append(msg2)
print("Y", msg2)
print("Make a plot now")
class ThreadedClient:
"""
Launch the main part of the GUI and the worker thread. periodic_call()
and end_application() could reside in the GUI part, but putting them
here means that you have all the thread controls in a single place.
"""
def __init__(self, master):
"""
Start the GUI and the asynchronous threads. We are in the main
(original) thread of the application, which will later be used by
the GUI as well. We spawn a new thread for the worker (I/O).
"""
self.master = master
# Create the queue
self.queue1 = queue.Queue()
self.queue2 = queue.Queue()
self.running=True
# Set up the GUI part
self.gui = GuiPart(master, self.queue1, self.queue2, self)
# Set up the thread to do asynchronous I/O
# More threads can also be created and used, if necessary
def start_thread1(self):
def f():
self.master.after_idle(self.gui.processIncoming)
thread1=threading.Thread(target=self.worker_thread1, args=(f, ))
thread1.start()
def worker_thread1(self, callback):
"""
This is where we handle the asynchronous I/O. For example, it may be
a 'select()'. One important thing to remember is that the thread has
to yield control pretty regularly, be it by select or otherwise.
"""
print(threading.current_thread())
if self.running:
time.sleep(1) # simulate a long-running process
for i in range(20):
msg = i
self.queue1.put(msg)
for j in range(20):
msg2 = j
self.queue2.put(msg2)
callback()
root = tk.Tk()
root.title('Matplotlib threading')
client = ThreadedClient(root)
root.mainloop()
so I need to implement a following scenario:
- Several tasks are running simultaneously as processes.
- Each task should display a progress bar with a "Cancel" button, clicking on which should terminate it.
To achieve responsive GUI, I run the task for each process in a separate thread, and it seems that I need to create a separate wx.App for each process as well, otherwise the thread seems to be not running. This setup works fine, however:
a) I am not sure whether multiple wx.App's is a good idea or
b) If there is a better way of achieving my goal.
MWE below (note: in this sample code I could have used Update method of wx.ProgressDialog to identify whether the "Cancel" button has been pressed, but can't do so for my real application).
import wx, multiprocessing, time, psutil
from multiprocessing import Queue
from threading import Thread
from wx.lib.pubsub import pub as Publisher
#runs the task
def task_runner(q):
pid = multiprocessing.current_process().pid
q.put(pid)
while True:
print("Process Running")
time.sleep(1)
wx.CallAfter(Publisher.sendMessage, "update") #call to update the bar
class TestPanel():
def __init__(self,name):
self.q = Queue()
self.count=0
max = 80
# dialog to see progress and cancel the task
self.dlg = wx.GenericProgressDialog(name,
"An informative message",
maximum = max,
parent=None,
style = wx.PD_CAN_ABORT
| wx.PD_APP_MODAL
| wx.PD_ELAPSED_TIME
)
#set listener to dialog's "Cancel" button
for child in self.dlg.GetChildren():
if isinstance(child, wx.Button):
cancel_function = lambda evt, parent=self.dlg: self.onClose(evt, parent)
child.Bind(wx.EVT_BUTTON, cancel_function)
#subscribe to update the progress bar from the thread
Publisher.subscribe(self.updateProgress, "update")
# start thread which runs some task
p = Thread(target=task_runner, args=(self.q,))
p.start()
#updates the progress bar
def updateProgress(self):
print("updating progress")
self.count=self.count+10
self.dlg.Update(self.count)
#kills the process
def kill(self, proc_pid):
process = psutil.Process(proc_pid)
for proc in process.children(recursive=True):
proc.kill()
process.kill()
#closing the dialog event
def onClose(self, event, dialog):
""""""
print "Closing dialog!"
pid = self.q.get()
self.kill(pid)
dialog.Destroy()
# run process, each process creates its own wx.App
def runProcess(name):
app = wx.App(False)
TestPanel(name)
app.MainLoop()
# worker class to use for multiprocessing pool
class Worker():
def __call__(self, name):
return runProcess(name)
if __name__ == '__main__':
items=['Bar1', 'Bar2']
pool = multiprocessing.Pool(processes=2)
result = pool.map(Worker(), items) #create two processes
pool.close()
No, having more than one wx.App in a single process is not a good idea. Even creating a new one after the prior is finished can sometimes be problematic.
However since you are using multiprocess it is not quite the same. Unless I'm missing something, each OS process does have just one wx.App in your case, and since the parent process is not creating a wx.App then they are not trying to inherit that one (which would likely cause even more problems.)
I am trying to create a basic console window using tkinter. My current code has two classes, one for the gui and one for adding text to the console. The text adding class is running in a different thread than the gui (which is running in the main thread). I have a queue set up to communicate between the two threads. Everything works except for the queue. When I add text to the queue, it can be read from the worker thread but not the main thread. Here is my code:
import threading, sys, os, time, Queue
from Tkinter import *
class consolegui:
"""Main GUI for the program"""
def __init__(self, root):
self.root=root
self.cout=Text(self.root, width=80)
self.root.after(100, self.process_queue)
self.cout.config(state=DISABLED)
self.cout.pack(side=TOP)
self.bottomf=Frame(self.root, height=1, width=80)
self.bottomf.pack(side=BOTTOM)
self.cin=Entry(self.bottomf, width=100)
self.cin.pack(side=LEFT)
self.enter=Button(self.bottomf, text="send", command=self.on_click)
self.cin.bind('<Return>', self.on_click)
self.enter.pack(side=BOTTOM)
self.queue = Queue.Queue()
worker(self.queue).start()
def on_click(self, *args):
self.cout.config(state=NORMAL)
self.cout.insert(END, self.cin.get()+'\n')
self.cout.config(state=DISABLED)
self.cin.delete(0, END)
def add_text(self, text):
self.cout.insert(END, text+'\n')
def process_queue(self):
print "reading queue"
try:
msg = self.queue.get(0)
print "found items in queue!"
self.add_text(msg)
with self.queue.mutex:
self.queue.queue.clear()
except Queue.Empty:
print "found nothing"
self.root.after(100, self.process_queue)
class worker(threading.Thread):
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
def run(self):
time.sleep(5)
print "adding to queue"
self.queue.put("Task finished")
print self.queue.get(0)
if __name__ == "__main__":
root = Tk()
console=consolegui(root)
root.mainloop()
Thanks for any help!
Are you sure it's not readable from the main thread? You're using a non-blocking get, and sleeping between checks. The worker only adds a single item to the queue, and reads it back immediately (which empties the queue). The race condition you've created would require the Python GIL to swap to the main thread exactly between the put and the get at the exact 100 second mark between checks (and as written, it looks like you're adding after five seconds so it's likely got ~95 seconds to go, and that race will never happen).
Short answer: You can only get an entry from a Queue once. If the worker reads it immediately, the main thread can't (and vice versa).
I am new to python and trying to implement multiprocess with Tkinter. I am having a main GUI process, with two other "test" processes. This code works fine in Windows, and the main window is displayed and the other two processes are also running. However when I run this code in Ubuntu, it does not work, the two test processes are running but the main GUI window is not displayed.
Can anyone help me on this?
from Tkinter import *
from multiprocessing import Process
import time
def mywind():
root=Tk()
root.title = "MyWindow"
frame=Frame(root)
root.mainloop()
def test1():
while True:
print "In test1"
time.sleep(1)
def test2():
while True:
print "In test2"
time.sleep(1)
if __name__ == '__main__':
p1 = Process(target=test1)
p1.start()
p2 = Process(target=test2)
p2.start()
p = Process(target=mywind)
p.start()
while True:
time.sleep(1)
Try this:
from Tkinter import *
from multiprocessing import Process
import time
root = None
def mywind():
root=Tk()
root.title = "MyWindow"
frame=Frame(root)
return root
def test1():
while True:
print "In test1"
time.sleep(1)
def test2():
while True:
print "In test2"
time.sleep(1)
if __name__ == '__main__':
p1 = Process(target=test1)
p1.start()
p2 = Process(target=test2)
p2.start()
root = mywind()
root.mainloop()
I can't perfectly explain why putting the main loop into the main process rather than a subprocess works. I assume it has something to do with the way Tk resources (and underlying native windowing resources) are managed by the Tkinter library, and collisions with the idea of running them in a separate process.
I am trying to build an GUI application with wxPy that perform some long scripted action. I have put the GUI and the script in different threads to prevent blocking. this worked well except after I close the GUI, the thread containing the script remain running.
below is a simplified version of my program(sorry if the code is hard to understand or missing parts, I am terrible at shortening programs)
class Form(wx.Frame):
...
def test(self, evt):
t2 = threading.Thread(target = self.runTest)
t2.start()
def runTest(self):
result = really_long_script()
def main():
app = wx.App(False)
form = form(app)
app.MainLoop()
t1 = threading.Thread(target = main)
t1.start()
Is there any way I can just kill the thread? right now the script still runs in background when I close the window.
Any help would be greatly appreciated!
Thanks,
John
If you set the thread to be a daemon thread, it will die with the main thread.
You can do this by adding the line t2.daemon = True before you call start
Edit:
Check this example, with the t.daemon = True line the thread dies when you close the frame, if you comment out that t.daemon = True line, the thread stays alive after the frame closes
import wx
import time
from threading import Thread
def print_something_forever(something):
while True:
print something
time.sleep(1)
class Frame(wx.Frame):
def __init__(self,parent):
wx.Frame.__init__(self,parent)
self.panel= wx.Panel(self)
t= Thread(target=print_something_forever,args=("Thread alive!",))
t.daemon= True
t.start()
self.Show()
if __name__ == "__main__":
app= wx.App(False)
Frame(None)
app.MainLoop()
Python doesn't support killing/destroying threads, probably because memory leaks, resources loss, etc.
try this "Threadizing class" :D
class Run_Other_Thread(threading.Thread):
"Raises a child thread \
I'm busy dying, rather lying - _P0W !"
def __init__(self,func_name,*args): #Constructor
self._func=func_name
self._func_args=args
threading.Thread.__init__(self)
def run(self): # Start Dying
try:
print("\n** Running New Thread :"+self._func.func_name)
except:
print("\n** Running New Thread :"+self._func.__name__)
self._func(*self._func_args)
def stop(self):
print('!! Stopped')
def __del__(self):#Constructor
try:
print('\n ## Farewell :'+self._func.func_name)
except:
print('\n ## Farewell :'+self._func.__name__)
You may run GUI as:(and try closing)
def GUI():
app= wx.App(False)
Frame(None)
app.MainLoop()
if __name__ == '__main__':
"Tkinter GUI in Thread helps in De-bugging"
Run_Other_Thread(GUI).start() # Release command window control