Multithreading with matplotlib and wxpython - python

Brief description on what I'm trying to achieve:
I'm working on a analytics software built using Python, wxPython, and matplotlib. I'm trying to implement a function where the program can plot the results after performing some analytical calculations. At the moment, the program freezes when the it's performing the calculations (and the calculation time takes up to 10 seconds depending on the amount of data) so I'm trying to use threading to create a non-blocking program to improve user experience.
Problem I'm getting
I keep getting this error :
(PyAssertionError: C++ assertion "hdcDst && hdcSrc" failed at ...... \src\msw\dc.cpp(2559) in AlphaBlt():AlphaBlt():invalid HDC)
and googling hasn't really help with identifying the cause.
I'll post the full traceback at the bottom of the post.
Here's my code:
import wx
import time
import matplotlib.pyplot as plt
from wx.lib.pubsub import Publisher as pub
from threading import Thread
def plotgraph(x,y,sleeptime):
plt.plot(x,y)
#Simulate long process using time.sleep
time.sleep(sleep time)
#Send out a message once process is completed
pub.sendMessage('PLOT','empty')
class listener():
def __init__(self,name):
self.name = name
#Listens to message
pub.subscribe(self.Plot,'PLOT')
pass
def Plot(self,message):
print self.name
plt.show()
print 'printed'
waiting = listener('Bob')
t1 = Thread(target=plotgraph,args=([1,2,3],[1,2,3],5))
t1.start()
t2 = Thread(target=plotgraph,args=([1,2,3],[1,2,3],3))
t2.start()
Basically, the user will be clicking an icon on the GUI and that will trigger a function to perform some analytical calculation simulated by 'plotgraph()' here. At the moment, without using threads, plotgraph() will block my entire program, so I'm trying to use threads to perform the calculations to free up my GUI.
However when I tried to plot my GUI within the thread, i.e. have plt.show() in plotgraph(), the plot appears then disappears again. When I click the button on the GUI to spawn the thread a second time, I get the same error.
So I've tried to work around it by sending a message after the thread's ended so that the plt.show() will happen outside the thread but I'm still getting the same error.
I can't seem to be able to find a similar error online, except for one thread posted in 2008. If anyone could help that would be awesome!
In a nutshell
I need a way to implement sort of a callback function that allows me to perform the analytic calculation in a thread, then plot the graph once the calculations are completed to free up my GUI. It'd be great if someone could explain to me what's wrong here, or could suggest an alternative method to do it. Thanks very much!!
Here's the full traceback:
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\App\appdata\canopy-1.
.5.3123.win-x86\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\App\appdata\canopy-1.
.5.3123.win-x86\lib\threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "<ipython-input-5-0cb01f87e97a>", line 13, in plotgraph
pub.sendMessage('PLOT','empty')
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\wx\lib\pubsub.py", line 811, in sendMessage
self.__topicTree.sendMessage(aTopic, message, onTopicNeverCreated)
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\wx\lib\pubsub.py", line 498, in sendMessage
deliveryCount += node.sendMessage(message)
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\wx\lib\pubsub.py", line 336, in sendMessage
listener(message)
File "<ipython-input-5-0cb01f87e97a>", line 24, in Plot
plt.show()
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\matplotlib\pyplot.py", line 155, in show
return _show(*args, **kw)
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\matplotlib\backend_bases.py", line 154, in __call__
manager.show()
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\matplotlib\backends\backend_wx.py", line 1414, in show
self.canvas.draw()
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\matplotlib\backends\backend_wxagg.py", line 50, in draw
self.gui_repaint(drawDC=drawDC)
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\matplotlib\backends\backend_wx.py", line 911, in gui_repaint
drawDC.DrawBitmap(self.bitmap, 0, 0)
File "C:\Users\chaishen\AppData\Local\Enthought\Canopy32\User\lib\site-package
\wx\_gdi.py", line 3460, in DrawBitmap
return _gdi_.DC_DrawBitmap(*args, **kwargs)
yAssertionError: C++ assertion "hdcDst && hdcSrc" failed at ..\..\src\msw\dc.cp
(2559) in AlphaBlt(): AlphaBlt(): invalid HDC

I think what you need is the wx.PyEventBinder.
It works like this:
anEVT_CALCULATED = wx.NewEventType()
EVT_CALCULATED = wx.PyEventBinder(anEVT_CALCULATED, 1)
def onCalculate(self, event): # this is your click
calc_thread = CalculatorThread(self, params)
calc_thread.start()
return
def onConnected(self, event):
''' this is where your thread comes back '''
self.doSomeThingLikePlotting(event.resultdata)
class CalcEvent(wx.PyCommandEvent):
''' Event to signal that the thread has calculated'''
def __init__(self, etype, eid, resultdata):
wx.PyCommandEvent.__init__(self, etype, eid)
self.resultdata = resultdata
class CalculatorThread(threading.Thread):
''' This is the thread doing your calculation and handing it back'''
def __init__(self, listener, params):
threading.Thread.__init__(self)
self.listener = listener
self.params = params
def run(self):
resultdata = calculate(params) # this is your calculation
event = CalcEvent(anEVT_CALCULATED, -1, resultdata=resultdata)
wx.PostEvent(self.listener, event)
return
And of course you need to add one line to your __init__
self.Bind(EVT_CONNECTED, self.onCalculated)

Related

Starting n threads with same function at diffrent times

I'm currently working with Tkinter to create a GUI for my Script. Among other things there is a function which writes and save some Data in some Files. To visualize the progress for the User i got an label which shows the progress. When i press the button to execute the function
button_download_data = tk.Button(text="Get Data",command=gatherData)
the window freezes and beside the counter for the progress increasaes it isnt shown due to the window is frozen. My solution was to start the function in a new thread using the threading modul.
button_download_data = tk.Button(text="Get Data",command=threading.Thread(target=gatherData).start)
Now the progress is showed but i cant press the button again because i get an error:
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\WPy64-31050\python-3.10.5.amd64\lib\tkinter\__init__.py", line 1921, in __call__
return self.func(*args)
File "C:\WPy64-31050\python-3.10.5.amd64\lib\threading.py", line 930, in start
raise RuntimeError("threads can only be started once")
RuntimeError: threads can only be started once
I tried to "kill" the thread when the function is done with raise Exception() and sys.Exit() but it doesn't work at all.
I figuerd out that i can outsource the thread start out of the tkinter button line with:
def gatherDate():
do something
def threadStart():
threading.Thread(target=gatherData).start
button_download_data = tk.Button(text="Get Data",command=threadStart)
and i think it might help to start a new thread on button press and not the same again but i cant imagine how.
You should be able to handle this by creating a separate function to spawn new worker threads as needed - here's a very basic example
import tkinter as tk
from threading import Thread
from time import sleep # for example - simulate a long-running process
def get_data():
print('foo') # do whatever you need to do here
sleep(2.0) # simulate the thread 'working' on something...for example
def spawn_thread():
t = Thread(target=get_data, daemon=True)
t.start()
root = tk.Tk()
button_download_data = tk.Button(root, text='Get Data', command=spawn_thread)
button_download_data.pack()
if __name__ == '__main__':
root.mainloop()
You could simplify spawn_thread a little by skipping the variable assignment t=... and doing Thread(target=get_data, daemon=True).start() instead (as long as you don't need access to the Thread object t for anything else)

Screenshot error: XDefaultRootWindow() failed after closing a Tkinter Toplevel --> Python MSS Ubuntu Linux

Ubuntu v22.04
MSS v7.0.1
Python 3.10
This issue is practically identical to: Python: Tkinter + MSS = mss.exception.ScreenShotError: XDefaultRootWindow() failed - however, their question is unsolved. For a minimum reproducible example, see there.
How my project functions
I have a project that involves controlling video game automation scripts/macros via a Python GUI. Using the tool goes something like this:
Run the program, which opens a navigable single-page GUI.
Select the game you'd like to automate.
Select the script you'd like to run.
Configure the options for the script (this involves opening a new Toplevel window in Tkinter).
Press Play to run the script.
System: When the Play button is pressed, my program screenshots the video game client to locate its window position as well as various UI element positions. This occurs in a function called win.initialize().
Optional: The user may pause, resume, or stop the script, or switch to a different script on the fly.
The problem
Please see this video for a demonstration:
The issue I'm experiencing happens between step 4 and 6. Strangely, the first time I open the options menu, select/save my options, and close that little pop-up window, the script will run successfully - i.e., MSS will not encounter an error when screenshotting the game client. If I stop/restart the script (which does not require me to re-select options), it will still work properly. However, if I change the options by opening the pop-up window again, MSS will throw this error when it attempts to screenshot:
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python3.10/tkinter/__init__.py", line 1921, in __call__
return self.func(*args)
File "/home/my_project/env/lib/python3.10/site-packages/customtkinter/widgets/ctk_button.py", line 372, in clicked
self.command()
File "/home/my_project/src/view/info_frame.py", line 139, in play_btn_clicked
self.controller.play_pause()
File "/home/my_project/src/controller/bot_controller.py", line 21, in play_pause
self.model.play_pause()
File "/home/my_project/src/model/bot.py", line 103, in play_pause
if not self.__initialize_window():
File "/home/my_project/src/model/bot.py", line 132, in __initialize_window
self.win.initialize()
File "/home/my_project/src/model/runelite_bot.py", line 47, in initialize
if not super().initialize():
File "/home/my_project/src/utilities/window.py", line 128, in initialize
a = self.__locate_minimap(client_rect)
File "/home/my_project/src/utilities/window.py", line 257, in __locate_minimap
if m := imsearch.search_img_in_rect(imsearch.BOT_IMAGES.joinpath("minimap.png"), client_rect):
File "/home/my_project/src/utilities/imagesearch.py", line 52, in search_img_in_rect
im = rect.screenshot()
File "/home/my_project/src/utilities/geometry.py", line 64, in screenshot
with mss.mss() as sct:
File "/home/my_project/env/lib/python3.10/site-packages/mss/factory.py", line 34, in mss
return linux.MSS(**kwargs)
File "/home/my_project/env/lib/python3.10/site-packages/mss/linux.py", line 297, in __init__
self.root = self.xlib.XDefaultRootWindow(self._get_display(display))
File "/home/my_project/env/lib/python3.10/site-packages/mss/linux.py", line 184, in validate
raise ScreenShotError(f"{func.__name__}() failed", details=details)
mss.exception.ScreenShotError: XDefaultRootWindow() failed
Screenshot error: XDefaultRootWindow() failed, {'retval': <mss.linux.LP_XWindowAttributes object at 0x7f31d8d38ac0>,
'args': (<mss.linux.LP_Display object at 0x7f31d8d38740>,)}
This indicates that the issue has something to do with opening and closing the TkToplevel (pop-up) widget. I cannot explain why this works the first time I select options. From what I can see, the Toplevel widget is being destroyed properly whenever the user closes it or presses the save button.
Does anyone know what might be going on here?
It's a bug in the current MSS. Please report it on this.
The current implementation changes the current X11 error handler and leaves it afterwards, and it causes a conflict with Tcl/Tk(the backend of Python tkinter).(See this for detail.)
To avoid this, you can initialize an MSS instance before calling any tkinter API and reuse it, so not disturbing the tkinter module, like the following example.
import tkinter as tk
import mss
mss_obj = mss.mss()
root = tk.Tk()
...
try:
root.mainloop()
finally:
mss_obj.close()

I am having trouble to create a thread and running it through another definition in python

I've been trying to build an application which takes input as text and gives output as speech.
I referred this site to get to know about Text-To-Speech modules in python:
https://pythonprogramminglanguage.com/text-to-speech/
when i ran the program it did the job perfectly but i couldn't use other functions like pause or resume. So i tried to create a new thread for the speech function so that i can alter it speech whenever i want to.
Here is the program:
import threading
import win32com.client as wincl
speak = wincl.Dispatch("SAPI.SpVoice")
t=threading.Event()
def s():
global t
t.set()
data="""This is a story of two tribal Armenian boys who belonged to the
Garoghlanian tribe. """
s=speak.Speak(data)
t1=threading.Thread(target=s)
t1.start
However i am trying to implement the program in GUI using tkinter.
I want the application to read the text when the user is clicking the button.
Since tkinter's button takes command as a function, i made a function for the initialization and starting of the new thread but it is producing an error which i could not interpret and find a solution.
Here is the program thats making error:
import threading
import win32com.client as wincl
speak = wincl.Dispatch("SAPI.SpVoice")
t=threading.Event()
def s():
global t
t.set()
data="""This is a story of two tribal Armenian boys who belonged to the
Garoghlanian tribe. """
s=speak.Speak(data)
def strt():
t1=threading.Thread(target=s)
t1.start()
Here is the error:
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Application\Python\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Application\Python\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\absan\Desktop\Python\Project-SpeakIt\SI-1.py", line 32, in
speakITheart
s=speak.Speak(data)
File "C:\Users\absan\AppData\Local\Temp\gen_py\3.6\C866CA3A-32F7-11D2-9602-
00C04F8EE628x0x5x4.py", line 2980, in Speak
, 0)
pywintypes.com_error: (-2147352567, 'Exception occurred.', (0, None, None,
None, 0, -2147221008), None)
EDIT:
Guys i somehow found a way to fix it when i was writing this post. I just added these lines to the program
import pyttsx3
engine = pyttsx3.init()
i really don't know how or why it fixed the error but it works!!
So this post might be helpful for someone who is facing the same problem.
Cheers!!
I like the solution from the comments. Just include in non-main thread this line to be able to use COM in it (right before the Speak call)
pythoncom.CoInitialize()
self.engine.Speak(self.msg)

Why matplotlib requires to plot only in the main thread?

I'm trying to plot live the output of a generator.
The following code works as expected (Ctrl-C terminates execution):
import numpy as np
import pylab as p
from Queue import Queue
from threading import Thread
import time
def dataGenerator():
while True:
yield np.random.random()
def populate():
f = dataGenerator()
while True:
x = f.next(); y = f.next()
q.put([x,y])
q = Queue()
p.figure(); p.hold(True); p.show(block=False)
populatorThread = Thread(target=populate)
populatorThread.daemon = True
populatorThread.start()
while True:
data = q.get()
x = data[0]
y = data[1]
p.plot(x,y,'o')
p.draw()
q.task_done()
populatorThread.join()
However, if instead I put the plotting in a thread, I get RuntimeError: main thread is not in main loop:
import numpy as np
import pylab as p
from Queue import Queue
from threading import Thread
import time
def dataGenerator():
while True:
yield np.random.random()
def plotter():
while True:
data = q.get()
x = data[0]
y = data[1]
p.plot(x,y,'o')
p.draw()
print x,y
q.task_done()
q = Queue()
p.figure(); p.hold(True); p.show(block=False)
plotThread = Thread(target=plotter)
plotThread.daemon = True
plotThread.start()
f = dataGenerator()
while True:
x = f.next()
y = f.next()
q.put([x,y])
plotThread.join()
Why does matplotlib care which thread does the plotting?
EDIT: I'm not asking how to solve this but rather why is this happening in the first place.
It's probably the GUI that you're using for backend. The GUI likely expects to find itself in the main thread, but it isn't when matplotlib calls get_current_fig_manager().canvas.draw().
For example, when I do this, I get the following traceback:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "tmp.py", line 18, in plotter
p.draw()
File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 555, in draw
get_current_fig_manager().canvas.draw()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 349, in draw
tkagg.blit(self._tkphoto, self.renderer._renderer, colormode=2)
File "/usr/lib/pymodules/python2.7/matplotlib/backends/tkagg.py", line 13, in blit
tk.call("PyAggImagePhoto", photoimage, id(aggimage), colormode, id(bbox_array))
RuntimeError: main thread is not in main loop
Note the tk.call(...) line. The exception you get is not raised from matplotlib, it's raised from TkInter.
Why does matplotlib care which thread does the plotting?
I'm not asking how to solve this but rather why is this happening in the first place.
#Evert is right, it's not matplotlib, it's your GUI toolkit (one of the backends that matplotlib uses to create a window with a plot for you). It happens because GUI toolkits are event-driven (you don't want blocking behavior for the user interface, right?) and they have internal event loop, that controls program execution. The idea is that events are monitored by the event loop and dispatched to the callbacks. To do this, event loop should be started in the main thread, while callbacks for long-running tasks are moved to separate threads.

Pass a counter from a python thread to a wxpython gage?

I am trying to pass a counter from a python COM thread back into a wxpython tabpanel. Anyone have a solution that would work. Some of my code below:
This class is seperate from my wxPython code but is called out and data sent to this thread.
class Log_COM_thread(Thread):
............
def run(self):
.............
int_log_cnt = int_log_cnt + 1
int_log.TabPanel.gauge.SetValue(int_log_cnt)
if int_log_cnt == 64:
int_log.TabPanel.Int_Log_Status.SetLabel('Extraction Complete')
The seperate module, int_log, this counter value is being sent to is called out like this:
class TabPanel(wx.Panel):
def __init__(self, parent):
self.gauge = wx.Gauge(self, range=72, size=(250, 25))
.....and then more wxpython code.
The error I run into is:
Exception in thread Thread-8:
Traceback (most recent call last):
File "F:\Python27\lib\threading.py", line 551, in __bootstrap_inner
self.run()
File "F:\Documents and Settings\swhite\Desktop\OG GUI Working Jan 13\nbm.py", line 267, in run
int_log.TabPanel.gauge.SetValue(int_log_cnt)
AttributeError: type object 'TabPanel' has no attribute 'gauge'
How would I send my counter back to that gauge in wxpython from my external thread. Any help in direction of how to do this would be extremely helpful.
You cannot call wx methods directly from a separate thread. You need to use one of wxPython thread-safe methods to communicate back to the main GUI thread. They are wx.CallAfter, wx.CallLater and wx.PostEvent.
There is lots of information on threads and wx on the wxPython wiki. I also wrote a tutorial on the subject that might help you.

Categories

Resources