I am seeking help with understanding why multithreading does not work correctly in this example:
import time
import matplotlib.pyplot as plt
import concurrent.futures
voltage = list()
stop_flag = False
fig, ax = plt.subplots()
fig.show()
def plotter(ax, arr):
while True:
if stop_flag:
print("stream interupted")
return
arr.append(1)
ax.clear()
ax.plot(arr)
ax.get_figure().canvas.draw()
ax.get_figure().canvas.flush_events()
time.sleep(1)
with concurrent.futures.ThreadPoolExecutor() as executor:
plot_thread = executor.submit(plotter, ax, voltage)
for _ in range(15):
res = input("Type: ")
if res == "q":
stop_flag = True
break
print(res)
Expected behaviour: Graph is being spawned by matplotlib and the line at y=1 is continuously growing in the x direction. In parallel terminal waits for user input. If input is q, the function plotter exits and everything finishes.
Actual behaviour: Graph is appearing, but not updating at all, while user input is being processed and reprinted to console. It seems that one point does exist on the graph, as it is being rescaled. When pressing q or reaching 15 iterations program terminates.
I do not understand why parallel thread does not run and updates the graph independently from for loop. I did manage to make it run with simpler functions on different threads, that just use print and sleep. But when it comes to matplotlib it does not work.
Edit:
Changing plotting to the main program and running input loop in another thread also does not work:
import time
import matplotlib.pyplot as plt
import concurrent.futures
import matplotlib
matplotlib.use("tkAgg") # lock tk backend
voltage = list()
stop_flag = False
fig, ax = plt.subplots()
fig.show()
def fetch_input():
global stop_flag
for _ in range(15):
res = input("Type: ")
if res == "q":
stop_flag = True
break
print(res)
with concurrent.futures.ThreadPoolExecutor() as executor:
new_thread = executor.submit(fetch_input)
while True:
if stop_flag:
print("main loop interupted")
break
voltage.append(1)
ax.clear()
ax.plot(voltage)
ax.get_figure().canvas.draw()
ax.get_figure().canvas.flush_events()
plt.pause(1) # run eventloop for 1 second
# time.sleep(1)
First the function fetching the input starts and continues until it finished and only then the figure appears. It is even weirder now, since fig.show() happens in the beginning before the thread is even started. So I would expect to see at least empty figure together with input query in the terminal.
this isn't really because of matplotlib itself, but due to the backends matplotlib uses to draw the GUI, most backends are single-threaded, starting the figure on the other thread may cause the gui eventloop to run on the other thread (depending on the backend), which allows it to work independent from the main thread.
import matplotlib.pyplot as plt
import concurrent.futures
import matplotlib
matplotlib.use("tkAgg") # lock tk backend
voltage = list()
stop_flag = False
def plotter(arr):
fig, ax = plt.subplots()
fig.show()
while True:
# prevent tcl interpreter from crashing
if stop_flag or not plt.fignum_exists(fig.number):
print("stream interupted")
return
arr.append(1)
ax.clear()
ax.plot(arr)
ax.get_figure().canvas.draw()
ax.get_figure().canvas.flush_events()
plt.pause(1) # run eventloop for 1 second
with concurrent.futures.ThreadPoolExecutor() as executor:
plot_thread = executor.submit(plotter, voltage)
for _ in range(15):
res = input("Type: ")
if res == "q":
stop_flag = True
break
print(res)
note that matplotlib will still warn you that what you are trying to do may fail (which again depends on the backend).
Related
I am trying to display a pyplot that updates every 3 seconds with data from another thread. I need to close the plot and reopen with the updated variables. I do not need an animation, I want to close the figure and open a new one.
This does not work unless I manually close the Figure window.
import numpy as np
from matplotlib import pyplot as plt
import threading as th
from time import sleep
Xplot = []
YPlot = []
def RandomXYGen():
x = np.random.normal(5, .75)
y = np.random.normal(10, 1.5)
return x, y
def LoopData():
while True:
xdata, ydata = RandomXYGen()
Xplot.append(xdata)
YPlot.append(ydata)
print(xdata, ydata)
sleep(0.1)
def Plotter(xToPlot, yToPlot):
plt.close('all')
plt.plot(xToPlot, yToPlot)
plt.show()
t = th.Thread(target=LoopData, daemon=True)
t.start()
while True:
Plotter(Xplot, YPlot)
sleep(3)
Any ideas why it is not closing?
[python 3.10.8 on Windows 10]
After many attempts on figuring it out I was finally able to get help from the Python Discord server. This is what solves the above problem:
plt.show(block=False)
plt.pause(0.01)
instead of:
plt.show()
Leaving this here in case someone runs into the same issue as me.
I have some program which updates global Dataframe every few second. I want to plot live graph with that dataframe.
Dataframe updated on threading. On internet I found, to plot graph with updated dataframe we need multiprocessing.Process with Queue.
How to keep plot open with responsive window and update if dataframe changed?
My demo Code:
import threading
import time
from multiprocessing import Process,Queue
import pandas as pd
from matplotlib import pyplot as plt
print("Starting")
global df
def live_df():
global df
df = pd.DataFrame({"demo": range(10)})
for n in range(10, 20):
df = df.append([{"demo": n}], ignore_index=True)
q.put(df)
time.sleep(5)
print("DataFrame Updated:", n)
def live_graph(qq):
plt.ion()
fig = plt.figure()
ax = fig.add_subplot(111)
for a in range(30):
data = qq.get()
print("inside Graph", a)
ax.plot(data)
plt.title(a)
fig.canvas.draw()
fig.canvas.flush_events()
if __name__ == '__main__':
q = Queue()
threading.Thread(target=live_df, daemon=True).start()
p = Process(target=live_graph, args=(q,))
p.start()
p.join()
Is this efficient approch or I need some changes? I tried FuncAnimation but with no success.
Graph plot is not responding on time.sleep inside demo_df function, Why?
My OS is Windows.
I want to plot data in matplotlib in real time. I want to open a figure once at the start of the programme, then update the figure when new data is acquired. Despite there being a few similar questions out there, none quite answer my specific question.
I want each set of data points new_data1 and new_data2 to be plotted on the same figure at the end of each while loop i.e. one line after the first while loop, two lines on the same figure after the second while loop etc. Currently they are all plotted together, but only right at the end of the programme, which is no use for real time data acquisition.
import matplotlib.pyplot as plt
import numpy
hl, = plt.plot([], [])
def update_line(hl, new_datax, new_datay):
hl.set_xdata(numpy.append(hl.get_xdata(), new_datax))
hl.set_ydata(numpy.append(hl.get_ydata(), new_datay))
plt.xlim(0, 50)
plt.ylim(0,200)
plt.draw()
x = 1
while x < 5:
new_data1 = []
new_data2 = []
for i in range(500):
new_data1.append(i * x)
new_data2.append(i ** 2 * x)
update_line(hl, new_data1, new_data2)
x += 1
else:
print("DONE")
This programme plots all 5 lines, but at the end of the programme. I want each line to be plotted after one another, after the while loop is completed. I have tried putting in plt.pause(0.001) in the function, but it has not worked.
This programme is different from the one that has been put forward - that programme only plots one graph and does not update with time.
If I correctly understood your specifications, you can modify just a bit your MWE as follows:
import matplotlib.pyplot as plt
import numpy
fig = plt.figure(figsize=(11.69,8.27))
ax = fig.gca()
ax.set_xlim(0, 50)
ax.set_ylim(0,200)
hl, = plt.plot([], [])
def update_line(hl, new_datax, new_datay):
# re initialize line object each time if your real xdata is not contiguous else comment next line
hl, = plt.plot([], [])
hl.set_xdata(numpy.append(hl.get_xdata(), new_datax))
hl.set_ydata(numpy.append(hl.get_ydata(), new_datay))
fig.canvas.draw_idle()
fig.canvas.flush_events()
x = 1
while x < 10:
new_data1 = []
new_data2 = []
for i in range(500):
new_data1.append(i * x)
new_data2.append(i ** 2 * x)
update_line(hl, new_data1, new_data2)
# adjust pause duration here
plt.pause(0.5)
x += 1
else:
print("DONE")
which displays :
Not sure, if I am reading the requirements right but below is a blueprint. Please change it to suit your requirements. You may want to change the function Redraw_Function and edit the frames (keyword parameter, which is np.arange(1,5,1) ) in the FuncAnimation call. Also interval=1000 means 1000 milliseconds of delay.
If you are using Jupyter then comment out the second last line (where it says plt.show()) and uncomment the last line. This will defeat your purpose of real time update but I am sorry I had trouble making it work real time in Jupyter. However if you are using python console or official IDLE please run the code as it is. It should work nicely.
import numpy as np
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
fig, ax = plt.subplots()
plot, = plt.plot([],[])
def init_function():
ax.set_xlim(0,50)
ax.set_ylim(0,250)
return plot,
def Redraw_Function(UpdatedVal):
new_x = np.arange(500)*UpdatedVal
new_y = np.arange(500)**2*UpdatedVal
plot.set_data(new_x,new_y)
return plot,
Animated_Figure = FuncAnimation(fig,Redraw_Function,init_func=init_function,frames=np.arange(1,5,1),interval=1000)
plt.show()
# Animated_Figure.save('MyAnimated.gif',writer='imagemagick')
When you run the code, you obtain the below result. I tried to keep very little code but I am sorry, if your requirement was totally different.
Best Wishes,
I've started to learn about matplotlib functions because i wanted to visualize data i was receiving via websocket. For that i made a dummy program that mimics the behaviour of my main program but has added the functionality of mathplotlib. what i noticed is the program takes more and more time to finish each loop and eventually 'freezes'. i managed to extend it life by changing the interval in animation.FuncAnimation from 1000 to 10000. But that just the program to plot sometimes up to 9s for 1 new peace of data. I believe the problem lays in a inappropriate way of cleaning old plots. But i don't know where exactly i did the mistake
import time
import datetime
import timeit
import queue
import os
import random
import copy
import matplotlib.pyplot as plt
import matplotlib.animation as animation
q = queue.Queue()
beta=[0,]
b=False
czas=[]
produkty=["primo"]
cena=[[] for _ in range(len(produkty))]
fig=plt.figure()
#ax1=fig.add_subplot(1,1,1)
#ax2=fig.add_subplot(1,1,1)
ax1=plt.subplot(1,1,1)
ax2=plt.subplot(1,1,1)
def animate(i):
ax1.clear()
ax2.clear()
ax1.plot(czas,cena[0])
ax2.plot(czas,beta)
while True:
time.sleep(1)
alpfa=time.time()
#input('press enter')
rand_produkt=random.choice(produkty)
rand_price=random.randint(1,10)
rand_czas=time.ctime()
alfa={'type':'ticker','price':rand_price,'product_id':rand_produkt,'time':rand_czas}
q.put(alfa)
if q.not_empty:
dane=q.get()
typ=dane.get('type',None)
if typ=='ticker':
price=dane.get('price', None)
pair=dane.get('product_id',None)
t=dane.get('time', None)
b=True
if b==True:
b=False
produkt_id=produkty.index(pair)
cena[produkt_id].append(float(price))
czas.append(t)
plt.ion()
ani=animation.FuncAnimation(fig,animate,interval=1000)#, blit=True)repeat=True)
plt.show()
plt.pause(0.001)
#fig.clf()
beta.append(time.time()-alpfa)
print(beta[-1])
The problem with your code is that you call a new animation in you while loop. Hence this will cause slow down down the line. It is better to initiate your plot ones. One trick may be to update the object data directly as such:
from matplotlib.pyplot import subplots, pause, show
from numpy import sin, pi
fig, ax = subplots()
x = [0]
y = [sin(2 * pi * x[-1])]
p1, = ax.plot(x, y)
show(block = False)
while True:
# update data
x.append(x[-1] + .1)
y.append(sin(2 * pi * x[-1]))
p1.set_data(x, y) # update data
ax.relim() # rescale axis
ax.autoscale_view()# update view
pause(1e-3)
I have a strange problem, with matplotlib. If I run this program, I'm able to open and close several time the same figure.
import numpy
from pylab import figure, show
X = numpy.random.rand(100, 1000)
xs = numpy.mean(X, axis=1)
ys = numpy.std(X, axis=1)
fig = figure()
ax = fig.add_subplot(111)
ax.set_title('click on point to plot time series')
line, = ax.plot(xs, ys, 'o', picker=5) # 5 points tolerance
def onpick(event):
figi = figure()
ax = figi.add_subplot(111)
ax.plot([1,2,3,4])
figi.show()
fig.canvas.mpl_connect('pick_event', onpick)
show()
On the contrary, if I use the same code of onpick function into my custom widget it opens the figure only the first time, into the other events it enters into the functions but doesn't display the figure:
from PyQt4 import QtGui, QtCore
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.figure import Figure
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.backends.backend_qt4 import NavigationToolbar2QT as NavigationToolbar
import time
STEP = 0.000152
class MplCanvas(FigureCanvas):
def __init__(self):
# initialization of the canvas
FigureCanvas.__init__(self, Figure())
self.queue = []
self.I_data = np.array([])
self.T_data = np.array([])
self.LvsT = self.figure.add_subplot(111)
self.LvsT.set_xlabel('Time, s')
self.LvsT.set_ylabel('PMT Voltage, V')
self.LvsT.set_title("Light vs Time")
self.LvsT.grid(True)
self.old_size = self.LvsT.bbox.width, self.LvsT.bbox.height
self.LvsT_background = self.copy_from_bbox(self.LvsT.bbox)
self.LvsT_plot, = self.LvsT.plot(self.T_data,self.I_data)
#self.LvsT_plot2, = self.LvsT.plot(self.T_data2,self.I_data2)
self.mpl_connect('axes_enter_event', self.enter_axes)
self.mpl_connect('button_press_event', self.onpick)
self.count = 0
self.draw()
def enter_axes(self,event):
print "dentro"
def onpick(self,event):
print "click"
print 'you pressed', event.canvas
a = np.arange(10)
print a
print self.count
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(a)
fig.show()
def Start_Plot(self,q,Vmin,Vmax,ScanRate,Cycles):
self.queue = q
self.LvsT.clear()
self.LvsT.set_xlim(0,abs(Vmin-Vmax)/ScanRate*Cycles)
self.LvsT.set_ylim(-3, 3)
self.LvsT.set_autoscale_on(False)
self.LvsT.clear()
self.draw()
self.T_data = np.array([])
self.I_data = np.array([])
# call the update method (to speed-up visualization)
self.timerEvent(None)
# start timer, trigger event every 1000 millisecs (=1sec)
self.timerLvsT = self.startTimer(3)
def timerEvent(self, evt):
current_size = self.LvsT.bbox.width, self.LvsT.bbox.height
if self.old_size != current_size:
self.old_size = current_size
self.LvsT.clear()
self.LvsT.grid()
self.draw()
self.LvsT_background = self.copy_from_bbox(self.LvsT.bbox)
self.restore_region(self.LvsT_background, bbox=self.LvsT.bbox)
result = self.queue.get()
if result == 'STOP':
self.LvsT.draw_artist(self.LvsT_plot)
self.killTimer(self.timerLvsT)
print "Plot finito LvsT"
else:
# append new data to the datasets
self.T_data = np.append(self.T_data,result[0:len(result)/2])
self.I_data = np.append(self.I_data,result[len(result)/2:len(result)])
self.LvsT_plot.set_data(self.T_data,self.I_data)#L_data
#self.LvsT_plot2.set_data(self.T_data2,self.I_data2)#L_data
self.LvsT.draw_artist(self.LvsT_plot)
self.blit(self.LvsT.bbox)
class LvsT_MplWidget(QtGui.QWidget):
def __init__(self, parent = None):
QtGui.QWidget.__init__(self, parent)
self.canvas = MplCanvas()
self.vbl = QtGui.QVBoxLayout()
self.vbl.addWidget(self.canvas)
self.setLayout(self.vbl)
This widget is needed for an animation plot and when the experiment is finished if I click on the plot it should appear a figure, that appears only the first time.
Do you have any clue?
Thank you very much.
At the start of your code, enable interactive mode via
plt.ion()
I have new information about this that a google search turned up
This is from the writer of matplotlib. This came from http://old.nabble.com/calling-show%28%29-twice-in-a-row-td24276907.html
Hi Ondrej,
I'm not sure where to find a good
explanation of that, but let me give
you some hints. It is intended to use
show only once per program. Namely
'show' should be the last line in your
script. If you want interactive
plotting you may consider interactive
mode (pyplot.ion-ioff) like in the
example below.
Furthermore for dynamic plotting all
animation demos might be useful.
Maybe you want to have also a look at
http://matplotlib.sourceforge.net/users/shell.html
.
best regards Matthias
So it seems it is an undocumented "feature" (bug?).
Edit: here is his code block:
from pylab import *
t = linspace(0.0, pi, 100)
x = cos(t)
y = sin(t)
ion() # turn on interactive mode
figure(0)
subplot(111, autoscale_on=False, xlim=(-1.2, 1.2), ylim=(-.2, 1.2))
point = plot([x[0]], [y[0]], marker='o', mfc='r', ms=3)
for j in arange(len(t)):
# reset x/y-data of point
setp(point[0], data=(x[j], y[j]))
draw() # redraw current figure
ioff() # turn off interactive mode
show()
So maybe by using draw() you can get what you want. I haven't tested this code, I'd like to know its behavior.
I had the same issue with show() only working the first time. Are you still on version 0.99.3 or thereabouts? I was able to resolve my problem recently, if you're still interested in changing the behaviour of show(), try this:
I noticed this paragraph titled multiple calls to show supported on the what's new part of the matplotlib download site.
A long standing request is to support multiple calls to show(). This has been difficult because it is hard to get consistent behavior across operating systems, user interface toolkits and versions. Eric Firing has done a lot of work on rationalizing show across backends, with the desired behavior to make show raise all newly created figures and block execution until they are closed. Repeated calls to show should raise newly created figures since the last call. Eric has done a lot of testing on the user interface toolkits and versions and platforms he has access to, but it is not possible to test them all, so please report problems to the mailing list and bug tracker.
This was 'what's new' for version 1.0.1, at time of writing the version in synaptic was still on 0.99.3. I was able to download and build from source v1.0.1. The additional packages I also required to satisfy dependencies were libfreetype6-dev tk-dev tk8.5-dev tcl8.5-dev python-gtk2-dev; your mileage may vary.
Now that i have matplotlib.__version__ == 1.0.1 , the following code works how I would expect:
from matplotlib import pyplot as p
from scipy import eye
p.imshow(eye(3))
p.show()
print 'a'
p.imshow(eye(6))
p.show()
print 'b'
p.imshow(eye(9))
p.show()
print 'c'
def onpick(self,event):
print "click"
print 'you pressed', event.canvas
...
ax.plot(a)
fig.show() # <--- this blocks the entire loop
Try:
def onpick(self,event):
print "click"
print 'you pressed', event.canvas
...
ax.plot(a)
self.draw()
self.update()
My workaround to this problem is to never call close.
I'm pretty sure you can control the transparency of a widget in PyQt. You might try controlling the visibility using Qt instead of matplotlib. I'm sure someone else who knows more about matplotlib can give a better answer than that though :D
You can create a figure instance by:
fig = plt.figure(0)
And draw your stuff by manipulate this fig.
You can use fig.show() for anytime to show your figure.