I am plotting iteratively using matplotlib in python. I am setting the axis of the plot, so as to display e.g. only 50 lines at a time. A pseudo code is given below as an example:
x = 0
y = 1
line_plot = 50
axis.set_ylim(0 , line_plot)
while True:
plot(x,y)
y = y+1
if y > line_plot :
axis.set_ylim(y , y+line_plot)
This code will run indefinitely, and eventually the memory required for the plot will get quite large, even if only 50 lines are present on the graph (since all data points are kept in memory). I would like to know if there is a command in python to delete all data that is out of axis limits, freeing some memory space.
Thank you,
Gaelle
This will depend a little bit on how exactly your script looks like. You need some method to determine the y-coordinates of every line, and based on some criteria remove them or not. But if you do something like:
x = np.arange(1)
y = np.ones(1)
pl.figure()
l1 = pl.plot(x,y)[0]
y[:] += 1
l2 = pl.plot(x,y)[0]
and call get_ydata() on both lines, they will have the same y-values, so get_ydata() seems to return the original array, not necessarily the values drawn in the plot (which apparently is a bug, see: this matplotlib issue). If, instead of y[:] += 1 you make an actual copy of the array (y = y.copy()+1), you can use get_ydata(). If this is the case in your real-world problem, such a solution might work:
import matplotlib
import matplotlib.pylab as pl
import numpy as np
pl.close('all')
x = np.arange(100000)
y = np.ones(x.size)
pl.figure()
ax = pl.gca()
line_plot = 50
ax.set_ylim(0, line_plot)
for i in range(200):
pl.plot(x, y)
y = y.copy() + 1
if y[0] > line_plot:
ax.set_ylim(y[0]-line_plot, y[0])
for l in ax.get_lines():
yval = l.get_ydata()[0]
if(yval < ax.get_ylim()[0]):
l.remove()
If I remove the for l in ax.get_lines part, the memory usage scales with i, with this part included the memory usage stays constant, even for very large values of i
You want look at the animation examples
# make a figure and axes object
fig, ax = plt.subplots()
# make a Line2D artist
ln, = ax.plot([], [], linestyle='', marker='o')
# local version of the data
xdata, ydata = [], []
for j in range(200):
# update your copy of the data
xdata.append(j)
ydata.append(j*j)
xdata = xdata[-50:]
ydata = ydata[-50:]
# update the Line2D objects copy of the data
ln.set_data(xdata, ydata)
# autoscale limits to new data
ax.relim()
ax.autoscale()
# needed in non-interactive mode and/or mpl < 1.5
# fig.canvas.draw_idle()
# sleep, but run the GUI event loop
plt.pause(.1)
Related
I am trying to plot real time data. I managed to plot the data but I would like for the bar graph to go up and down on a single x-value rather than produce new x-values for every new datapoint. I believe I have to replace the function x.append(i) with something like a replace, any ideas? Thank you!!
So far this is what I came up with:
import time
import psutil
import matplotlib.pyplot as plt
%matplotlib notebook
fig = plt.figure()
ax = fig.add_subplot(111)
fig.show()
plt.axis('off')
i = 0
x, y = [], []
while True:
x.append(i)
y.append(psutil.cpu_percent())
ax.bar(x, y, color='b')
fig.canvas.draw()
ax.set_xlim(left=max(0, i-50), right=i+50)
time.sleep(0.1)
i += 1
For the bar graph you can create a list inside the while loop, and instantly update it there. First you need to import a random in order get random value for y axis, or you can use cpu_percent.
import psutil
import random
These two should work.
And then:
while True:
x_axis = [str(_) for _ in range(100, 200)]
y_axis = [8 * random.random() for _ in range(100, 200)]
ax.bar(x, y, color='b')
fig.canvas.draw()
time.sleep(0.1)
However, matplotlib is not convenient for real data plotting, I strongly recommend you to use bokeh. You can find bokeh documentation here. It is really cool for creating any kind of real time plot. And at the same time, you can integrate it with your web browser. Hope this will help you)
If you just want to display the latest value, you can consider doing something like:
plt.ion()
graph = plt.bar(["Now"], [0])[0]
plt.axis('off')
i = 0
data = {}
while True:
cpu_percent = psutil.cpu_percent()
graph.set_ydata(cpu_percent)
plt.draw()
plt.pause(0.1)
data[i] = cpu_percent
i += 1
This way, you still have a record of all the datapoints to play with later (x, y) but you will only display 1 x value at a time on the graph.
Further reading
I am new to Matplotlib and that's why there might be a more efficient way to run my program.
It is plotting a bunch of points with different colours (depending on some factors). It is constantly producing new pictures in a loop of the current colour state.
Basically it looks like this:
import matplotlib.pyplot as plt
def getColour():
#calculate some stuff with x and y and the changing factors
while True:
fig = plt.figure(figsize=(17,10))
plt.scatter(x, y , c=getColour())
plt.show()
plt.close(fig)
I was trying out clf() as well. However, it didn't change the pace at all. Does anyone have ideas? What am I doing wrong?
Thank you!
Edit:
The target is to produce a picture each time it goes through the loop. Since my program is doing this quite slowly, my question is whether there is a way to make it run faster.
I am working with python 2.7
Something like an animation:
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
import numpy as np
ms_between_frames = 100
n_points = 100
x = np.arange(n_points, dtype=float) #EDIT
y = np.random.random(n_points)
z = np.random.random(n_points)
def getColour(x, y, z):
c = np.empty((len(x),3))
for i in range(len(x)):
c[i] = [x[i]/n_points, z[i], 1.-z[i]]
return c
def update(frame_number):
global x, y
z = np.random.random(n_points)
c = getColour(x, y, z)
graph.set_color(c)
fig = plt.figure(figsize=(17,10))
ax = fig.add_subplot(111)
graph = ax.scatter(x, y , c=getColour(x, y, z))
animation = FuncAnimation(fig, update, interval=ms_between_frames)
plt.show()
EDIT: made x hold floats so the division inside getColour would not return 0 (could also have made /float(n_points))
By the way, it should be possible to define only one function to update the colours, depending on the arguments you require to do so, to avoid the call overhead.
I am used to work with plots that change over the time in order to show differences when a parameter is changed. Here I provide an easy example
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(111)
ax.grid(True)
x = np.arange(-3, 3, 0.01)
for j in range(1, 15):
y = np.sin(np.pi*x*j) / (np.pi*x*j)
line, = ax.plot(x, y)
plt.draw()
plt.pause(0.5)
line.remove()
You can clearly see that increasing the paramter j the plot becames narrower and narrower.
Now if I want to do the some job with a counter plot than I just have to remove the comma after "line". From my understanding this little modification comes from the fact that the counter plot is not an element of a tuple anymore, but just an attribute as the counter plot completely "fill up" all the space available.
But it looks like there is no way to remove (and plot again) an histogram. Infact if type
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(111)
ax.grid(True)
x = np.random.randn(100)
for j in range(15):
hist, = ax.hist(x, 40)*j
plt.draw()
plt.pause(0.5)
hist.remove()
It doesn't matter whether I type that comma or not, I just get a message of error.
Could you help me with this, please?
ax.hist doesn't return what you think it does.
The returns section of the docstring of hist (access via ax.hist? in an ipython shell) states:
Returns
-------
n : array or list of arrays
The values of the histogram bins. See **normed** and **weights**
for a description of the possible semantics. If input **x** is an
array, then this is an array of length **nbins**. If input is a
sequence arrays ``[data1, data2,..]``, then this is a list of
arrays with the values of the histograms for each of the arrays
in the same order.
bins : array
The edges of the bins. Length nbins + 1 (nbins left edges and right
edge of last bin). Always a single array even when multiple data
sets are passed in.
patches : list or list of lists
Silent list of individual patches used to create the histogram
or list of such list if multiple input datasets.
So you need to unpack your output:
counts, bins, bars = ax.hist(x, 40)*j
_ = [b.remove() for b in bars]
Here the right way to iteratively draw and delete histograms in matplotlib
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure(figsize = (20, 10))
ax = fig.add_subplot(111)
ax.grid(True)
for j in range(1, 15):
x = np.random.randn(100)
count, bins, bars = ax.hist(x, 40)
plt.draw()
plt.pause(1.5)
t = [b.remove() for b in bars]
I am aware of these questions: (A), (B) and (C) - all of which address parts of my problem.
I have also read the Animations Cookbook
My questions, however, seem not to be addressed in any of the above.
I wish to plot objective functions returned by an optimizer while the optimizer is running. I do not know in advance how many iterations the optimizer will run. Independent of how I get the array containing the objective functions, the problem can be isolated in this minimal example:
import numpy as np
import matplotlib.pyplot as plt
SIZE = 50
R1 = 0.5
R2 = 0.75
plt.ion()
fig = plt.figure()
ax = fig.add_subplot(111)
fig.canvas.set_window_title('broken spiral')
for i in range(0, SIZE):
A.append(R1 * i * np.sin(i))
B.append(R2 * i * np.cos(i))
line1, = ax.plot(A,'-k',label='black')
line2, = ax.plot(B,'-r',label='red')
legend = ax.legend(loc=0)
plt.draw()
plt.savefig('test_broken_spiral.png')
Here the plot is only 'pseudo'-updated. What really happens is that for each iteration a new line for A and B is generated, overlying with the original one, but also generating a new legend entry. After 50 iterations, I have 100 lines and 100 legend entries.
I tried this next:
for i in range(0, SIZE):
A.append(R1 * i * np.sin(i))
B.append(R2 * i * np.cos(i))
if i == 0:
line1, = ax.plot(A,'-k',label='black')
line2, = ax.plot(B,'-r',label='red')
legend = ax.legend(loc=0)
plt.draw()
else:
line1.set_ydata(A)
line2.set_ydata(B)
plt.draw()
plt.savefig('test_broken_spiral.png')
Unfortunately, this plot has completely messed up axis.
I put the if i == 0 statement in, because I do not know the number of iterations in advance (yes, I do in this case, but not in the application this is targeted at) and somehow have to 'initialize' the plot and legend.
My questions can be summarized as follows:
1.) How do I update my plot? If I run the optimizer for 10,000 iterations, I don't want 10,000 overlying lines in my plot (filesize).
2.) Where do I place the legend command?
I'm running python 2.6.6 and matplotlib 0.99.1.1
Edit:
This seems to be a similar question, with the same unanswered issue.
Just make the line objects with empty data outside of your loop:
line1, = ax.plot([], [],'-k',label='black')
line2, = ax.plot([], [],'-r',label='red')
ax.legend()
for i in range(0, SIZE):
A.append(R1 * i * np.sin(i))
B.append(R2 * i * np.cos(i))
line1.set_ydata(A)
line1.set_xdata(range(len(A)))
line2.set_ydata(B)
line2.set_xdata(range(len(B)))
ax.relim()
ax.autoscale_view()
plt.draw()
You can probably be a bit more clever about updating your xdata.
For a more complete example see here and the full gallery of animation examples.
I'm using matplotlib to display data that is constantly being updated (changes roughly 10 times per second). I'm using a 3D scatter plot, and I would like the axes to be fixed to a specific range, since the location of the data with respect to the edges of the plot is what is important.
Currently whenever I add new data, the axes will reset to being scaled by the data, rather than the size I want (when I have hold=False). If I set hold=True, the axes will remain the right size, but the new data will be overlayed on the old data, which is not what I want.
I can get it to work if I rescale the axes everytime I get new data, but this seems like an inefficient way to do this, especially since I need to do all other formatting again as well (adding titles, legends, etc)
Is there some way in which I can specify the properties of the plot just once, and this will remain fixed as I add new data?
Here is a rough outline of my code, to help explain what I mean:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
X_MAX = 50
Y_MAX = 50
Z_MAX = 50
fig = plt.figure(1)
ax = fig.add_subplot(111, projection='3d')
ax.set_title("My Title")
ax.set_xlim3d([0, X_MAX])
ax.set_ylim3d([0, Y_MAX])
ax.set_zlim3d([0, Z_MAX])
ax.set_autoscale_on(False)
# This is so the new data replaces the old data
# seems to be replacing the axis ranges as well, maybe a different method should be used?
ax.hold(False)
plt.ion()
plt.show()
a = 0
while a < 50:
a += 1
ax.scatter( a, a/2+1, 3, s=1 )
# If I don't set the title and axes ranges again here, they will be reset each time
# I want to know if there is a way to only set them once and have it persistent
ax.set_title("My Title")
ax.set_xlim3d([0, X_MAX])
ax.set_ylim3d([0, Y_MAX])
ax.set_zlim3d([0, Z_MAX])
plt.pause(0.001)
EDIT:
1. I have also tried ax.set_autoscale_on(False), but with no success
2. I tried this with a regular 2D scatter plot, and the same issue still exists
3. Found a related question which also still doesn't have an answer
I would do something like this (note removal of hold(False) ):
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
X_MAX = 50
Y_MAX = 50
Z_MAX = 50
fig = plt.figure(1)
ax = fig.add_subplot(111, projection='3d')
ax.set_title("My Title")
ax.set_xlim3d([0, X_MAX])
ax.set_ylim3d([0, Y_MAX])
ax.set_zlim3d([0, Z_MAX])
ax.set_autoscale_on(False)
plt.ion()
plt.show()
a = 0
sct = None
while a < 50:
a += 1
if sct is not None:
sct.remove()
sct = ax.scatter( a, a/2+1, 3, s=1 )
fig.canvas.draw()
plt.pause(0.001)
Where you remove just the added scatter plot each time through the loop.