Develop Reference

Matplotlib figure does not update in every loop

I would like to update a figure of matplotlib for every iteration in a loop. It works for about the first 30 iterations, but then the updates stop although there are more iterations.
Following you can find my code for the figure:
import numpy as np
import matplotlib.pyplot as plt
class SimpOutput:
fig = None
ax_l = None
ax_r = None
it_container = []
obj_container = []
def __init__(self):
self.fig, (self.ax_l, self.ax_r) = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))
self.ax_l.set_title("Flexibility $c$")
self.ax_l.set_ylabel("obj. value $c$")
self.ax_l.set_xlabel("Iteration")
self.ax_r.set_title("Shape")
self.fig.show()
def update(self, iteration, obj, x):
self.it_container.append(iteration)
self.obj_container.append(obj)
self.ax_l.plot(self.it_container, self.obj_container, c="r")
x = x.reshape((4, 4))
x = x.T
x = np.flip(x, 0)
self.ax_r.imshow(x, cmap="binary")
plt.pause(0.1)
self.fig.show()
plt.pause(0.1)
if __name__ == "__main__":
out = SimpOutput()
for i in range(50):
out.update(i, 1000 * np.random.rand(), np.random.rand(16))
update is called in every loop.
Using fig.canvas.show() and different values for pause does not affect on the update. Furthermore, the methods set_array() and set_data() do not fix the problem either. While debugging, the figure is updated for every iteration. I write the code with PyCharm.
Does anyone had the same issue or rather does anyone has an idea how to solve this issue?
Thanks in advance!
Bests,
Sebastian

I tried to run the following simple code:
for i in range(50):
plt.clf()
pd.DataFrame([3,4,i]).plot.line(title=f'iteration {i}')
plt.show()
Not updating any figure, just clearing the old one and plotting a new figure.
It stops creating new figures after 30 iterations as well. Same problem as you.
I suspected the problem is in pycharm, so I tried one more thing.
I replaced plt.show() with plt.savefig(f'example_{i}.jpg'). It indeed saved all 50 figures in the folder (means it worked okay), and in addition, it gave me a warning:
envs\my_main_env\lib\site-packages\pandas\plotting\_matplotlib\core.py:337: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
fig = self.plt.figure(figsize=self.figsize)
indicating the problem is in the number of figures open at parallel. You can google the warning find some answers to it (e.g. warning about too many open figures).
I added the parameter at the beginning of the code: plt.rcParams.update({'figure.max_open_warning': 60})
now, running again. In pycharm in scientific mode, it didn't work. So I turned off scientific mode (opening a new figure for each plot) and now it works well! so I think that the limitation is a pycharm limitation for scientific mode - how many figures can be open at the same time.
turn off scientific mode for plotting by going to Settings->Tools->Python Scientific-> uncheck "Show plots in tool window" box.
That worked for me. I don't know how to change the limit in scientific mode (if possible at all). I believe it gives enough insights for you to solve your issue - whether by saving the figures or by plotting them without scientific mode.

Real-time plot in matplotlib - python

I'm trying to get real-time spectrum analyzer type plot in matplotlib. I've got some code working (with help from other posts on StackOverflow) as follows:
import time
import numpy as np
import matplotlib.pyplot as plt
plt.axis([0, 1000, 0, 1])
plt.ion()
plt.show()
i=0
np.zeros([1,500],'float')
lines=plt.plot(y[0])
while 1:
i=i+1
lines.pop(0).remove()
y = np.random.rand(1,100)
lines=plt.plot(y[0])
plt.draw()
The code works and I'm getting what I want, but there is a serious problem. The plot window would freeze after some time. I know the program is still running by inspecting the i variable (I'm running the code in Anaconda/Spyder so I can see the variables). However the plot window would show "Non responding" and if I terminate the python program in Spyder by ctrl+c, the plot window comes back to life and show the latest plot.
I'm out of wits here as how to further debug the issue. Anyone to help?
Thanks

I am not sure that adding plt.pause will entirely solve your issue. It may just take longer before the application crash. The memory used by your application seems to constantly increase over time (even after adding plt.pause). Below are two suggestions that may help you with your current issue:
Instead of removing/recreating the lines artists with each iteration with remove and plot, I would use the same artist throughout the whole animation and simply update its ydata.
I'll use explicit handlers for the axe and figure and call show and draw explicitly on the figure manager and canvas instead of going with implicit calls through pyplot, following the advices given in a post by tcaswell.
Following the above, the code would look something like this:
import numpy as np
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([0, 100, 0, 1])
y = np.random.rand(100)
lines = ax.plot(y)
fig.canvas.manager.show()
i=0
while 1:
i=i+1
y = np.random.rand(100)
lines[0].set_ydata(y)
fig.canvas.draw()
fig.canvas.flush_events()
I've run the above code for a good 10 minutes and the memory used by the application remained stable the whole time, while the memory used by your current code (without plt.pause) increased by about 30MiB over the same period.

To answer myself, I solved the issue by adding
plt.pause(0.01)
after the
plt.draw()
This probably allows the GUI to finish the drawing and clear the buffer somewhere (my guess) before the new data comes in.

I know I'm late to answer this question, but for your issue you could look into the "joystick" package. It is based on the line.set_data() and canvas.draw() methods, with optional axes re-scaling, hence most probably faster than removing a line and adding a new one. It also allows for interactive text logging or image plotting (in addition to graph plotting).
No need to do your own loops in a separate thread, the package takes care of it, just give the update frequency you wish. Plus the terminal remains available for more monitoring commands while live plotting, which is not possible with a "while True" loop.
See http://www.github.com/ceyzeriat/joystick/ or https://pypi.python.org/pypi/joystick (use pip install joystick to install)
try:
import joystick as jk
import numpy as np
import time
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the doc
"""
self._t0 = time.time() # initialize time
self.xdata = np.array([self._t0]) # time x-axis
self.ydata = np.array([0.0]) # fake data y-axis
# create a graph frame
self.mygraph = self.add_frame(jk.Graph(name="test", size=(500, 500), pos=(50, 50), fmt="go-", xnpts=100, xnptsmax=1000, xylim=(None, None, 0, 1)))
#_infinite_loop(wait_time=0.2)
def _generate_data(self): # function looped every 0.2 second to read or produce data
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.2 seconds
"""
# concatenate data on the time x-axis
self.xdata = jk.core.add_datapoint(self.xdata, time.time(), xnptsmax=self.mygraph.xnptsmax)
# concatenate data on the fake data y-axis
self.ydata = jk.core.add_datapoint(self.ydata, np.random.random(), xnptsmax=self.mygraph.xnptsmax)
self.mygraph.set_xydata(t, self.ydata)
t = test()
t.start()
t.stop()

Is there a better way to re-use plots Matplotlib?

The below code block (A) solves my overall problem of being able to re-use plots, but i'm wondering if there is a better way (one that doesn't involve creating a function for each plt.plot, as is shown below.
Code block A:
import maptplotlib.pyplot as plt
#create a function just to call the plt.plot
def first_plot(): plt.plot(x,y)
first_plot()
# Now i can just show the first plot
plt.show()
def second_plot(): plt.plot(x,z)
first_plot() # instead of plt.plot(x,y)
second_plot() # instead of plt.plot(x,z)
# now i can show both plots
plt.show()
if the plots are complicated:
plot.plot(lots of details)
and their are many:
plots = [first,second,third,fourth,...]
I would think this would be advantageous because it avoids code re-use.
However, creating a function just to call plt.plot() indicates to me their might be a better way.what i would like to be doing is something like
first_plot = plt.plot(x,y)
first_plot()
#now i want to show just the first plot
plt.show() # first call
second_plot = plt.plot(x,z)
# now i want to show them together combined
first_plot()
second_plot()
plt.show() # second call
But this doesn't seem to work/e.g the second call to plt.show() wouldn't produce the first plot. (even if you unpack first_plot (which in reality, from code block B, is actual a list):
In [17]: first_plot
Out[17]: [<matplotlib.lines.Line2D at 0x7fd33ac8ff50>]
This cannot be used to produce the plot again in anyway I can see. This might be because of the relationship between plt.show and plt.plot, which i don't fully understand. Plt.plot seems to put the plot in a que, that then plt.show pops out. However,
plt.shows description talks about a blocking and unblocking mode and interactive and non-interactive mode:
show(*args, **kw)
When running in ipython with its pylab mode, display all
figures and return to the ipython prompt.
In non-interactive mode, display all figures and block until
the figures have been closed; in interactive mode it has no
effect unless figures were created prior to a change from
non-interactive to interactive mode (not recommended). In
that case it displays the figures but does not block.
A single experimental keyword argument, *block*, may be
set to True or False to override the blocking behavior
described above.
Which i don't understand. But regardless of how i call plt.show() vs plt.show(False) (blocking?) it doesn't seem to have an impact e.g the output is the same in the context of code block A and B.
So put another way, is there a way to select which plots created to show/overlay at different points in the code?

However awkward, it seems the best way to "re-use" plots as described in my question is to do as I original suggested and put it inside a function.
def some_plot(): return plot(x,y)
and then if you want to re-use the plot simply call the function again:
some_plot()

matplotlib figures disappearing between show() and savefig()

I've kept a set of references to figures in a dictionary so that I could save them later if desired. I am troubled that the saved figures are blank if invoke a show() command and look at them first. Since the show() command blocks and I am not using a spyder-like interpreter, I have to close the figures before I get to savefig()
figures['myfig_1'] = figure()
...
figures['myfig_n'] = figure()
...
#show() #disabling this makes the problem go away
print "Saving:"
for fig in figures:
figure(figures[fig].number)
savefig(fig)
print "Figure " + str(figures[fig].number) + ": " + fig
The print statement here has given me the indication that the dictionary is still intact, which I think means that I have not lost the figure references (they are still returning meaningful numbers in their .number attribute.)
Another twist I have noticed is that when I have done a similar thing in a class, storing the dictionary as a member and dividing the store and save functions into their own methods, this does not happen. Is there something about the way I am closing the figures or storing the data which is making the figures loose their data?

Generally speaking, in cases like this don't use the interactive matlab-ish state machine interface to matplotlib. It's meant for interactive use.
You're trying to make a figure "active", and creating a new figure instead. It doesn't matter which figure is active, if you just retain the returned figure and/or axis objects and use them directly. (Also, don't use wildcard imports! You will regret it at some later point when you're maintaining your code!)
Just do something like this:
import matplotlib.pyplot as plt
figures = {}
figures['a'] = plt.figure()
ax = figures['a'].add_subplot(111)
ax.plot(range(10), 'ro-')
figures['b'] = plt.figure()
ax = figures['b'].add_subplot(111)
ax.plot(range(10), 'bo-')
plt.show()
for name, fig in figures.iteritems():
fig.savefig('figure-%s.png' % name)

From the documentation, whether or not the drawing elements are destroyed from show() depends on the backend, and the version of matplotlib. Not having the figures destroyed seems to be available with version 1.1.0. To figure out which backend is in use, use the get_backend() function. In my case, I was using the Qt4Agg backend. By invoking the TkAgg backend, with the call matplotlib.use('TkAgg') the figures were not destroyed before the save. Now to find out how to change the behavior of the Qt4Agg...

Matplotlib canvas drawing

Let's say I define a few functions to do certain matplotlib actions, such as
def dostuff(ax):
ax.scatter([0.],[0.])
Now if I launch ipython, I can load these functions and start a new figure:
In [1]: import matplotlib.pyplot as mpl
In [2]: fig = mpl.figure()
In [3]: ax = fig.add_subplot(1,1,1)
In [4]: run functions # run the file with the above defined function
If I now call dostuff, then the figure does not refresh:
In [6]: dostuff(ax)
I have to then explicitly run:
In [7]: fig.canvas.draw()
To get the canvas to draw. Now I can modify dostuff to be
def dostuff(ax):
ax.scatter([0.],[0.])
ax.get_figure().canvas.draw()
This re-draws the canvas automatically. But now, say that I have the following code:
def dostuff1(ax):
ax.scatter([0.],[0.])
ax.get_figure().canvas.draw()
def dostuff2(ax):
ax.scatter([1.],[1.])
ax.get_figure().canvas.draw()
def doboth(ax):
dostuff1(ax)
dostuff2(ax)
ax.get_figure().canvas.draw()
I can call each of these functions, and the canvas will be redrawn, but in the case of doboth(), it will get redrawn multiple times.
My question is: how could I code this, such that the canvas.draw() only gets called once? In the above example it won't change much, but in more complex cases with tens of functions that can be called individually or grouped, the repeated drawing is much more obvious, and it would be nice to be able to avoid it. I thought of using decorators, but it doesn't look as though it would be simple.
Any ideas?

Why doesn't my answer to this SO question of yours about "refresh decorator" make it simple? I showed exactly what to do what you're again requesting here (by keeping a count of nestings -- incidentally, one that's also thread-safe) and you completely ignored my answer... peculiar behavior!-)