I am using Python and the matplotlib library.
I run through a very long code creating multiple figures along the way.
I do something like this, many many times:
plt.figure()
plt.plot(x, y)
plt.grid('on')
plt.close()
Then, at some point, I get the error:
More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
The error is clear to me, but: I do call "plt.close()".
Now that I am writing this I am realizing that maybe plt.close() has to take a specific qualifier for what figure to close? Like plt.close(1) etc. I guess I can use plt.close('all'), but what if I just want to close the most recent figure?
The code from the question should work fine. Since you close the figure in every loop step, there will only ever be one single figure open.
Minimal example, which does not produce any error:
import matplotlib.pyplot as plt
for i in range(30):
plt.figure()
plt.plot(range(i+3), range(i+3))
plt.grid('on')
plt.close()
plt.show() # doesn't show anything since no figure is open
So the reason for the error must be somewhere else in the code.
You should operate on matplotlib objects directly. It's so much less ambiguous:
fig, ax = plt.subplots()
ax.plot(x, y)
...
plt.close(fig)
Related
I would like to update a figure of matplotlib for every iteration in a loop. It works for about the first 30 iterations, but then the updates stop although there are more iterations.
Following you can find my code for the figure:
import numpy as np
import matplotlib.pyplot as plt
class SimpOutput:
fig = None
ax_l = None
ax_r = None
it_container = []
obj_container = []
def __init__(self):
self.fig, (self.ax_l, self.ax_r) = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))
self.ax_l.set_title("Flexibility $c$")
self.ax_l.set_ylabel("obj. value $c$")
self.ax_l.set_xlabel("Iteration")
self.ax_r.set_title("Shape")
self.fig.show()
def update(self, iteration, obj, x):
self.it_container.append(iteration)
self.obj_container.append(obj)
self.ax_l.plot(self.it_container, self.obj_container, c="r")
x = x.reshape((4, 4))
x = x.T
x = np.flip(x, 0)
self.ax_r.imshow(x, cmap="binary")
plt.pause(0.1)
self.fig.show()
plt.pause(0.1)
if __name__ == "__main__":
out = SimpOutput()
for i in range(50):
out.update(i, 1000 * np.random.rand(), np.random.rand(16))
update is called in every loop.
Using fig.canvas.show() and different values for pause does not affect on the update. Furthermore, the methods set_array() and set_data() do not fix the problem either. While debugging, the figure is updated for every iteration. I write the code with PyCharm.
Does anyone had the same issue or rather does anyone has an idea how to solve this issue?
Thanks in advance!
Bests,
Sebastian
I tried to run the following simple code:
for i in range(50):
plt.clf()
pd.DataFrame([3,4,i]).plot.line(title=f'iteration {i}')
plt.show()
Not updating any figure, just clearing the old one and plotting a new figure.
It stops creating new figures after 30 iterations as well. Same problem as you.
I suspected the problem is in pycharm, so I tried one more thing.
I replaced plt.show() with plt.savefig(f'example_{i}.jpg'). It indeed saved all 50 figures in the folder (means it worked okay), and in addition, it gave me a warning:
envs\my_main_env\lib\site-packages\pandas\plotting\_matplotlib\core.py:337: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
fig = self.plt.figure(figsize=self.figsize)
indicating the problem is in the number of figures open at parallel. You can google the warning find some answers to it (e.g. warning about too many open figures).
I added the parameter at the beginning of the code: plt.rcParams.update({'figure.max_open_warning': 60})
now, running again. In pycharm in scientific mode, it didn't work. So I turned off scientific mode (opening a new figure for each plot) and now it works well! so I think that the limitation is a pycharm limitation for scientific mode - how many figures can be open at the same time.
turn off scientific mode for plotting by going to Settings->Tools->Python Scientific-> uncheck "Show plots in tool window" box.
That worked for me. I don't know how to change the limit in scientific mode (if possible at all). I believe it gives enough insights for you to solve your issue - whether by saving the figures or by plotting them without scientific mode.
I'd like to know the correct way to release memory after a plot is done since I'm getting a RuntimeError: Could not allocate memory for image error when plotting multiple images in a loop.
Currently I have the following commands in another code to supposedly do just that:
import matplotlib.pyplot as plt
# The code
.....
# Make plot
fig = plt.figure()
# Plotting stuff.
plt.imshow(...)
plt.plot(...)
plt.scatter(...)
# Save plot to file.
plt.savefig(...)
# Release memory.
plt.clf()
plt.close()
A comment in this answer states that the correct syntax is actually plt.close(fig) but the highest voted answer given here says that plt.clf() is enough and doesn't mention .close.
The questions are: what is(are) the correct command(s) to release memory after the plot is saved to file? Do I need both .clf and .close or is one of them enough?
I would like to suggest for you an alternate approach. Note that imshow returns a handle for you. Grab a reference on this, and use the set_data method on that object for subsequent iterations.
>>> h = plt.imshow(np.zeros([480, 640]))
>>> h
<matplotlib.image.AxesImage at 0x47a03d0>
>>> for img in my_imgs:
... h.set_data(img) #etc
This answer has beautifully showed how to reverse the y-axis. However, I now wish to draw all my dots, etc. with respect to this reversed version of coordinate system.
I find the following all fail this purpose:
plt.figure()
plt.gca().invert_yaxis()
plt.plot([1,2],[1,3]) # just a random line
plt.figure()
plt.plot([1,2],[1,3]) # just a random line
plt.gca().invert_yaxis()
How may I fix it and let it work?
For me, if I use an OOP-style figure, i.e.
fig = plt.figure()
axes = fig.add_axes([0.1, 0.1, 0.8, 0.8])
axes.plot([1,2],[1,3]) # just a random line
axes.invert_yaxis()
it works.
But for the current two non-OOP styles listed above, a new figure with a reversed y-axis is created, but the line is not there.
I still can't reproduce your original error using the snippet you posted (is that really all of your code?), but what you're describing sounds like it could be caused by a race condition when you call plt.gca() twice in quick succession. You could perhaps try inserting a short pause between plotting your two figures:
import time
plt.figure()
plt.gca().invert_yaxis()
plt.plot([1,2],[1,3]) # just a random line
time.sleep(0.1)
plt.figure()
plt.plot([1,2],[1,3]) # just a random line
plt.gca().invert_yaxis()
However, as a more general point I would strongly recommend that you avoid using gca() and gcf() except for convenience during interactive sessions - it's much more Pythonic to pass the axes or figure objects explicitly, and it makes it way easier to keep track of exactly which axes/figures are being modified.
Matplotlib offers these functions:
cla() # Clear axis
clf() # Clear figure
close() # Close a figure window
When should I use each function and what exactly does it do?
They all do different things, since matplotlib uses a hierarchical order in which a figure window contains a figure which may consist of many axes. Additionally, there are functions from the pyplot interface and there are methods on the Figure class. I will discuss both cases below.
pyplot interface
pyplot is a module that collects a couple of functions that allow matplotlib to be used in a functional manner. I here assume that pyplot has been imported as import matplotlib.pyplot as plt.
In this case, there are three different commands that remove stuff:
See matplotlib.pyplot Functions:
plt.cla() clears an axis, i.e. the currently active axis in the current figure. It leaves the other axes untouched.
plt.clf() clears the entire current figure with all its axes, but leaves the window opened, such that it may be reused for other plots.
plt.close() closes a window, which will be the current window, if not specified otherwise.
Which functions suits you best depends thus on your use-case.
The close() function furthermore allows one to specify which window should be closed. The argument can either be a number or name given to a window when it was created using figure(number_or_name) or it can be a figure instance fig obtained, i.e., usingfig = figure(). If no argument is given to close(), the currently active window will be closed. Furthermore, there is the syntax close('all'), which closes all figures.
methods of the Figure class
Additionally, the Figure class provides methods for clearing figures.
I'll assume in the following that fig is an instance of a Figure:
fig.clf() clears the entire figure. This call is equivalent to plt.clf() only if fig is the current figure.
fig.clear() is a synonym for fig.clf()
Note that even del fig will not close the associated figure window. As far as I know the only way to close a figure window is using plt.close(fig) as described above.
There is just a caveat that I discovered today.
If you have a function that is calling a plot a lot of times you better use plt.close(fig) instead of fig.clf() somehow the first does not accumulate in memory. In short if memory is a concern use plt.close(fig) (Although it seems that there are better ways, go to the end of this comment for relevant links).
So the the following script will produce an empty list:
for i in range(5):
fig = plot_figure()
plt.close(fig)
# This returns a list with all figure numbers available
print(plt.get_fignums())
Whereas this one will produce a list with five figures on it.
for i in range(5):
fig = plot_figure()
fig.clf()
# This returns a list with all figure numbers available
print(plt.get_fignums())
From the documentation above is not clear to me what is the difference between closing a figure and closing a window. Maybe that will clarify.
If you want to try a complete script there you have:
import numpy as np
import matplotlib.pyplot as plt
x = np.arange(1000)
y = np.sin(x)
for i in range(5):
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.plot(x, y)
plt.close(fig)
print(plt.get_fignums())
for i in range(5):
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.plot(x, y)
fig.clf()
print(plt.get_fignums())
If memory is a concern somebody already posted a work-around in SO see:
Create a figure that is reference counted
plt.cla() means clear current axis
plt.clf() means clear current figure
also, there's plt.gca() (get current axis) and plt.gcf() (get current figure)
Read more here: Matplotlib, Pyplot, Pylab etc: What's the difference between these and when to use each?
I've kept a set of references to figures in a dictionary so that I could save them later if desired. I am troubled that the saved figures are blank if invoke a show() command and look at them first. Since the show() command blocks and I am not using a spyder-like interpreter, I have to close the figures before I get to savefig()
figures['myfig_1'] = figure()
...
figures['myfig_n'] = figure()
...
#show() #disabling this makes the problem go away
print "Saving:"
for fig in figures:
figure(figures[fig].number)
savefig(fig)
print "Figure " + str(figures[fig].number) + ": " + fig
The print statement here has given me the indication that the dictionary is still intact, which I think means that I have not lost the figure references (they are still returning meaningful numbers in their .number attribute.)
Another twist I have noticed is that when I have done a similar thing in a class, storing the dictionary as a member and dividing the store and save functions into their own methods, this does not happen. Is there something about the way I am closing the figures or storing the data which is making the figures loose their data?
Generally speaking, in cases like this don't use the interactive matlab-ish state machine interface to matplotlib. It's meant for interactive use.
You're trying to make a figure "active", and creating a new figure instead. It doesn't matter which figure is active, if you just retain the returned figure and/or axis objects and use them directly. (Also, don't use wildcard imports! You will regret it at some later point when you're maintaining your code!)
Just do something like this:
import matplotlib.pyplot as plt
figures = {}
figures['a'] = plt.figure()
ax = figures['a'].add_subplot(111)
ax.plot(range(10), 'ro-')
figures['b'] = plt.figure()
ax = figures['b'].add_subplot(111)
ax.plot(range(10), 'bo-')
plt.show()
for name, fig in figures.iteritems():
fig.savefig('figure-%s.png' % name)
From the documentation, whether or not the drawing elements are destroyed from show() depends on the backend, and the version of matplotlib. Not having the figures destroyed seems to be available with version 1.1.0. To figure out which backend is in use, use the get_backend() function. In my case, I was using the Qt4Agg backend. By invoking the TkAgg backend, with the call matplotlib.use('TkAgg') the figures were not destroyed before the save. Now to find out how to change the behavior of the Qt4Agg...