I would like to update a figure of matplotlib for every iteration in a loop. It works for about the first 30 iterations, but then the updates stop although there are more iterations.
Following you can find my code for the figure:
import numpy as np
import matplotlib.pyplot as plt
class SimpOutput:
fig = None
ax_l = None
ax_r = None
it_container = []
obj_container = []
def __init__(self):
self.fig, (self.ax_l, self.ax_r) = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))
self.ax_l.set_title("Flexibility $c$")
self.ax_l.set_ylabel("obj. value $c$")
self.ax_l.set_xlabel("Iteration")
self.ax_r.set_title("Shape")
self.fig.show()
def update(self, iteration, obj, x):
self.it_container.append(iteration)
self.obj_container.append(obj)
self.ax_l.plot(self.it_container, self.obj_container, c="r")
x = x.reshape((4, 4))
x = x.T
x = np.flip(x, 0)
self.ax_r.imshow(x, cmap="binary")
plt.pause(0.1)
self.fig.show()
plt.pause(0.1)
if __name__ == "__main__":
out = SimpOutput()
for i in range(50):
out.update(i, 1000 * np.random.rand(), np.random.rand(16))
update is called in every loop.
Using fig.canvas.show() and different values for pause does not affect on the update. Furthermore, the methods set_array() and set_data() do not fix the problem either. While debugging, the figure is updated for every iteration. I write the code with PyCharm.
Does anyone had the same issue or rather does anyone has an idea how to solve this issue?
Thanks in advance!
Bests,
Sebastian
I tried to run the following simple code:
for i in range(50):
plt.clf()
pd.DataFrame([3,4,i]).plot.line(title=f'iteration {i}')
plt.show()
Not updating any figure, just clearing the old one and plotting a new figure.
It stops creating new figures after 30 iterations as well. Same problem as you.
I suspected the problem is in pycharm, so I tried one more thing.
I replaced plt.show() with plt.savefig(f'example_{i}.jpg'). It indeed saved all 50 figures in the folder (means it worked okay), and in addition, it gave me a warning:
envs\my_main_env\lib\site-packages\pandas\plotting\_matplotlib\core.py:337: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
fig = self.plt.figure(figsize=self.figsize)
indicating the problem is in the number of figures open at parallel. You can google the warning find some answers to it (e.g. warning about too many open figures).
I added the parameter at the beginning of the code: plt.rcParams.update({'figure.max_open_warning': 60})
now, running again. In pycharm in scientific mode, it didn't work. So I turned off scientific mode (opening a new figure for each plot) and now it works well! so I think that the limitation is a pycharm limitation for scientific mode - how many figures can be open at the same time.
turn off scientific mode for plotting by going to Settings->Tools->Python Scientific-> uncheck "Show plots in tool window" box.
That worked for me. I don't know how to change the limit in scientific mode (if possible at all). I believe it gives enough insights for you to solve your issue - whether by saving the figures or by plotting them without scientific mode.
Related
I am following this tutorial, and I stumbled upon something I don't understand.
The idea is to have a function, that plots an image. This function is then called in a loop where subplots are defined:
minimal example
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
def show_image(image):
"""Show image"""
plt.imshow(image)
def show_image_wait(image):
"""show image, and wait a little bit. similar implementation than in the tutorial"""
plt.imshow(image)
plt.pause(0.001)
Now, calling both functions in a loop:
no waiting:
for i in range(4):
image = np.random.randint(0,3, (10,10))
plt.subplot(1, 4, i+1)
show_image(image)
# expected output: 1 row, with 4 images side by side
# actual output: 1 row, with 4 images, side by side
with waiting, however:
for i in range(4):
image = np.random.randint(0,3, (10,10))
plt.subplot(1, 4, i+1)
show_image_wait(image)
# expected output: 1 row, with 4 images side by side
# actual output: 4 rows, with 1 images each
A function similar to show_image_wait is used in the tutorial linked above, where all images appear correctly positioned.
I don't understand why waiting for a tiny bit overrides the subplot positioning in my case, and not in the linked example.
All of this happens in a jupyter notebook
Help is greatly appreciated!
The reason you see several rows is that for each loop run a new figure is produced. Independent figures are placed below each other in a jupyter output cell.
This in turn is caused by the one from the last loop iteration being drawn on screen and hence when plt.subplot is called another time, no active figure is present - therefore a new one is created.
The underlying cause of all of this is that plt.pause(..) does a bit more than only pausing. Instead it handles possible events on the figure and eventually draws and shows the figure in interactive mode.
The source of plt.pause is
manager = _pylab_helpers.Gcf.get_active()
if manager is not None:
canvas = manager.canvas
if canvas.figure.stale:
canvas.draw_idle()
show(block=False) # <---- here the figure is shown.
canvas.start_event_loop(interval)
else:
time.sleep(interval)
where I marked the crucial line with a comment.
So in total, if you want a true pause as in "Do not do anything for x seconds", plt.pause is not well suited. In general it is also a bit questionable how useful it is in jupyter notebooks with inline backend, because that backend does not provide any interactivity.
I think it would work if you use plt.show() after the for loop but in JuPyTer notebook, the plotting is inline. A work around solution could be to use time.sleep(0.001). You may try and see if it serves your purpose.
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import time
def show_image_wait(image):
"""show image, and wait a little bit. similar implementation than in the tutorial"""
plt.imshow(image)
time.sleep(0.001)
for i in range(4):
image = np.random.randint(0,3, (10,10))
plt.subplot(1, 4, i+1)
show_image_wait(image)
I am using Python and the matplotlib library.
I run through a very long code creating multiple figures along the way.
I do something like this, many many times:
plt.figure()
plt.plot(x, y)
plt.grid('on')
plt.close()
Then, at some point, I get the error:
More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
The error is clear to me, but: I do call "plt.close()".
Now that I am writing this I am realizing that maybe plt.close() has to take a specific qualifier for what figure to close? Like plt.close(1) etc. I guess I can use plt.close('all'), but what if I just want to close the most recent figure?
The code from the question should work fine. Since you close the figure in every loop step, there will only ever be one single figure open.
Minimal example, which does not produce any error:
import matplotlib.pyplot as plt
for i in range(30):
plt.figure()
plt.plot(range(i+3), range(i+3))
plt.grid('on')
plt.close()
plt.show() # doesn't show anything since no figure is open
So the reason for the error must be somewhere else in the code.
You should operate on matplotlib objects directly. It's so much less ambiguous:
fig, ax = plt.subplots()
ax.plot(x, y)
...
plt.close(fig)
I am using matplotlib 1.4.3 with python 3.3. I would like to draw multiple figures with multiples sub-plot in it. I am facing some kind of bug that is really boring me:
When I use fill() and boxplot() methods whithin a figure, results of those functions are hidden if the figure is not the first one created.
This bug seems to be related somehow to polygon display and matplotlib environment state.
When a parse only one single figure, everything is working fine. When I parse multiple figures, the first one is ok. But, in every other subsequent figures, everything is all-right except wiskerbox and polygons that are hidden.
Each plot code is wrapped into a function, which accepts positional arguments, *args and **kwargs. Lets say signature are:
def myplot(t, x, *args, *kwargs):
# [...]
hFig = plt.figure()
# [...]
return hFig
As far as I understand python mechanisms, after the function call is resolved, there must be nothing alive (I do not use global variables) except what matplotlib environment has stored into its global namespace variables.
In every call, I close() my figure, I also have tried hFig.clf() in addition before leaving function, but it does not solve the problem.
Each plot is wrapped into printer (decorator) to add generic functionalities:
def myprint(func):
def inner(*args, **kwargs)
# [...]
hFig = func(*args, **kwargs)
# [...]
return inner
What I have tried so far:
Increased zscore of wiskerbox and polygons, not working;
Execute plot generation in different threads, not working;
Execute plot generation in different processes, working but I have to change my function signature because it can be pickled.
I do not want use dill and pathos, even if I would I cannot.
It looks like it is a matplotlib environment bug, because when I run different processes, this environment is recreated from scratch and it works the way it should. I would like to know if there is a way to reset matplotlib environment state within a python script. If not, what can I do for solving this issue.
Obs.: I am using GridSpecs object and subplot() method to create my figures. The problem was not present when I computed boxes myself and used add_axes() method.
Update: Here you can find a MCVE of my problem. By doing this simple example, I found the line which makes my bug happens (looks like I have old bad Matlab behaviours). It seems that plt.hold(False) alters the way of polygons and boxplot are displayed. And, as I pointed out, it was related to matplotlib global namespace variable. I just misunderstood the way this method works, and in each sub-process, it was reset.
import datetime
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gspec
def bloodyplotit(t_, x_):
hFig = plt.figure()
gs = gspec.GridSpec(1, 4, height_ratios=[1], width_ratios=[15, 2, 3, 1])
gs.update(left=0.10, right=0.90, top=0.90, bottom=0.25, hspace=0.05, wspace=0.05)
plt.hold(True)
hAxe = plt.subplot(gs[0,0])
hAxe.plot(t_, x_)
#plt.hold(False) # <------------------------ This line make the scirpt bug
hAxe = plt.subplot(gs[0,1])
hAxe.hist(x_, orientation='horizontal')
hAxe = plt.subplot(gs[0,3])
hAxe.boxplot(x_)
plt.show()
n = 1000
t = datetime.datetime.utcnow() + np.arange(n)*datetime.timedelta(minutes=1)
x = np.random.randn(1000,1)
for i in range(10):
bloodyplotit(t, x)
Here's an even more minimal script that produces the error:
x = np.random.randn(1000)
fig, ax = plt.subplots(1, 2)
ax[0].hold(True)
ax[0].boxplot(x);
ax[1].hold(False)
ax[1].boxplot(x);
As far as I can tell, this is expected behavior. According to the documentation of plt.hold,
When hold is True, subsequent plot commands will be added to the current axes. When hold is False, the current axes and figure will be cleared on the next plot command.
Boxplot is a compound object: it is created by calling multiple plotting commands. If hold is False, the axes are cleared between each of those commands, so parts of the boxplot don't show up.
Personally, I've never found a reason to toggle the hold state in 10+ years of using matplotlib. My experience is that doing it (especially globally) just causes confusion, and I'd recommend avoiding it.
I'm trying to get real-time spectrum analyzer type plot in matplotlib. I've got some code working (with help from other posts on StackOverflow) as follows:
import time
import numpy as np
import matplotlib.pyplot as plt
plt.axis([0, 1000, 0, 1])
plt.ion()
plt.show()
i=0
np.zeros([1,500],'float')
lines=plt.plot(y[0])
while 1:
i=i+1
lines.pop(0).remove()
y = np.random.rand(1,100)
lines=plt.plot(y[0])
plt.draw()
The code works and I'm getting what I want, but there is a serious problem. The plot window would freeze after some time. I know the program is still running by inspecting the i variable (I'm running the code in Anaconda/Spyder so I can see the variables). However the plot window would show "Non responding" and if I terminate the python program in Spyder by ctrl+c, the plot window comes back to life and show the latest plot.
I'm out of wits here as how to further debug the issue. Anyone to help?
Thanks
I am not sure that adding plt.pause will entirely solve your issue. It may just take longer before the application crash. The memory used by your application seems to constantly increase over time (even after adding plt.pause). Below are two suggestions that may help you with your current issue:
Instead of removing/recreating the lines artists with each iteration with remove and plot, I would use the same artist throughout the whole animation and simply update its ydata.
I'll use explicit handlers for the axe and figure and call show and draw explicitly on the figure manager and canvas instead of going with implicit calls through pyplot, following the advices given in a post by tcaswell.
Following the above, the code would look something like this:
import numpy as np
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([0, 100, 0, 1])
y = np.random.rand(100)
lines = ax.plot(y)
fig.canvas.manager.show()
i=0
while 1:
i=i+1
y = np.random.rand(100)
lines[0].set_ydata(y)
fig.canvas.draw()
fig.canvas.flush_events()
I've run the above code for a good 10 minutes and the memory used by the application remained stable the whole time, while the memory used by your current code (without plt.pause) increased by about 30MiB over the same period.
To answer myself, I solved the issue by adding
plt.pause(0.01)
after the
plt.draw()
This probably allows the GUI to finish the drawing and clear the buffer somewhere (my guess) before the new data comes in.
I know I'm late to answer this question, but for your issue you could look into the "joystick" package. It is based on the line.set_data() and canvas.draw() methods, with optional axes re-scaling, hence most probably faster than removing a line and adding a new one. It also allows for interactive text logging or image plotting (in addition to graph plotting).
No need to do your own loops in a separate thread, the package takes care of it, just give the update frequency you wish. Plus the terminal remains available for more monitoring commands while live plotting, which is not possible with a "while True" loop.
See http://www.github.com/ceyzeriat/joystick/ or https://pypi.python.org/pypi/joystick (use pip install joystick to install)
try:
import joystick as jk
import numpy as np
import time
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the doc
"""
self._t0 = time.time() # initialize time
self.xdata = np.array([self._t0]) # time x-axis
self.ydata = np.array([0.0]) # fake data y-axis
# create a graph frame
self.mygraph = self.add_frame(jk.Graph(name="test", size=(500, 500), pos=(50, 50), fmt="go-", xnpts=100, xnptsmax=1000, xylim=(None, None, 0, 1)))
#_infinite_loop(wait_time=0.2)
def _generate_data(self): # function looped every 0.2 second to read or produce data
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.2 seconds
"""
# concatenate data on the time x-axis
self.xdata = jk.core.add_datapoint(self.xdata, time.time(), xnptsmax=self.mygraph.xnptsmax)
# concatenate data on the fake data y-axis
self.ydata = jk.core.add_datapoint(self.ydata, np.random.random(), xnptsmax=self.mygraph.xnptsmax)
self.mygraph.set_xydata(t, self.ydata)
t = test()
t.start()
t.stop()
At the moment I am working with Spyder and doing my plotting with matplotlib. I have two monitors, one for development and another for (data) browsing and other stuff. Since I am doing some calculations and my code often changes, I often (re)execute the code and have a look at the plots to check if the results are valid.
Is there any way to place my matplotlib plots on a second monitor and refresh them from the main monitor?
I have already searched for a solution but could not find anything. It would be really helpful for me!
Here's some additional information:
OS: Ubuntu 14.04 (64 Bit)
Spyder-Version: 2.3.2
Matplotlib-Version: 1.3.1.-1.4.2.
I know it's an old question but I came across a similar problem and found this question. I managed to move my plots to a second display using the QT4Agg backend.
import matplotlib.pyplot as plt
plt.switch_backend('QT4Agg')
# a little hack to get screen size; from here [1]
mgr = plt.get_current_fig_manager()
mgr.full_screen_toggle()
py = mgr.canvas.height()
px = mgr.canvas.width()
mgr.window.close()
# hack end
x = [i for i in range(0,10)]
plt.figure()
plt.plot(x)
figManager = plt.get_current_fig_manager()
# if px=0, plot will display on 1st screen
figManager.window.move(px, 0)
figManager.window.showMaximized()
figManager.window.setFocus()
plt.show()
[1] answer from #divenex: How do you set the absolute position of figure windows with matplotlib?
This has to do with matplotlib, not Spyder. Placing the location of a figure explicitly appears to be one of those things for which there's really just workarounds ... see the answers to the question here. That's an old question, but I'm not sure there's been change since then (any matplotlib devs, feel free to correct me!).
The second monitor shouldn't make any difference, it sounds like the issue is just that the figure is being replaced with a new one.
Fortunately you can update figures you've moved to where you want them pretty easily, by using the object interface specifically, and updating the Axes object without creating a new figure. An example is below:
import matplotlib.pyplot as plt
import numpy as np
# Create the figure and axes, keeping the object references
fig = plt.figure()
ax = fig.add_subplot(111)
p, = ax.plot(np.linspace(0,1))
# First display
plt.show()
# Some time to let you look at the result and move/resize the figure
plt.pause(3)
# Replace the contents of the Axes without making a new window
ax.cla()
p, = ax.plot(2*np.linspace(0,1)**2)
# Since the figure is shown already, use draw() to update the display
plt.draw()
plt.pause(3)
# Or you can get really fancy and simply replace the data in the plot
p.set_data(np.linspace(-1,1), 10*np.linspace(-1,1)**3)
ax.set_xlim(-1,1)
ax.set_ylim(-1,1)
plt.draw()