It seems that the standard way of creating a figure in matplotlib doesn't behave as I'd expect in python: by default calling fig = matplotlib.figure() in a loop will hold on to all the figures created, and eventually run out of memory.
There are quite a few posts which deal with workarounds, but requiring explicit calls to matplotlib.pyplot.close(fig) seems a bit hackish. What I'd like is a simple way to make fig reference counted, so I won't have to worry about memory leaks. Is there some way to do this?
If you create the figure without using plt.figure, then it should be reference counted as you expect. For example (This is using the non-interactive Agg backend, as well.)
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
# The pylab figure manager will be bypassed in this instance.
# This means that `fig` will be garbage collected as you'd expect.
fig = Figure()
canvas = FigureCanvas(fig)
ax = fig.add_subplot(111)
If you're only going to be saving figures rather than displaying them, you can use:
def savefig(*args, **kwargs):
plt.savefig(*args, **kwargs)
plt.close(plt.gcf())
This is arguably no less hacky, but whatever.
If you want to profit from the use of pyplot and have the possibility to wrap the figure in a class of your own, you can use the __del__ method of your class to close the figure.
Something like:
import matplotlib.pyplot as plt
class MyFigure:
"""This is my class that just wraps a matplotlib figure"""
def __init__(self):
# Get a new figure using pyplot
self.figure = plt.figure()
def __del__(self):
# This object is getting deleted, close its figure
plt.close(self.figure)
Whenever the garbage collector decides to delete your figure because it is inaccessible, the matplotlib figure will be closed.
However note that if someone has grabbed the figure (but not your wrapper) they might be annoyed that you closed it. It probably can be solved with some more thought or imposing some restrictions on usage.
Related
I am trying to use a python process to animate a plot as shown below:
from multiprocessing import Process
import datetime as dt
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import numpy as np
process_enabled = 1;
print("Process enabled: ", process_enabled)
x = []
y = []
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
def start_animation():
# Set up plot to call animate() function periodically
ani = animation.FuncAnimation(fig, animate, fargs=(x, y), interval=1000)
print("Called animate function")
plt.show()
# This function is called periodically from FuncAnimation
def animate(i, xs, ys):
fx=[0.045,0.02,0.0,0.04,0.015,-0.01,0.015,0.045,0.035,0.01,
0.055,0.04,0.02,0.025,0.0,-0.005,-0.005,-0.02,-0.05,-0.03] # fx values
# Add x and y to lists
xs.append(dt.datetime.now().strftime('%H:%M:%S.%f'))
if(i<len(fx)):
ys.append(fx[i])
# Draw x and y lists
ax.clear()
if(i<len(fx)):
ys_stacked = np.stack((np.array(ys),0.1+np.array(ys)),axis=1)
ax.plot(xs, ys_stacked)
print("Animating")
# Format plot
if(i<len(fx)):
plt.xticks(rotation=45, ha='right')
plt.subplots_adjust(bottom=0.30)
plt.title('Force/Torque Sensor Data')
plt.ylabel('Fx (N)')
if(process_enabled):
p_graph = Process(name='Graph', target=start_animation)
print("Created graph process")
p_graph.start()
print("Started graph process")
else:
start_animation()
When I disable the process, the start_animation() function works fine and the plot is displayed and the animation begins. However, when the process is enabled, the process starts and then the code breaks at print("Called animate function"). There is no plot window and there are no error messages in the terminal).
I'm new to both multiprocessing in python and indeed matplotlib. Any direction would be much appreciated.
Cheers,
Tony
I'm trying to solve this same problem, but haven't quite figured it out completely. However, I think I can provide a few useful comments on your question.
To start, is there any reason why you want to handle the animation in a separate process? Your approach seems to work fine within a single process. There's a number of issues you'll need to address to do this. If you truly do require a separate process, then the following might be useful.
First, you won't be able to use your global variables in the 'graph' process, as that process doesn't share the same instances of those variables (see Globals variables and Python multiprocessing).
You can share state between processes, but this is difficult for complex objects that you'd want to share (i.e. plt.figure()). See the multiprocessing reference for more information (https://docs.python.org/3/library/multiprocessing.html#sharing-state-between-processes)
One final suggestion would be to do away with the pyplot interface. This is handy for straightforward scripts and interactive data analysis, but it obfuscates a lot of important things - like knowing which figure, axis etc you're dealing with when you call plt methods.
I've provided an alternative, object-oriented approach using a custom class, that can run your animation (without a separate process):
import sys
from multiprocessing import Process, Queue
import datetime as dt
from matplotlib.figure import Figure
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg
from matplotlib.backends.qt_compat import QtWidgets
import matplotlib.animation as animation
class StripChart(FigureCanvasQTAgg):
def __init__(self):
self.fig = Figure(figsize=(8,5), dpi=100)
self.ax = self.fig.add_subplot(111)
# hold a copy of our torque data
self.fx = [0.045,0.02,0.0,0.04,0.015,-0.01,0.015,0.045,0.035,0.01,
0.055,0.04,0.02,0.025,0.0,-0.005,-0.005,-0.02,-0.05,-0.03]
super().__init__(self.fig)
# instantiate the data arrays
self.xs = []
self.ys = []
def start_animation(self):
print("starting animation")
# set up the animation
self.ani = animation.FuncAnimation(self.fig, self.animate, init_func=self.clear_frame,
frames=100, interval=500, blit=False)
def clear_frame(self):
self.ax.clear()
self.ax.plot([], [])
def animate(self, i):
print("animate frame")
# get the current time
t_now = dt.datetime.now()
# update trace values
self.xs.append(t_now.strftime("%H:%M:%S.%f"))
self.ys.append(self.fx[i % len(self.fx)])
# keep max len(self.fx) points
if len(self.xs) > len(self.fx):
self.xs.pop(0)
self.ys.pop(0)
self.ax.clear()
self.ax.plot(self.xs, self.ys)
# need to reapply format after clearing axes
self.fig.autofmt_xdate(rotation=45)
self.fig.subplots_adjust(bottom=0.30)
self.ax.set_title('Force/Torque Sensor Data')
self.ax.set_ylabel('Fx (N)')
if __name__=='__main__':
# start a new qapplication
qapp = QtWidgets.QApplication(sys.argv)
# create our figure in the main process
strip_chart = StripChart()
strip_chart.show()
strip_chart.start_animation()
# start qt main loop
qapp.exec()
Things of note in this example:
you'll need to have a backend installed in your environment (i.e. pip install pyqt5)
I've added an init_func to the animation, you don't really need this as you can call self.ax.clear() in the animate method.
If you need better performance for your animation, you can use blit=True but you'll need to modify the clear_frame and animate methods to return the artists that you want to update (see https://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/ for more info). One drawback is that you won't be able to update the axis labels with that approach.
I've set it up to run infinitely until you close the window
I'm assuming that the reason you want to run the animation in a separate process is that there is some time consuming/CPU intensive task that is involved in either updating the graph data, or drawing all the points. Perhaps you have this embedded in some other UI?
I've tried to execute the animation in a separate process, but you need to pass the instance of the figure that's displayed. As I mentioned this isn't straightforward, although there do appear to be ways to do it (https://stackoverflow.com/a/57793267/13752965). I'll update if I find a working solution.
I am using matplotlib 1.4.3 with python 3.3. I would like to draw multiple figures with multiples sub-plot in it. I am facing some kind of bug that is really boring me:
When I use fill() and boxplot() methods whithin a figure, results of those functions are hidden if the figure is not the first one created.
This bug seems to be related somehow to polygon display and matplotlib environment state.
When a parse only one single figure, everything is working fine. When I parse multiple figures, the first one is ok. But, in every other subsequent figures, everything is all-right except wiskerbox and polygons that are hidden.
Each plot code is wrapped into a function, which accepts positional arguments, *args and **kwargs. Lets say signature are:
def myplot(t, x, *args, *kwargs):
# [...]
hFig = plt.figure()
# [...]
return hFig
As far as I understand python mechanisms, after the function call is resolved, there must be nothing alive (I do not use global variables) except what matplotlib environment has stored into its global namespace variables.
In every call, I close() my figure, I also have tried hFig.clf() in addition before leaving function, but it does not solve the problem.
Each plot is wrapped into printer (decorator) to add generic functionalities:
def myprint(func):
def inner(*args, **kwargs)
# [...]
hFig = func(*args, **kwargs)
# [...]
return inner
What I have tried so far:
Increased zscore of wiskerbox and polygons, not working;
Execute plot generation in different threads, not working;
Execute plot generation in different processes, working but I have to change my function signature because it can be pickled.
I do not want use dill and pathos, even if I would I cannot.
It looks like it is a matplotlib environment bug, because when I run different processes, this environment is recreated from scratch and it works the way it should. I would like to know if there is a way to reset matplotlib environment state within a python script. If not, what can I do for solving this issue.
Obs.: I am using GridSpecs object and subplot() method to create my figures. The problem was not present when I computed boxes myself and used add_axes() method.
Update: Here you can find a MCVE of my problem. By doing this simple example, I found the line which makes my bug happens (looks like I have old bad Matlab behaviours). It seems that plt.hold(False) alters the way of polygons and boxplot are displayed. And, as I pointed out, it was related to matplotlib global namespace variable. I just misunderstood the way this method works, and in each sub-process, it was reset.
import datetime
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gspec
def bloodyplotit(t_, x_):
hFig = plt.figure()
gs = gspec.GridSpec(1, 4, height_ratios=[1], width_ratios=[15, 2, 3, 1])
gs.update(left=0.10, right=0.90, top=0.90, bottom=0.25, hspace=0.05, wspace=0.05)
plt.hold(True)
hAxe = plt.subplot(gs[0,0])
hAxe.plot(t_, x_)
#plt.hold(False) # <------------------------ This line make the scirpt bug
hAxe = plt.subplot(gs[0,1])
hAxe.hist(x_, orientation='horizontal')
hAxe = plt.subplot(gs[0,3])
hAxe.boxplot(x_)
plt.show()
n = 1000
t = datetime.datetime.utcnow() + np.arange(n)*datetime.timedelta(minutes=1)
x = np.random.randn(1000,1)
for i in range(10):
bloodyplotit(t, x)
Here's an even more minimal script that produces the error:
x = np.random.randn(1000)
fig, ax = plt.subplots(1, 2)
ax[0].hold(True)
ax[0].boxplot(x);
ax[1].hold(False)
ax[1].boxplot(x);
As far as I can tell, this is expected behavior. According to the documentation of plt.hold,
When hold is True, subsequent plot commands will be added to the current axes. When hold is False, the current axes and figure will be cleared on the next plot command.
Boxplot is a compound object: it is created by calling multiple plotting commands. If hold is False, the axes are cleared between each of those commands, so parts of the boxplot don't show up.
Personally, I've never found a reason to toggle the hold state in 10+ years of using matplotlib. My experience is that doing it (especially globally) just causes confusion, and I'd recommend avoiding it.
I haven't used Matplotlib much. Based on someone's advice, I'm trying to write some plotting codes using object-oriented paradigms as much as possible---and therefore trying to use pure Matplotlib (i.e. not relying on pyplot) to generate some simple figures.
A stripped-down version of my code looks like this:
import matplotlib as mpl
time = [0,1,2,3,4]
cell = [1,2,1,2,1]
sample = [3,2,3,4,4]
(figHt, figWd) = (5, 8) # in
lBorderWidth = bBorderWidth = rBorderWidth = tBorderWidth = 0.1
lbwh = (lBorderWidth, bBorderWidth,
(1-lBorderWidth-rBorderWidth),
(1-tBorderWidth-bBorderWidth)) # left, bottom, width, height
fig = mpl.figure.Figure(figsize=(figHt, figWd))
ax = fig.add_axes(lbwh)
lines1, = ax.plot(time,cell,'k--')
lines2, = ax.plot(time,sample,'k-')
fig.legend([lines1,lines2],['p','q'],'upper left')
fig.canvas.draw()
But when I run it, Python complains when it reaches fig.canvas.draw() that canvas type is None.
Based on a reading of the Matplotlib Artists tutorial, it seems like pyplot takes care of a few behind-the-scenes setup tasks, most notably establishing the connection between the Figure object and the desired renderer/backend. The tutorial says:
In the example below, we create a Figure instance using matplotlib.pyplot.figure(), which is a convenience method for
instantiating Figure instances and connecting them with your user
interface or drawing toolkit FigureCanvas. As we will discuss below,
this is not necessary – you can work directly with PostScript, PDF
Gtk+, or wxPython FigureCanvas instances, instantiate your Figures
directly and connect them yourselves – but since we are focusing here
on the Artist API we’ll let pyplot handle some of those details for us
Unfortunately, that particular page doesn't proceed beyond generating plots with pyplot.figure(), so I am still trying to discover what the required steps are. Again, I realize pyplot can simplify this task---just trying to grok how all the pieces fit together.
I saw this description of a base class used by backends, FigureCanvasBase, and I assume that I need to assign fig.canvas one of FigureCanvasBase's subclasses.
Also, I verified that Python is using a default backend. So I know the problem isn't caused by lack of a backend.
>>> matplotlib.backends.backend
'Qt4Agg'
Thanks in advance for any help. In summary, two questions:
What am I missing that is causing this to fail? Is it because I didn't assign the figure object a renderer?
I mentioned that I suspected I needed a subclass of FigureCanvasBase to move forward. Even if the problem can probably be solved more elegantly, is there a way to search the Python environment for subclasses that inherit from FigureCanvasBase? This might come in handy in other problems.
You need to create a FigureCanvasAgg in order to plot manually, try this:
import matplotlib as mpl
mpl.use('Agg') #setup the backend
import matplotlib.figure as mfigure
from matplotlib.backends.backend_agg import FigureCanvasAgg #canvas
time = [0,1,2,3,4]
cell = [1,2,1,2,1]
sample = [3,2,3,4,4]
(figHt, figWd) = (5, 8) # in
lBorderWidth = bBorderWidth = rBorderWidth = tBorderWidth = 0.1
lbwh = (lBorderWidth, bBorderWidth,
(1-lBorderWidth-rBorderWidth),
(1-tBorderWidth-bBorderWidth)) # left, bottom, width, height
fig = mfigure.Figure(figsize=(figHt, figWd))
canvas = FigureCanvasAgg(fig) #create the canvas
ax = fig.add_axes(lbwh)
lines1, = ax.plot(time,cell,'k--')
lines2, = ax.plot(time,sample,'k-')
fig.legend([lines1,lines2],['p','q'],'upper left')
fig.savefig('test.png') #save the figure
Note: You can find the subclasses of FigureCanvasBase in matplotlib.backends.<your backend>.FigureCanvas<your backend>
I've kept a set of references to figures in a dictionary so that I could save them later if desired. I am troubled that the saved figures are blank if invoke a show() command and look at them first. Since the show() command blocks and I am not using a spyder-like interpreter, I have to close the figures before I get to savefig()
figures['myfig_1'] = figure()
...
figures['myfig_n'] = figure()
...
#show() #disabling this makes the problem go away
print "Saving:"
for fig in figures:
figure(figures[fig].number)
savefig(fig)
print "Figure " + str(figures[fig].number) + ": " + fig
The print statement here has given me the indication that the dictionary is still intact, which I think means that I have not lost the figure references (they are still returning meaningful numbers in their .number attribute.)
Another twist I have noticed is that when I have done a similar thing in a class, storing the dictionary as a member and dividing the store and save functions into their own methods, this does not happen. Is there something about the way I am closing the figures or storing the data which is making the figures loose their data?
Generally speaking, in cases like this don't use the interactive matlab-ish state machine interface to matplotlib. It's meant for interactive use.
You're trying to make a figure "active", and creating a new figure instead. It doesn't matter which figure is active, if you just retain the returned figure and/or axis objects and use them directly. (Also, don't use wildcard imports! You will regret it at some later point when you're maintaining your code!)
Just do something like this:
import matplotlib.pyplot as plt
figures = {}
figures['a'] = plt.figure()
ax = figures['a'].add_subplot(111)
ax.plot(range(10), 'ro-')
figures['b'] = plt.figure()
ax = figures['b'].add_subplot(111)
ax.plot(range(10), 'bo-')
plt.show()
for name, fig in figures.iteritems():
fig.savefig('figure-%s.png' % name)
From the documentation, whether or not the drawing elements are destroyed from show() depends on the backend, and the version of matplotlib. Not having the figures destroyed seems to be available with version 1.1.0. To figure out which backend is in use, use the get_backend() function. In my case, I was using the Qt4Agg backend. By invoking the TkAgg backend, with the call matplotlib.use('TkAgg') the figures were not destroyed before the save. Now to find out how to change the behavior of the Qt4Agg...
Let's say I define a few functions to do certain matplotlib actions, such as
def dostuff(ax):
ax.scatter([0.],[0.])
Now if I launch ipython, I can load these functions and start a new figure:
In [1]: import matplotlib.pyplot as mpl
In [2]: fig = mpl.figure()
In [3]: ax = fig.add_subplot(1,1,1)
In [4]: run functions # run the file with the above defined function
If I now call dostuff, then the figure does not refresh:
In [6]: dostuff(ax)
I have to then explicitly run:
In [7]: fig.canvas.draw()
To get the canvas to draw. Now I can modify dostuff to be
def dostuff(ax):
ax.scatter([0.],[0.])
ax.get_figure().canvas.draw()
This re-draws the canvas automatically. But now, say that I have the following code:
def dostuff1(ax):
ax.scatter([0.],[0.])
ax.get_figure().canvas.draw()
def dostuff2(ax):
ax.scatter([1.],[1.])
ax.get_figure().canvas.draw()
def doboth(ax):
dostuff1(ax)
dostuff2(ax)
ax.get_figure().canvas.draw()
I can call each of these functions, and the canvas will be redrawn, but in the case of doboth(), it will get redrawn multiple times.
My question is: how could I code this, such that the canvas.draw() only gets called once? In the above example it won't change much, but in more complex cases with tens of functions that can be called individually or grouped, the repeated drawing is much more obvious, and it would be nice to be able to avoid it. I thought of using decorators, but it doesn't look as though it would be simple.
Any ideas?
Why doesn't my answer to this SO question of yours about "refresh decorator" make it simple? I showed exactly what to do what you're again requesting here (by keeping a count of nestings -- incidentally, one that's also thread-safe) and you completely ignored my answer... peculiar behavior!-)