I am trying to use a python process to animate a plot as shown below:
from multiprocessing import Process
import datetime as dt
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import numpy as np
process_enabled = 1;
print("Process enabled: ", process_enabled)
x = []
y = []
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
def start_animation():
# Set up plot to call animate() function periodically
ani = animation.FuncAnimation(fig, animate, fargs=(x, y), interval=1000)
print("Called animate function")
plt.show()
# This function is called periodically from FuncAnimation
def animate(i, xs, ys):
fx=[0.045,0.02,0.0,0.04,0.015,-0.01,0.015,0.045,0.035,0.01,
0.055,0.04,0.02,0.025,0.0,-0.005,-0.005,-0.02,-0.05,-0.03] # fx values
# Add x and y to lists
xs.append(dt.datetime.now().strftime('%H:%M:%S.%f'))
if(i<len(fx)):
ys.append(fx[i])
# Draw x and y lists
ax.clear()
if(i<len(fx)):
ys_stacked = np.stack((np.array(ys),0.1+np.array(ys)),axis=1)
ax.plot(xs, ys_stacked)
print("Animating")
# Format plot
if(i<len(fx)):
plt.xticks(rotation=45, ha='right')
plt.subplots_adjust(bottom=0.30)
plt.title('Force/Torque Sensor Data')
plt.ylabel('Fx (N)')
if(process_enabled):
p_graph = Process(name='Graph', target=start_animation)
print("Created graph process")
p_graph.start()
print("Started graph process")
else:
start_animation()
When I disable the process, the start_animation() function works fine and the plot is displayed and the animation begins. However, when the process is enabled, the process starts and then the code breaks at print("Called animate function"). There is no plot window and there are no error messages in the terminal).
I'm new to both multiprocessing in python and indeed matplotlib. Any direction would be much appreciated.
Cheers,
Tony
I'm trying to solve this same problem, but haven't quite figured it out completely. However, I think I can provide a few useful comments on your question.
To start, is there any reason why you want to handle the animation in a separate process? Your approach seems to work fine within a single process. There's a number of issues you'll need to address to do this. If you truly do require a separate process, then the following might be useful.
First, you won't be able to use your global variables in the 'graph' process, as that process doesn't share the same instances of those variables (see Globals variables and Python multiprocessing).
You can share state between processes, but this is difficult for complex objects that you'd want to share (i.e. plt.figure()). See the multiprocessing reference for more information (https://docs.python.org/3/library/multiprocessing.html#sharing-state-between-processes)
One final suggestion would be to do away with the pyplot interface. This is handy for straightforward scripts and interactive data analysis, but it obfuscates a lot of important things - like knowing which figure, axis etc you're dealing with when you call plt methods.
I've provided an alternative, object-oriented approach using a custom class, that can run your animation (without a separate process):
import sys
from multiprocessing import Process, Queue
import datetime as dt
from matplotlib.figure import Figure
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg
from matplotlib.backends.qt_compat import QtWidgets
import matplotlib.animation as animation
class StripChart(FigureCanvasQTAgg):
def __init__(self):
self.fig = Figure(figsize=(8,5), dpi=100)
self.ax = self.fig.add_subplot(111)
# hold a copy of our torque data
self.fx = [0.045,0.02,0.0,0.04,0.015,-0.01,0.015,0.045,0.035,0.01,
0.055,0.04,0.02,0.025,0.0,-0.005,-0.005,-0.02,-0.05,-0.03]
super().__init__(self.fig)
# instantiate the data arrays
self.xs = []
self.ys = []
def start_animation(self):
print("starting animation")
# set up the animation
self.ani = animation.FuncAnimation(self.fig, self.animate, init_func=self.clear_frame,
frames=100, interval=500, blit=False)
def clear_frame(self):
self.ax.clear()
self.ax.plot([], [])
def animate(self, i):
print("animate frame")
# get the current time
t_now = dt.datetime.now()
# update trace values
self.xs.append(t_now.strftime("%H:%M:%S.%f"))
self.ys.append(self.fx[i % len(self.fx)])
# keep max len(self.fx) points
if len(self.xs) > len(self.fx):
self.xs.pop(0)
self.ys.pop(0)
self.ax.clear()
self.ax.plot(self.xs, self.ys)
# need to reapply format after clearing axes
self.fig.autofmt_xdate(rotation=45)
self.fig.subplots_adjust(bottom=0.30)
self.ax.set_title('Force/Torque Sensor Data')
self.ax.set_ylabel('Fx (N)')
if __name__=='__main__':
# start a new qapplication
qapp = QtWidgets.QApplication(sys.argv)
# create our figure in the main process
strip_chart = StripChart()
strip_chart.show()
strip_chart.start_animation()
# start qt main loop
qapp.exec()
Things of note in this example:
you'll need to have a backend installed in your environment (i.e. pip install pyqt5)
I've added an init_func to the animation, you don't really need this as you can call self.ax.clear() in the animate method.
If you need better performance for your animation, you can use blit=True but you'll need to modify the clear_frame and animate methods to return the artists that you want to update (see https://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/ for more info). One drawback is that you won't be able to update the axis labels with that approach.
I've set it up to run infinitely until you close the window
I'm assuming that the reason you want to run the animation in a separate process is that there is some time consuming/CPU intensive task that is involved in either updating the graph data, or drawing all the points. Perhaps you have this embedded in some other UI?
I've tried to execute the animation in a separate process, but you need to pass the instance of the figure that's displayed. As I mentioned this isn't straightforward, although there do appear to be ways to do it (https://stackoverflow.com/a/57793267/13752965). I'll update if I find a working solution.
Related
Basically what I want to do is draw a graph in real-time that updates/animates the data that is being shared across multiple processes. I have used Manager from the python multiprocessing module and shared a wave list (that is an array) that is updated/modified in another process in a while loop.
It seems that when I run the FuncAnimation function I can only update the data displayed according to local or global variables within the same process. Here is my code:
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
x = np.arange(0, 4408)
wave = np.arange(0, 4408)
def animate(i):
plt.cla()
global x
x = [a + 4408 for a in x] # x-axis continues to shift
plt.plot(x, wave)
# wave array from another process
def graph(shared_wave):
global wave
wave = shared_wave # Here it gets updated once but never again
ani = FuncAnimation(plt.gcf(), animate, interval=100)
plt.tight_layout()
plt.show()
Any help would be greatly appreciated. One thing I have been thinking about is using multithreading in one process to run the animation in one thread and another thread to update a global variable.
I have a Python program that generates graphs using matplotlib. I am trying to get the program to generate a bunch of plots in one program run (the user is asked if they want to generate another graph) all in separate windows. Any way I can do this?
To generate a new figure, you can add plt.figure() before any plotting that your program does.
import matplotlib.pyplot as plt
import numpy as np
def make_plot(slope):
x = np.arange(1,10)
y = slope*x+3
plt.figure()
plt.plot(x,y)
make_plot(2)
make_plot(3)
Using the latest matlibplot, I found the following to work for my purposes:
# create figure (will only create new window if needed)
plt.figure()
# Generate plot1
plt.plot(range(10, 20))
# Show the plot in non-blocking mode
plt.show(block=False)
# create figure (will only create new window if needed)
plt.figure()
# Generate plot2
plt.plot(range(10, 20))
# Show the plot in non-blocking mode
plt.show(block=False)
...
# Finally block main thread until all plots are closed
plt.show()
The easiest way to ensure all of your lines go to the correct figure window is something like:
from six.moves import input
import matplotlib.pyplot as plt
another = True
while another:
fig, ax = plt.subplots()
ax.plot(range(5))
fig.canvas.manager.show()
# this makes sure that the gui window gets shown
# if this is needed depends on rcparams, this is just to be safe
fig.canvas.flush_events()
# this make sure that if the event loop integration is not
# set up by the gui framework the plot will update
another = bool(input("would you like another? "))
If you want to run this with a non-gui backend you will need to drop the flush_events call or wrap it in a try: ... except NotImplementedError. Much of this complication is defensive programming because GUIs can be difficult and the behavior of this code may be dependent on many factors which are not obvious from the code shown.
Using the implicit axes of pyplot can cause problems as the 'current axes' is set by the last axes the user clicked on. You should really only use pyplot when interactively typing at the rpel and almost never (other than plt.subplots) in scripts/programs.
Use the .figure() function to create a new window, the following code makes two windows:
import matplotlib.pyplot as plt
plt.plot(range(10)) # Creates the plot. No need to save the current figure.
plt.draw() # Draws, but does not block
plt.figure() # New window, if needed. No need to save it, as pyplot uses the concept of current figure
plt.plot(range(10, 20))
plt.draw()
You can repeat this as many times as you want
I am using Matplotlib on MacOS with Sulime Text.
I use Python 3.5 and Matplotlib 2.0.
When I work on a figure, I usually have a script that plot the data, and save the figure in a .pdf file with plt.savefig(). Then I use Skim (a pdf viewer) in order to refresh the file each time I modify and run the script. This allows me to set my working layout as clean as: there is one window for the script, and one window for the figure which is automatically refreshing.
I would like to do keep the same layout, but using the Matplotlib figures (because they are interactive). I am looking for a way to use plt.show() but always in the same figure that has been created the first time I've run the script.
For instance:
1. First run
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.figure()
noise = np.random.rand(1, 100)
ax(noise)
plt.show()
2. Following runs
import matplotlib.pyplot as plt
import numpy as np
# This is the super command I am looking for
fig = plt.get_previous_run_figure()
ax = fig.axes
noise = np.random.rand(1, 100)
ax.plot(noise)
plt.draw()
In that case of course, I would have to do a first-run script separately from the main script. Does anyone know if it is possible ?
You want to have multiple consecutive python sessions share a common Matplotlib window. I see no way to share this windows from separate processes, especially when the original owner may terminate at any point in time.
However, you could do something similar to your current workflow in which you have an external pdf viewer to view a output file which you update from multiple python instances.
See this question/answer on how to pickle a matplotlib figure:
Store and reload matplotlib.pyplot object
In every script, output your matplotlib figure as a pickled object, rather than calling plt.show():
import matplotlib.pyplot as plt
import numpy as np
import pickle
ax = plt.subplot(111)
x = np.linspace(0, 10)
y = np.exp(x)
plt.plot(x, y)
pickle.dump(ax, file('myplot.pickle', 'w'))
Then, start a dedicated python session which loads this pickled object and calls plt.show(). Have this script run in a loop, checking for updates of the pickled file on disk, and reloading when necessary:
import matplotlib.pyplot as plt
import pickle
while True:
ax = pickle.load(file('myplot.pickle'))
plt.show()
Alternative
Instead of having separate python sessions, I usually have a single Ipython session in which I run different script. By selecting the same figure windows, I end up with a mostly similar setup as you describe, in which the same figure window is reused throughout the day.
import matplotlib.pyplot as plt
fig = plt.figure(0)
fig.clf()
plt.show()
In principle establishing a connection between two different scripts could be done using a system-wide clipboard. (As far as I know the clipboard in windows and macos are system-wide.)
So the idea can be to set up an application using tk or pyqt, and implement a generic FigureCanvas. This application could have an event listener for changes in the clipboard.
The other main workflow script would then call some function that wraps the current figure into a pickle object and sends it to the clipboard, from where it gets caught by the GUI application, is unpickled and shown in the canvas.
This sounds like a little bit of work, but should meet the very restrictive requirements from the question.
The alternative from Daan worked for me. Here's a bit more code. I used this in a Tkinter interactive GUI for reusing/updating a matplotlib figure window:
fig1 = None
if fig1:
#if exists, clear figure 1
plt.figure(1).clf()
plt.suptitle("New Fig Title", fontsize=18, fontweight='bold')
#reuse window of figure 1 for new figure
fig1 = plt.scatter(points.T[0], points.T[1], color='red', **plot_kwds)
else:
#initialize
fig1 = plt.figure(num=1,figsize=(7, int(7*imgRatio)), dpi=80)
plt.tick_params(axis='both', which='major', labelsize=14)
plt.tick_params(axis='both', which='minor', labelsize=14)
plt.suptitle("Title", fontsize=18, fontweight='bold')
fig1 = plt.scatter(points.T[0], points.T[1], color='red', **plot_kwds)
The figure is reusing the (interactive) plt window. For this to work, I had to set interactive : True in the matplotlibrc file (see my comment here)
I'm trying to create a live plot which updates as more data is available.
import os,sys
import matplotlib.pyplot as plt
import time
import random
def live_plot():
fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_xlabel('Time (s)')
ax.set_ylabel('Utilization (%)')
ax.set_ylim([0, 100])
ax.set_xlim(left=0.0)
plt.ion()
plt.show()
start_time = time.time()
traces = [0]
timestamps = [0.0]
# To infinity and beyond
while True:
# Because we want to draw a line, we need to give it at least two points
# so, we pick the last point from the previous lists and append the
# new point to it. This should allow us to create a continuous line.
traces = [traces[-1]] + [random.randint(0, 100)]
timestamps = [timestamps[-1]] + [time.time() - start_time]
ax.set_xlim(right=timestamps[-1])
ax.plot(timestamps, traces, 'b-')
plt.draw()
time.sleep(0.3)
def main(argv):
live_plot()
if __name__ == '__main__':
main(sys.argv)
The above code works. However, I'm unable to interact with the window generated by plt.show()
How can I plot live data while still being able to interact with the plot window?
Use plt.pause() instead of time.sleep().
The latter simply holds execution of the main thread and the GUI event loop does not run. Instead, plt.pause runs the event loop and allows you to interact with the figure.
From the documentation:
Pause for interval seconds.
If there is an active figure it will be updated and displayed, and the
GUI event loop will run during the pause.
If there is no active figure, or if a non-interactive backend is in
use, this executes time.sleep(interval).
Note
The event loop that allows you to interact with the figure only runs during the pause period. You will not be able to interact with the figure during computations. If the computations take a long time (say 0.5s or more) the interaction will feel "laggy". In that case it may make sense to let the computations run in a dedicated worker thread or process.
I'm trying to get real-time spectrum analyzer type plot in matplotlib. I've got some code working (with help from other posts on StackOverflow) as follows:
import time
import numpy as np
import matplotlib.pyplot as plt
plt.axis([0, 1000, 0, 1])
plt.ion()
plt.show()
i=0
np.zeros([1,500],'float')
lines=plt.plot(y[0])
while 1:
i=i+1
lines.pop(0).remove()
y = np.random.rand(1,100)
lines=plt.plot(y[0])
plt.draw()
The code works and I'm getting what I want, but there is a serious problem. The plot window would freeze after some time. I know the program is still running by inspecting the i variable (I'm running the code in Anaconda/Spyder so I can see the variables). However the plot window would show "Non responding" and if I terminate the python program in Spyder by ctrl+c, the plot window comes back to life and show the latest plot.
I'm out of wits here as how to further debug the issue. Anyone to help?
Thanks
I am not sure that adding plt.pause will entirely solve your issue. It may just take longer before the application crash. The memory used by your application seems to constantly increase over time (even after adding plt.pause). Below are two suggestions that may help you with your current issue:
Instead of removing/recreating the lines artists with each iteration with remove and plot, I would use the same artist throughout the whole animation and simply update its ydata.
I'll use explicit handlers for the axe and figure and call show and draw explicitly on the figure manager and canvas instead of going with implicit calls through pyplot, following the advices given in a post by tcaswell.
Following the above, the code would look something like this:
import numpy as np
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([0, 100, 0, 1])
y = np.random.rand(100)
lines = ax.plot(y)
fig.canvas.manager.show()
i=0
while 1:
i=i+1
y = np.random.rand(100)
lines[0].set_ydata(y)
fig.canvas.draw()
fig.canvas.flush_events()
I've run the above code for a good 10 minutes and the memory used by the application remained stable the whole time, while the memory used by your current code (without plt.pause) increased by about 30MiB over the same period.
To answer myself, I solved the issue by adding
plt.pause(0.01)
after the
plt.draw()
This probably allows the GUI to finish the drawing and clear the buffer somewhere (my guess) before the new data comes in.
I know I'm late to answer this question, but for your issue you could look into the "joystick" package. It is based on the line.set_data() and canvas.draw() methods, with optional axes re-scaling, hence most probably faster than removing a line and adding a new one. It also allows for interactive text logging or image plotting (in addition to graph plotting).
No need to do your own loops in a separate thread, the package takes care of it, just give the update frequency you wish. Plus the terminal remains available for more monitoring commands while live plotting, which is not possible with a "while True" loop.
See http://www.github.com/ceyzeriat/joystick/ or https://pypi.python.org/pypi/joystick (use pip install joystick to install)
try:
import joystick as jk
import numpy as np
import time
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the doc
"""
self._t0 = time.time() # initialize time
self.xdata = np.array([self._t0]) # time x-axis
self.ydata = np.array([0.0]) # fake data y-axis
# create a graph frame
self.mygraph = self.add_frame(jk.Graph(name="test", size=(500, 500), pos=(50, 50), fmt="go-", xnpts=100, xnptsmax=1000, xylim=(None, None, 0, 1)))
#_infinite_loop(wait_time=0.2)
def _generate_data(self): # function looped every 0.2 second to read or produce data
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.2 seconds
"""
# concatenate data on the time x-axis
self.xdata = jk.core.add_datapoint(self.xdata, time.time(), xnptsmax=self.mygraph.xnptsmax)
# concatenate data on the fake data y-axis
self.ydata = jk.core.add_datapoint(self.ydata, np.random.random(), xnptsmax=self.mygraph.xnptsmax)
self.mygraph.set_xydata(t, self.ydata)
t = test()
t.start()
t.stop()