Develop Reference

Updating matplotlib figures in real time for data acquisition

I want to plot data in matplotlib in real time. I want to open a figure once at the start of the programme, then update the figure when new data is acquired. Despite there being a few similar questions out there, none quite answer my specific question.
I want each set of data points new_data1 and new_data2 to be plotted on the same figure at the end of each while loop i.e. one line after the first while loop, two lines on the same figure after the second while loop etc. Currently they are all plotted together, but only right at the end of the programme, which is no use for real time data acquisition.
import matplotlib.pyplot as plt
import numpy
hl, = plt.plot([], [])
def update_line(hl, new_datax, new_datay):
hl.set_xdata(numpy.append(hl.get_xdata(), new_datax))
hl.set_ydata(numpy.append(hl.get_ydata(), new_datay))
plt.xlim(0, 50)
plt.ylim(0,200)
plt.draw()
x = 1
while x < 5:
new_data1 = []
new_data2 = []
for i in range(500):
new_data1.append(i * x)
new_data2.append(i ** 2 * x)
update_line(hl, new_data1, new_data2)
x += 1
else:
print("DONE")
This programme plots all 5 lines, but at the end of the programme. I want each line to be plotted after one another, after the while loop is completed. I have tried putting in plt.pause(0.001) in the function, but it has not worked.
This programme is different from the one that has been put forward - that programme only plots one graph and does not update with time.

If I correctly understood your specifications, you can modify just a bit your MWE as follows:
import matplotlib.pyplot as plt
import numpy
fig = plt.figure(figsize=(11.69,8.27))
ax = fig.gca()
ax.set_xlim(0, 50)
ax.set_ylim(0,200)
hl, = plt.plot([], [])
def update_line(hl, new_datax, new_datay):
# re initialize line object each time if your real xdata is not contiguous else comment next line
hl, = plt.plot([], [])
hl.set_xdata(numpy.append(hl.get_xdata(), new_datax))
hl.set_ydata(numpy.append(hl.get_ydata(), new_datay))
fig.canvas.draw_idle()
fig.canvas.flush_events()
x = 1
while x < 10:
new_data1 = []
new_data2 = []
for i in range(500):
new_data1.append(i * x)
new_data2.append(i ** 2 * x)
update_line(hl, new_data1, new_data2)
# adjust pause duration here
plt.pause(0.5)
x += 1
else:
print("DONE")
which displays :

Not sure, if I am reading the requirements right but below is a blueprint. Please change it to suit your requirements. You may want to change the function Redraw_Function and edit the frames (keyword parameter, which is np.arange(1,5,1) ) in the FuncAnimation call. Also interval=1000 means 1000 milliseconds of delay.
If you are using Jupyter then comment out the second last line (where it says plt.show()) and uncomment the last line. This will defeat your purpose of real time update but I am sorry I had trouble making it work real time in Jupyter. However if you are using python console or official IDLE please run the code as it is. It should work nicely.
import numpy as np
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
fig, ax = plt.subplots()
plot, = plt.plot([],[])
def init_function():
ax.set_xlim(0,50)
ax.set_ylim(0,250)
return plot,
def Redraw_Function(UpdatedVal):
new_x = np.arange(500)*UpdatedVal
new_y = np.arange(500)**2*UpdatedVal
plot.set_data(new_x,new_y)
return plot,
Animated_Figure = FuncAnimation(fig,Redraw_Function,init_func=init_function,frames=np.arange(1,5,1),interval=1000)
plt.show()
# Animated_Figure.save('MyAnimated.gif',writer='imagemagick')
When you run the code, you obtain the below result. I tried to keep very little code but I am sorry, if your requirement was totally different.
Best Wishes,

Replacing part of a plot with a dotted line

I would like to replace part of my plot where the function dips down to '-1' with a dashed line carrying on from the previous point (see plots below).
Here's some code I've written, along with its output:
import numpy as np
import matplotlib.pyplot as plt
y = [5,6,8,3,5,7,3,6,-1,3,8,5]
plt.plot(np.linspace(1,12,12),y,'r-o')
plt.show()
for i in range(1,len(y)):
if y[i]!=-1:
plt.plot(np.linspace(i-1,i,2),y[i-1:i+1],'r-o')
else:
y[i]=y[i-1]
plt.plot(np.linspace(i-1,i,2),y[i-1:i+1],'r--o')
plt.ylim(-1,9)
plt.show()
Here's the original plot
Modified plot:
The code I've written works (it produces the desired output), but it's inefficient and takes a long time when I actually run it on my (much larger) dataset. Is there a smarter way to go about doing this?

You can achieve something similar without the loops:
import pandas as pd
import matplotlib.pyplot as plt
# Create a data frame from the list
a = pd.DataFrame([5,6,-1,-1, 8,3,5,7,3,6,-1,3,8,5])
# Prepare a boolean mask
mask = a > 0
# New data frame with missing values filled with the last element of
# the previous segment. Choose 'bfill' to use the first element of
# the next segment.
a_masked = a[mask].fillna(method = 'ffill')
# Prepare the plot
fig, ax = plt.subplots()
line, = ax.plot(a_masked, ls = '--', lw = 1)
ax.plot(a[mask], color=line.get_color(), lw=1.5, marker = 'o')
plt.show()
You can also highlight the negative regions by choosing a different colour for the lines:
My answer is based on a great post from July, 2017. The latter also tackles the case when the first element is NaN or in your case a negative number:
Dotted lines instead of a missing value in matplotlib

I would use numpy functionality to cut your line into segments and then plot all solid and dashed lines separately. In the example below I added two additional -1s to your data to see that this works universally.
import numpy as np
import matplotlib.pyplot as plt
Y = np.array([5,6,-1,-1, 8,3,5,7,3,6,-1,3,8,5])
X = np.arange(len(Y))
idxs = np.where(Y==-1)[0]
sub_y = np.split(Y,idxs)
sub_x = np.split(X,idxs)
fig, ax = plt.subplots()
##replacing -1 values and plotting dotted lines
for i in range(1,len(sub_y)):
val = sub_y[i-1][-1]
sub_y[i][0] = val
ax.plot([sub_x[i-1][-1], sub_x[i][0]], [val, val], 'r--')
##plotting rest
for x,y in zip(sub_x, sub_y):
ax.plot(x, y, 'r-o')
plt.show()
The result looks like this:
Note, however, that this will fail if the first value is -1, as then your problem is not well defined (no previous value to copy from). Hope this helps.

Not too elegant, but here's something that doesn't use loops which I came up with (based on the above answers) which works. #KRKirov and #Thomas Kühn , thank you for your answers, I really appreciate them
import pandas as pd
import matplotlib.pyplot as plt
# Create a data frame from the list
a = pd.DataFrame([5,6,-1,-1, 8,3,5,7,3,6,-1,3,8,5])
b=a.copy()
b[2]=b[0].shift(1,axis=0)
b[4]=(b[0]!=-1) & (b[2]==-1)
b[5]=b[4].shift(-1,axis=0)
b[0] = (b[5] | b[4])
c=b[0]
d=pd.DataFrame(c)
# Prepare a boolean mask
mask = a > 0
# New data frame with missing values filled with the last element of
# the previous segment. Choose 'bfill' to use the first element of
# the next segment.
a_masked = a[mask].fillna(method = 'ffill')
# Prepare the plot
fig, ax = plt.subplots()
line, = ax.plot(a_masked, 'b:o', lw = 1)
ax.plot(a[mask], color=line.get_color(), lw=1.5, marker = 'o')
ax.plot(a_masked[d], color=line.get_color(), lw=1.5, marker = 'o')
plt.show()

matplotlib one legend entry too much

I am trying to do an errorplot with different marker-colors in python 2.7. Additionally I am including to line plots.
I found a way here: matplotlib errorbar plot - using a custom colormap using a scatter plot for the colors and errorbar() for the bars.
As you can see in my example code, in the legend I always get one entry too much (just at the top). I cannot figure out, why. Tried to exclude it, which did not work. Did not find something helpful either, as I cannot really call the first legend entry.
Any ideas?
Here's my code:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
data = pd.DataFrame()
data['x'] = range(10)
data['y'] = data.x
data['err'] = .5
data['col'] = np.where(data.y<5,'r','b')
### setup 1-1 line
lin = pd.DataFrame() # setting 1-1 line
lin['x'] = range(10)
lin['y'] = range(10)
### setup 1-2 line
lin['x2'] = lin.x
lin['y2'] = lin.y
plt.errorbar(data.x, data.y, yerr = data.err, \
xerr = .3, fmt=' ', markersize=4, zorder = 1)
plt.scatter(data.x,data.y, marker='o', color = data.col, zorder = 2)
plt.plot(lin.x,lin.y,'g-')
plt.plot(lin.x2,1.8*lin.y2,'r-')
plt.legend(['','1-1 line', '1-1.8 line','holla','molla'], loc=4)
What I get is:
Thanks for your help!

To clean this whole thing up, I post a proper answer instead of comments.
The problem could be solved by upgrading matplotlib from 1.3.1 to 1.5.1. Easy as that.

Remove axis scale

I've spent some time fruitlessly searching for an answer to my question, so I think a new question is in order. Consider this plot:
The axes labels use scientific notation. On the y-axis, all is well. However, I have tried and failed to get rid off the scaling factor that Python added in the lower-right corner. I would like to either remove this factor completely and simply indicate it by the units in the axis title or have it multiplied to every tick label. Everything would look better than this ugly 1e14.
Here's the code:
import numpy as np data_a = np.loadtxt('exercise_2a.txt')
import matplotlib as mpl
font = {'family' : 'serif',
'size' : 12}
mpl.rc('font', **font)
import matplotlib.pyplot as plt
fig = plt.figure()
subplot = fig.add_subplot(1,1,1)
subplot.plot(data_a[:,0], data_a[:,1], label='$T(t)$', linewidth=2)
subplot.set_yscale('log')
subplot.set_xlabel("$t[10^{14}s]$",fontsize=14)
subplot.set_ylabel("$T\,[K]$",fontsize=14)
plt.xlim(right=max(data_a [:,0]))
plt.legend(loc='upper right')
plt.savefig('T(t).pdf', bbox_inches='tight')
Update: Incorporating Will's implementation of scientificNotation into my script, the plot now looks like
Much nicer if you ask me. Here's the complete code for anyone wanting to adopt some part of it:
import numpy as np
data = np.loadtxt('file.txt')
import matplotlib as mpl
font = {'family' : 'serif',
'size' : 16}
mpl.rc('font', **font)
import matplotlib.pyplot as plt
fig = plt.figure()
subplot = fig.add_subplot(1,1,1)
subplot.plot(data[:,0], data[:,1], label='$T(t)$', linewidth=2)
subplot.set_yscale('log')
subplot.set_xlabel("$t[s]$",fontsize=20)
subplot.set_ylabel("$T\,[K]$",fontsize=20)
plt.xlim(right=max(data [:,0]))
plt.legend(loc='upper right')
def scientificNotation(value):
if value == 0:
return '0'
else:
e = np.log10(np.abs(value))
m = np.sign(value) * 10 ** (e - int(e))
return r'${:.0f} \cdot 10^{{{:d}}}$'.format(m, int(e))
formatter = mpl.ticker.FuncFormatter(lambda x, p: scientificNotation(x))
plt.gca().xaxis.set_major_formatter(formatter)
plt.savefig('T(t).pdf', bbox_inches='tight', transparent=True)

Just divide the x-values by 1e14:
subplot.plot(data_a[:,0] / 1e14, data_a[:,1], label='$T(t)$', linewidth=2)
If you want to add the label to each individual tick, you'll have to provide a custom formatter, like in tom's answer.
If you want it to look like as nice as the ticks on your y-axis, you could provide a function to format it with LaTeX:
def scientificNotation(value):
if value == 0:
return '0'
else:
e = np.log10(np.abs(value))
m = np.sign(value) * 10 ** (e - int(e))
return r'${:.0f} \times 10^{{{:d}}}$'.format(m, int(e))
# x is the tick value; p is the position on the axes.
formatter = mpl.ticker.FuncFormatter(lambda x, p: scientificNotation(x))
plt.gca().xaxis.set_major_formatter(formatter)
Of course, this will clutter your x-axis up quite a bit, so you might end up needing to display them at an angle, for example.

You can also change the tick formatter with the ticker module.
An example would be to use a FormatStrFormatter:
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
fig,ax = plt.subplots()
ax.semilogy(np.linspace(0,5e14,50),np.logspace(3,7,50),'b-')
ax.xaxis.set_major_formatter(ticker.FormatStrFormatter('%.0e'))
Also see the answers here with lots of good ideas for ways to solve this.

In addition to the good answer from Will Vousden, you can set what you write in your ticks with:
plt.xticks(range(6), range(6))
the first range(6) is the location and the second is the label.

Get the list of figures in matplotlib

I would like to:
pylab.figure()
pylab.plot(x)
pylab.figure()
pylab.plot(y)
# ...
for i, figure in enumerate(pylab.MagicFunctionReturnsListOfAllFigures()):
figure.savefig('figure%d.png' % i)
What is the magic function that returns a list of current figures in pylab?
Websearch didn't help...

Pyplot has get_fignums method that returns a list of figure numbers. This should do what you want:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(100)
y = -x
plt.figure()
plt.plot(x)
plt.figure()
plt.plot(y)
for i in plt.get_fignums():
plt.figure(i)
plt.savefig('figure%d.png' % i)

The following one-liner retrieves the list of existing figures:
import matplotlib.pyplot as plt
figs = list(map(plt.figure, plt.get_fignums()))

Edit: As Matti Pastell's solution shows, there is a much better way: use plt.get_fignums().
import numpy as np
import pylab
import matplotlib._pylab_helpers
x=np.random.random((10,10))
y=np.random.random((10,10))
pylab.figure()
pylab.plot(x)
pylab.figure()
pylab.plot(y)
figures=[manager.canvas.figure
for manager in matplotlib._pylab_helpers.Gcf.get_all_fig_managers()]
print(figures)
# [<matplotlib.figure.Figure object at 0xb788ac6c>, <matplotlib.figure.Figure object at 0xa143d0c>]
for i, figure in enumerate(figures):
figure.savefig('figure%d.png' % i)

This should help you (from the pylab.figure doc):
call signature::
figure(num=None, figsize=(8, 6),
dpi=80, facecolor='w', edgecolor='k')
Create a new figure and return a
:class:matplotlib.figure.Figure
instance. If num = None, the
figure number will be incremented and
a new figure will be created.** The
returned figure objects have a
number attribute holding this number.
If you want to recall your figures in a loop then a good aproach would be to store your figure instances in a list and to call them in the loop.
>> f = pylab.figure()
>> mylist.append(f)
etc...
>> for fig in mylist:
>> fig.savefig()

Assuming you haven't manually specified num in any of your figure constructors (so all of your figure numbers are consecutive) and all of the figures that you would like to save actually have things plotted on them...
import matplotlib.pyplot as plt
plot_some_stuff()
# find all figures
figures = []
for i in range(maximum_number_of_possible_figures):
fig = plt.figure(i)
if fig.axes:
figures.append(fig)
else:
break
Has the side effect of creating a new blank figure, but better if you don't want to rely on an unsupported interface

I tend to name my figures using strings rather than using the default (and non-descriptive) integer. Here is a way to retrieve that name and save your figures with a descriptive filename:
import matplotlib.pyplot as plt
figures = []
figures.append(plt.figure(num='map'))
# Make a bunch of figures ...
assert figures[0].get_label() == 'map'
for figure in figures:
figure.savefig('{0}.png'.format(figure.get_label()))