Develop Reference

Change colour scheme label to log scale without changing the axis in matplotlib

I am quite new to python programming. I have a script with me that plots out a heat map using matplotlib. Range of X-axis value = (-180 to +180) and Y-axis value =(0 to 180). The 2D heatmap colours areas in Rainbow according to the number of points occuring in a specified area in the x-y graph (defined by the 'bin' (see below)).
In this case, x = values_Rot and y = values_Tilt (see below for code).
As of now, this script colours the 2D-heatmap in the linear scale. How do I change this script such that it colours the heatmap in the log scale? Please note that I only want to change the heatmap colouring scheme to log-scale, i.e. only the number of points in a specified area. The x and y-axis stay the same in linear scale (not in logscale).
A portion of the code is here.
rot_number = get_header_number(headers, AngleRot)
tilt_number = get_header_number(headers, AngleTilt)
psi_number = get_header_number(headers, AnglePsi)
values_Rot = []
values_Tilt = []
values_Psi = []
for line in data:
try:
values_Rot.append(float(line.split()[rot_number]))
values_Tilt.append(float(line.split()[tilt_number]))
values_Psi.append(float(line.split()[psi_number]))
except:
print ('This line didnt work, it may just be a blank space. The line is:' + line)
# Change the values here if you want to plot something else, such as psi.
# You can also change how the data is binned here.
plt.hist2d(values_Rot, values_Tilt, bins=25,)
plt.colorbar()
plt.show()
plt.savefig('name_of_output.png')

You can use a LogNorm for the colors, using plt.hist2d(...., norm=LogNorm()). Here is a comparison.
To have the ticks in base 2, the developers suggest adding the base to the LogLocator and the LogFormatter. As in this case the LogFormatter seems to write the numbers with one decimal (.0), a StrMethodFormatter can be used to show the number without decimals. Depending on the range of numbers, sometimes the minor ticks (shorter marker lines) also get a string, which can be suppressed assigning a NullFormatter for the minor colorbar ticks.
Note that base 2 and base 10 define exactly the same color transformation. The position and the labels of the ticks are different. The example below creates two colorbars to demonstrate the different look.
import matplotlib.pyplot as plt
from matplotlib.ticker import NullFormatter, StrMethodFormatter, LogLocator
from matplotlib.colors import LogNorm
import numpy as np
from copy import copy
# create some toy data for a standalone example
values_Rot = np.random.randn(100, 10).cumsum(axis=1).ravel()
values_Tilt = np.random.randn(100, 10).cumsum(axis=1).ravel()
fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(15, 4))
cmap = copy(plt.get_cmap('hot'))
cmap.set_bad(cmap(0))
_, _, _, img1 = ax1.hist2d(values_Rot, values_Tilt, bins=40, cmap='hot')
ax1.set_title('Linear norm for the colors')
fig.colorbar(img1, ax=ax1)
_, _, _, img2 = ax2.hist2d(values_Rot, values_Tilt, bins=40, cmap=cmap, norm=LogNorm())
ax2.set_title('Logarithmic norm for the colors')
fig.colorbar(img2, ax=ax2) # default log 10 colorbar
cbar2 = fig.colorbar(img2, ax=ax2) # log 2 colorbar
cbar2.ax.yaxis.set_major_locator(LogLocator(base=2))
cbar2.ax.yaxis.set_major_formatter(StrMethodFormatter('{x:.0f}'))
cbar2.ax.yaxis.set_minor_formatter(NullFormatter())
plt.show()
Note that log(0) is minus infinity. Therefore, the zero values in the left plot (darkest color) are left empty (white background) on the plot with the logarithmic color values. If you just want to use the lowest color for these zeros, you need to set a 'bad' color. In order not the change a standard colormap, the latest matplotlib versions wants you to first make a copy of the colormap.
PS: When calling plt.savefig() it is important to call it before plt.show() because plt.show() clears the plot.
Also, try to avoid the 'jet' colormap, as it has a bright yellow region which is not at the extreme. It may look nice, but can be very misleading. This blog article contains a thorough explanation. The matplotlib documentation contains an overview of available colormaps.
Note that to compare two plots, plt.subplots() needs to be used, and instead of plt.hist2d, ax.hist2d is needed (see this post). Also, with two colorbars, the elements on which the colorbars are based need to be given as parameter. A minimal change to your code would look like:
from matplotlib.ticker import NullFormatter, StrMethodFormatter, LogLocator
from matplotlib.colors import LogNorm
from matplotlib import pyplot as plt
from copy import copy
# ...
# reading the data as before
cmap = copy(plt.get_cmap('magma'))
cmap.set_bad(cmap(0))
plt.hist2d(values_Rot, values_Tilt, bins=25, cmap=cmap, norm=LogNorm())
cbar = plt.colorbar()
cbar.ax.yaxis.set_major_locator(LogLocator(base=2))
cbar.ax.yaxis.set_major_formatter(StrMethodFormatter('{x:.0f}'))
cbar.ax.yaxis.set_minor_formatter(NullFormatter())
plt.savefig('name_of_output.png') # needs to be called prior to plt.show()
plt.show()

Using percentiles of a timeseries to set colour gradient in Python's matplotlib

I have a time series which will have over 10,000 daily values of a variable over the course of a year array size (365, 10000). Because I will have so much data (many time series for many variables), I was hoping to save only the percentiles (0, 10, 20,..., 90, 100) and use these later in plots to set a color gradient showing the density of values (obviously being darkest at the median and lightest at the min and max). The purpose of this is to avoid excessive file sizes in the saved simulation outputs, since I'll have millions of outputs to process. This would reduce the file sizes significantly if I can get it to work.
I was able to compute the percentiles of a sample data set (just using 50 values for now) and plot them as shown in the attached figure (using an array with size 365,11). How would I use this information to then set up a plot showing the colour gradient (or density of values)? Is this possible? Or is there some other way of going about it? I'm using matplotlib...
import numpy as np
import matplotlib.pyplot as plt
SampleData=(375-367)*np.random.random_sample((365, 50))+367
SDist=np.zeros((365,11))
for i in range(11):
for t in range(365):
SDist[t,i]=np.percentile(SampleData[t,:],i*10)
fig, (ax1) = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(8,4))
ax1.plot(np.arange(0,365,1), SDist)
ax1.set_title("SampleData", fontsize=15)
ax1.tick_params(labelsize=11.5)
ax1.set_xlabel('Day', fontsize=14)
ax1.set_ylabel('SampleData', fontsize=14)
fig.tight_layout()
EDIT
Here is a good example of what I'm going for (though obviously it will look different with my sample data) - I think it's similar to a fan chart:

You can use a matplotlib cm object to get the colormaps and manually calculate the color to plot based on a value. The below example calculates the color to plot based on line index (0-11). However, you can calculate the color based on anything, such as number of observations used to calculate the percentile, so long as you plot them individually and call the correct color value.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
n = 11 # change this value for the number of iterations/percentiles
colormap = cm.Blues # change this for the colormap of choice
percentiles = np.linspace(0,100,n)
SampleData=(375-367)*np.random.random_sample((365, 50))+367
SDist=np.zeros((365,n))
for i in range(n):
for t in range(365):
SDist[t,i]=np.percentile(SampleData[t,:],percentiles[i])
half = int((n-1)/2)
fig, (ax1) = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(8,4))
ax1.plot(np.arange(0,365,1), SDist[:,half],color='k')
for i in range(half):
ax1.fill_between(np.arange(0,365,1), SDist[:,i],SDist[:,-(i+1)],color=colormap(i/half))
ax1.set_title("SampleData", fontsize=15)
ax1.tick_params(labelsize=11.5)
ax1.set_xlabel('Day', fontsize=14)
ax1.set_ylabel('SampleData', fontsize=14)
fig.tight_layout()
The result should look like this:

fill_between ended up solving the problem:
import numpy as np
import matplotlib.pyplot as plt
SampleData=(375-367)*np.random.random_sample((365, 50))+367
SDist=np.zeros((365,11))
for i in range(11):
for t in range(365):
SDist[t,i]=np.percentile(SampleData[t,:],i*10)
x=np.arange(0,365,1)
fig, (ax1) = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(8,4))
ax1.set_color_cycle(['red'])
ax1.plot(x, SDist[:,5])
for i in range(6):
alph=0.05+(i/10.)
ax1.fill_between(x, SDist[:,0+i], SDist[:,10-i], color="red", alpha=alph)
ax1.set_title("SampleData", fontsize=15)
ax1.tick_params(labelsize=11.5)
ax1.set_xlabel('Day', fontsize=14)
ax1.set_ylabel('SampleData', fontsize=14)
fig.tight_layout()

How to plot a Spectrogram with very small values? [duplicate]

I am using matplotlib.pyplot.specgram and matplotlib.pyplot.pcolormesh to make spectrogram plots of a seismic signal.
Background information -The reason for using pcolormesh is that I need to do arithmitic on the spectragram data array and then replot the resulting spectrogram (for a three-component seismogram - east, north and vertical - I need to work out the horizontal spectral magnitude and divide the vertical spectra by the horizontal spectra). It is easier to do this using the spectrogram array data than on individual amplitude spectra
I have found that the plots of the spectrograms after doing my arithmetic have unexpected values. Upon further investigation it turns out that the spectrogram plot made using the pyplot.specgram method has different values compared to the spectrogram plot made using pyplot.pcolormesh and the returned data array from the pyplot.specgram method. Both plots/arrays should contain the same values, I cannot work out why they do not.
Example:
The plot of
plt.subplot(513)
PxN, freqsN, binsN, imN = plt.specgram(trN.data, NFFT = 20000, noverlap = 0, Fs = trN.stats.sampling_rate, detrend = 'mean', mode = 'magnitude')
plt.title('North')
plt.xlabel('Time [s]')
plt.ylabel('Frequency [Hz]')
plt.clim(0, 150)
plt.colorbar()
#np.savetxt('PxN.txt', PxN)
looks different to the plot of
plt.subplot(514)
plt.pcolormesh(binsZ, freqsZ, PxN)
plt.clim(0,150)
plt.colorbar()
even though the "PxN" data array (that is, the spectrogram data values for each segment) is generated by the first method and re-used in the second.
Is anyone aware why this is happening?
P.S. I realise that my value for NFFT is not a square number, but it's not important at this stage of my coding.
P.P.S. I am not aware of what the "imN" array (fourth returned variable from pyplot.specgram) is and what it is used for....

First off, let's show an example of what you're describing so that other folks
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1)
# Brownian noise sequence
x = np.random.normal(0, 1, 10000).cumsum()
fig, (ax1, ax2) = plt.subplots(nrows=2, figsize=(8, 10))
values, ybins, xbins, im = ax1.specgram(x, cmap='gist_earth')
ax1.set(title='Specgram')
fig.colorbar(im, ax=ax1)
mesh = ax2.pcolormesh(xbins, ybins, values, cmap='gist_earth')
ax2.axis('tight')
ax2.set(title='Raw Plot of Returned Values')
fig.colorbar(mesh, ax=ax2)
plt.show()
Magnitude Differences
You'll immediately notice the difference in magnitude of the plotted values.
By default, plt.specgram doesn't plot the "raw" values it returns. Instead, it scales them to decibels (in other words, it plots the 10 * log10 of the amplitudes). If you'd like it not to scale things, you'll need to specify scale="linear". However, for looking at frequency composition, a log scale is going to make the most sense.
With that in mind, let's mimic what specgram does:
plotted = 10 * np.log10(values)
fig, ax = plt.subplots()
mesh = ax.pcolormesh(xbins, ybins, plotted, cmap='gist_earth')
ax.axis('tight')
ax.set(title='Plot of $10 * log_{10}(values)$')
fig.colorbar(mesh)
plt.show()
Using a Log Color Scale Instead
Alternatively, we could use a log norm on the image and get a similar result, but communicate that the color values are on a log scale more clearly:
from matplotlib.colors import LogNorm
fig, ax = plt.subplots()
mesh = ax.pcolormesh(xbins, ybins, values, cmap='gist_earth', norm=LogNorm())
ax.axis('tight')
ax.set(title='Log Normalized Plot of Values')
fig.colorbar(mesh)
plt.show()
imshow vs pcolormesh
Finally, note that the examples we've shown have had no interpolation applied, while the original specgram plot did. specgram uses imshow, while we've been plotting with pcolormesh. In this case (regular grid spacing) we can use either.
Both imshow and pcolormesh are very good options, in this case. However,imshow will have significantly better performance if you're working with a large array. Therefore, you might consider using it instead, even if you don't want interpolation (e.g. interpolation='nearest' to turn off interpolation).
As an example:
extent = [xbins.min(), xbins.max(), ybins.min(), ybins.max()]
fig, ax = plt.subplots()
mesh = ax.imshow(values, extent=extent, origin='lower', aspect='auto',
cmap='gist_earth', norm=LogNorm())
ax.axis('tight')
ax.set(title='Log Normalized Plot of Values')
fig.colorbar(mesh)
plt.show()

matplotlib spectrogram, intensity scale [duplicate]

I using matplotlib to plot some data in python and the plots require a standard colour bar. The data consists of a series of NxM matrices containing frequency information so that a simple imshow() plot gives a 2D histogram with colour describing frequency. Each matrix contains data in different, but overlapping ranges. Imshow normalizes the data in each matrix to the range 0-1 which means that, for example, the plot of matrix A, will appear identical to the plot of the matrix 2*A (though the colour bar will show double the values). What I would like is for the colour red, for example, to correspond to the same frequency in all of the plots. In other words, a single colour bar would suffice for all the plots. Any suggestions would be greatly appreciated.

Not to steal #ianilis's answer, but I wanted to add an example...
There are multiple ways, but the simplest is just to specify the vmin and vmax kwargs to imshow. Alternately, you can make a matplotlib.cm.Colormap instance and specify it, but that's more complicated than necessary for simple cases.
Here's a quick example with a single colorbar for all images:
import numpy as np
import matplotlib.pyplot as plt
# Generate some data that where each slice has a different range
# (The overall range is from 0 to 2)
data = np.random.random((4,10,10))
data *= np.array([0.5, 1.0, 1.5, 2.0])[:,None,None]
# Plot each slice as an independent subplot
fig, axes = plt.subplots(nrows=2, ncols=2)
for dat, ax in zip(data, axes.flat):
# The vmin and vmax arguments specify the color limits
im = ax.imshow(dat, vmin=0, vmax=2)
# Make an axis for the colorbar on the right side
cax = fig.add_axes([0.9, 0.1, 0.03, 0.8])
fig.colorbar(im, cax=cax)
plt.show()

Easiest solution is to call clim(lower_limit, upper_limit) with the same arguments for each plot.

This only answer half of the question, or rather starts a new one.
If you change
data *= np.array([0.5, 1.0, 1.5, 2.0])[:,None,None]
to
data *= np.array([2.0, 1.0, 1.5, 0.5])[:,None,None]
your colorbar will go from 0 to 0.5 which in this case is dark blue to slightly lighter blue and will not cover the whole range (0 to 2).
The colorbar will only show the colors from the last image or contour regardless of vmin and vmax.

I wasn't happy with the solutions that suggested to manually set vmin and vmax, so I decided to read the limits of each plot and automatically set vmin and vmax.
The example below shows three plots of samples taken from normal distributions with increasing mean value.
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import ImageGrid
import numpy as np
numberOfPlots = 3
data = []
for i in range(numberOfPlots):
mean = i
data.append(np.random.normal(mean, size=(100,100)))
fig = plt.figure()
grid = ImageGrid(fig, 111, nrows_ncols=(1,numberOfPlots), cbar_mode='single')
ims = []
for i in range(numberOfPlots):
ims.append(grid[i].imshow(data[i]))
grid[i].set_title("Mean = " + str(i))
clims = [im.get_clim() for im in ims]
vmin = min([clim[0] for clim in clims])
vmax = max([clim[1] for clim in clims])
for im in ims:
im.set_clim(vmin=np.floor(vmin),vmax=np.ceil(vmax))
grid[0].cax.colorbar(ims[0]) # with cbar_mode="single", cax attribute of all axes are identical
fig.show()

Python: subplots with different total sizes

Original Post
I need to make several subplots with different sizes.
I have simulation areas of the size (x y) 35x6µm to 39x2µm and I want to plot them in one figure. All subplots have the same x-ticklabels (there is a grid line every 5µm on the x-axis).
When I plot the subplots into one figure, then the graphs with the small x-area are streched, so that the x-figuresize is completely used. Therefore, the x-gridlines do not match together anymore.
How can I achieve that the subplots aren't streched anymore and are aligned on the left?
Edit: Here is some code:
size=array([[3983,229],[3933,350],[3854,454],[3750,533],[3500,600]], dtype=np.float)
resolution=array([[1024,256],[1024,320],[1024,448],[1024,512],[1024,640]], dtype=np.float)
aspect_ratios=(resolution[:,0]/resolution[:,1])*(size[:,1]/size[:,0])
number_of_graphs=len(data)
fig, ax=plt.subplots(nrows=number_of_graphs, sharex=xshare)
fig.set_size_inches(12,figheight)
for i in range(number_of_graphs):
temp=np.rot90(np.loadtxt(path+'/'+data[i]))
img=ax[i].imshow(temp,
interpolation="none",
cmap=mapping,
norm=specific_norm,
aspect=aspect_ratios[i]
)
ax[i].set_adjustable('box-forced')
#Here I have to set some ticks and labels....
ax[i].xaxis.set_ticks(np.arange(0,int(size[i,0]),stepwidth_width)*resolution[i,0]/size[i,0])
ax[i].set_xticklabels((np.arange(0, int(size[i,0]), stepwidth_width)))
ax[i].yaxis.set_ticks(np.arange(0,int(size[i,1]),stepwidth_height)*resolution[i,1]/size[i,1])
ax[i].set_yticklabels((np.arange(0, int(size[i,1]), stepwidth_height)))
ax[i].set_title(str(mag[i]))
grid(True)
savefig(path+'/'+name+'all.pdf', bbox_inches='tight', pad_inches=0.05) #saves graph
Here are some examples:
If I plot different matrices in a for loop, the iPhython generates an output which is pretty much what I want. The y-distande between each subplot is constant, and the size of each figure is correct. You can see, that the x-labels match to each other:
When I plot the matrices in one figure using subplots, then this is not the case: The x-ticks do not fit together, and every subplot has the same size on the canvas (which means, that for thin subplots there is more white space reservated on the canvas...).
I simply want the first result from iPython in one output file using subplots.
Using GridSpec
After the community told me to use GridSpec to determine the size of my subplots directly I wrote a code for automatic plotting:
size=array([[3983,229],[3933,350],[3854,454],[3750,533],[3500,600]], dtype=np.float)
#total size of the figure
total_height=int(sum(size[:,1]))
total_width=int(size.max())
#determines steps of ticks
stepwidth_width=500
stepwidth_height=200
fig, ax=plt.subplots(nrows=len(size))
fig.set_size_inches(size.max()/300., total_height/200)
gs = GridSpec(total_height, total_width)
gs.update(left=0, right=0.91, hspace=0.2)
height=0
for i in range (len(size)):
ax[i] = plt.subplot(gs[int(height):int(height+size[i,1]), 0:int(size[i,0])])
temp=np.rot90(np.loadtxt(path+'/'+FFTs[i]))
img=ax[i].imshow(temp,
interpolation="none",
vmin=-100,
vmax=+100,
aspect=aspect_ratios[i],
)
#Some rescaling
ax[i].xaxis.set_ticks(np.arange(0,int(size[i,0]),stepwidth_width)*resolution[i,0]/size[i,0])
ax[i].set_xticklabels((np.arange(0, int(size[i,0]), stepwidth_width)))
ax[i].yaxis.set_ticks(np.arange(0,int(size[i,1]),stepwidth_height)*resolution[i,1]/size[i,1])
ax[i].set_yticklabels((np.arange(0, int(size[i,1]), stepwidth_height)))
ax[i].axvline(antenna[i]) #at the antenna position a vertical line is plotted
grid(True)
#colorbar
cbaxes = fig.add_axes([0.93, 0.2, 0.01, 0.6]) #[left, bottom, width, height]
cbar = plt.colorbar(img, cax = cbaxes, orientation='vertical')
tick_locator = ticker.MaxNLocator(nbins=3)
cbar.locator = tick_locator
cbar.ax.yaxis.set_major_locator(matplotlib.ticker.AutoLocator())
cbar.set_label('Intensity',
#fontsize=12
)
cbar.update_ticks()
height=height+size[i,1]
plt.show()
And here is the result....
Do you have any ideas?

What about using matplotlib.gridspec.GridSpec? Docs.
You could try something like
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
gs = GridSpec(8, 39)
ax1 = plt.subplot(gs[:6, :35])
ax2 = plt.subplot(gs[6:, :])
data1 = np.random.rand(6, 35)
data2 = np.random.rand(2, 39)
ax1.imshow(data1)
ax2.imshow(data2)
plt.show()