Develop Reference

draw a color grid based on points density using python matplotlib

The question is to read 10,000 coordinate points from a file and create a colored grid based on the density of each block on the grid. The range of x-axis is [-73.59, -73.55] and the y-axis is [45.49,45.530]. My code will plot a grid with many different colors, now I need a feature to only color the grid that has a specific density n, for example, The n = 100, only the grid with 100 points or higher will be colored to yellow, and other grids will be black.
I just added a link to my shapefile
https://drive.google.com/open?id=1H-8FhfonnPrYW9y7RQZDtiNLxVEiC6R8
import numpy as np
import matplotlib.pyplot as plt
import shapefile
grid_size = 0.002
x1 = np.arange(-73.59,-73.55,grid_size)
y1 = np.arange(45.49,45.530,grid_size)
shape = shapefile.Reader("Shape/crime_dt.shp",encoding='ISO-8859-1')
shapeRecords = shape.shapeRecords()
x_coordinates=[]
y_coordinates=[]
# read all points in .shp file, and store them in 2 lists.
for k in range(len(shapeRecords)):
x = float(shapeRecords[k].shape.__geo_interface__["coordinates"][0])
y = float(shapeRecords[k].shape.__geo_interface__["coordinates"][1])
x_coordinates.append(x)
y_coordinates.append(y)
plt.hist2d(x_coordinates,y_coordinates,bins=[x1,y1])
plt.show()

You can create a colormap with just two colors, and set vmin and vmax to be symmetrical around your desired pivot value.
Optionally you put the value of each bin inside the cells, while the pivot value decides the text color.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import ListedColormap
grid_size = 0.002
x1 = np.arange(-73.59, -73.55, grid_size)
y1 = np.arange(45.49, 45.530, grid_size)
# read coordinates from file and put them into two lists, similar to this
x_coordinates = np.random.uniform(x1.min(), x1.max(), size=40000)
y_coordinates = np.random.uniform(y1.min(), y1.max(), size=40000)
pivot_value = 100
# create a colormap with two colors, vmin and vmax are chosen so that their center is the pivot value
cmap = ListedColormap(['indigo', 'gold'])
# create a 2d histogram with xs and ys as bin boundaries
binvalues, _, _, _ = plt.hist2d(x_coordinates, y_coordinates, bins=[x1, y1], cmap=cmap, vmin=0, vmax=2*pivot_value)
binvalues = binvalues.astype(np.int)
for i in range(len(x1) - 1):
for j in range(len(y1) - 1):
plt.text((x1[i] + x1[i + 1]) / 2, (y1[j] + y1[j + 1]) / 2, binvalues[i, j],
color='white' if binvalues[i, j] < pivot_value else 'black',
ha='center', va='center', size=8)
plt.show()
PS: If the bin values are very important, you can add them all as ticks. Then, their positions can also be used to draw gridlines as a division between the cells.
plt.yticks(y1)
plt.xticks(x1, rotation=90)
plt.grid(True, ls='-', lw=1, color='black')
To obtain contours based on these data, you could plt.contourf with the generated matrix. (You might want to use np.histogram2d to directly create the matrix.)
plt.contourf((x1[1:]+x1[:-1])/2, (y1[1:]+y1[:-1])/2, binvalues.T, levels=[0,100,1000], cmap=cmap)

Draw linewidth inside rectangle matplotlib

I'm having trouble drawing rectangles in matplotlib using Patches. When linewidth is supplied to patches.Rectangle, the border is drawn on the outside of the rectangle. Here's an example:
import matplotlib.pyplot as plt
import matplotlib.patches as patches
fig, ax = plt.subplots(1)
rect = patches.Rectangle((1, 1), 1, 1, facecolor = 'blue')
rect2 = patches.Rectangle((1, 2.1), 1, 1, facecolor = 'none', edgecolor = 'black', linewidth = 6)
ax.add_patch(rect)
ax.add_patch(rect2)
ax.set_xlim([0, 3.5])
ax.set_ylim([0, 3.5])
here's the result:
Note that the border is drawn on the outside of the box such that the box + border now exceeds the size of the blue box. I would like the border to be drawn inside the box, such that it is always the same size as the blue box regardless of linewidth.
I've tried this in two different ways and neither was satisfying:
Convert the linewidth from absolute units into units of the data, then calculate a smaller box that could be drawn with a normal border that matches the other box.
Explore some of the offsetbox functionality as in this post, though I didn't get very far as I couldn't figure out how to specify pad correctly.
Any help would be appreciated!

The easiest approach is to set a clip rectangle that hides everything outside the rectangle. As you already have a rectangle, it can be used to clip itself.
As the border is drawn centered on the border line, half of it will be clipped away. This can be tackled by setting the width to double the desired width.
Note that for clipping to work as desired, the rectangle already needs to be transformed to axes coordinates. So, first add the rectangle patch to the ax and only then use is to set the clipping.
Also note that with default parameters, a rectangle uses the same color for the inside as well as for a thin border. Setting the linewidth to zero ensures that it doesn't draw outside the rectangle.
Similarly, ellipses can be drawn with the line only at the inside.
The code below uses a thickness of 10 and some extra dotted red lines to illustrate what's happening.
import matplotlib.pyplot as plt
import matplotlib.patches as patches
fig, ax = plt.subplots()
pad = 0.1 # distance between the rectangles
for i in range(3):
for j in range(2):
x = .5 + i * (1 + pad)
y = .5 + j * (1 + pad)
if i == j:
patch = patches.Rectangle((x, y), 1, 1, facecolor='blue', linewidth=0)
elif i < 2:
patch = patches.Rectangle((x, y), 1, 1, facecolor='none', edgecolor='black',
linewidth=10*2 if j == 0 else 10)
else:
patch = patches.Ellipse((x+0.5, y+0.5), 1, 1, facecolor='none', edgecolor='black',
linewidth=10*2 if j == 0 else 10)
ax.add_patch(patch)
if j == 0:
patch.set_clip_path(patch)
for i in range(3):
x = .5 + i * (1 + pad)
for s in 0,1:
ax.axvline (x+s, color='crimson', ls=':', lw=1)
for j in range(2):
y = .5 + j * (1 + pad)
for s in 0,1:
ax.axhline (y+s, color='crimson', ls=':', lw=1)
ax.set_xlim([0, 4.0])
ax.set_ylim([0, 3.0])
ax.set_aspect('equal')
plt.show()
The image below shows the standard way of drawing at the top, and clipping with double linewidth at the bottom.

Matplotlib legend makes the image too large

I'm plotting this figure with matplotlib, the for loop just color the background:
fig, ax = plt.subplots()
ax.set_ylabel('Number of contacts')
ax.set_xlabel('Time [s]')
for m in range(len(data[node])):
if data[node][m] == -1:
ax.axvline(m,color='r',linewidth=5,alpha=0.2,label="OUT")
if data[node][m] == 0:
ax.axvline(m,color='g',linewidth=5,alpha=0.2,label="RZ0")
if data[node][m] == 1:
ax.axvline(m,color='y',linewidth=5,alpha=0.2,label="RZ1")
ax.plot(x, y, 'b+')
# ax.legend() # HERE is the problem
plt.show()
Which plots the following:
What I want now is a legend to indicate each color of the background meaning, but when I include ax.legend() I get the following error:
ValueError: Image size of 392x648007 pixels is too large. It must be less than 2^16 in each
direction.
<Figure size 432x288 with 1 Axes>
<Figure size 432x288 with 0 Axes>
How am I supposed to name each color of the background, there are 43200 vertical lines but only 3 colors, does it have anything to do with the number of lines?

The trick is to set the label only once. You can add a variable for each label and replace it with None once it's used. Note that using axvline to draw a background has the problem that the line width is measured in pixel space, so neighboring lines will either overlap or have a small white space inbetween. Better to use axvspan. To avoid the white space at the left and at the right, you can explicitly set the x-limits.
The code can be simplified somewhat using a loop.
Updated code:
group consecutive spans together for drawing
precalculate the effect of alpha so the background can be drawn without the need for transparency
from matplotlib import pyplot as plt
from matplotlib import colors as mcolors
import numpy as np
import pandas as pd
import itertools
fig, ax = plt.subplots()
# create some random data
x = np.arange(100)
y = np.sinh(x/20)
indicators = [-1, 0, 1]
node = 0
data = [np.random.choice(indicators, len(x), p=[10/16,1/16,5/16])]
labels = ["OUT", "RZ0", "RZ1"]
colors = ['lime', 'purple', 'gold']
alpha = 0.4
# precalculate the effect of alpha so the colors can be applied with alpha=1
colors = [[1 + (x - 1) * alpha for x in mcolors.to_rgb(c)] for c in colors]
m = 0
for val, group in itertools.groupby(data[node]):
width = len(list(group))
ind = indicators.index(val)
ax.axvspan(m, m + width, color=colors[ind], linewidth=0, alpha=1, label=labels[ind])
labels[ind] = None # reset the label to make sure it is only used once
m += width
ax.plot(x, y, 'b+')
ax.set_xlim(0, len(data[node]))
ax.legend(framealpha=1) # to make the legend background opaque
plt.show()

Do something like hrz1 = ax.axvline(m,color='y',linewidth=5,alpha=0.2) for each of your classes, and then ax.legend((hrz1, hrz0, hout), ('RZ1', 'RZ0', 'OUT'). The hrz1 pointer will be rewritten for each line you make, and then legend will only make one label for each of the handles.

How to create multiple 1D axes showing intervals with colored lines?

I want to visualise mathematical domains, or intervals. Equivalently, I want to visualise a boolean array. There are multiple such arrays, that ideally are plotted one above the other.
What I have is some data: several recordings, over a period of, say, 100 min. Each recording satisfies a given condition only part of the time. I want to visualise the times at which each recording is "True". Some simpler variant of:
In my case, each recording can be the union of multiple intervals. For example:
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sb
sb.set_context("paper")
times = np.arange(0, 100)
mask1 = (times >= 0) * (times <= 30) + (times >= 70) * (times <= 100)
mask2 = (times >= 20) * (times <= 80)
I can plot each recording separately, with these two functions I have written:
def bool2extreme(mask, times) :
"""return xmins and xmaxs for intervals in times"""
binary = 1*mask
slope = np.diff(binary)
extr = (slope != 0)
signs = slope[extr]
mins = list(times[1:][slope==1])
maxs = list(times[:-1][slope==-1])
if signs[0]==-1:
mins = [times[0]] + mins
if signs[-1]==1:
maxs = maxs + [times[-1]]
return mins, maxs
def plot_interval(mask, times, y=0, color='k', ax=None) :
if ax==None:
print('None')
ax = plt.gca()
xmins, xmaxs = bool2extreme(mask, times)
for xmin, xmax in zip(xmins, xmaxs):
ax.plot([xmin, xmax], [y,y], lw=6, color=color)
return ax
My problem is to control the vertical spacing between the various intervals. Indeed, when I plot one of them, there is a vertical axis which I don't want. Even if I set its visibility to False, it exists and takes space. So, when I put each recording on a different subplot, the vertical spacing between them is much too big:
masks = [mask1, mask2]
labels = ['domain1', 'domain2']
n_plots = len(masks)
fig, axs = plt.subplots(n_plots, sharex=True)
for i, mask in enumerate(masks) :
axs[i] = plot_interval(mask, times, ax=axs[i])
axs[-1].set_xlabel('Time (min)')
sb.despine()
Another option I tried: have all the intervals in the same axis, but at different y values. But the problem of the vertical spacing between the intervals remains the same.
masks = [mask1, mask2]
labels = ['domain1', 'domain2']
n_plots = len(masks)
fig, ax = plt.subplots(sharex=True)
for i, mask in enumerate(masks) :
ax = plot_interval(mask, times, y=i, ax=ax)
ax.set_xlabel('Time (min)')
ax.set_yticks(range(n_plots))
ax.set_yticklabels(labels)
ax.grid(axis="x")
sb.despine(left=True)
How can I control the vertical spacing between these intervals?

Some ideas:
figsize with a small height when creating the subplots; the height of figsize controls the distance between the horizontal axes: they will be height/num_axes separated when measured in inches
ax.yaxis.set_visible(False) to hide the ticks from the y-axis
ax.spines['left'].set_color('None') to make the spine of the y-axis invisible
ax.spines['bottom'].set_position(('data', 0)) to place the x-axis at the y=0 height
(optionally) ax.tick_params(labelbottom=True) to have labels for the xticks on all subplots (instead of only on the last)
use a rectangle instead of a thick line to better control the exact start and end of the line as well as the thickness above and under the axis
to control the height of the rectangle, the ylims need to be fixed; I propose (-1.5, .5) so a thickness can be chosen appropriately; there is more space below making room for the labels of the xticks
as drawing a rectangle doesn't automatically update the xlims, they need to be set explicitly
(optionally) ax.tick_params(which='both', direction='in') to get tick marks above instead below (both mayor and minor ticks)
To have labels on the left, the following worked for me:
# ax.yaxis.set_visible(False) # removed, as it also hides the ylabel
ax.set_ylabel('my ylabel', rotation=0, ha='right', labelpad=10)
ax.set_yticks([]) # to remove the ticks, the spine was already removed
In the demo code, more xticks and some type of arrow at the ends are added. There are 7 masks in the demo, to better see the effect of distance between the axes. Trying to get the axes as close as possible, a distance of 0.4 inches seems doable. (The bool2extreme function is untouched, as it is closely related to the format used as input.)
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle, Polygon
import matplotlib.ticker as plticker
import seaborn as sbs
sbs.set_context("paper")
times = np.arange(0, 101)
num_masks = 7
masks = [np.zeros_like(times, dtype=bool) for _ in range(num_masks)]
for i in range(num_masks):
for j in range(50):
masks[i] += (times >= (i+3)*j) * (times <= (i+3)*j+i+1)
masks = masks[::-1] # reverse to get the masks plotted from bottom to top
def bool2extreme(mask, times) :
"""return xmins and xmaxs for intervals in times"""
binary = 1*mask
slope = np.diff(binary)
extr = (slope != 0)
signs = slope[extr]
mins = list(times[1:][slope==1])
maxs = list(times[:-1][slope==-1])
if signs[0]==-1:
mins = [times[0]] + mins
if signs[-1]==1:
maxs = maxs + [times[-1]]
return mins, maxs
def plot_interval(mask, times, xlim=None, y=0, thickness=0.4, color='k', ax=None):
if ax is None:
ax = plt.gca()
ax.yaxis.set_visible(False)
ax.spines['left'].set_color('None')
ax.spines['right'].set_color('None')
ax.spines['top'].set_color('None')
ax.spines['bottom'].set_position(('data', 0))
ax.tick_params(labelbottom=True) # to get tick labels on all axes
# ax.tick_params(which='both', direction='in')` # tick marks above instead below the axis
ax.xaxis.set_major_locator(plticker.MultipleLocator(base=10)) # major ticks in steps of 10
ax.xaxis.set_minor_locator(plticker.MultipleLocator(base=1)) # minor ticks in steps of 1
ax.set_ylim(-1.5,.5)
if xlim is None:
xlim = (times[0]-0.9, times[-1]+0.9)
ax.set_xlim(xlim)
xmins, xmaxs = bool2extreme(mask, times)
for xmin, xmax in zip(xmins, xmaxs):
#ax.add_patch(Rectangle((xmin, y-thickness), xmax-xmin, 2*thickness, linewidth=0, color=color))
ax.add_patch(Rectangle((xmin, y), xmax-xmin, thickness, linewidth=0, color=color))
triangle1 = [(xlim[0]-0.5, y), (xlim[0], y-thickness), (xlim[0], y+thickness)]
ax.add_patch(Polygon(triangle1, linewidth=0, color='black', clip_on=False))
triangle2 = [(xlim[1]+0.5, y), (xlim[1], y-thickness), (xlim[1], y+thickness)]
ax.add_patch(Polygon(triangle2, linewidth=0, color='black', clip_on=False))
return ax
n_plots = len(masks)
dist_between_axis_in_inches = 0.4
fig, axs = plt.subplots(n_plots, sharex=True, figsize=(10, dist_between_axis_in_inches*len(masks)))
for i, mask in enumerate(masks) :
axs[i] = plot_interval(mask, times, xlim=(times[0]-0.5, times[-1]+0.5), ax=axs[i], color='lime')
axs[-1].set_xlabel('Time (min)')
plt.show()
Result with axes close together:
PS: This post contains more proposals about adding arrows.

Laying out several plots in matplotlib + numpy

I am pretty new to python and want to plot a dataset using a histogram and a heatmap below. However, I am a bit confused about
How to put a title above both plots and
How to insert some text into bots plots
How to reference the upper and the lower plot
For my first task I used the title instruction, which inserted a caption in between both plots instead of putting it above both plots
For my second task I used the figtext instruction. However, I could not see the text anywhere in the plot. I played a bit with the x, y and fontsize parameters without any success.
Here is my code:
def drawHeatmap(xDim, yDim, plot, threshold, verbose):
global heatmapList
stableCells = 0
print("\n[I] - Plotting Heatmaps ...")
for currentHeatmap in heatmapList:
if -1 in heatmapList[currentHeatmap]:
continue
print("[I] - Plotting heatmap for PUF instance", currentHeatmap,"(",len(heatmapList[currentHeatmap])," values)")
# Convert data to ndarray
#floatMap = list(map(float, currentHeatmap[1]))
myArray = np.array(heatmapList[currentHeatmap]).reshape(xDim,yDim)
# Setup two plots per page
fig, ax = plt.subplots(2)
# Histogram
weights = np.ones_like(heatmapList[currentHeatmap]) / len(heatmapList[currentHeatmap])
hist, bins = np.histogram(heatmapList[currentHeatmap], bins=50, weights=weights)
width = 0.7 * (bins[1] - bins[0])
center = (bins[:-1] + bins[1:]) / 2
ax[0].bar(center, hist, align='center', width=width)
stableCells = calcPercentageStable(threshold, verbose)
plt.figtext(100,100,"!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!", fontsize=40)
heatmap = ax[1].pcolor(myArray, cmap=plt.cm.Blues, alpha=0.8, vmin=0, vmax=1)
cbar = fig.colorbar(heatmap, shrink=0.8, aspect=10, fraction=.1,pad=.01)
#cbar.ax.tick_params(labelsize=40)
for y in range(myArray.shape[0]):
for x in range(myArray.shape[1]):
plt.text(x + 0.5, y + 0.5, '%.2f' % myArray[y, x],
horizontalalignment='center',
verticalalignment='center',
fontsize=(xDim/yDim)*5
)
#fig = plt.figure()
fig = matplotlib.pyplot.gcf()
fig.set_size_inches(60.5,55.5)
plt.savefig(dataDirectory+"/"+currentHeatmap+".pdf", dpi=800, papertype="a3", format="pdf")
#plt.title("Heatmap for PUF instance "+str(currentHeatmap[0][0])+" ("+str(numberOfMeasurements)+" measurements; "+str(sizeOfMeasurements)+" bytes)")
if plot:
plt.show()
print("\t[I] - Done ...")
And here is my current output:

Perhaps this example will make things easier to understand. Things to note are:
Use fig.suptitle to add a title to the top of a figure.
Use ax[i].text(x, y, str) to add text to an Axes object
Each Axes object, ax[i] in your case, holds all the information about a single plot. Use them instead of calling plt, which only really works well with one subplot per figure or to modify all subplots at once. For example, instead of calling plt.figtext, call ax[0].text to add text to the top plot.
Try following the example code below, or at least read through it to get a better idea how to use your ax list.
import numpy as np
import matplotlib.pyplot as plt
histogram_data = np.random.rand(1000)
heatmap_data = np.random.rand(10, 100)
# Set up figure and axes
fig = plt.figure()
fig.suptitle("These are my two plots")
top_ax = fig.add_subplot(211) #2 rows, 1 col, 1st plot
bot_ax = fig.add_subplot(212) #2 rows, 1 col, 2nd plot
# This is the same as doing 'fig, (top_ax, bot_ax) = plt.subplots(2)'
# Histogram
weights = np.ones_like(histogram_data) / histogram_data.shape[0]
hist, bins = np.histogram(histogram_data, bins=50, weights=weights)
width = 0.7 * (bins[1] - bins[0])
center = (bins[:-1] + bins[1:]) / 2
# Use top_ax to modify anything with the histogram plot
top_ax.bar(center, hist, align='center', width=width)
# ax.text(x, y, str). Make sure x,y are within your plot bounds ((0, 1), (0, .5))
top_ax.text(0.5, 0.5, "Here is text on the top plot", color='r')
# Heatmap
heatmap_params = {'cmap':plt.cm.Blues, 'alpha':0.8, 'vmin':0, 'vmax':1}
# Use bot_ax to modify anything with the heatmap plot
heatmap = bot_ax.pcolor(heatmap_data, **heatmap_params)
cbar = fig.colorbar(heatmap, shrink=0.8, aspect=10, fraction=.1,pad=.01)
# See how it looks
plt.show()