Develop Reference

Seaborn clustermap fixed cell size

I am using the seaborn clustermap function and I would like to make multiple plots where the cell sizes are exactly identical. Also the size of the axis labels should be the same. This means figure size and aspect ratio will need to change, the rest needs to stay identical.
import pandas
import seaborn
import numpy as np
dataFrameA = pd.DataFrame([ [1,2],[3,4] ])
dataFrameB = pd.DataFrame( np.arange(3*6).reshape(3,-1))
Then decide how big the clustermap itself needs to be, something along the lines of:
dpi = 72
cellSizePixels = 150
This decides that dataFrameA should be should be 300 by 300 pixels. I think that those need to be converted to the size units of the figure, which will be cellSizePixels/dpi units per pixel. So for dataFrameA that will be a heatmap size of ~2.01 inches. Here I am introducing a problem: there is stuff around the heatmap, which will also take up some space, and I don't know how much space those will exactly take.
I tried to parametrize the heatmap function with a guess of the image size using the formula above:
def fixedWidthClusterMap( dpi, cellSizePixels, dataFrame):
clustermapParams = {
'square':False # Tried to set this to True before. Don't: the dendograms do not scale well with it.
}
figureWidth = (cellSizePixels/dpi)*dataFrame.shape[1]
figureHeight= (cellSizePixels/dpi)*dataFrame.shape[0]
return sns.clustermap( dataFrame, figsize=(figureWidth,figureHeight), **clustermapParams)
fixedWidthClusterMap(dpi, cellSizePixels, dataFrameA)
plt.show()
fixedWidthClusterMap(dpi, cellSizePixels, dataFrameB)
plt.show()
This yields:
My question: how do I obtain square cells which are exactly the size I want?

This is a bit tricky, because there are quite a few things to take into consideration, and in the end, it depends how "exact" you need the sizes to be.
Looking at the code for clustermap the heatmap part is designed to have a ratio of 0.8 compared to the axes used for the dendrograms. But we also need to take into account the margins used to place the axes. If one knows the size of the heatmap axes, one should therefore be able to calculate the desired figure size that would produce the right shape.
dpi = matplotlib.rcParams['figure.dpi']
marginWidth = matplotlib.rcParams['figure.subplot.right']-matplotlib.rcParams['figure.subplot.left']
marginHeight = matplotlib.rcParams['figure.subplot.top']-matplotlib.rcParams['figure.subplot.bottom']
Ny,Nx = dataFrame.shape
figWidth = (Nx*cellSizePixels/dpi)/0.8/marginWidth
figHeigh = (Ny*cellSizePixels/dpi)/0.8/marginHeight
Unfortunately, it seems matplotlib must adjust things a bit during plotting, because that was not enough the get perfectly square heatmap cells. So I choose to resize the various axes create by clustermap after the fact, starting with the heatmap, then the dendrogram axes.
I think the resulting image is pretty close to what you were trying to get, but my tests sometime show some errors by 1-2 px, which I attribute to rounding errors due to all the conversions between sizes in inches and pixels.
dataFrameA = pd.DataFrame([ [1,2],[3,4] ])
dataFrameB = pd.DataFrame( np.arange(3*6).reshape(3,-1))
def fixedWidthClusterMap(dataFrame, cellSizePixels=50):
# Calulate the figure size, this gets us close, but not quite to the right place
dpi = matplotlib.rcParams['figure.dpi']
marginWidth = matplotlib.rcParams['figure.subplot.right']-matplotlib.rcParams['figure.subplot.left']
marginHeight = matplotlib.rcParams['figure.subplot.top']-matplotlib.rcParams['figure.subplot.bottom']
Ny,Nx = dataFrame.shape
figWidth = (Nx*cellSizePixels/dpi)/0.8/marginWidth
figHeigh = (Ny*cellSizePixels/dpi)/0.8/marginHeight
# do the actual plot
grid = sns.clustermap(dataFrame, figsize=(figWidth, figHeigh))
# calculate the size of the heatmap axes
axWidth = (Nx*cellSizePixels)/(figWidth*dpi)
axHeight = (Ny*cellSizePixels)/(figHeigh*dpi)
# resize heatmap
ax_heatmap_orig_pos = grid.ax_heatmap.get_position()
grid.ax_heatmap.set_position([ax_heatmap_orig_pos.x0, ax_heatmap_orig_pos.y0,
axWidth, axHeight])
# resize dendrograms to match
ax_row_orig_pos = grid.ax_row_dendrogram.get_position()
grid.ax_row_dendrogram.set_position([ax_row_orig_pos.x0, ax_row_orig_pos.y0,
ax_row_orig_pos.width, axHeight])
ax_col_orig_pos = grid.ax_col_dendrogram.get_position()
grid.ax_col_dendrogram.set_position([ax_col_orig_pos.x0, ax_heatmap_orig_pos.y0+axHeight,
axWidth, ax_col_orig_pos.height])
return grid # return ClusterGrid object
grid = fixedWidthClusterMap(dataFrameA, cellSizePixels=75)
plt.show()
grid = fixedWidthClusterMap(dataFrameB, cellSizePixels=75)
plt.show()

Not a complete answer (not dealing with pixels) but I suspect OP has moved on after 4 years.
def reshape_clustermap(cmap, cell_width=0.02, cell_height=0.02):
ny, nx = cmap.data2d.shape
hmap_width = nx * cell_width
hmap_height = ny * cell_height
hmap_orig_pos = cmap.ax_heatmap.get_position()
cmap.ax_heatmap.set_position(
[hmap_orig_pos.x0, hmap_orig_pos.y0, hmap_width, hmap_height]
)
top_dg_pos = cmap.ax_col_dendrogram.get_position()
cmap.ax_col_dendrogram.set_position(
[hmap_orig_pos.x0, hmap_orig_pos.y0 + hmap_height, hmap_width, top_dg_pos.height]
)
left_dg_pos = cmap.ax_row_dendrogram.get_position()
cmap.ax_row_dendrogram.set_position(
[left_dg_pos.x0, left_dg_pos.y0, left_dg_pos.width, hmap_height]
)
if cmap.ax_cbar:
cbar_pos = cmap.ax_cbar.get_position()
hmap_pos = cmap.ax_heatmap.get_position()
cmap.ax_cbar.set_position(
[cbar_pos.x0, hmap_pos.y1, cbar_pos.width, cbar_pos.height]
)
cmap = sns.clustermap(dataFrameA)
reshape_clustermap(cmap)

pyqtgraph plotwidget multiple Y axis plots in wrong area

I have an embedded widget in a QT5 gui connected via PlotWidget.
I am trying to plot 2 streams of live data Voltage (self.p1) & Current (self.p2). Voltage on the left axis & Current on the right. So far I have each data stream associated with its relevant axis.
However my problem is that the Current plot (self.p2) is not in the correct area of the display. This particular plot appears in the upper left hand corner of the widget, it appears before the LHD axis. Its best to view the image to view the problem.
View Me
.
I know the problem lies in the setup function & self.p2 (Current) is being placed in the wrong location but my searching hasn't produced an answer.
Could someone please help?
Code used to generate graph, is called once on start up:
def pg_plot_setup(self): # not working still on left axis
self.p1 = self.graphicsView.plotItem
# x axis
self.p1.setLabel('bottom', 'Time', units='s', color='g', **{'font-size':'12pt'})
self.p1.getAxis('bottom').setPen(pg.mkPen(color='g', width=3))
# Y1 axis
self.p1.setLabel('left', 'Voltage', units='V', color='r', **{'font-size':'12pt'})
self.p1.getAxis('left').setPen(pg.mkPen(color='r', width=3))
self.p2 = pg.ViewBox()
self.p1.showAxis('right')
self.p1.scene().addItem(self.p2)
self.p1.getAxis('right').linkToView(self.p2)
self.p2.setXLink(self.p1)
# Y2 axis
self.p1.setLabel('right', 'Current', units="A", color='c', **{'font-size':'12pt'}) #<font>Ω</font>
self.p1.getAxis('right').setPen(pg.mkPen(color='c', width=3))
and code used to update display, is called via QTimer:
def update_graph_plot(self):
start = time.time()
X = np.asarray(self.graph_X, dtype=np.float32)
Y1 = np.asarray(self.graph_Y1, dtype=np.float32)
Y2 = np.asarray(self.graph_Y2, dtype=np.float32)
pen1=pg.mkPen(color='r',width=1.0)
pen2=pg.mkPen(color='c',width=1.0)
self.p1.plot(X,Y1,pen=pen1, name="V", clear=True)
self.p2.addItem(pg.PlotCurveItem(X,Y2,pen=pen2, name="I"))

I found the answer buried in here MultiplePlotAxes.py
adding this self.p2.setGeometry(self.p1.vb.sceneBoundingRect()) to function 'update_graph_plot' adjusts the size of viewbox each time the scene is updated, but it has to be in the update loop.
or add this self.p1.vb.sigResized.connect(self.updateViews) to function 'pg_plot_setup' as part of the setup, which shall then automatically call this function
def updateViews(self):
self.p2.setGeometry(self.p1.vb.sceneBoundingRect())
to resize viewbox (self.p2) each time self.p1 updates.

shapely parallel_offset sometimes does not generate closed ring

I am using the parallel_offset function of the shapely package to get offset structures to some polygons that are closed rings. I have several polygons at once, many with similar shapes. Around 10-25% of them, however, do not generate a closed ring from the parallel_offset. Here is a MWE of a shape that does not work:
import matplotlib.pyplot as plt
from shapely.geometry.polygon import LinearRing
def plot_line(ax, ob, color):
x, y = ob.xy
ax.plot(x, y, color=color, alpha=0.7, linewidth=3,
solid_capstyle='round', zorder=2)
polygon = [[-29.675, -30.675],
[-28.4094, -29.4094],
[-28.325, -29.325],
[-28.325, -29.764],
[-28.325, -29.7933],
[-28.4587, -29.8274],
[-28.4676, -29.8297],
[-28.5956, -29.8814],
[-28.6041, -29.8848],
[-28.724, -29.953],
[-28.732, -29.9576],
[-28.8417, -30.0413],
[-28.849, -30.0469],
[-28.9466, -30.1445],
[-28.9531, -30.151],
[-29.0368, -30.2607],
[-29.0424, -30.268],
[-29.1106, -30.3879],
[-29.1152, -30.3959],
[-29.1669, -30.5239],
[-29.1703, -30.5324],
[-29.2044, -30.6661],
[-29.2067, -30.675],
[-29.6457, -30.675],
[-29.675, -30.675]]
poly_line = LinearRing(polygon)
poly_line_offset = poly_line.parallel_offset(0.05, side="left", resolution=16,
join_style=2, mitre_limit=1)
fig = plt.figure()
ax = fig.add_subplot(111)
plot_line(ax, poly_line, "blue")
plot_line(ax, poly_line_offset, "green")
plt.show()
As you can see, the green offset polygon does not close at the point that is first/last in the list of vertices. Other very similar shapes, however, do work as intended. They have the same data structure and also have the same start/end point, as does my example above. The join_style attribute does not change the outcome to what I want. Changing the resolution or distance does not help either. The documentation also does not address this issue.
Do you have any guidance? I am using shapely 1.6.3.

not completely sure why this happens, nevertheless you might use a workaround based on the buffer method:
poly_line = LinearRing(polygon)
poly_line_offset = poly_line.buffer(0.05,
resolution=16, join_style=2, mitre_limit=1).exterior
With your data, this produces the (probably) desired result:

Here's a work around I did in my code.
I basically rolled the LinearRing (shifting the start point along the ring),
applied two offsets, and then added them back together.
It's probably not an ideal solution, but hopefully can work as a starting point:
from shapely import ops, geometry
import numpy as np
# test geo:
ring_coords = [(0,0.1),(0,2),(4,2),(4,0)]
ring = geometry.LinearRing(ring_coords)
# shifts the ring by one point
rolled = LinearRing(np.roll(ring.coords[:-1], 2))
# apply the offsets
offset_ring = ring.parallel_offset(-0.2, side='right', resolution=3, join_style=2, mitre_limit=3)
offset_rolled = rolled.parallel_offset(-0.2, side='right', resolution=3, join_style=2, mitre_limit=3)
# combine the points
# assuming you started with two rings, backward should be empty
forward, backward = ops.shared_paths(offset_rolled, offset_ring)
combined = geometry.LinearRing(ops.linemerge(forward))

Transformations in matplotlib

I'm trying to draw objects (lines/patches) with a fixed size (in device coordinates) at a certain position (in data coordinates). This behavior is akin to markers and the tips of annotation arrows, both of which are (size-) invariant under zoom and pan.
Why does the following example not work as expected?
The expected output is two crossed lines forming the diagonals of a 50x50 point square (device coordinates). The left lower corner of said square should be at point (1,0) in data coordinates.
While the computed points appear to be correct, the second diagonal is simply not visible.
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import matplotlib.path as mpath
import matplotlib.transforms as mtrans
import matplotlib as mpl
import numpy as np
class FixedPointOffsetTransform(mtrans.Transform):
"""
Always returns the same transformed point plus
the given point in device coordinates as an offset.
"""
def __init__(self, trans, fixed_point):
mtrans.Transform.__init__(self)
self.input_dims = self.output_dims = 2
self.has_inverse = False
self.trans = trans
self.fixed_point = np.array(fixed_point).reshape(1, 2)
def transform(self, values):
fp = self.trans.transform(self.fixed_point)
values = np.array(values)
if values.ndim == 1:
return fp.flatten() + values
else:
return fp + values
fig , ax = plt.subplots(1,1)
ax.set_xlim([-1,10])
ax.set_ylim([-1,10])
# this transformation shifts the input by the given offset
# the offset is transformed with the given transformation
# and then added to the input
fixed_pt_trans = FixedPointOffsetTransform(ax.transData, (1, 0))
# these values are in device coordinates i.e. points
height = 50
width = 50
# two points in device coordinates, that are modified with the above transformation
A = fixed_pt_trans.transform((0,0))
B = fixed_pt_trans.transform((width,height))
l1 = mpl.lines.Line2D([A[0],B[0]], [A[1],B[1]])
ax.add_line(l1)
# already in device coordinates with the offset applied,
# no further transformation nessesary
l1.set_transform(None)
print(A)
print(B)
print(l1.get_transform().transform(A))
print(l1.get_transform().transform(B))
# two points in device coordinates (unmodified)
A = (width,0)
B = (0,height)
l2 = mpl.lines.Line2D([A[0],B[0]], [A[1],B[1]])
ax.add_line(l2)
# apply transformation to add offset
l2.set_transform(fixed_pt_trans)
print(l2.get_transform().transform(A))
print(l2.get_transform().transform(B))
fig.show()

According to matplotlib API Changes documentation, starting with matplotlib 1.2.x:
Transform subclassing behaviour is now subtly changed. If your transform implements a non-affine transformation, then it should override the transform_non_affine method, rather than the generic transform method.
Therefore, simply reimplementing the transform_non_affine instead of the transform method, as said above, in the FixedPointOffsetTransform class seems to solve the issue:
class FixedPointOffsetTransform(mtrans.Transform):
"""
Always returns the same transformed point plus
the given point in device coordinates as an offset.
"""
def __init__(self, trans, fixed_point):
mtrans.Transform.__init__(self)
self.input_dims = self.output_dims = 2
self.has_inverse = False
self.trans = trans
self.fixed_point = np.array(fixed_point).reshape(1, 2)
def transform_non_affine(self, values):
fp = self.trans.transform(self.fixed_point)
values = np.array(values)
if values.ndim == 1:
return fp.flatten() + values
else:
return fp + values

get bins coordinates with hexbin in matplotlib

I use matplotlib's method hexbin to compute 2d histograms on my data.
But I would like to get the coordinates of the centers of the hexagons in order to further process the results.
I got the values using get_array() method on the result, but I cannot figure out how to get the bins coordinates.
I tried to compute them given number of bins and the extent of my data but i don't know the exact number of bins in each direction. gridsize=(10,2) should do the trick but it does not seem to work.
Any idea?

I think this works.
from __future__ import division
import numpy as np
import math
import matplotlib.pyplot as plt
def generate_data(n):
"""Make random, correlated x & y arrays"""
points = np.random.multivariate_normal(mean=(0,0),
cov=[[0.4,9],[9,10]],size=int(n))
return points
if __name__ =='__main__':
color_map = plt.cm.Spectral_r
n = 1e4
points = generate_data(n)
xbnds = np.array([-20.0,20.0])
ybnds = np.array([-20.0,20.0])
extent = [xbnds[0],xbnds[1],ybnds[0],ybnds[1]]
fig=plt.figure(figsize=(10,9))
ax = fig.add_subplot(111)
x, y = points.T
# Set gridsize just to make them visually large
image = plt.hexbin(x,y,cmap=color_map,gridsize=20,extent=extent,mincnt=1,bins='log')
# Note that mincnt=1 adds 1 to each count
counts = image.get_array()
ncnts = np.count_nonzero(np.power(10,counts))
verts = image.get_offsets()
for offc in xrange(verts.shape[0]):
binx,biny = verts[offc][0],verts[offc][1]
if counts[offc]:
plt.plot(binx,biny,'k.',zorder=100)
ax.set_xlim(xbnds)
ax.set_ylim(ybnds)
plt.grid(True)
cb = plt.colorbar(image,spacing='uniform',extend='max')
plt.show()

I would love to confirm that the code by Hooked using get_offsets() works, but I tried several iterations of the code mentioned above to retrieve center positions and, as Dave mentioned, get_offsets() remains empty. The workaround that I found is to use the non-empty 'image.get_paths()' option. My code takes the mean to find centers but which means it is just a smidge longer, but it does work.
The get_paths() option returns a set of x,y coordinates embedded that can be looped over and then averaged to return the center position for each hexagram.
The code that I have is as follows:
counts=image.get_array() #counts in each hexagon, works great
verts=image.get_offsets() #empty, don't use this
b=image.get_paths() #this does work, gives Path([[]][]) which can be plotted
for x in xrange(len(b)):
xav=np.mean(b[x].vertices[0:6,0]) #center in x (RA)
yav=np.mean(b[x].vertices[0:6,1]) #center in y (DEC)
plt.plot(xav,yav,'k.',zorder=100)

I had this same problem. I think what needs to be developed is a framework to have a HexagonalGrid object which can then be applied to many different data sets (and it would be awesome to do it for N dimensions). This is possible and it surprises me that neither Scipy or Numpy has anything for it (furthermore there seems to be nothing else like it except perhaps binify)
That said, I assume you want to use hexbinning to compare multiple binned data sets. This requires some common base. I got this to work using matplotlib's hexbin the following way:
import numpy as np
import matplotlib.pyplot as plt
def get_data (mean,cov,n=1e3):
"""
Quick fake data builder
"""
np.random.seed(101)
points = np.random.multivariate_normal(mean=mean,cov=cov,size=int(n))
x, y = points.T
return x,y
def get_centers (hexbin_output):
"""
about 40% faster than previous post only cause you're not calculating the
min/max every time
"""
paths = hexbin_output.get_paths()
v = paths[0].vertices[:-1] # adds a value [0,0] to the end
vx,vy = v.T
idx = [3,0,5,2] # index for [xmin,xmax,ymin,ymax]
xmin,xmax,ymin,ymax = vx[idx[0]],vx[idx[1]],vy[idx[2]],vy[idx[3]]
half_width_x = abs(xmax-xmin)/2.0
half_width_y = abs(ymax-ymin)/2.0
centers = []
for i in xrange(len(paths)):
cx = paths[i].vertices[idx[0],0]+half_width_x
cy = paths[i].vertices[idx[2],1]+half_width_y
centers.append((cx,cy))
return np.asarray(centers)
# important parts ==>
class Hexagonal2DGrid (object):
"""
Used to fix the gridsize, extent, and bins
"""
def __init__ (self,gridsize,extent,bins=None):
self.gridsize = gridsize
self.extent = extent
self.bins = bins
def hexbin (x,y,hexgrid):
"""
To hexagonally bin the data in 2 dimensions
"""
fig = plt.figure()
ax = fig.add_subplot(111)
# Note mincnt=0 so that it will return a value for every point in the
# hexgrid, not just those with count>mincnt
# Basically you fix the gridsize, extent, and bins to keep them the same
# then the resulting count array is the same
hexbin = plt.hexbin(x,y, mincnt=0,
gridsize=hexgrid.gridsize,
extent=hexgrid.extent,
bins=hexgrid.bins)
# you could close the figure if you don't want it
# plt.close(fig.number)
counts = hexbin.get_array().copy()
return counts, hexbin
# Example ===>
if __name__ == "__main__":
hexgrid = Hexagonal2DGrid((21,5),[-70,70,-20,20])
x_data,y_data = get_data((0,0),[[-40,95],[90,10]])
x_model,y_model = get_data((0,10),[[100,30],[3,30]])
counts_data, hexbin_data = hexbin(x_data,y_data,hexgrid)
counts_model, hexbin_model = hexbin(x_model,y_model,hexgrid)
# if you want the centers, they will be the same for both
centers = get_centers(hexbin_data)
# if you want to ignore the cells with zeros then use the following mask.
# But if want zeros for some bins and not others I'm not sure an elegant way
# to do this without using the centers
nonzero = counts_data != 0
# now you can compare the two data sets
variance_data = counts_data[nonzero]
square_diffs = (counts_data[nonzero]-counts_model[nonzero])**2
chi2 = np.sum(square_diffs/variance_data)
print(" chi2={}".format(chi2))