Develop Reference

How to bidirectionally link X axis of HoloViews (hvplot) plot with panel DatetimePicker (or DatetimeRangePicker) widget

Question:
I am struggling for a more than a week now to do something probably pretty simple:
I want to make a time series plot in which i can control the x axis
range/zoom with a datetime picker widget.
I also want the datetime picker to be updated when the x range is
changed with the plot zoom controls
So far I can do either but not both. It did work for other widgets such as the intslider etc.
Requirements:
If the solution has 1 DatetimeRangePicker to define the x range or 2 DatetimePicker widgets (one for start one for end) would both work great for me.
my datasets are huge so it would be great if it works with datashader
Any help is much appreciated :)
What I tried:
MRE & CODE BELOW
Create a DatetimeRangePicker widget, plot the data using pvplot and set the xlim=datatimerangepicker.
Result: the zoom changes with the selected dates on the widget, but zooming / panning the plot does not change the values of the widget.
Use hv.streams.RangeX stream to capture changes in x range when panning / zooming. Use a pn.depends function to generate plot when changing DatetimeRangePicker widget.
Result: the figure loads and zooming/panning changes the widget (but is very slow), but setting the widget causes AttributeError.
Create a DatetimePicker widget for start and end. Link them with widget.jslink() bidirectionally to x_range.start and x_range.end of the figure.
Result: figure loads but nothing changes when changing values on the widget or panning/zooming.
MRE & Failed Attempts
Create Dataset
import pandas as pd
import numpy as np
import panel as pn
import holoviews as hv
import hvplot.pandas
hv.extension('bokeh')
df = pd.DataFrame({'data': np.random.randint(0, 100, 100)}, index=pd.date_range(start="2022", freq='D', periods=100))
Failed Method 1:
plot changes with widget, but widget does not change with plot
range_select = pn.widgets.DatetimeRangePicker(value=(df.index[0], df.index[-1]))
pn.Column(df.data.hvplot.line(datashade=True, xlim=range_select), range_select)
Failed Method 2:
Slow and causes AttributeError: 'NoneType' object has no attribute 'id' when changing widget
range_select = pn.widgets.DatetimeRangePicker(value=(df.index[0], df.index[-1]))
#pn.depends(range_x=range_select.param.value)
def make_fig(range_x):
fig = df.data.hvplot.line(datashade=True, xlim=range_x)
pointer = hv.streams.RangeX(source=fig)
tabl = hv.DynamicMap(show_x, streams=[pointer]) # plot useless table to make it work
return fig + tabl
def show_x(x_range):
if x_range is not None:
range_select.value = tuple([pd.Timestamp(i).to_pydatetime() for i in x_range])
return hv.Table({"start": [x_range[0]], "stop": [x_range[1]]}, ["start", "stop"]) if x_range else hv.Table({})
pn.Column(range_select, make_fig)
Failed Method 3:
does not work with DatetimePicker widget, but does work other widgets (e.g. intslider)
pn.widgets.DatetimePicker._source_transforms = ({}) # see https://discourse.holoviz.org/t/using-jslink-with-pn-widgets-datepicker/1116
# datetime range widgets
range_strt = pn.widgets.DatetimePicker()
range_end = pn.widgets.DatetimePicker()
# int sliders as example that some widgets work
int_start_widget = pn.widgets.IntSlider(start=0, step=int(1e6), end=int(1.7e12))
int_end_widget = pn.widgets.IntSlider(start=0, step=int(1e6), end=int(1.7e12))
points = df.data.hvplot.line(datashade=True) # generate plot
# link widgets to plot:
int_start_widget.jslink(points, value="x_range.start", bidirectional=True)
int_end_widget.jslink(points, value="x_range.end", bidirectional=True)
range_strt.jslink(points, value="x_range.start", bidirectional=True)
range_end.jslink(points, value="x_range.end", bidirectional=True)
pn.Row(points,pn.Column( range_strt, range_end, int_start_widget, int_end_widget,))

Here is what I came up with:
range_select = pn.widgets.DatetimeRangePicker(value=(df.index[0].to_pydatetime(), df.index[-1].to_pydatetime()))
curve = df.data.hvplot.line(datashade=True).apply.opts(xlim=range_select, framewise=True)
rxy = hv.streams.RangeX(source=curve)
def update_widget(event):
new_dates = tuple([pd.Timestamp(i).to_pydatetime() for i in event.new])
if new_dates != range_select.value:
range_select.value = new_dates
rxy.param.watch(update_widget, 'x_range')
pn.Column(range_select, curve)
Basically we use .apply.opts to apply current widget value as the xlim dynamically (and set framewise=True so the plot ranges update dynamically). Then we instantiate a RangeX stream which we use to update the widget value. Annoyingly we have to do some datetime conversions because np.datetime64 and Timestamps aren't supported in some cases.

How can i Plot arrows in a existing mplsoccer pitch?

I tried to do the tutorial of McKay Johns on YT (reference to the Jupyter Notebook to see the data (https://github.com/mckayjohns/passmap/blob/main/Pass%20map%20tutorial.ipynb).
I understood everything but I wanted to do a little change. I wanted to change plt.plot(...) with:
plt.arrow(df['x'][x],df['y'][x], df['endX'][x] - df['x'][x], df['endY'][x]-df['y'][x],
shape='full', color='green')
But the problem is, I still can't see the arrows. I tried multiple changes but I've failed. So I'd like to ask you in the group.
Below you can see the code.
## Read in the data
df = pd.read_csv('...\Codes\Plotting_Passes\messibetis.csv')
#convert the data to match the mplsoccer statsbomb pitch
#to see how to create the pitch, watch the video here: https://www.youtube.com/watch?v=55k1mCRyd2k
df['x'] = df['x']*1.2
df['y'] = df['y']*.8
df['endX'] = df['endX']*1.2
df['endY'] = df['endY']*.8
# Set Base
fig ,ax = plt.subplots(figsize=(13.5,8))
# Change background color of base
fig.set_facecolor('#22312b')
# Change color of base inside
ax.patch.set_facecolor('#22312b')
#this is how we create the pitch
pitch = Pitch(pitch_type='statsbomb',
pitch_color='#22312b', line_color='#c7d5cc')
# Set the axes to our Base
pitch.draw(ax=ax)
# X-Achsen => 0 to 120
# Y-Achsen => 80 to 0
# Lösung: Y-Achse invertieren:
plt.gca().invert_yaxis()
#use a for loop to plot each pass
for x in range(len(df['x'])):
if df['outcome'][x] == 'Successful':
#plt.plot((df['x'][x],df['endX'][x]),(df['y'][x],df['endY'][x]),color='green')
plt.scatter(df['x'][x],df['y'][x],color='green')
**plt.arrow(df['x'][x],df['y'][x], df['endX'][x] - df['x'][x], df['endY'][x]-df['y'][x],
shape='full', color='green')** # Here is the problem!
if df['outcome'][x] == 'Unsuccessful':
plt.plot((df['x'][x],df['endX'][x]),(df['y'][x],df['endY'][x]),color='red')
plt.scatter(df['x'][x],df['y'][x],color='red')
plt.title('Messi Pass Map vs Real Betis',color='white',size=20)
It always shows:

The problem is that plt.arrow has default values for head_width and head_length, which are too small for your figure. I.e. it is drawing arrows, the arrow heads are just way too tiny to see them (even if you zoom out). E.g. try something as follows:
import pandas as pd
import matplotlib.pyplot as plt
from mplsoccer.pitch import Pitch
df = pd.read_csv('https://raw.githubusercontent.com/mckayjohns/passmap/main/messibetis.csv')
...
# create a dict for the colors to avoid repetitive code
colors = {'Successful':'green', 'Unsuccessful':'red'}
for x in range(len(df['x'])):
plt.scatter(df['x'][x],df['y'][x],color=colors[df.outcome[x]], marker=".")
plt.arrow(df['x'][x],df['y'][x], df['endX'][x] - df['x'][x],
df['endY'][x]-df['y'][x], color=colors[df.outcome[x]],
head_width=1, head_length=1, length_includes_head=True)
# setting `length_includes_head` to `True` ensures that the arrow head is
# *part* of the line, not added on top
plt.title('Messi Pass Map vs Real Betis',color='white',size=20)
Result:
Note that you can also use plt.annotate for this, passing specific props to the parameter arrowprops. E.g.:
import pandas as pd
import matplotlib.pyplot as plt
from mplsoccer.pitch import Pitch
df = pd.read_csv('https://raw.githubusercontent.com/mckayjohns/passmap/main/messibetis.csv')
...
# create a dict for the colors to avoid repetitive code
colors = {'Successful':'green', 'Unsuccessful':'red'}
for x in range(len(df['x'])):
plt.scatter(df['x'][x],df['y'][x],color=colors[df.outcome[x]], marker=".")
props= {'arrowstyle': '-|>,head_width=0.25,head_length=0.5',
'color': colors[df.outcome[x]]}
plt.annotate("", xy=(df['endX'][x],df['endY'][x]),
xytext=(df['x'][x],df['y'][x]), arrowprops=props)
plt.title('Messi Pass Map vs Real Betis',color='white',size=20)
Result (a bit sharper, if you ask me, but maybe some tweaking with params in plt.arrow can also achieve that):

How to animate a plot in python using the VisVis package?

I am trying to animate a plot using visvis.
This is the example code they have:
import visvis as vv
# read image
ims = [vv.imread('astronaut.png')]
# make list of images: decrease red channel in subsequent images
for i in range(9):
im = ims[i].copy()
im[:,:,0] = im[:,:,0]*0.9
ims.append(im)
# create figure, axes, and data container object
a = vv.gca()
m = vv.MotionDataContainer(a)
# create textures, loading them into opengl memory, and insert into container.
for im in ims:
t = vv.imshow(im)
t.parent = m
and I added:
app = vv.use()
app.Run()
This worked. But I needed to animate a plot, not an image, so I tried doing this:
import visvis as vv
from visvis.functions import getframe
# create figure, axes, and data container object
a = vv.gca()
m = vv.MotionDataContainer(a, interval=100)
for i in range(3):
vv.plot([0, 2+i*10], [0, 2+i*10])
f = getframe(a)
t = vv.imshow(f)
t.parent = m
a.SetLimits(rangeX=[-2, 25], rangeY=[-2, 25])
app = vv.use()
app.Run()
The axes are being initialized very big, that is why I am using set limits, and the output is not animated. I am getting only the last frame so a line from (0,0) to (22, 22).
Does anyone know a way of doing this with visvis?

It turns out adding the frame as a child of MotionDataContainer was not the way to go. The function vv.plot returns an instance of the class Line, and one should add the line as a child. If anyone is having the same problem, I could write a more detailed answer.
EDIT Adding a more detailed answer as requested:
To animate a plot made of lines, one must simply add the lines as children of MotionDataContainer. Taking my example in the question above, one would write:
import visvis as vv
# create figure, axes, and data container object
a = vv.gca()
m = vv.MotionDataContainer(a, interval=100)
for i in range(3):
line = vv.plot([0, 2+i*10], [0, 2+i*10])
line.parent = m
app = vv.use()
app.Run()
In my special case, I even needed to animate multiple lines being drawn at the same time.
To do this, I ended up defining a new class that, like MotionDataContainer, also inherits from MotionMixin, and change the class attribute delta which specifies how many objects should be made visible at the same time. For that, one has to also rewrite the function _SetMotionIndex.
(See visvis official source code: https://github.com/almarklein/visvis/blob/master/wobjects/motion.py)
Disclaimer: Concerning the animation of multiple objects, I have no idea if this is the intended use or if this is the easiest solution, but this is what worked for me.

call back from slider change not updating my plot in bokeh in jupyter lab?

I am working on a Bokeh visualisation of datasets across a number of categories. The initial part of the visual is a donut chart of the categories showing the total number of items in each category. I am trying to get the chart to update based on a min-max range using RangeSlider - but the chart does not update.
The input source for the glyphs is the output from a create_cat_df - which is returned as a Pandas DF, then converted into a CDS using ColumnDataSource.from_df().
The chart appears okay when this code is run (with slider alongside) - but moving the slider changes nothing.
There is a similar post here.
The answer here was useful in putting me onto from_df - but even after following this I can't get the code to work.
def create_doc(doc):
### INPUT widget
cat_min_max = RangeSlider(start=0, end=1000, value=[0, 1000], step=1, title="Category min-max items (m)")
inputs = column(cat_min_max, width=300, height=850) # in preparation for multiple widgets
### Tooltip & tools
TOOLTIPS_2 = [("Item", "$item") # a sample
]
hover_2 = HoverTool(tooltips=TOOLTIPS_2, names = ['cat'])
tools = [hover_2, TapTool(), WheelZoomTool(), PanTool(), ResetTool()]
### Create Figure
p = figure(plot_width=width, plot_height=height, title="",
x_axis_type=None, y_axis_type=None,
x_range=(-420, 420), y_range=(-420, 420),
min_border=0, outline_line_color=None,
background_fill_color="#f0e1d2",
tools = tools, toolbar_location="left")
p.xgrid.grid_line_color = None
p.ygrid.grid_line_color = None
# taptool
url = "https://google.com/" #dummy URL
taptool = p.select(type=TapTool)
taptool.callback = OpenURL(url=url)
# create cat_source CDS using create_cat_df function (returns pandas df) and 'from_df' method
cat_source = ColumnDataSource.from_df(create_cat_df(cat_min_max.value[0], cat_min_max.value[1]))
## plot category wedges
p.annular_wedge('centre_x', 'centre_y', 'inner', 'outer', 'start', 'end', color='color',
alpha='alpha', direction='clock', source=cat_source, name='cat')
r = row([inputs, p])
def callback(attr, old, new):
cat_source.data = ColumnDataSource.from_df(create_cat_df(cat_min_max.value[0], cat_min_max.value[1]))
cat_min_max.on_change('value', callback)
doc.add_root(r)
show(create_doc)
I would like to get the code working and the chart updating. There are a number more glyphs & different data layers to layer in, but I want to get the basics working first.

According to Bokeh documentation the ColumnDataSource.from_df() method returns a dictionary while you need to pass a ColumnDatSource to the source argument in p.annular_wedge(source = cat_source)
So instead of:
cat_source = ColumnDataSource.from_df(create_cat_df(cat_min_max.value[0], cat_min_max.value[1]))
You should do:
cat_source = ColumnDataSource(data = ColumnDataSource.from_df(create_cat_df(cat_min_max.value[0], cat_min_max.value[1])))

Seaborn clustermap fixed cell size

I am using the seaborn clustermap function and I would like to make multiple plots where the cell sizes are exactly identical. Also the size of the axis labels should be the same. This means figure size and aspect ratio will need to change, the rest needs to stay identical.
import pandas
import seaborn
import numpy as np
dataFrameA = pd.DataFrame([ [1,2],[3,4] ])
dataFrameB = pd.DataFrame( np.arange(3*6).reshape(3,-1))
Then decide how big the clustermap itself needs to be, something along the lines of:
dpi = 72
cellSizePixels = 150
This decides that dataFrameA should be should be 300 by 300 pixels. I think that those need to be converted to the size units of the figure, which will be cellSizePixels/dpi units per pixel. So for dataFrameA that will be a heatmap size of ~2.01 inches. Here I am introducing a problem: there is stuff around the heatmap, which will also take up some space, and I don't know how much space those will exactly take.
I tried to parametrize the heatmap function with a guess of the image size using the formula above:
def fixedWidthClusterMap( dpi, cellSizePixels, dataFrame):
clustermapParams = {
'square':False # Tried to set this to True before. Don't: the dendograms do not scale well with it.
}
figureWidth = (cellSizePixels/dpi)*dataFrame.shape[1]
figureHeight= (cellSizePixels/dpi)*dataFrame.shape[0]
return sns.clustermap( dataFrame, figsize=(figureWidth,figureHeight), **clustermapParams)
fixedWidthClusterMap(dpi, cellSizePixels, dataFrameA)
plt.show()
fixedWidthClusterMap(dpi, cellSizePixels, dataFrameB)
plt.show()
This yields:
My question: how do I obtain square cells which are exactly the size I want?

This is a bit tricky, because there are quite a few things to take into consideration, and in the end, it depends how "exact" you need the sizes to be.
Looking at the code for clustermap the heatmap part is designed to have a ratio of 0.8 compared to the axes used for the dendrograms. But we also need to take into account the margins used to place the axes. If one knows the size of the heatmap axes, one should therefore be able to calculate the desired figure size that would produce the right shape.
dpi = matplotlib.rcParams['figure.dpi']
marginWidth = matplotlib.rcParams['figure.subplot.right']-matplotlib.rcParams['figure.subplot.left']
marginHeight = matplotlib.rcParams['figure.subplot.top']-matplotlib.rcParams['figure.subplot.bottom']
Ny,Nx = dataFrame.shape
figWidth = (Nx*cellSizePixels/dpi)/0.8/marginWidth
figHeigh = (Ny*cellSizePixels/dpi)/0.8/marginHeight
Unfortunately, it seems matplotlib must adjust things a bit during plotting, because that was not enough the get perfectly square heatmap cells. So I choose to resize the various axes create by clustermap after the fact, starting with the heatmap, then the dendrogram axes.
I think the resulting image is pretty close to what you were trying to get, but my tests sometime show some errors by 1-2 px, which I attribute to rounding errors due to all the conversions between sizes in inches and pixels.
dataFrameA = pd.DataFrame([ [1,2],[3,4] ])
dataFrameB = pd.DataFrame( np.arange(3*6).reshape(3,-1))
def fixedWidthClusterMap(dataFrame, cellSizePixels=50):
# Calulate the figure size, this gets us close, but not quite to the right place
dpi = matplotlib.rcParams['figure.dpi']
marginWidth = matplotlib.rcParams['figure.subplot.right']-matplotlib.rcParams['figure.subplot.left']
marginHeight = matplotlib.rcParams['figure.subplot.top']-matplotlib.rcParams['figure.subplot.bottom']
Ny,Nx = dataFrame.shape
figWidth = (Nx*cellSizePixels/dpi)/0.8/marginWidth
figHeigh = (Ny*cellSizePixels/dpi)/0.8/marginHeight
# do the actual plot
grid = sns.clustermap(dataFrame, figsize=(figWidth, figHeigh))
# calculate the size of the heatmap axes
axWidth = (Nx*cellSizePixels)/(figWidth*dpi)
axHeight = (Ny*cellSizePixels)/(figHeigh*dpi)
# resize heatmap
ax_heatmap_orig_pos = grid.ax_heatmap.get_position()
grid.ax_heatmap.set_position([ax_heatmap_orig_pos.x0, ax_heatmap_orig_pos.y0,
axWidth, axHeight])
# resize dendrograms to match
ax_row_orig_pos = grid.ax_row_dendrogram.get_position()
grid.ax_row_dendrogram.set_position([ax_row_orig_pos.x0, ax_row_orig_pos.y0,
ax_row_orig_pos.width, axHeight])
ax_col_orig_pos = grid.ax_col_dendrogram.get_position()
grid.ax_col_dendrogram.set_position([ax_col_orig_pos.x0, ax_heatmap_orig_pos.y0+axHeight,
axWidth, ax_col_orig_pos.height])
return grid # return ClusterGrid object
grid = fixedWidthClusterMap(dataFrameA, cellSizePixels=75)
plt.show()
grid = fixedWidthClusterMap(dataFrameB, cellSizePixels=75)
plt.show()

Not a complete answer (not dealing with pixels) but I suspect OP has moved on after 4 years.
def reshape_clustermap(cmap, cell_width=0.02, cell_height=0.02):
ny, nx = cmap.data2d.shape
hmap_width = nx * cell_width
hmap_height = ny * cell_height
hmap_orig_pos = cmap.ax_heatmap.get_position()
cmap.ax_heatmap.set_position(
[hmap_orig_pos.x0, hmap_orig_pos.y0, hmap_width, hmap_height]
)
top_dg_pos = cmap.ax_col_dendrogram.get_position()
cmap.ax_col_dendrogram.set_position(
[hmap_orig_pos.x0, hmap_orig_pos.y0 + hmap_height, hmap_width, top_dg_pos.height]
)
left_dg_pos = cmap.ax_row_dendrogram.get_position()
cmap.ax_row_dendrogram.set_position(
[left_dg_pos.x0, left_dg_pos.y0, left_dg_pos.width, hmap_height]
)
if cmap.ax_cbar:
cbar_pos = cmap.ax_cbar.get_position()
hmap_pos = cmap.ax_heatmap.get_position()
cmap.ax_cbar.set_position(
[cbar_pos.x0, hmap_pos.y1, cbar_pos.width, cbar_pos.height]
)
cmap = sns.clustermap(dataFrameA)
reshape_clustermap(cmap)