Develop Reference

Python: scatter plot with non-linear x axis

I have data with lots of x values around zero and only a few as you go up to around 950,
I want to create a plot with a non-linear x axis so that the relationship can be seen in a 'straight line' form. Like seen in this example,
I have tried using plt.xscale('log') but it does not achieve what I want.
I have not been able to use the log scale function with a scatter plot as it then only shows 3 values rather than the thousands that exist.
I have tried to work around it using
plt.plot(retper, aep_NW[y], marker='o', linewidth=0)
to replicate the scatter function which plots but does not show what I want.
plt.figure(1)
plt.scatter(rp,aep,label="SSI sum")
plt.show()
Image 3:
plt.figure(3)
plt.scatter(rp, aep)
plt.xscale('log')
plt.show()
Image 4:
plt.figure(4)
plt.plot(rp, aep, marker='o', linewidth=0)
plt.xscale('log')
plt.show()
ADDITION:
Hi thank you for the response.
I think you are right that my x axis is truncated but I'm not sure why or how...
I'm not really sure what to post code wise as the data is all large and coming from a server so can't really give you the data to see it with.
Basically aep_NW is a one dimensional array with 951 elements, values from 0-~140, with most values being small and only a few larger values. The data represents a storm severity index for 951 years.
Then I want the x axis to be the return period for these values, so basically I made a rp array, of the same size, which is given values from 951 down decreasing my a half each time.
I then sort the aep_NW values from lowest to highest with the highest value being associated with the largest return value (951), then the second highest aep_NW value associated with the second largest return period value (475.5) ect.
So then when I plot it I need the x axis scale to be similar to the example you showed above or the first image I attatched originally.
rp = [0]*numseas.shape[0]
i = numseas.shape[0] - 1
rp[i] = numseas.shape[0]
i = i - 1
while i != 0:
rp[i] = rp[i+1]/2
i = i - 1
y = np.argsort(aep_NW)
fig, ax = plt.subplots()
ax.scatter(rp,aep_NW[y],label="SSI sum")
ax.set_xscale('log')
ax.set_xlabel("Return period")
ax.set_ylabel("SSI score")
plt.title("AEP for NW Europe: total loss per entire extended winter season")
plt.show()

It looks like in your "Image 3" the x axis is truncated, so that you don't see the data you are interested in. It appears this is due to there being 0's in your 'rp' array. I updated the examples to show the error you are seeing, one way to exclude the zeros, and one way to clip them and show them on a different scale.
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
n = 100
numseas = np.logspace(-5, 3, n)
aep_NW = np.linspace(0, 140, n)
rp = [0]*numseas.shape[0]
i = numseas.shape[0] - 1
rp[i] = numseas.shape[0]
i = i - 1
while i != 0:
rp[i] = rp[i+1] /2
i = i - 1
y = np.argsort(aep_NW)
fig, axes = plt.subplots(1, 3, figsize=(14, 5))
ax = axes[0]
ax.scatter(rp, aep_NW[y], label="SSI sum")
ax.set_xscale('log')
ax.set_xlabel("Return period")
ax.set_ylabel("SSI score")
ax = axes[1]
rp = np.array(rp)[y]
mask = rp > 0
ax.scatter(rp[mask], aep_NW[y][mask], label="SSI sum")
ax.set_xscale('log')
ax.set_xlabel("Return period (0 values excluded)")
ax = axes[2]
log2_clipped_rp = np.log2(rp.clip(2**-100, None))[y]
ax.scatter(log2_clipped_rp, aep_NW[y], label="SSI sum")
xticks = list(range(-110, 11, 20))
xticklabels = [f'$2^{{{i}}}$' for i in xticks]
ax.set_xticks(xticks)
ax.set_xticklabels(xticklabels)
ax.set_xlabel("log$_2$ Return period (values clipped to 2$^{-100}$)")
plt.show()

Shift plots of different lengths in the same x-axis

I have a very specific something I want to do with matplotlib I don't even know if it's possible, but I figured it was worth asking. Maybe the answers will give me an alternate idea about how to go about it.
I have 4 arrays of similar, but different lengths that I want to plot in the same x-axis. This question suggests creating the values for x using range(), and it worked:
plt.figure(figsize=(8, 6), dpi=300)
x_5 = range(len(all_data_float[0]))
plt.plot(x_5, all_data_float[0], color='b', marker='.')
x_10 = range(len(all_data_float[1]))
plt.plot(x_10, all_data_float[1], color='r', marker='.')
x_15 = range(len(all_data_float[2]))
plt.plot(x_15, all_data_float[2], color='g', marker='.')
x_20 = range(len(all_data_float[3]))
plt.plot(x_20, all_data_float[3], color='c', marker='.')
plt.show()
But I wanted to do something else, I want to plot a vertical line in the middle aligned by a point, for example there are 4 plots with:
plot1: 101 points
plot2: 99 points
plot3: 100 points
plot4: 101 points
So for plot1, that point would be index 51, which means 51 points before and 49 after with the line crossing point 51. For plot2, that middle point is index 49, which means 49 points before and 50 after, and so forth.
My difficulty is that the vertical line has a different index for each plot. I know plt.vlines() accepts an array, but in this case it plots multiple lines, and I wanted a single line.
Is there a way to "shift" each plot relative to the x-axis? so index 51 of plot1 aligns with index 49 of plot2, etc? Or is there a better strategy to do this?

From the set up of the question I am going to assume that the the x-values do not have any numerical meaning so it is safe from a data-point-of-view to shift them around. Instead of plotting your data against range(len(...)), do the shift there!
import matpoltlib.pyplot as plt
import numpy as np
def synthetic_data(length):
"make some variable length synthetic data to plot."
return np.exp(-((np.linspace(-5, 5, length)) ** 2))
data = [synthetic_data(51), synthetic_data(75), synthetic_data(105)]
fig, ax = plt.subplots(constrained_layout=True)
for d in data:
x_vector = np.arange(len(d)) - len(d) // 2
ax.plot(x_vector, d)
ax.axvline(0, color="k", ls="--")
ax.set_xlabel("delta from center")
ax.set_ylabel("synthetic data!")

How to plot scipy.hierarchy.dendrogram using polar coordinates?

I'm trying to adapt the following resources to this question:
Python conversion between coordinates
https://matplotlib.org/gallery/pie_and_polar_charts/polar_scatter.html
I can't seem to get the coordinates to transfer the dendrogram shape over to polar coordinates.
Does anyone know how to do this? I know there is an implementation in networkx but that requires building a graph and then using pygraphviz backend to get the positions.
Is there a way to convert dendrogram cartesian coordinates to polar coordinates with matplotlib and numpy?
import requests
from ast import literal_eval
import matplotlib.pyplot as plt
import numpy as np
def read_url(url):
r = requests.get(url)
return r.text
def cartesian_to_polar(x, y):
rho = np.sqrt(x**2 + y**2)
phi = np.arctan2(y, x)
return(rho, phi)
def plot_dendrogram(icoord,dcoord,figsize, polar=False):
if polar:
icoord, dcoord = cartesian_to_polar(icoord, dcoord)
with plt.style.context("seaborn-white"):
fig = plt.figure(figsize=figsize)
ax = fig.add_subplot(111, polar=polar)
for xs, ys in zip(icoord, dcoord):
ax.plot(xs,ys, color="black")
ax.set_title(f"Polar= {polar}", fontsize=15)
# Load the dendrogram data
string_data = read_url("https://pastebin.com/raw/f953qgdr").replace("\r","").replace("\n","").replace("\u200b\u200b","")
# Convert it to a dictionary (a subset of the output from scipy.hierarchy.dendrogram)
dendrogram_data = literal_eval(string_data)
icoord = np.asarray(dendrogram_data["icoord"], dtype=float)
dcoord = np.asarray(dendrogram_data["dcoord"], dtype=float)
# Plot the cartesian version
plot_dendrogram(icoord,dcoord, figsize=(8,3), polar=False)
# Plot the polar version
plot_dendrogram(icoord,dcoord, figsize=(5,5), polar=True)
I just tried this and it's closer but still not correct:
import matplotlib.transforms as mtransforms
with plt.style.context("seaborn-white"):
fig, ax = plt.subplots(figsize=(5,5))
for xs, ys in zip(icoord, dcoord):
ax.plot(xs,ys, color="black",transform=trans_offset)
ax_polar = plt.subplot(111, projection='polar')
trans_offset = mtransforms.offset_copy(ax_polar.transData, fig=fig)
for xs, ys in zip(icoord, dcoord):
ax_polar.plot(xs,ys, color="black",transform=trans_offset)

You can make the "root" of the tree start in the middle and have the leaves outside. You also have to add more points to the "bar" part for it to look nice and round.
We note that each element of icoord and dcoord (I will call this seg) has four points:
seg[1] seg[2]
+-------------+
| |
+ seg[0] + seg[3]
The vertical bars are fine as straight lines between the two points, but we need more points between seg[1] and seg[2] (the horizontal bar, which will need to become an arc).
This function will add more points in those positions and can be called on both xs and ys in the plotting function:
def smoothsegment(seg, Nsmooth=100):
return np.concatenate([[seg[0]], np.linspace(seg[1], seg[2], Nsmooth), [seg[3]]])
Now we must modify the plotting function to calculate the radial coordinates. Some experimentation has led to the log formula I am using, based on the other answer which also uses log scale. I've left a gap open on the right for the radial labels and done a very rudimentary mapping of the "icoord" coordinates to the radial ones so that the labels correspond to the ones in the rectangular plot. I don't know exactly how to handle the radial dimension. The numbers are correct for the log, but we probably want to map them as well.
def plot_dendrogram(icoord,dcoord,figsize, polar=False):
if polar:
dcoord = -np.log(dcoord+1)
# avoid a wedge over the radial labels
gap = 0.1
imax = icoord.max()
imin = icoord.min()
icoord = ((icoord - imin)/(imax - imin)*(1-gap) + gap/2)*2*numpy.pi
with plt.style.context("seaborn-white"):
fig = plt.figure(figsize=figsize)
ax = fig.add_subplot(111, polar=polar)
for xs, ys in zip(icoord, dcoord):
if polar:
xs = smoothsegment(xs)
ys = smoothsegment(ys)
ax.plot(xs,ys, color="black")
ax.set_title(f"Polar= {polar}", fontsize=15)
if polar:
ax.spines['polar'].set_visible(False)
ax.set_rlabel_position(0)
Nxticks = 10
xticks = np.linspace(gap/2, 1-gap/2, Nxticks)
ax.set_xticks(xticks*np.pi*2)
ax.set_xticklabels(np.round(np.linspace(imin, imax, Nxticks)).astype(int))
Which results in the following figure:

First, I think you might benefit from this question.
Then, let's break down the objective: it is not very clear to me what you want to do, but I assume you want to get something that looks like this
(source, page 14)
To render something like this, you need to be able to render horizontal lines that appear as hemi-circles in polar coordinates. Then, it's a matter of mapping your horizontal lines to polar plot.
First, note that your radius are not normalized in this line:
if polar:
icoord, dcoord = cartesian_to_polar(icoord, dcoord)
you might normalize them by simply remapping icoord to [0;2pi).
Now, let's try plotting something simpler, instead of your complex plot:
icoord, dcoord = np.meshgrid(np.r_[1:10], np.r_[1:4])
# Plot the cartesian version
plot_dendrogram(icoord, dcoord, figsize=(8, 3), polar=False)
# Plot the polar version
plot_dendrogram(icoord, dcoord, figsize=(5, 5), polar=True)
Result is the following:
as you can see, the polar code does not map horizontal lines to semi-circles, therefore that is not going to work. Let's try with plt.polar instead:
plt.polar(icoord.T, dcoord.T)
produces
which is more like what we need. We need to fix the angles first, and then we shall consider that Y coordinate goes inward (while you probably want it going from center to border). It boils down to this
nic = (icoord.T - icoord.min()) / (icoord.max() - icoord.min())
plt.polar(2 * np.pi * nic, -dcoord.T)
which produces the following
Which is similar to what you need. Note that straight lines remain straight, and are not replaced with arcs, so you might want to resample them in your for loop.
Also, you might benefit from single color and log-scale to make reading easier
plt.subplots(figsize=(10, 10))
ico = (icoord.T - icoord.min()) / (icoord.max() - icoord.min())
plt.polar(2 * np.pi * ico, -np.log(dcoord.T), 'b')

Scale colormap for contour and contourf

I'm trying to plot the contour map of a given function f(x,y), but since the functions output scales really fast, I'm losing a lot of information for lower values of x and y. I found on the forums to work that out using vmax=vmax, it actually worked, but only when plotted for a specific limit of x and y and levels of the colormap.
Say I have this plot:
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
u = np.linspace(-2,2,1000)
x,y = np.meshgrid(u,u)
z = (1-x)**2+100*(y-x**2)**2
cont = plt.contour(x,y,z,500,colors='black',linewidths=.3)
cont = plt.contourf(x,y,z,500,cmap="jet",vmax=100)
plt.colorbar(cont)
plt.show
I want to uncover whats beyond the axis limits keeping the same scale, but if I change de x and y limits to -3 and 3 I get:
See how I lost most of my levels since my max value for the function at these limits are much higher. A work around to this problem is to increase the levels to 1000, but that takes a lot of computational time.
Is there a way to plot only the contour levels that I need? That is, between 0 and 100.
An example of a desired output would be:
With the white space being the continuation of the plot without resizing the levels.
The code I'm using is the one given after the first image.

There are a few possible ideas here. The one I very much prefer is a logarithmic representation of the data. An example would be
from matplotlib import ticker
fig = plt.figure(1)
cont1 = plt.contourf(x,y,z,cmap="jet",locator=ticker.LogLocator(numticks=10))
plt.colorbar(cont1)
plt.show()
fig = plt.figure(2)
cont2 = plt.contourf(x,y,np.log10(z),100,cmap="jet")
plt.colorbar(cont2)
plt.show()
The first example uses matplotlibs LogLocator functions. The second one just directly computes the logarithm of the data and plots that normally.
The third example just caps all data above 100.
fig = plt.figure(3)
zcapped = z.copy()
zcapped[zcapped>100]=100
cont3 = plt.contourf(x,y,zcapped,100,cmap="jet")
cbar = plt.colorbar(cont3)
plt.show()

matplotlib pyplot 2 plots with different axes in same figure

I have a small issue with matplotlib.pyplot and I hope someone might have come across it before.
I have data that contain X,Y,e values that are the X, Y measurements of a variable and e are the errors of the measurements in Y. I need to plot them in a log log scale.
I use the plt.errorbars function to plot them and then set yscale and xscale to log and this works fine. But I need to also plot a line on the same graph that needs to be in linear scale.
I am able to have the plots done separately just fine but I would like to have them in the same image if possible. Do you have any ideas? I am posting what I have done for now.
Cheers,
Kimon
tdlist = np.array([0.01,0.02,0.05,0.1,0.2,0.3,0.4,0.5,0.8,1,2,5,10,15,20,25,30,40,60,80,100,150,200,250,300,400])
freqlist=np.array([30,40,50,60,70,80,90,100,110,120,140,160,180,200,220,250,300,350,400,450])
filename=opts.filename
data = reader(filename)
data2 = logconv(data)
#x,y,e the data. Calculating usefull sums
x = data2[0]
y = data2[1]
e = data2[2]
xoe2 = np.sum(x/e**2)
yoe2 = np.sum(y/e**2)
xyoe2 = np.sum(x*y/e**2)
oe2 = np.sum(1/e**2)
x2oe2 = np.sum(x**2/e**2)
aslope = (xoe2*yoe2-xyoe2*oe2)/(xoe2**2-x2oe2*oe2)
binter = (xyoe2-aslope*x2oe2)/xoe2
aerr = np.sqrt(oe2/(x2oe2*oe2-xoe2**2))
berr = np.sqrt(x2oe2/(x2oe2*oe2-xoe2**2))
print('slope is ',aslope,' +- ', aerr)
print('inter is ',binter,' +- ', berr)
fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
ax2 = fig.add_axes(ax1.get_position(), frameon=False)
ax1.errorbar(data[0],data[1],yerr=data[2],fmt='o')
ax1.set_xscale('log',basex=10)
ax1.set_yscale('log',basey=10)
ax1.set_yticks([])
ax1.set_xticks([])
ax2.plot(x,aslope*x+binter,'r')
ax2.plot(x,(aslope-aerr)*x+(binter+berr),'--')
ax2.plot(x,(aslope+aerr)*x+(binter-berr),'--')
ax2.set_xscale('linear')
ax2.set_yscale('linear')
plt.xticks(np.log10(freqlist),freqlist.astype('int'))
plt.yticks(np.log10(tdlist),tdlist.astype('float'))
plt.xlabel('Frequency (MHz)')
plt.ylabel('t_s (msec)')
fitndx1 = 'Fit slope '+"{0:.2f}".format(aslope)+u"\u00B1"+"{0:.2f}".format(aerr)
plt.legend(('Data',fitndx1))
plt.show()
Following Molly's suggestion I managed to get closer to my goal but still not there. I am adding a bit more info for what I am trying to do and it might clarify things a bit.
I am setting ax1 to the errobar plot that uses loglog scale. I need to use errorbar and not loglog plot so that I can display the errors with my points.
I am using ax2 to plot the linear fit in linealinear scale.
Moreover I do not want the x and y axes to display values that are 10,100,1000 powers of ten but my own axes labels that have the spacing I want therefore I am using the plt.xticks. I tried ax1.set_yticks and ax1.set_yticklabes but with no success. Below is the image I am getting.
I do not have enough reputation to post an image but here is the link of it uploaded
http://postimg.org/image/uojanigab/
The values of my points should be x range = 40 - 80 and y range = 5 -200 as the fit lines are now.

You can create two overlapping axes using the add_suplot method of figure. Here's an example:
from matplotlib import pyplot as plt
fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
ax2 = fig.add_axes(ax1.get_position(), frameon=False)
ax1.loglog([1,10,100,1000],[1000,1,100,10])
ax2.plot([5,10,11,13],'r')
plt.show()
You can then turn off the x and y ticks for the linear scale plot like this:
ax2.set_xticks([])
ax2.set_yticks([])

I was not able to get two sets of axis working with the errorbar function so I had to convert everything to log scale including my linear plot. Below is the code I use to get it might be useful to someone.
plt.errorbar(data[0],data[1],yerr=data[2],fmt='o')
plt.xscale('log',basex=10)
plt.yscale('log',basey=10)
plt.plot(data[0],data[0]**aslope*10**binter,'r')
plt.plot(data[0],data[0]**(aslope-aerr)*10**(binter+berr),'--')
plt.plot(data[0],data[0]**(aslope+aerr)*10**(binter-berr),'--')
plt.xticks(freqlist,freqlist.astype('int'))
plt.yticks(tdlist,tdlist.astype('float'))
plt.xlabel('Frequency (MHz)')
plt.ylabel('t_s (msec)')
fitndx1 = 'Fit slope '+"{0:.2f}".format(aslope)+u"\u00B1"+"{0:.2f}".format(aerr)
plt.legend(('Data',fitndx1))
plt.show()
And here is the link to the final image
http://postimg.org/image/bevj2k6nf/