Develop Reference

Python matplotlib polar coordinate is not plotting as it is supposed to be

I am plotting from a CSV file that contains Cartesian coordinates and I want to change it to Polar coordinates, then plot using the Polar coordinates.
Here is the code
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
df = pd.read_csv('test_for_plotting.csv',index_col = 0)
x_temp = df['x'].values
y_temp = df['y'].values
df['radius'] = np.sqrt( np.power(x_temp,2) + np.power(y_temp,2) )
df['theta'] = np.arctan2(y_temp,x_temp)
df['degrees'] = np.degrees(df['theta'].values)
df['radians'] = np.radians(df['degrees'].values)
ax = plt.axes(polar = True)
ax.set_aspect('equal')
ax.axis("off")
sns.set(rc={'axes.facecolor':'white', 'figure.facecolor':'white','figure.figsize':(10,10)})
# sns.scatterplot(data = df, x = 'x',y = 'y', s= 1,alpha = 0.1, color = 'black',ax = ax)
sns.scatterplot(data = df, x = 'radians',y = 'radius', s= 1,alpha = 0.1, color = 'black',ax = ax)
plt.tight_layout()
plt.show()
Here is the dataset
If you run this command using polar = False and use this line to plot sns.scatterplot(data = df, x = 'x',y = 'y', s= 1,alpha = 0.1, color = 'black',ax = ax) it will result in this picture
now after setting polar = True and run this line to plot sns.scatterplot(data = df, x = 'radians',y = 'radius', s= 1,alpha = 0.1, color = 'black',ax = ax) It is supposed to give you this
But it is not working as if you run the actual code the shape in the Polar format is the same as Cartesian which does not make sense and it does not match the picture I showed you for polar (If you are wondering where did I get the second picture from, I plotted it using R)
I would appreciate your help and insights and thanks in advance!

For a polar plot, the "x-axis" represents the angle in radians. So, you need to switch x and y, and convert the angles to radians (I also added ax=ax, as the axes was created explicitly):
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
data = {'radius': [0, 0.5, 1, 1.5, 2, 2.5], 'degrees': [0, 25, 75, 155, 245, 335]}
df_temp = pd.DataFrame(data)
ax = plt.axes(polar=True)
sns.scatterplot(x=np.radians(df_temp['degrees']), y=df_temp['radius'].to_numpy(),
s=100, alpha=1, color='black', ax=ax)
for deg, y in zip(df_temp['degrees'], df_temp['radius']):
x = np.radians(deg)
ax.axvline(x, color='skyblue', ls=':')
ax.text(x, y, f' {deg}', color='crimson')
ax.set_rlabel_position(-15) # Move radial labels away from plotted dots
plt.tight_layout()
plt.show()
About your new question: if you have an xy plot, and you convert these xy values to polar coordinates, and then plot these on a polar plot, you'll get again the same plot.
After some more testing with the data, I decided to create the plot directly with matplotlib, as seaborn makes some changes that don't have exactly equal effects across seaborn and matplotlib versions.
What seems to be happening in R:
The angles (given by "x") are spread out to fill the range (0,2 pi). This either requires a rescaling of x, or change how the x-values are mapped to angles. One way to get this, is subtracting the minimum. And with that result divide by the new maximum and multiply by 2 pi.
The 0 of the angles it at the top, and the angles go clockwise.
The following code should create the plot with Python. You might want to experiment with alpha and with s in the scatter plot options. (Default the scatter dots get an outline, which often isn't desired when working with very small dots, and can be removed by lw=0.)
ax = plt.axes(polar=True)
ax.set_aspect('equal')
ax.axis('off')
x_temp = df['x'].to_numpy()
y_temp = df['y'].to_numpy()
x_temp -= x_temp.min()
x_temp = x_temp / x_temp.max() * 2 * np.pi
ax.scatter(x=x_temp, y=y_temp, s=0.05, alpha=1, color='black', lw=0)
ax.set_rlim(y_temp.min(), y_temp.max())
ax.set_theta_zero_location("N") # set zero at the north (top)
ax.set_theta_direction(-1) # go clockwise
plt.show()
At the left the resulting image, at the right using the y-values for coloring (ax.scatter(..., c=y_temp, s=0.05, alpha=1, cmap='plasma_r', lw=0)):

Data points falling outside the meshgrid being interpolated over, while the meshgrid certainly covers those points

I am trying to interpolate sparse data over a meshgrid, but am observing some rather odd behavior. The white dots are precisely where I have values, and I am relying on the linear interpolation algorithm to fill in the other grids where possible. I recognize that this type of interpolation is not perfect due to the obvious lack of data, but how come some of the points where I have data fall outside the meshgrid that I am interpolating over? Is this a common phenomenon? This doesn't change even if I make the grid coarser.
I would appreciate some insight into why this happens, (perhaps how the linear interpolation works), or if there are any ways to fix this. See the red circles in the picture below for example:
Data points provided for interpolation falling outside the meshgrid that is interpolated over
The following is some code on the interpolation that generated the gridded data.
#mesh grid
xg = np.linspace(-130, -60, num=70)
yg = np.linspace(20,50,num=30)
Xg,Yg = np.meshgrid(xg,yg)
zg1 = griddata(points1, df2['tempratio'], (Xg, Yg), method = 'linear')
from mpl_toolkits.basemap import Basemap
lon_0 = xg.mean()
lat_0 = yg.mean()
m = Basemap(width=5000000, height=3500000,
resolution='l', projection='stere',\
lat_ts=40, lat_0=lat_0, lon_0=lon_0)
xm, ym = m(Xg, Yg)
cs = m.pcolormesh(xm,ym,zg1,shading='flat',cmap=plt.cm.Reds)

griddata assigns values to the vertices of a grid, so 70x30 points. pcolormesh doesn't color vertices, but the rectangles in-between. There are only 69x29 rectangles formed by the given vertices. So, one row and one column of zg1 will be dropped. To counter that, an extra row and extra column can be added to the coordinates and shifting everything half a rectangle in each direction.
It still doesn't force griddata to include all given points, but goes a step towards the desired outcome. A denser grid can also help. (Choosing 'nearest' instead of 'linear' interpolation would fill the complete grid.)
Here is some code to illustrate what's happening:
import numpy as np
from scipy.interpolate import griddata
from matplotlib import pyplot as plt
def extend_range(x):
dx = (x[1] - x[0]) / 2
return np.append( x - dx, x[-1] + dx)
N = 10
points1 = np.vstack([np.random.randint(-130, -60, N), np.random.randint(20, 50, N)]).T
tempratio = np.random.randint(0, 20, N)
xg = np.linspace(-130, -60, num=15)
yg = np.linspace(20, 50, num=10)
Xg, Yg = np.meshgrid(xg, yg)
zg1 = griddata(points1, tempratio, (Xg, Yg), method='linear')
fig, axs = plt.subplots(ncols=2, figsize=(12, 4))
for ax in axs:
ax.scatter(Xg, Yg, c=zg1, cmap='coolwarm', ec='g', s=80, zorder=2, label='griddata')
ax.scatter(points1[:,0], points1[:,1], c=tempratio, cmap='coolwarm', ec='black', s=150, zorder=3, label='given data')
if ax == axs[0]:
ax.pcolormesh(xg, yg, zg1, shading='flat', cmap='coolwarm')
ax.set_title('given x and y ranges')
else:
#todo: convert xg and yg to map coordinates
ax.pcolormesh(extend_range(xg), extend_range(yg), zg1, shading='flat', cmap='coolwarm')
ax.set_title('extended x and y ranges')
ax.legend()
plt.show()

Density scatter plot for huge dataset in matplotlib

I wrote some code a while ago that used gaussian kde to make simple density scatter plots. However, for datasets larger than about 100,000 points, it just ran 'forever' (I killed it after a few days). A friend gave me some code in R that could create such a density plot in seconds (plot_fun.R), and it seems like matplotlib should be able to do the same thing.
I think the right place to look is 2d histograms, but I am struggling to get the density to be 'right'. I modified code I found at this question to accomplish this, but the density is not showing, it looks like only the densist posible points are getting any color.
Here is approximately the code I am using:
# initial data
x = -np.log10(np.random.random_sample(10000))
y = -np.log10(np.random.random_sample(10000))
#histogram definition
bins = [1000, 1000] # number of bins
thresh = 3 #density threshold
#data definition
mn = min(x.min(), y.min())
mx = max(x.max(), y.max())
mn = mn-(mn*.1)
mx = mx+(mx*.1)
xyrange = [[mn, mx], [mn, mx]]
# histogram the data
hh, locx, locy = np.histogram2d(x, y, range=xyrange, bins=bins)
posx = np.digitize(x, locx)
posy = np.digitize(y, locy)
#select points within the histogram
ind = (posx > 0) & (posx <= bins[0]) & (posy > 0) & (posy <= bins[1])
hhsub = hh[posx[ind] - 1, posy[ind] - 1] # values of the histogram where the points are
xdat1 = x[ind][hhsub < thresh] # low density points
ydat1 = y[ind][hhsub < thresh]
hh[hh < thresh] = np.nan # fill the areas with low density by NaNs
f, a = plt.subplots(figsize=(12,12))
c = a.imshow(
np.flipud(hh.T), cmap='jet',
extent=np.array(xyrange).flatten(), interpolation='none',
origin='upper'
)
f.colorbar(c, ax=ax, orientation='vertical', shrink=0.75, pad=0.05)
s = a.scatter(
xdat1, ydat1, color='darkblue', edgecolor='', label=None,
picker=True, zorder=2
)
That produces this plot:
The KDE code is here:
f, a = plt.subplots(figsize=(12,12))
xy = np.vstack([x, y])
z = sts.gaussian_kde(xy)(xy)
# Sort the points by density, so that the densest points are
# plotted last
idx = z.argsort()
x2, y2, z = x[idx], y[idx], z[idx]
s = a.scatter(
x2, y2, c=z, s=50, cmap='jet',
edgecolor='', label=None, picker=True, zorder=2
)
That produces this plot:
The problem is, of course, that this code is unusable on large data sets.
My question is: how can I use the 2d histogram to produce a scatter plot like that? ax.hist2d does not produce a useful output, because it colors the whole plot, and all my efforts to get the above 2d histogram data to actually color the dense regions of the plot correctly have failed, I always end up with either no coloring or a tiny percentage of the densest points being colored. Clearly I just don't understand the code very well.

Your histogram code assigns a unique color (color='darkblue') so what are you expecting?
I think you are also over complicating things. This much simpler code works fine:
import numpy as np
import matplotlib.pyplot as plt
x, y = -np.log10(np.random.random_sample((2,10**6)))
#histogram definition
bins = [1000, 1000] # number of bins
# histogram the data
hh, locx, locy = np.histogram2d(x, y, bins=bins)
# Sort the points by density, so that the densest points are plotted last
z = np.array([hh[np.argmax(a<=locx[1:]),np.argmax(b<=locy[1:])] for a,b in zip(x,y)])
idx = z.argsort()
x2, y2, z2 = x[idx], y[idx], z[idx]
plt.figure(1,figsize=(8,8)).clf()
s = plt.scatter(x2, y2, c=z2, cmap='jet', marker='.')

contour deformation in python

I have a contour map and I want to make a deformation of all the contour lines, where the contour of level 0.5 will be deformed around the blue point situated in his line and then pass on the blue point on the contour of level 1, and so on.
Original map :
Deformed map :
I think there are two steps, the first one is the delete some parts of the map and the second is to redraw the contour map.
I think i have to iterate through the contour map like this:
CS = plt.contour(X, Y, Z)
for level in CS.collections:
for kp, path in list(enumerate(level.get_paths())):
But I have no idea how to use kp and path
Any tips for doing this would be appreciated!

Here is an example on how you could change the contour plot to achieve the intended deformation.
It generates some data x,y,z which should later be modified. Then it specifies a deformation function, which when multiplied to z deforms the data in the desired way. This deformation function takes the x and y data as input as well as the angle of the line along which to perform the deformation and a width (spread) of the deformation. Finally a parameter i is used for stearing the degree of deformation (i.e. i=0 means no deformation). Of course you can use any other function to deform your contour.
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.animation
#### generate some x,y,z data ####
r = np.linspace(0,6, num=100)
phi = np.linspace(0, 2*np.pi, num=200)
R, Phi = np.meshgrid(r,phi)
x = R*np.cos(Phi)
y = R*np.sin(Phi)
z = R
##################################
fig, ax=plt.subplots()
ax.set_aspect("equal")
def update(i):
ax.clear()
f = lambda x,y, offs, width, i: 1-i*np.exp(-(np.arctan2(x,y)-offs)**2/width)
z_deformed = z*f(x,y, np.pi/4, 1./8., i=i)
ax.contour(x,y,z_deformed, 10, linewidths=4)
ax.contourf(x,y,z_deformed, 10, alpha=0.3)
ax.set_xlim([-4,4])
ax.set_ylim([-4,4])
update(0) #plot the original data
anipath = 0.5*np.sin(np.linspace(0, np.pi, num=20))**2
ani = matplotlib.animation.FuncAnimation(fig, update, frames=anipath, interval = 100)
plt.show()
Of course you can use other shapes of deformation. E.g. to get a triangular shape use
f = lambda x, A, a, b: A*(1.-np.abs((x-b)/a))*(np.abs((x-b)) < a )
z_deformed = z - f(np.arctan2(x,y), i, 1./8., np.pi/4 )

How can I plot non-rectangular 2D data using matplotlib?

I am working on a calculation in Python which calculates the physical properties of an object for angles in the range 0 < θ < π/2 and 0 < φ < π/2 (i.e. the first octant). To visualize the properties I am currently plotting them as color values on a 3D unit sphere. Producing this plot for reasonable resolution is a fairly resource intensive process, but I have no interest in being able to look at the plot from any other angle.
What I would like to create instead is a 2D image plot similar to what imshow would create except that it should have the triangular outline of a sphere octant being projected into 2D. Note that I am not asking how to project the 3D data into 2D, but rather how to display the 2D data in a manner which looks similar to a sphere octant viewed from θ = π/4, φ = π/4.
My current code is below. The specifics may not be that relevant to an answer, but it gives an idea of what I am trying to do.
'''
The code above this point produces three arrays stored in a dictionary
called phs with the three entries using the keys 'I', 'II', 'III. Each
array is a function of theta and phi where
theta = np.linspace( 0, 90, nPoints)
phi = np.linspace( 0, 90, nPoints)
also
types = ('I', 'II', 'III')
'''
# Colormaps
mx = np.maximum( np.maximum( phs['I'], phs['II']), phs['III'])
cmap = cm.ScalarMappable( cmap='BuPu')
cmap.set_array( mx)
clrs = dict()
for type in types:
clrs[type] = cmap.to_rgba( phs[type])
# Convert to Cartesian coordinates with unit radius
xM, yM, zM = plotCartesianFixedR( thetaM, phiM)
# Plot
fig = plt.figure( figsize=(16,7))
ax = dict()
ax['I'] = plt.subplot( 131, projection='3d')
ax['II'] = plt.subplot( 132, projection='3d')
ax['III'] = plt.subplot( 133, projection='3d')
surf = dict()
for type in types:
surf[type] = ax[type].plot_surface( xM, yM, zM, rstride=1, cstride=1,
facecolors=clrs[type], shade=False)
# Set axis properties
ax[type].set_xticklabels([])
ax[type].set_yticklabels([])
ax[type].set_zticklabels([])
ax[type].view_init(elev=45, azim=45)
# Colorbar
plt.colorbar( cmap, shrink=1)
ax['I'].set_title( 'Log$_{10}(|\Delta k|)$ Type I (ssf)')
ax['II'].set_title( 'Log$_{10}(|\Delta k|)$ Type II (sff)')
ax['III'].set_title( 'Log$_{10}(|\Delta k|)$ Type III (fsf)')
# Add title
if title:
plt.suptitle(title)
The output looks like:
Just to restate the problem; I would like to reproduce this plot almost exactly but in 2D without including the background axes.