Develop Reference

Python matplotlib polar coordinate is not plotting as it is supposed to be

I am plotting from a CSV file that contains Cartesian coordinates and I want to change it to Polar coordinates, then plot using the Polar coordinates.
Here is the code
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
df = pd.read_csv('test_for_plotting.csv',index_col = 0)
x_temp = df['x'].values
y_temp = df['y'].values
df['radius'] = np.sqrt( np.power(x_temp,2) + np.power(y_temp,2) )
df['theta'] = np.arctan2(y_temp,x_temp)
df['degrees'] = np.degrees(df['theta'].values)
df['radians'] = np.radians(df['degrees'].values)
ax = plt.axes(polar = True)
ax.set_aspect('equal')
ax.axis("off")
sns.set(rc={'axes.facecolor':'white', 'figure.facecolor':'white','figure.figsize':(10,10)})
# sns.scatterplot(data = df, x = 'x',y = 'y', s= 1,alpha = 0.1, color = 'black',ax = ax)
sns.scatterplot(data = df, x = 'radians',y = 'radius', s= 1,alpha = 0.1, color = 'black',ax = ax)
plt.tight_layout()
plt.show()
Here is the dataset
If you run this command using polar = False and use this line to plot sns.scatterplot(data = df, x = 'x',y = 'y', s= 1,alpha = 0.1, color = 'black',ax = ax) it will result in this picture
now after setting polar = True and run this line to plot sns.scatterplot(data = df, x = 'radians',y = 'radius', s= 1,alpha = 0.1, color = 'black',ax = ax) It is supposed to give you this
But it is not working as if you run the actual code the shape in the Polar format is the same as Cartesian which does not make sense and it does not match the picture I showed you for polar (If you are wondering where did I get the second picture from, I plotted it using R)
I would appreciate your help and insights and thanks in advance!

For a polar plot, the "x-axis" represents the angle in radians. So, you need to switch x and y, and convert the angles to radians (I also added ax=ax, as the axes was created explicitly):
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
data = {'radius': [0, 0.5, 1, 1.5, 2, 2.5], 'degrees': [0, 25, 75, 155, 245, 335]}
df_temp = pd.DataFrame(data)
ax = plt.axes(polar=True)
sns.scatterplot(x=np.radians(df_temp['degrees']), y=df_temp['radius'].to_numpy(),
s=100, alpha=1, color='black', ax=ax)
for deg, y in zip(df_temp['degrees'], df_temp['radius']):
x = np.radians(deg)
ax.axvline(x, color='skyblue', ls=':')
ax.text(x, y, f' {deg}', color='crimson')
ax.set_rlabel_position(-15) # Move radial labels away from plotted dots
plt.tight_layout()
plt.show()
About your new question: if you have an xy plot, and you convert these xy values to polar coordinates, and then plot these on a polar plot, you'll get again the same plot.
After some more testing with the data, I decided to create the plot directly with matplotlib, as seaborn makes some changes that don't have exactly equal effects across seaborn and matplotlib versions.
What seems to be happening in R:
The angles (given by "x") are spread out to fill the range (0,2 pi). This either requires a rescaling of x, or change how the x-values are mapped to angles. One way to get this, is subtracting the minimum. And with that result divide by the new maximum and multiply by 2 pi.
The 0 of the angles it at the top, and the angles go clockwise.
The following code should create the plot with Python. You might want to experiment with alpha and with s in the scatter plot options. (Default the scatter dots get an outline, which often isn't desired when working with very small dots, and can be removed by lw=0.)
ax = plt.axes(polar=True)
ax.set_aspect('equal')
ax.axis('off')
x_temp = df['x'].to_numpy()
y_temp = df['y'].to_numpy()
x_temp -= x_temp.min()
x_temp = x_temp / x_temp.max() * 2 * np.pi
ax.scatter(x=x_temp, y=y_temp, s=0.05, alpha=1, color='black', lw=0)
ax.set_rlim(y_temp.min(), y_temp.max())
ax.set_theta_zero_location("N") # set zero at the north (top)
ax.set_theta_direction(-1) # go clockwise
plt.show()
At the left the resulting image, at the right using the y-values for coloring (ax.scatter(..., c=y_temp, s=0.05, alpha=1, cmap='plasma_r', lw=0)):

Add horizontal line with conditional coloring

I make a contourf plot using matplotlib.pyplot. Now I want to have a horizontal line (or something like ax.vspan would work too) with conditional coloring at y = 0. I will show you what I have and what I would like to get. I want to do this with an array, let's say landsurface that represents either land, ocean or ice. This array is filled with 1 (land), 2 (ocean) or 3 (ice) and has the len(locs) (so the x-axis).
This is the plot code:
plt.figure()
ax=plt.axes()
clev=np.arange(0.,50.,.5)
plt.contourf(locs,height-surfaceheight,var,clev,extend='max')
plt.xlabel('Location')
plt.ylabel('Height above ground level [m]')
cbar = plt.colorbar()
cbar.ax.set_ylabel('o3 mixing ratio [ppb]')
plt.show()
This is what I have so far:
This is what I want:
Many thanks in advance!

Intro
I'm going to use a line collection .
Because I have not your original data, I faked some data using a simple sine curve and plotting on the baseline the color codes corresponding to small, middle and high values of the curve
Code
Usual boilerplate, we need to explicitly import LineCollection
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.collections import LineCollection
Just to plot something, a sine curve (x r
x = np.linspace(0, 50, 101)
y = np.sin(0.3*x)
The color coding from the curve values (corresponding to your surface types) to the LineCollection colors, note that LineCollection requires that the colors are specified as RGBA tuples but I have seen examples using color strings, bah!
# 1 when near min, 2 when near 0, 3 when near max
z = np.where(y<-0.5, 1, np.where(y<+0.5, 2, 3))
col_d = {1:(0.4, 0.4, 1.0, 1), # blue, near min
2:(0.4, 1.0, 0.4, 1), # green, near zero
3:(1.0, 0.4, 0.4, 1)} # red, near max
# prepare the list of colors
colors = [col_d[n] for n in z]
In a line collection we need a sequence of segments, here I have decided to place my coded line at y=0 but you can just add a constant to s to move it up and down.
I admit that forming the sequence of segments is a bit tricky...
# build the sequence of segments
s = np.zeros(101)
segments=np.array(list(zip(zip(x,x[1:]),zip(s,s[1:])))).transpose((0,2,1))
# and fill the LineCollection
lc = LineCollection(segments, colors=colors, linewidths=5,
antialiaseds=0, # to prevent artifacts between lines
zorder=3 # to force drawing over the curve) lc = LineCollection(segments, colors=colors, linewidths=5) # possibly add zorder=...
Finally, we put everything on the canvas
# plot the function and the line collection
fig, ax = plt.subplots()
ax.plot(x,y)
ax.add_collection(lc)

I would suggest adding an imshow() with proper extent, e.g.:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colorbar as colorbar
import matplotlib.colors as colors
### generate some data
np.random.seed(19680801)
npts = 50
x = np.random.uniform(0, 1, npts)
y = np.random.uniform(0, 1, npts)
X,Y=np.meshgrid(x,y)
z = x * np.exp(-X**2 - Y**2)*100
### create a colormap of three distinct colors for each landmass
landmass_cmap=colors.ListedColormap(["b","r","g"])
x_land=np.linspace(0,1,len(x)) ## this should be scaled to your "location"
## generate some fake landmass types (either 0, 1, or 2) with probabilites
y_land=np.random.choice(3, len(x), p=[0.1, 0.6, 0.3])
print(y_land)
fig=plt.figure()
ax=plt.axes()
clev=np.arange(0.,50.,.5)
## adjust the "height" of the landmass
x0,x1=0,1
y0,y1=0,0.05 ## y1 is the "height" of the landmass
## make sure that you're passing sensible zorder here and in your .contourf()
im = ax.imshow(y_land.reshape((-1,len(x))),cmap=landmass_cmap,zorder=2,extent=(x0,x1,y0,y1))
plt.contourf(x,y,z,clev,extend='max',zorder=1)
ax.set_xlim(0,1)
ax.set_ylim(0,1)
ax.plot()
ax.set_xlabel('Location')
ax.set_ylabel('Height above ground level [m]')
cbar = plt.colorbar()
cbar.ax.set_ylabel('o3 mixing ratio [ppb]')
## add a colorbar for your listed colormap
cax = fig.add_axes([0.2, 0.95, 0.5, 0.02]) # x-position, y-position, x-width, y-height
bounds = [0,1,2,3]
norm = colors.BoundaryNorm(bounds, landmass_cmap.N)
cb2 = colorbar.ColorbarBase(cax, cmap=landmass_cmap,
norm=norm,
boundaries=bounds,
ticks=[0.5,1.5,2.5],
spacing='proportional',
orientation='horizontal')
cb2.ax.set_xticklabels(['sea','land','ice'])
plt.show()
yields:

Not getting the heatmap in the background using Matplotlib Python

I have tried this and got the result as in the image:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list("", ["red","grey","green"])
df = pd.read_csv('t.csv', header=0)
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax = ax1.twiny()
# Scatter plot of positive points, coloured blue (C0)
ax.scatter(np.argwhere(df['real'] > 0), df.loc[df['real'] > 0, 'real'], color='C2')
# Scatter plot of negative points, coloured red (C3)
ax.scatter(np.argwhere(df['real'] < 0), df.loc[df['real'] < 0, 'real'], color='C3')
# Scatter neutral values in grey (C7)
ax.scatter(np.argwhere(df['real'] == 0), df.loc[df['real'] == 0, 'real'], color='C7')
ax.set_ylim([df['real'].min(), df['real'].max()])
index = len(df.index)
ymin = df['prediction'].min()
ymax= df['prediction'].max()
ax1.imshow([np.arange(index),df['prediction']],cmap=cmap,
extent=(0,index-1,ymin, ymax), alpha=0.8)
plt.show()
Image:
I was expecting one output where the color is placed according to the figure. I am getting green color and no reds or greys.
I want to get the image or contours spread as the values are. How I can do that? See the following image, something similar:
Please let me know how I can achieve this. The data I used is here: t.csv
For a live version, have a look at Tensorflow Playground

There are essentially 2 tasks required in a solution like this:
Plot the heatmap as the background;
Plot the scatter data;
Output:
Source code:
import numpy as np
import matplotlib.pyplot as plt
###
# Plot heatmap in the background
###
# Setting up input values
x = np.arange(-6.0, 6.0, 0.1)
y = np.arange(-6.0, 6.0, 0.1)
X, Y = np.meshgrid(x, y)
# plot heatmap colorspace in the background
fig, ax = plt.subplots(nrows=1)
im = ax.imshow(X, cmap=plt.cm.get_cmap('RdBu'), extent=(-6, 6, -6, 6), interpolation='bilinear')
cax = fig.add_axes([0.21, 0.95, 0.6, 0.03]) # [left, bottom, width, height]
fig.colorbar(im, cax=cax, orientation='horizontal') # add colorbar at the top
###
# Plot data as scatter
###
# generate the points
num_samples = 150
theta = np.linspace(0, 2 * np.pi, num_samples)
# generate inner points
circle_r = 2
r = circle_r * np.random.rand(num_samples)
inner_x, inner_y = r * np.cos(theta), r * np.sin(theta)
# generate outter points
circle_r = 4
r = circle_r + np.random.rand(num_samples)
outter_x, outter_y = r * np.cos(theta), r * np.sin(theta)
# plot data
ax.scatter(inner_x, inner_y, s=30, marker='o', color='royalblue', edgecolors='white', linewidths=0.8)
ax.scatter(outter_x, outter_y, s=30, marker='o', color='crimson', edgecolors='white', linewidths=0.8)
ax.set_ylim([-6,6])
ax.set_xlim([-6,6])
plt.show()
To keep things simple, I kept the colorbar range (-6, 6) to match the data range.
I'm sure this code can be changed to suit your specific needs. Good luck!

Here is a possible solution.
A few notes and questions:
What are the 'prediction' values in your data file? They do not seem to correlate with the values in the 'real' column.
Why do you create a second axis? What is represented on the bottom X-axis in your plot? I removed the second axis and labelled the remaining axes (index and real).
When you slice a pandas DataFrame, the index comes with it. You don't need to create a separate index (argwhere and arange(index) in your code). I simplified the first part of the code, where scatterplots are produced.
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list("", ["red","grey","green"])
df = pd.read_csv('t.csv', header=0)
print(df)
fig = plt.figure()
ax = fig.add_subplot(111)
# Data limits
xmin = 0
xmax = df.shape[0]
ymin = df['real'].min()
ymax = df['real'].max()
# Scatter plots
gt0 = df.loc[df['real'] > 0, 'real']
lt0 = df.loc[df['real'] < 0, 'real']
eq0 = df.loc[df['real'] == 0, 'real']
ax.scatter(gt0.index, gt0.values, edgecolor='white', color='C2')
ax.scatter(lt0.index, lt0.values, edgecolor='white', color='C3')
ax.scatter(eq0.index, eq0.values, edgecolor='white', color='C7')
ax.set_ylim((ymin, ymax))
ax.set_xlabel('index')
ax.set_ylabel('real')
# We want 0 to be in the middle of the colourbar,
# because gray is defined as df['real'] == 0
if abs(ymax) > abs(ymin):
lim = abs(ymax)
else:
lim = abs(ymin)
# Create a gradient that runs from -lim to lim in N number of steps,
# where N is the number of colour steps in the cmap.
grad = np.arange(-lim, lim, 2*lim/cmap.N)
# Arrays plotted with imshow must be 2D arrays. In this case it will be
# 1 pixel wide and N pixels tall. Set the aspect ratio to auto so that
# each pixel is stretched out to the full width of the frame.
grad = np.expand_dims(grad, axis=1)
im = ax.imshow(grad, cmap=cmap, aspect='auto', alpha=1, origin='bottom',
extent=(xmin, xmax, -lim, lim))
fig.colorbar(im, label='real')
plt.show()
This gives the following result:

Problems with pcolormesh in Cartopy

I'm trying to adapt the Cartopy example plot for circular South Polar Stereographic plots to the North Pole and add data to it. I have a couple questions.
First, in the example code, the land feature is added before the ocean feature. When I did that, I got a map with only ocean. I reversed the order of the call in the code below and get a map with land and ocean. Why did the other order work with the South Polar example?
Second, and more importantly, I can't figure out why my pcolormesh call isn't having any effect.
I'm using Python 2.7.7, matplotlib 1.5.1, and Cartopy 0.15.1.
import matplotlib.path as mpath
import matplotlib.pyplot as plt
import numpy as np
import cartopy.crs as ccrs
import cartopy.feature
lats = np.linspace(60,90,30)
lons = np.linspace(0,360,200)
X,Y = np.meshgrid(lons,lats)
Z = np.random.normal(size = X.shape)
def main():
fig = plt.figure(figsize=[10, 5])
ax = plt.subplot(1, 1, 1, projection=ccrs.NorthPolarStereo())
fig.subplots_adjust(bottom=0.05, top=0.95,
left=0.04, right=0.95, wspace=0.02)
# Limit the map to -60 degrees latitude and below.
ax.set_extent([-180, 180, 60, 60], ccrs.PlateCarree())
ax.gridlines()
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.LAND)
# Compute a circle in axes coordinates, which we can use as a boundary
# for the map. We can pan/zoom as much as we like - the boundary will be
# permanently circular.
theta = np.linspace(0, 2*np.pi, 100)
center, radius = [0.5, 0.5], 0.5
verts = np.vstack([np.sin(theta), np.cos(theta)]).T
circle = mpath.Path(verts * radius + center)
ax.set_boundary(circle, transform=ax.transAxes)
ax.pcolormesh(X,Y,Z,transform=ccrs.PlateCarree())
plt.show()
if __name__ == '__main__':
main()

Your code leaves cartopy to dictate the order of feature plots on the map, as a result, some features can be hidden with no clues. It is possible to specify the order of plots explicitly.
The order of features plot is controlled by zorder, which can be specified with zorder=integer in most plotting statements. Here is a modified code that produces a better plot.
# your data
lats = np.linspace(60, 90, 30)
lons = np.linspace(0, 360, 160)
X,Y = np.meshgrid(lons, lats)
Z = np.random.normal(size = X.shape)
# new data for pcolormesh plot
latz = np.linspace(75, 90, 15)
lonz = np.linspace(0, 360, 160)
X1,Y1 = np.meshgrid(lonz, latz)
Z1 = np.random.normal(size = X1.shape)
def main():
fig = plt.figure(figsize=[10, 10])
ax = plt.subplot(1, 1, 1, projection=ccrs.NorthPolarStereo())
fig.subplots_adjust(bottom=0.05, top=0.95,
left=0.04, right=0.95, wspace=0.02)
# Limit the map to -60 degrees latitude and below.
ax.set_extent([-180, 180, 60, 60], ccrs.PlateCarree())
ax.gridlines()
# zorder can be used to arrange what is on top
ax.add_feature(cartopy.feature.LAND, zorder=4) # land is specified to plot above ...
ax.add_feature(cartopy.feature.OCEAN, zorder=1) # ... the ocean
# Compute a circle in axes coordinates, which we can use as a boundary
# for the map. We can pan/zoom as much as we like - the boundary will be
# permanently circular.
theta = np.linspace(0, 2*np.pi, 100)
center, radius = [0.5, 0.5], 0.5
verts = np.vstack([np.sin(theta), np.cos(theta)]).T
circle = mpath.Path(verts * radius + center)
ax.set_boundary(circle, transform=ax.transAxes)
# pcolormesh is specified to plot on top of the ocean but below land
ax.pcolormesh(X1, Y1, Z1, transform=ccrs.PlateCarree(), zorder=3)
plt.show()
if __name__ == '__main__':
main()

How do I offset lines in matplotlib by X points

I'm using matplotlib to plot some data that I wish to annotate with arrows (distance markers). These arrows should be offset by several points so as not to overlap with the plotted data:
import matplotlib.pyplot as plt
import matplotlib.transforms as transforms
fig, ax = plt.subplots()
x = [0, 1]
y = [0, 0]
# Plot horizontal line
ax.plot(x, y)
dy = 5/72
offset = transforms.ScaledTranslation(0, dy, ax.get_figure().dpi_scale_trans)
verttrans = ax.transData+offset
# Plot horizontal line 5 points above (works!)
ax.plot(x, y, transform = verttrans)
# Draw arrow 5 points above line (doesn't work--not vertically translated)
ax.annotate("", (0,0), (1,0),
size = 10,
transform=verttrans,
arrowprops = dict(arrowstyle = '<|-|>'))
plt.show()
Is there any way to make lines drawn by ax.annotate() be offset by X points? I wish to use absolute coordinates (e.g., points or inches) instead of data coordinates because the axis limits are prone to changing.
Thanks!

The following code does what I desired. It uses ax.transData and figure.get_dpi():
import matplotlib.pyplot as plt
import matplotlib.transforms as transforms
fig, ax = plt.subplots()
x = [0, 1]
y = [0, 0]
ax.plot(x, y)
dy = 5/72
i = 1 # 0 for dx
tmp = ax.transData.transform([(0,0), (1,1)])
tmp = tmp[1,i] - tmp[0,i] # 1 unit in display coords
tmp = 1/tmp # 1 pixel in display coords
tmp = tmp*dy*ax.get_figure().get_dpi() # shift pixels in display coords
ax.plot(x, y)
ax.annotate("", [0,tmp], [1,tmp],
size = 10,
arrowprops = dict(arrowstyle = '<|-|>'))
plt.show()

What's your expected output? If you're just looking to move the arrow you're drawing vertically, the API for annotate is
annotate(s, xy, xytext=None, ...)
so you can draw something like
ax.annotate("", (0,0.01), (1,0.01),
size = 10,
arrowprops = dict(arrowstyle = '<|-|>'))
which is moved up by 0.01 in data coordinates in the y direction. You can also specify coordinates as a fraction of the total figure size in annotate (see doc). Is that what you wanted?