Develop Reference

Matplotlib Contour/Contourf in Cartopy singularity at North Pole/ dateline

I am trying to produce heatmaps showing atmospheric attenuation values for a RF link to a satellite above the North Pole, but I have issues with the interpolation done by the Matplotlib contour/contourf functions.
The linear interpolation done by the contourf function does not work well around the N.Pole, as I suspect it does not know to interpolate between values which go from (-180 deg to +180 deg) - i.e. cross the dateline, or cross the pole.
Any suggestions on a different approach to generate the heatmap, to avoid this horrible hole at the centre?!
Code below to generate plot.
import cartopy.crs as ccrs
import cartopy.feature
plt.figure(figsize=(10,10))
# Initialise Cartopy Axes.
proj=ccrs.LambertAzimuthalEqualArea(central_longitude=0, central_latitude=90)
ax = plt.axes(projection = proj)
ax.set_extent([-180,180,45,90], ccrs.PlateCarree())
ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.gridlines(ls=":",color="grey",lw=0.5)
x0,x1 = attenuation_df.lon.min(), attenuation_df.lon.max()
y0,y1 = attenuation_df.lat.min(), attenuation_df.lat.max()
x,y = np.linspace(x0,x1,1000), np.linspace(y0,y1,1000)
X,Y = np.meshgrid(x,y)
Z = scipy.interpolate.griddata(
attenuation_df[["lon","lat"]],
attenuation_df["attenuation"],
(X,Y),
method="linear",
)
plt.contourf(X,Y,Z,transform=ccrs.PlateCarree(),alpha=0.5)
plt.colorbar(shrink=0.5)
plt.title("Attenuation")
plt.show()
Attenuation_df is a Pandas Dataframe which contains an attenuation value at approximately 3500 sample points, which are equally spaced around the globe. Here is the location of the sample points:
Here is the header of attenuation_df:
lon
lat
attenuation
0
-30.8538
48.8813
0.860307
1
-29.0448
49.5026
0.783662
2
-27.2358
50.1317
0.720165
3
-32.6628
48.2676
0.947662
4
37.4226
46.0322
0.27495
The link to the csv of attenuation_df is here: https://pastebin.com/NYA1jFgt

A solution is to reproject your data to a different coordinate system, my suggestion is to use a Polar Stereographic system. However, the large "hole" centered at the North Pole is not coming from the coordinate system in use but to the presence of some nans in your dataset, so you first have to remove those values.
Here a working solution:
from pyproj import Proj
# Define a pyproj function to reproject data
def coordinate_conv(x, y, inverse = True):
p = Proj('+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs')
return p(x, y, inverse = inverse)
# Drop null values
attenuation_df.dropna(how = 'any', inplace = True)
# Reproject data
rpjx, rpjy = coordinate_conv(attenuation_df.lon, attenuation_df.lat, False)
rpj_cord = pd.DataFrame({'x': rpjx, 'y': rpjy})
# Interpoolate data
x,y = np.linspace(rpjx.min(),rpjx.max(),1000), np.linspace(rpjy.min(),rpjy.max(),1000)
X,Y = np.meshgrid(x,y)
Z = interpolate.griddata(
rpj_cord,
attenuation_df["attenuation"],
(X,Y),
method="linear",
)
# Figure
plt.figure(figsize=(10,10))
# Initialise Cartopy Axes.
proj=ccrs.LambertAzimuthalEqualArea(central_longitude=0, central_latitude=90)
ax = plt.axes(projection = proj)
ax.set_extent([-180,180,45,90], ccrs.PlateCarree())
ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.gridlines(ls=":",color="grey",lw=0.5)
kw = dict(central_latitude=90, central_longitude=-45, true_scale_latitude=70)
plt.contourf(X,Y,Z, transform=ccrs.Stereographic(**kw),alpha=0.5)
plt.colorbar(shrink=0.5)
plt.title("Attenuation")
And this is the output figure:

How to use geopandas to plot latitude and longitude on a more detailed map with by using basemaps?

I am trying to plot some latitude and longitudes on the map of delhi which I am able to do by using a shape file in python3.8 using geopandas
Here is the link for the shape file:
https://drive.google.com/file/d/1CEScjlcsKFCgdlME21buexHxjCbkb3WE/view?usp=sharing
Following is my code to plot points on the map:
lo=[list of longitudes]
la=[list of latitudes]
delhi_map = gpd.read_file(r'C:\Users\Desktop\Delhi_Wards.shp')
fig,ax = plt.subplots(figsize = (15,15))
delhi_map.plot(ax = ax)
geometry = [Point(xy) for xy in zip(lo,la)]
geo_df = gpd.GeoDataFrame(geometry = geometry)
print(geo_df)
g = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi')
plt.show()
Following is the result:
Now this map is not very clear and anyone will not be able to recognise the places marked so i tried to use basemap for a more detailed map through the following code:
df = gpd.read_file(r'C:\Users\Jojo\Desktop\Delhi_Wards.shp')
new_df = df.to_crs(epsg=3857)
print(df.crs)
print(new_df.crs)
ax = new_df.plot()
ctx.add_basemap(ax)
plt.show()
And following is the result:
I am getting the basemap but my shapefile is overlapping it. Can i get a map to plot my latitudes and longitudes where the map is much more detailed with names of places or roads or anything similar to it like in google maps or even something like the map which is being overlapped by the blue shapefile map?
Is it possible to plot on a map like this??
https://www.researchgate.net/profile/P_Jops/publication/324715366/figure/fig3/AS:618748771835906#1524532611545/Map-of-Delhi-reproduced-from-Google-Maps-12.png

use zorder parameter to adjust the layers' orders (lower zorder means lower layer), and alpha to the polygon. anyway, I guess, you're plotting df twice, that's why it's overlapping.
here's my script and the result
import geopandas as gpd
import matplotlib.pyplot as plt
import contextily as ctx
from shapely.geometry import Point
long =[77.2885437011719, 77.231931, 77.198767, 77.2750396728516]
lat = [28.6877899169922, 28.663863, 28.648287, 28.5429172515869]
geometry = [Point(xy) for xy in zip(long,lat)]
wardlink = "New Folder/wards delimited.shp"
ward = gpd.read_file(wardlink, bbox=None, mask=None, rows=None)
geo_df = gpd.GeoDataFrame(geometry = geometry)
ward.crs = {'init':"epsg:4326"}
geo_df.crs = {'init':"epsg:4326"}
# plot the polygon
ax = ward.plot(alpha=0.35, color='#d66058', zorder=1)
# plot the boundary only (without fill), just uncomment
#ax = gpd.GeoSeries(ward.to_crs(epsg=3857)['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
ax = gpd.GeoSeries(ward['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
# plot the marker
ax = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi', zorder=3)
ctx.add_basemap(ax, crs=geo_df.crs.to_string(), source=ctx.providers.OpenStreetMap.Mapnik)
plt.show()
I don't know about google maps being in the contextily, I don't think it's available. alternatively, you can use OpenStreetMap base map which shows quite the same toponym, or any other basemap you can explore. use `source` keyword in the argument, for example, `ctx.add_basemap(ax, source=ctx.providers.OpenStreetMap.Mapnik)` . here's how to check the available providers and the map each providers provides:
>>> ctx.providers.keys()
dict_keys(['OpenStreetMap', 'OpenSeaMap', 'OpenPtMap', 'OpenTopoMap', 'OpenRailwayMap', 'OpenFireMap', 'SafeCast', 'Thunderforest', 'OpenMapSurfer', 'Hydda', 'MapBox', 'Stamen', 'Esri', 'OpenWeatherMap', 'HERE', 'FreeMapSK', 'MtbMap', 'CartoDB', 'HikeBike', 'BasemapAT', 'nlmaps', 'NASAGIBS', 'NLS', 'JusticeMap', 'Wikimedia', 'GeoportailFrance', 'OneMapSG'])
>>> ctx.providers.OpenStreetMap.keys()
dict_keys(['Mapnik', 'DE', 'CH', 'France', 'HOT', 'BZH'])

I don't know geopandas. The idea I'm suggesting uses only basic python and matplotlib. I hope you can adapt it to your needs.
The background is the following map. I figured out the GPS coordinates of its corners using google-maps.
The code follows the three points of my remark. Note that the use of imread and imshow reverses the y coordinate. This is why the function coordinatesOnFigur looks non-symmetrical in x and y.
Running the code yields the map with a red bullet near Montijo (there is a small test at the end).
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib import patches
from matplotlib.widgets import Button
NE = (-8.9551, 38.8799)
SE = (-8.9551, 38.6149)
SW = (-9.4068, 38.6149)
NW = (-9.4068, 38.8799)
fig = plt.figure(figsize=(8, 6))
axes = fig.add_subplot(1,1,1, aspect='equal')
img_array = plt.imread("lisbon_2.jpg")
axes.imshow(img_array)
xmax = axes.get_xlim()[1]
ymin = axes.get_ylim()[0] # the y coordinates are reversed, ymax=0
# print(axes.get_xlim(), xmax)
# print(axes.get_ylim(), ymin)
def coordinatesOnFigure(long, lat, SW=SW, NE=NE, xmax=xmax, ymin=ymin):
px = xmax/(NE[0]-SW[0])
qx = -SW[0]*xmax/(NE[0]-SW[0])
py = -ymin/(NE[1]-SW[1])
qy = NE[1]*ymin/(NE[1]-SW[1])
return px*long + qx, py*lat + qy
# plotting a red bullet that corresponds to a GPS location on the map
x, y = coordinatesOnFigure(-9, 38.7)
print("test: on -9, 38.7 we get", x, y)
axes.scatter(x, y, s=40, c='red', alpha=0.9)
plt.show()

basemap & contourf, Python

I have a set of satellite data file here, I created a grid for lat & lon and a 2D array for Ozone values.
I know that in order to plot the contourf of the data in a map I need the projection coordinates, but I can't get find a way around it as my grid is not square (144x24). I am covering the geographical area (0 to 360; -30 to 30) and I require square pixels.
The data is quite long to post it but this is my code so far,
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from mpl_toolkits.basemap import Basemap, cm
%matplotlib inline
path = '/home/rafaella/month_files_CSV/O3_COLUMNS_MATCHED_fv0005_200306.csv'
df = pd.read_csv(path, skiprows=1)
df = pd.read_csv(path, delim_whitespace=True)
lat = np.array(df['AVG_LAT'])
lon = np.array(df['AVG_LON'])
toc = np.array(df['TROP_COL'])
#new grid for lon[0,360] lat[-30,30]
lomin = 0
lomax = 360
lamin = -30
lamax = 30
stp = 2.5
loc_lon = np.zeros(int((lomax-lomin)/stp))
loc_lat = np.zeros(int((lamax-lamin)/stp))
for i in range(0,len(loc_lon)):
loc_lon[i] = i*stp +lomin
for j in range(0,len(loc_lat)):
loc_lat[j] = j*stp +lamin
mtoc_local = np.zeros((len(loc_lon),len(loc_lat)))
sdtoc_local = np.zeros((len(loc_lon),len(loc_lat)))
mtoc_local[:,:] = np.nan
sdtoc_local[:,:] = np.nan
for i in range (0, len(loc_lon)):
for j in range (0,len(loc_lat)):
ix = np.where((lon>=loc_lon[i])& (lat>=loc_lat[j]) & (lon<loc_lon[i]+stp) & (lat<loc_lat[j]+stp))[0]
mtoc_local[i,j]=np.nanmean(toc[ix])
sdtoc_local[i,j]=np.nanstd(toc[ix])
fig = plt.figure(figsize=(20, 5))
map = Basemap(llcrnrlon=0,llcrnrlat=-30, urcrnrlon=360.,urcrnrlat=30.,\
rsphere=(6378137.00,6356752.3142),\
resolution='l',projection='merc',\
lat_0=0,lon_0=-30.,lat_ts=30.)
map.drawcoastlines()
# draw parallels
map.drawparallels(np.arange(-30,30,10),labels=[1,1,0,1])
# draw meridians
map.drawmeridians(np.arange(-180,180,20),labels=[1,1,0,1])
map = plt.contourf(loc_lon, loc_lat , mtoc_local.T, vmin=210, vmax=350, cmap='RdPu')
plt.colorbar(orientation='horizontal', ticks=[200, 220, 240, 260, 280, 300, 320, 340] )
plt.title('Tropical TOC monthly mean 06,2009')
plt.show()
It plots very well the map OR the data but not both. here an image of both separately
map
real data
I am very new to python, I started a month ago, so it is still not familiar to me all the functions and libraries.

Your code has two problems. First you have to apply the projection on your coordinates which is done using x,y = map(lon, lat). However, this will raise an error in your case since the dimensions of loc_lon and loc_lat are different. Instead of passing x and y vectors to the contourf function you can pass arrays with the same shape as z (mtoc_local.T). You can use np.meshgrid to create those. Long story short, replace the line with the contourf command with the following three lines
X, Y = np.meshgrid(loc_lon, loc_lat)
x,y = map(X,Y)
map = plt.contourf(x, y , mtoc_local.T, vmin=210, vmax=350, cmap='RdPu')
and the result looks like this

Python irregular x,y data to contour plot on original domain

I have file containing points under the columns "x-cord", "y-cord", "value". These are irregularly spaced. I am trying to make a contour plot of "value" and overlay this over the original domain. I gave up trying to do this in both pgfplots and matlab and thought I would give python a go. An answer in any of these scripts would be fine. The python script is as follows
import numpy as np
from scipy.interpolate import griddata
import matplotlib.pyplot as plt
import numpy.ma as ma
from numpy.random import uniform, seed
from scipy.spatial import ConvexHull
#
# Loading data
filename = "strain.dat"
coordinates = []
x_c = []
y_c = []
z_c = []
xyz = open(filename)
title = xyz.readline()
for line in xyz:
x,y,z = line.split()
coordinates.append([float(x), float(y), float(z)])
x_c.append([float(x)])
y_c.append([float(y)])
z_c.append([float(z)])
xyz.close()
#
# Rehaping and translating data
x_c=np.ravel(np.array(x_c))
y_c=np.ravel(np.array(y_c))
z_c=np.ravel(np.array(z_c))
x_c = x_c-100.0
y_c = y_c-100.0
#
# Checking the convex hull
points=np.column_stack((x_c,y_c))
hull = ConvexHull(points);
plt.plot(points[hull.vertices,0], points[hull.vertices,1], 'r--', lw=2)
plt.scatter(x_c, y_c, marker='o', s=5, zorder=10)
#
# Mapping the irregular data onto a regular grid and plotting
xic = np.linspace(min(x_c), max(x_c), 1000)
yic = np.linspace(min(y_c), max(y_c), 1000)
zic = griddata((x_c, y_c), z_c, (xic[None,:], yic[:,None]))
CS = plt.contour(xic,yic,zic,15,linewidths=0.5,colors='k')
CS = plt.contourf(xic,yic,zic,15,cmap=plt.cm.summer)
plt.colorbar() # draw colorbar
#
#plt.scatter(x_c, y_c, marker='o', s=5, zorder=10)
plt.axis('equal')
plt.savefig('foo.pdf', bbox_inches='tight')
plt.show()
and the output looks like
The problem is that griddata uses a convex hull and this convex hull exceeds the edges of the irregular data. Is there any way to set the values of the griddata points which are outside the edges of the boundary of the original points to zero?
Edit
In the end I threw in the towel and reverted back to Matlab. I'll have to export the data to pgfplots to get a nice plot. The code I came up with was
x = strain.x;
y = strain.y;
z = strain.eps;
% Get the alpha shape (couldn't do this in python easily)
shp = alphaShape(x,y,.001);
% Get the boundary nodes
[bi, xy] = boundaryFacets(shp);
no_grid = 500;
xb=xy(:,1);
yb=xy(:,2);
[X,Y] = ndgrid(linspace(min(x),max(x),no_grid),linspace(min(y),max(y),no_grid));
Z = griddata(x,y,z,X,Y,'v4');
% Got through the regular grid and set the values which are outside the boundary of the original domain to Nans
for j = 1:no_grid
[in,on] = inpolygon(X(:,j),Y(:,j),xb,yb);
Z(~in,j) = NaN;
end
contourf(X,Y,Z,10),axis equal
colorbar
hold on
plot(xb,yb)
axis equal
hold off
Here is the resulting image.
If someone can do something similar in Python I'll happily accept the answer.

I had to plot interpolated data on a complex geometry (see the blue points on figure) P(x,z) (z is the horizontal coordinate). I used mask operations and it worked well. Without mask, the whole square (x=0..1 ; z=0..17.28) is covered by contourf.
## limiting values for geometry
xmax1=0.408
zmin1=6.
xmax2=0.064
zmin2=13.12
xmin=0.
xmax=1.
zmin=0.
zmax=17.28
# Grid for points
x1 = np.arange(xmin,xmax+dx,dx)
z1 = np.arange(zmin,zmax+dz,dz)
zi2,xi2 = np.meshgrid(z1,x1)
mask = (((zi2 > zmin2) & (xi2 > xmax2)) | ((zi2 > zmin1) & (zi2 <= zmin2) & (xi2 > xmax1)))
zim=np.ma.masked_array(zi2,mask)
xim=np.ma.masked_array(xi2,mask)
# Grid for P values
# npz=z coordinates of data, npx is the x coordinates and npp is P values
grid_p = scipy.interpolate.griddata((npz, npx), npp, (zim,xim),method='nearest')
pm=np.ma.masked_array(grid_p,mask)
# plot
plt.contour(zim, xim, pm, 25, linewidths=0.5, colors='k',corner_mask=False)
plt.contourf(zim, xim, pm, 25,vmax=grid_p.max(), vmin=grid_p.min(),corner_mask=False)
plt.colorbar()
# Scatter plot to check
plt.scatter(npz,npr, marker='x', s=2)
plt.show()
enter image description here

Color scale - close but not close enough

I'm trying to produce a plot which uses the same colorscale as the Met Office, so I can easily compare my plots to theirs. An example of theirs is at Here
My current closest effort is here:
Here
I appreciate my code is messy - I couldn't find a way to set a color for values above a certain threshold (otherwise it goes white),hence the loop.
I would upload the NetCDF File but I haven't got a high enough rep to do this.
Many, many thanks in advance for any help.
My code for plotting is shown below;
from Scientific.IO.NetCDF import NetCDFFile
from mpl_toolkits.basemap import Basemap
from matplotlib import pyplot as plt
import numpy as np
myfile = NetCDFFile('ERA_Dec_89-94.nc', 'r')
Lat = NetCDFFile('/home/james/Documents/Lat_Lon_NC_Files/latitudes_d02.nc','r')
Long = NetCDFFile('/home/james/Documents/Lat_Lon_NC_Files/longitudes_d02.nc','r')
XLAT = Lat.variables['XLAT'][:]
XLONG = Long.variables['XLONG'][:]
ERA_Data = myfile.variables['Monthlyrain'][:]
plot = np.zeros([1000,1730])
plot[:,:] = np.average(ERA_Data[:,:,:],axis=0)
m = Basemap(projection='merc',resolution='f',llcrnrlat=49,llcrnrlon=-11,urcrnrlat=61,urcrnrlon=3)
m.drawparallels(np.arange(-90., 91., 5.), labels=[1,0,0,0], fontsize=11)
m.drawmeridians(np.arange(-180., 181., 5.), labels=[0,0,0,1], fontsize=11)
m.drawcoastlines()
X, Y = m(XLONG, XLAT)
for i in range(0,1729):
for j in range(0,999):
if plot[j,i] >250:
plot[j,i] = 250.001
if plot[j,i] < 40:
plot[j,i] = 40
scale = [40,40.001,60,80,100,125,150,200,250, 250.001]
cs = m.contourf(X,Y,plot,scale, cmap='PuOr')
cbar = m.colorbar(cs, ticks= [40.0005,50,70,90,112.5,137.5,175,225,250.0005])
cbar.set_ticklabels(['<40','40-60', '60-80', '80-100', '100-125', '125-150', '150-200', '200-250', '>250'])
plt.title('Some Title')
cbar.set_label('Monthly average rainfall (mm)')
print "Finished"
plt.show()

If the issue is simply the colormap, you can pick the RGB components of the colors off your screen and turn them into a ListedColormap, mapped to the boundaries of the rainfall in the chart you give as an example. For example,
bounds = [0, 40, 60, 80, 100, 125, 150, 200, 250, 1000]
rgblist = [(51,0,0), (102,51,0), (153,102,51), (204,153,102), (255, 255, 255),
(204,204,255), (153,153,255), (51,102,255), (0,0,153)]
clist = [[c/255 for c in rgb] for rgb in rgblist]
from matplotlib import colors
cmap = colors.ListedColormap(clist)
norm = colors.BoundaryNorm(bounds, cmap.N)
ax.imshow(arr, cmap=cmap, norm=norm)
plt.show()

The first part (getting the colors right) was already answered. In order to restrict the values to a certain range you have several options.
Use cmap.set_over and cmap.set_under to set out-of-bounds colors, as described here
use np.clip instead of the loop to restrict the values to a certian range:
plot = np.clip(plot, 40, 250)