Develop Reference

Matplotlib Contour/Contourf in Cartopy singularity at North Pole/ dateline

I am trying to produce heatmaps showing atmospheric attenuation values for a RF link to a satellite above the North Pole, but I have issues with the interpolation done by the Matplotlib contour/contourf functions.
The linear interpolation done by the contourf function does not work well around the N.Pole, as I suspect it does not know to interpolate between values which go from (-180 deg to +180 deg) - i.e. cross the dateline, or cross the pole.
Any suggestions on a different approach to generate the heatmap, to avoid this horrible hole at the centre?!
Code below to generate plot.
import cartopy.crs as ccrs
import cartopy.feature
plt.figure(figsize=(10,10))
# Initialise Cartopy Axes.
proj=ccrs.LambertAzimuthalEqualArea(central_longitude=0, central_latitude=90)
ax = plt.axes(projection = proj)
ax.set_extent([-180,180,45,90], ccrs.PlateCarree())
ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.gridlines(ls=":",color="grey",lw=0.5)
x0,x1 = attenuation_df.lon.min(), attenuation_df.lon.max()
y0,y1 = attenuation_df.lat.min(), attenuation_df.lat.max()
x,y = np.linspace(x0,x1,1000), np.linspace(y0,y1,1000)
X,Y = np.meshgrid(x,y)
Z = scipy.interpolate.griddata(
attenuation_df[["lon","lat"]],
attenuation_df["attenuation"],
(X,Y),
method="linear",
)
plt.contourf(X,Y,Z,transform=ccrs.PlateCarree(),alpha=0.5)
plt.colorbar(shrink=0.5)
plt.title("Attenuation")
plt.show()
Attenuation_df is a Pandas Dataframe which contains an attenuation value at approximately 3500 sample points, which are equally spaced around the globe. Here is the location of the sample points:
Here is the header of attenuation_df:
lon
lat
attenuation
0
-30.8538
48.8813
0.860307
1
-29.0448
49.5026
0.783662
2
-27.2358
50.1317
0.720165
3
-32.6628
48.2676
0.947662
4
37.4226
46.0322
0.27495
The link to the csv of attenuation_df is here: https://pastebin.com/NYA1jFgt

A solution is to reproject your data to a different coordinate system, my suggestion is to use a Polar Stereographic system. However, the large "hole" centered at the North Pole is not coming from the coordinate system in use but to the presence of some nans in your dataset, so you first have to remove those values.
Here a working solution:
from pyproj import Proj
# Define a pyproj function to reproject data
def coordinate_conv(x, y, inverse = True):
p = Proj('+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs')
return p(x, y, inverse = inverse)
# Drop null values
attenuation_df.dropna(how = 'any', inplace = True)
# Reproject data
rpjx, rpjy = coordinate_conv(attenuation_df.lon, attenuation_df.lat, False)
rpj_cord = pd.DataFrame({'x': rpjx, 'y': rpjy})
# Interpoolate data
x,y = np.linspace(rpjx.min(),rpjx.max(),1000), np.linspace(rpjy.min(),rpjy.max(),1000)
X,Y = np.meshgrid(x,y)
Z = interpolate.griddata(
rpj_cord,
attenuation_df["attenuation"],
(X,Y),
method="linear",
)
# Figure
plt.figure(figsize=(10,10))
# Initialise Cartopy Axes.
proj=ccrs.LambertAzimuthalEqualArea(central_longitude=0, central_latitude=90)
ax = plt.axes(projection = proj)
ax.set_extent([-180,180,45,90], ccrs.PlateCarree())
ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.gridlines(ls=":",color="grey",lw=0.5)
kw = dict(central_latitude=90, central_longitude=-45, true_scale_latitude=70)
plt.contourf(X,Y,Z, transform=ccrs.Stereographic(**kw),alpha=0.5)
plt.colorbar(shrink=0.5)
plt.title("Attenuation")
And this is the output figure:

changing colorbar sensitivity

I re edited the question in order to make it more clear with what I need help. So I have the following code so far
#import the libraries
import netCDF4 as nc
from mpl_toolkits.basemap import Basemap
import numpy as np
import matplotlib.pyplot as plt
#check what variables are included
# in our case : dict_keys(['time', 'time_bnds', 'lat', 'lat_bnds', 'lon', 'lon_bnds', 'height', 'tas'])
print(data.variables.keys())
#extract the data
lats = data.variables['lat'][:]
lons = data.variables['lon'][:]
time = data.variables['time'][:]
te = data.variables['tas'][:]-273.5
#extract the coordinates we want to plot (Italy)by inputting lower #left and top right coordinates
italy = Basemap(projection='merc',
#ITALY
llcrnrlon= 6.63,
llcrnrlat= 35.29,
urcrnrlon= 18.78,
urcrnrlat= 47.09,
resolution= 'f')
#create the plot
lon,lat =np.meshgrid(lons,lats)
x,y=italy(lon,lat)
c_scheme = italy.pcolor(x,y,np.squeeze(te[0,:,:]), cmap ='jet')
italy.drawcoastlines()
italy.drawstates()
italy.drawcounties()
cbar = italy.colorbar(c_scheme, location='right',pad='10%')
plt.title('KIOST-ESM')
plt.show()
which creates the following heat map
Now what I want is to make the plot identify(colour) smaller differences between the temperatures. However I do not know exactly how to find the min and max value of my area that I am investigating (this time it is Italy). My variable te that has the temperatures is of the type : numpy.ma.core.MaskedArray with shape (9125, 160, 320) [time,lon,lat]

Setting vmin and vmax values should solve this. please try
min_height = int(np.min(np.squeeze(te[0,:,:])))
max_height = int(np.max(np.squeeze(te[0,:,:])))
c_scheme = italy.pcolor(x,y,np.squeeze(te[0,:,:]), cmap ='jet', vmin=min_height, vmax=max_height)
In the case that the te array holds is bigger than the mesh in x and y, I would try doing the following:
min_height = int(np.min(np.squeeze(te[0,x,y])))
max_height = int(np.max(np.squeeze(te[0,x,y])))

How to use geopandas to plot latitude and longitude on a more detailed map with by using basemaps?

I am trying to plot some latitude and longitudes on the map of delhi which I am able to do by using a shape file in python3.8 using geopandas
Here is the link for the shape file:
https://drive.google.com/file/d/1CEScjlcsKFCgdlME21buexHxjCbkb3WE/view?usp=sharing
Following is my code to plot points on the map:
lo=[list of longitudes]
la=[list of latitudes]
delhi_map = gpd.read_file(r'C:\Users\Desktop\Delhi_Wards.shp')
fig,ax = plt.subplots(figsize = (15,15))
delhi_map.plot(ax = ax)
geometry = [Point(xy) for xy in zip(lo,la)]
geo_df = gpd.GeoDataFrame(geometry = geometry)
print(geo_df)
g = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi')
plt.show()
Following is the result:
Now this map is not very clear and anyone will not be able to recognise the places marked so i tried to use basemap for a more detailed map through the following code:
df = gpd.read_file(r'C:\Users\Jojo\Desktop\Delhi_Wards.shp')
new_df = df.to_crs(epsg=3857)
print(df.crs)
print(new_df.crs)
ax = new_df.plot()
ctx.add_basemap(ax)
plt.show()
And following is the result:
I am getting the basemap but my shapefile is overlapping it. Can i get a map to plot my latitudes and longitudes where the map is much more detailed with names of places or roads or anything similar to it like in google maps or even something like the map which is being overlapped by the blue shapefile map?
Is it possible to plot on a map like this??
https://www.researchgate.net/profile/P_Jops/publication/324715366/figure/fig3/AS:618748771835906#1524532611545/Map-of-Delhi-reproduced-from-Google-Maps-12.png

use zorder parameter to adjust the layers' orders (lower zorder means lower layer), and alpha to the polygon. anyway, I guess, you're plotting df twice, that's why it's overlapping.
here's my script and the result
import geopandas as gpd
import matplotlib.pyplot as plt
import contextily as ctx
from shapely.geometry import Point
long =[77.2885437011719, 77.231931, 77.198767, 77.2750396728516]
lat = [28.6877899169922, 28.663863, 28.648287, 28.5429172515869]
geometry = [Point(xy) for xy in zip(long,lat)]
wardlink = "New Folder/wards delimited.shp"
ward = gpd.read_file(wardlink, bbox=None, mask=None, rows=None)
geo_df = gpd.GeoDataFrame(geometry = geometry)
ward.crs = {'init':"epsg:4326"}
geo_df.crs = {'init':"epsg:4326"}
# plot the polygon
ax = ward.plot(alpha=0.35, color='#d66058', zorder=1)
# plot the boundary only (without fill), just uncomment
#ax = gpd.GeoSeries(ward.to_crs(epsg=3857)['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
ax = gpd.GeoSeries(ward['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
# plot the marker
ax = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi', zorder=3)
ctx.add_basemap(ax, crs=geo_df.crs.to_string(), source=ctx.providers.OpenStreetMap.Mapnik)
plt.show()
I don't know about google maps being in the contextily, I don't think it's available. alternatively, you can use OpenStreetMap base map which shows quite the same toponym, or any other basemap you can explore. use `source` keyword in the argument, for example, `ctx.add_basemap(ax, source=ctx.providers.OpenStreetMap.Mapnik)` . here's how to check the available providers and the map each providers provides:
>>> ctx.providers.keys()
dict_keys(['OpenStreetMap', 'OpenSeaMap', 'OpenPtMap', 'OpenTopoMap', 'OpenRailwayMap', 'OpenFireMap', 'SafeCast', 'Thunderforest', 'OpenMapSurfer', 'Hydda', 'MapBox', 'Stamen', 'Esri', 'OpenWeatherMap', 'HERE', 'FreeMapSK', 'MtbMap', 'CartoDB', 'HikeBike', 'BasemapAT', 'nlmaps', 'NASAGIBS', 'NLS', 'JusticeMap', 'Wikimedia', 'GeoportailFrance', 'OneMapSG'])
>>> ctx.providers.OpenStreetMap.keys()
dict_keys(['Mapnik', 'DE', 'CH', 'France', 'HOT', 'BZH'])

I don't know geopandas. The idea I'm suggesting uses only basic python and matplotlib. I hope you can adapt it to your needs.
The background is the following map. I figured out the GPS coordinates of its corners using google-maps.
The code follows the three points of my remark. Note that the use of imread and imshow reverses the y coordinate. This is why the function coordinatesOnFigur looks non-symmetrical in x and y.
Running the code yields the map with a red bullet near Montijo (there is a small test at the end).
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib import patches
from matplotlib.widgets import Button
NE = (-8.9551, 38.8799)
SE = (-8.9551, 38.6149)
SW = (-9.4068, 38.6149)
NW = (-9.4068, 38.8799)
fig = plt.figure(figsize=(8, 6))
axes = fig.add_subplot(1,1,1, aspect='equal')
img_array = plt.imread("lisbon_2.jpg")
axes.imshow(img_array)
xmax = axes.get_xlim()[1]
ymin = axes.get_ylim()[0] # the y coordinates are reversed, ymax=0
# print(axes.get_xlim(), xmax)
# print(axes.get_ylim(), ymin)
def coordinatesOnFigure(long, lat, SW=SW, NE=NE, xmax=xmax, ymin=ymin):
px = xmax/(NE[0]-SW[0])
qx = -SW[0]*xmax/(NE[0]-SW[0])
py = -ymin/(NE[1]-SW[1])
qy = NE[1]*ymin/(NE[1]-SW[1])
return px*long + qx, py*lat + qy
# plotting a red bullet that corresponds to a GPS location on the map
x, y = coordinatesOnFigure(-9, 38.7)
print("test: on -9, 38.7 we get", x, y)
axes.scatter(x, y, s=40, c='red', alpha=0.9)
plt.show()

How to plot a shapefile centered in the Pacific with Basemap?

When plotting with Basemap's readshapefile, if the defined map is centered anywhere else than the longitudinal center of the shapefile, only a portion of it it's plotted. Here's an example using Natural Earth's coastlines:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = './NaturalEarth/ne_50m_land/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land')
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
Which produces:
Is there a workaround for this with Basemap or Python? I know some people re-center the shapefile in QGIS or similar, but it seems unpractical to do so every time you create a new map, and my QGIS skills are extremely basic.

One way to do it would be to tell readshapefile not to plot the coastlines directly and then to manipulate the line segments before plotting them yourself. Here an example based on your use case:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = 'shapefiles/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land', drawbounds = False)
boundary = 0.0
for info, shape in zip(m.ne_50m_land_info, m.ne_50m_land):
lons, lats = map(np.array, zip(*shape))
sep = (lons <= boundary).astype(int)
roots = np.where(sep[:-1]+sep[1:] == 1)[0]+1
lower = np.concatenate([[0],roots]).astype(int)
upper = np.concatenate([roots,[len(lons)]]).astype(int)
for low, high in zip(lower,upper):
lo_patch = lons[low:high]
la_patch = lats[low:high]
lo_patch[lo_patch<0] += 360
x,y = m(lo_patch,la_patch)
ax.plot(x,y,'k',lw=0.5)
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
plt.show()
In the example above, I iterate through the line segments of the shape file the way it is explained in the Basemap documentation. First I thought it would be enough to just add 360 to each point with a longitude smaller 0, but then you would get horizontal lines whenever a coast line crosses the 0 degree line. So, instead, one has to cut the lines into smaller segments whenever such a crossing appears. This is quite easily accomplished with numpy. I then use the plot command to draw the coast lines. If you want to do something more complex have a look at the Basemap documentation.
The final result looks like this:
Hope this helps.

Color scale - close but not close enough

I'm trying to produce a plot which uses the same colorscale as the Met Office, so I can easily compare my plots to theirs. An example of theirs is at Here
My current closest effort is here:
Here
I appreciate my code is messy - I couldn't find a way to set a color for values above a certain threshold (otherwise it goes white),hence the loop.
I would upload the NetCDF File but I haven't got a high enough rep to do this.
Many, many thanks in advance for any help.
My code for plotting is shown below;
from Scientific.IO.NetCDF import NetCDFFile
from mpl_toolkits.basemap import Basemap
from matplotlib import pyplot as plt
import numpy as np
myfile = NetCDFFile('ERA_Dec_89-94.nc', 'r')
Lat = NetCDFFile('/home/james/Documents/Lat_Lon_NC_Files/latitudes_d02.nc','r')
Long = NetCDFFile('/home/james/Documents/Lat_Lon_NC_Files/longitudes_d02.nc','r')
XLAT = Lat.variables['XLAT'][:]
XLONG = Long.variables['XLONG'][:]
ERA_Data = myfile.variables['Monthlyrain'][:]
plot = np.zeros([1000,1730])
plot[:,:] = np.average(ERA_Data[:,:,:],axis=0)
m = Basemap(projection='merc',resolution='f',llcrnrlat=49,llcrnrlon=-11,urcrnrlat=61,urcrnrlon=3)
m.drawparallels(np.arange(-90., 91., 5.), labels=[1,0,0,0], fontsize=11)
m.drawmeridians(np.arange(-180., 181., 5.), labels=[0,0,0,1], fontsize=11)
m.drawcoastlines()
X, Y = m(XLONG, XLAT)
for i in range(0,1729):
for j in range(0,999):
if plot[j,i] >250:
plot[j,i] = 250.001
if plot[j,i] < 40:
plot[j,i] = 40
scale = [40,40.001,60,80,100,125,150,200,250, 250.001]
cs = m.contourf(X,Y,plot,scale, cmap='PuOr')
cbar = m.colorbar(cs, ticks= [40.0005,50,70,90,112.5,137.5,175,225,250.0005])
cbar.set_ticklabels(['<40','40-60', '60-80', '80-100', '100-125', '125-150', '150-200', '200-250', '>250'])
plt.title('Some Title')
cbar.set_label('Monthly average rainfall (mm)')
print "Finished"
plt.show()

If the issue is simply the colormap, you can pick the RGB components of the colors off your screen and turn them into a ListedColormap, mapped to the boundaries of the rainfall in the chart you give as an example. For example,
bounds = [0, 40, 60, 80, 100, 125, 150, 200, 250, 1000]
rgblist = [(51,0,0), (102,51,0), (153,102,51), (204,153,102), (255, 255, 255),
(204,204,255), (153,153,255), (51,102,255), (0,0,153)]
clist = [[c/255 for c in rgb] for rgb in rgblist]
from matplotlib import colors
cmap = colors.ListedColormap(clist)
norm = colors.BoundaryNorm(bounds, cmap.N)
ax.imshow(arr, cmap=cmap, norm=norm)
plt.show()

The first part (getting the colors right) was already answered. In order to restrict the values to a certain range you have several options.
Use cmap.set_over and cmap.set_under to set out-of-bounds colors, as described here
use np.clip instead of the loop to restrict the values to a certian range:
plot = np.clip(plot, 40, 250)