Develop Reference

hvplot(geo=True) argument maps XY coordinates to incorrect latitude coordinates

I have a raster of the Northern beaches, Australia, in the UTM CRS projection for this region (56S). Now when I'm plotting this data using the hvplot.xarray extension as da.hvplot(x="x", y="y", geo=True), the argument geo=True triggers y coordinates [meters] to be converted to coordinates [lon, lat], but the latitudes are mapped as northern hemisphere coordinates. They become lat = 90 + (-y), whereas it should simply be lat = -y. Which function is triggered by geo=True that infers the coordinates from XY to lon/lat?
Plotting with matplotlib works fine, see: https://i.stack.imgur.com/zRC68.png
Aim
Overal objective is to overlay the vector data (shoreline) on top of the raster (ndwi).
Some software version
python==3.10.8
hvplot.__version__ # '0.8.2'
holoviews.__version__ # 1.15.3
Complete, minimal, self-contained example code that reproduces the issue
import cartopy.crs as ccrs
import geopandas as gpd
import geoviews as gv
import geoviews.tile_sources as gts
import holoviews as hv
import hvplot.xarray
import xarray as xr
ndwi = xr.open_dataset(
"https://s3.eu-central-1.amazonaws.com/floris.calkoen.open.data/narrabeen_ndwi_2023-01-24T00%3A02%3A21.tif",
engine="rasterio",
)
shoreline = gpd.read_file(
"https://s3.eu-central-1.amazonaws.com/floris.calkoen.open.data/narrabeen_shoreline_2023-01-24T00%3A02%3A21.geojson"
)
ndwi.rio.crs == shoreline.crs ## true
utm_zone = shoreline.crs.utm_zone
southern_hemisphere = True if utm_zone[-1] == "S" else False
utm_code = int(utm_zone[:-1])
utm_crs = ccrs.UTM(
utm_code, southern_hemisphere
) # why cartopy.crs.UTM() doesn't handle utm zone strings?
shoreline_plot = gv.Path([shoreline], crs=utm_crs)
ndwi_plot = ndwi["band_data"].squeeze().hvplot(x="x", y="y", geo=True, crs=utm_crs)
shoreline_plot * gts.EsriImagery # this seems correct
ndwi_plot # see how the XY coordinates are converted from XY UTM zone 56S to latitudes in the northern hemisphere (90 + (-y))
shoreline_plot * ndwi_plot # therefore the overlay doesn't work
# to plot with matplotblib
fig, ax = plt.subplots()
ndwi["band_data"].squeeze().plot(ax=ax)
shoreline.plot(ax=ax, color='black', linewidth=5, label="shoreline")
plt.legend()
plt.show()

How to use geopandas to plot latitude and longitude on a more detailed map with by using basemaps?

I am trying to plot some latitude and longitudes on the map of delhi which I am able to do by using a shape file in python3.8 using geopandas
Here is the link for the shape file:
https://drive.google.com/file/d/1CEScjlcsKFCgdlME21buexHxjCbkb3WE/view?usp=sharing
Following is my code to plot points on the map:
lo=[list of longitudes]
la=[list of latitudes]
delhi_map = gpd.read_file(r'C:\Users\Desktop\Delhi_Wards.shp')
fig,ax = plt.subplots(figsize = (15,15))
delhi_map.plot(ax = ax)
geometry = [Point(xy) for xy in zip(lo,la)]
geo_df = gpd.GeoDataFrame(geometry = geometry)
print(geo_df)
g = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi')
plt.show()
Following is the result:
Now this map is not very clear and anyone will not be able to recognise the places marked so i tried to use basemap for a more detailed map through the following code:
df = gpd.read_file(r'C:\Users\Jojo\Desktop\Delhi_Wards.shp')
new_df = df.to_crs(epsg=3857)
print(df.crs)
print(new_df.crs)
ax = new_df.plot()
ctx.add_basemap(ax)
plt.show()
And following is the result:
I am getting the basemap but my shapefile is overlapping it. Can i get a map to plot my latitudes and longitudes where the map is much more detailed with names of places or roads or anything similar to it like in google maps or even something like the map which is being overlapped by the blue shapefile map?
Is it possible to plot on a map like this??
https://www.researchgate.net/profile/P_Jops/publication/324715366/figure/fig3/AS:618748771835906#1524532611545/Map-of-Delhi-reproduced-from-Google-Maps-12.png

use zorder parameter to adjust the layers' orders (lower zorder means lower layer), and alpha to the polygon. anyway, I guess, you're plotting df twice, that's why it's overlapping.
here's my script and the result
import geopandas as gpd
import matplotlib.pyplot as plt
import contextily as ctx
from shapely.geometry import Point
long =[77.2885437011719, 77.231931, 77.198767, 77.2750396728516]
lat = [28.6877899169922, 28.663863, 28.648287, 28.5429172515869]
geometry = [Point(xy) for xy in zip(long,lat)]
wardlink = "New Folder/wards delimited.shp"
ward = gpd.read_file(wardlink, bbox=None, mask=None, rows=None)
geo_df = gpd.GeoDataFrame(geometry = geometry)
ward.crs = {'init':"epsg:4326"}
geo_df.crs = {'init':"epsg:4326"}
# plot the polygon
ax = ward.plot(alpha=0.35, color='#d66058', zorder=1)
# plot the boundary only (without fill), just uncomment
#ax = gpd.GeoSeries(ward.to_crs(epsg=3857)['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
ax = gpd.GeoSeries(ward['geometry'].unary_union).boundary.plot(ax=ax, alpha=0.5, color="#ed2518",zorder=2)
# plot the marker
ax = geo_df.plot(ax = ax, markersize = 20, color = 'red',marker = '*',label = 'Delhi', zorder=3)
ctx.add_basemap(ax, crs=geo_df.crs.to_string(), source=ctx.providers.OpenStreetMap.Mapnik)
plt.show()
I don't know about google maps being in the contextily, I don't think it's available. alternatively, you can use OpenStreetMap base map which shows quite the same toponym, or any other basemap you can explore. use `source` keyword in the argument, for example, `ctx.add_basemap(ax, source=ctx.providers.OpenStreetMap.Mapnik)` . here's how to check the available providers and the map each providers provides:
>>> ctx.providers.keys()
dict_keys(['OpenStreetMap', 'OpenSeaMap', 'OpenPtMap', 'OpenTopoMap', 'OpenRailwayMap', 'OpenFireMap', 'SafeCast', 'Thunderforest', 'OpenMapSurfer', 'Hydda', 'MapBox', 'Stamen', 'Esri', 'OpenWeatherMap', 'HERE', 'FreeMapSK', 'MtbMap', 'CartoDB', 'HikeBike', 'BasemapAT', 'nlmaps', 'NASAGIBS', 'NLS', 'JusticeMap', 'Wikimedia', 'GeoportailFrance', 'OneMapSG'])
>>> ctx.providers.OpenStreetMap.keys()
dict_keys(['Mapnik', 'DE', 'CH', 'France', 'HOT', 'BZH'])

I don't know geopandas. The idea I'm suggesting uses only basic python and matplotlib. I hope you can adapt it to your needs.
The background is the following map. I figured out the GPS coordinates of its corners using google-maps.
The code follows the three points of my remark. Note that the use of imread and imshow reverses the y coordinate. This is why the function coordinatesOnFigur looks non-symmetrical in x and y.
Running the code yields the map with a red bullet near Montijo (there is a small test at the end).
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib import patches
from matplotlib.widgets import Button
NE = (-8.9551, 38.8799)
SE = (-8.9551, 38.6149)
SW = (-9.4068, 38.6149)
NW = (-9.4068, 38.8799)
fig = plt.figure(figsize=(8, 6))
axes = fig.add_subplot(1,1,1, aspect='equal')
img_array = plt.imread("lisbon_2.jpg")
axes.imshow(img_array)
xmax = axes.get_xlim()[1]
ymin = axes.get_ylim()[0] # the y coordinates are reversed, ymax=0
# print(axes.get_xlim(), xmax)
# print(axes.get_ylim(), ymin)
def coordinatesOnFigure(long, lat, SW=SW, NE=NE, xmax=xmax, ymin=ymin):
px = xmax/(NE[0]-SW[0])
qx = -SW[0]*xmax/(NE[0]-SW[0])
py = -ymin/(NE[1]-SW[1])
qy = NE[1]*ymin/(NE[1]-SW[1])
return px*long + qx, py*lat + qy
# plotting a red bullet that corresponds to a GPS location on the map
x, y = coordinatesOnFigure(-9, 38.7)
print("test: on -9, 38.7 we get", x, y)
axes.scatter(x, y, s=40, c='red', alpha=0.9)
plt.show()

How to plot a shapefile centered in the Pacific with Basemap?

When plotting with Basemap's readshapefile, if the defined map is centered anywhere else than the longitudinal center of the shapefile, only a portion of it it's plotted. Here's an example using Natural Earth's coastlines:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = './NaturalEarth/ne_50m_land/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land')
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
Which produces:
Is there a workaround for this with Basemap or Python? I know some people re-center the shapefile in QGIS or similar, but it seems unpractical to do so every time you create a new map, and my QGIS skills are extremely basic.

One way to do it would be to tell readshapefile not to plot the coastlines directly and then to manipulate the line segments before plotting them yourself. Here an example based on your use case:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = 'shapefiles/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land', drawbounds = False)
boundary = 0.0
for info, shape in zip(m.ne_50m_land_info, m.ne_50m_land):
lons, lats = map(np.array, zip(*shape))
sep = (lons <= boundary).astype(int)
roots = np.where(sep[:-1]+sep[1:] == 1)[0]+1
lower = np.concatenate([[0],roots]).astype(int)
upper = np.concatenate([roots,[len(lons)]]).astype(int)
for low, high in zip(lower,upper):
lo_patch = lons[low:high]
la_patch = lats[low:high]
lo_patch[lo_patch<0] += 360
x,y = m(lo_patch,la_patch)
ax.plot(x,y,'k',lw=0.5)
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
plt.show()
In the example above, I iterate through the line segments of the shape file the way it is explained in the Basemap documentation. First I thought it would be enough to just add 360 to each point with a longitude smaller 0, but then you would get horizontal lines whenever a coast line crosses the 0 degree line. So, instead, one has to cut the lines into smaller segments whenever such a crossing appears. This is quite easily accomplished with numpy. I then use the plot command to draw the coast lines. If you want to do something more complex have a look at the Basemap documentation.
The final result looks like this:
Hope this helps.

Python irregular x,y data to contour plot on original domain

I have file containing points under the columns "x-cord", "y-cord", "value". These are irregularly spaced. I am trying to make a contour plot of "value" and overlay this over the original domain. I gave up trying to do this in both pgfplots and matlab and thought I would give python a go. An answer in any of these scripts would be fine. The python script is as follows
import numpy as np
from scipy.interpolate import griddata
import matplotlib.pyplot as plt
import numpy.ma as ma
from numpy.random import uniform, seed
from scipy.spatial import ConvexHull
#
# Loading data
filename = "strain.dat"
coordinates = []
x_c = []
y_c = []
z_c = []
xyz = open(filename)
title = xyz.readline()
for line in xyz:
x,y,z = line.split()
coordinates.append([float(x), float(y), float(z)])
x_c.append([float(x)])
y_c.append([float(y)])
z_c.append([float(z)])
xyz.close()
#
# Rehaping and translating data
x_c=np.ravel(np.array(x_c))
y_c=np.ravel(np.array(y_c))
z_c=np.ravel(np.array(z_c))
x_c = x_c-100.0
y_c = y_c-100.0
#
# Checking the convex hull
points=np.column_stack((x_c,y_c))
hull = ConvexHull(points);
plt.plot(points[hull.vertices,0], points[hull.vertices,1], 'r--', lw=2)
plt.scatter(x_c, y_c, marker='o', s=5, zorder=10)
#
# Mapping the irregular data onto a regular grid and plotting
xic = np.linspace(min(x_c), max(x_c), 1000)
yic = np.linspace(min(y_c), max(y_c), 1000)
zic = griddata((x_c, y_c), z_c, (xic[None,:], yic[:,None]))
CS = plt.contour(xic,yic,zic,15,linewidths=0.5,colors='k')
CS = plt.contourf(xic,yic,zic,15,cmap=plt.cm.summer)
plt.colorbar() # draw colorbar
#
#plt.scatter(x_c, y_c, marker='o', s=5, zorder=10)
plt.axis('equal')
plt.savefig('foo.pdf', bbox_inches='tight')
plt.show()
and the output looks like
The problem is that griddata uses a convex hull and this convex hull exceeds the edges of the irregular data. Is there any way to set the values of the griddata points which are outside the edges of the boundary of the original points to zero?
Edit
In the end I threw in the towel and reverted back to Matlab. I'll have to export the data to pgfplots to get a nice plot. The code I came up with was
x = strain.x;
y = strain.y;
z = strain.eps;
% Get the alpha shape (couldn't do this in python easily)
shp = alphaShape(x,y,.001);
% Get the boundary nodes
[bi, xy] = boundaryFacets(shp);
no_grid = 500;
xb=xy(:,1);
yb=xy(:,2);
[X,Y] = ndgrid(linspace(min(x),max(x),no_grid),linspace(min(y),max(y),no_grid));
Z = griddata(x,y,z,X,Y,'v4');
% Got through the regular grid and set the values which are outside the boundary of the original domain to Nans
for j = 1:no_grid
[in,on] = inpolygon(X(:,j),Y(:,j),xb,yb);
Z(~in,j) = NaN;
end
contourf(X,Y,Z,10),axis equal
colorbar
hold on
plot(xb,yb)
axis equal
hold off
Here is the resulting image.
If someone can do something similar in Python I'll happily accept the answer.

I had to plot interpolated data on a complex geometry (see the blue points on figure) P(x,z) (z is the horizontal coordinate). I used mask operations and it worked well. Without mask, the whole square (x=0..1 ; z=0..17.28) is covered by contourf.
## limiting values for geometry
xmax1=0.408
zmin1=6.
xmax2=0.064
zmin2=13.12
xmin=0.
xmax=1.
zmin=0.
zmax=17.28
# Grid for points
x1 = np.arange(xmin,xmax+dx,dx)
z1 = np.arange(zmin,zmax+dz,dz)
zi2,xi2 = np.meshgrid(z1,x1)
mask = (((zi2 > zmin2) & (xi2 > xmax2)) | ((zi2 > zmin1) & (zi2 <= zmin2) & (xi2 > xmax1)))
zim=np.ma.masked_array(zi2,mask)
xim=np.ma.masked_array(xi2,mask)
# Grid for P values
# npz=z coordinates of data, npx is the x coordinates and npp is P values
grid_p = scipy.interpolate.griddata((npz, npx), npp, (zim,xim),method='nearest')
pm=np.ma.masked_array(grid_p,mask)
# plot
plt.contour(zim, xim, pm, 25, linewidths=0.5, colors='k',corner_mask=False)
plt.contourf(zim, xim, pm, 25,vmax=grid_p.max(), vmin=grid_p.min(),corner_mask=False)
plt.colorbar()
# Scatter plot to check
plt.scatter(npz,npr, marker='x', s=2)
plt.show()
enter image description here

How to plot gridded data in basemap and find grid points along a specific azimuth

Update: I am trying to map some data. I have a set of measured back-azimuths (baz) from a reference point in a grid. I want to find all points on the grid that a great circle along the baz would cross. To do this I iterate through each point in the grid, calculate expected back-azimuth between that point and the reference point and compare to each measured baz. If the difference between the two is small (less than 2 degrees) I weight that point. I then put it all on a map. The code I use is below but the results look a bit strange, does anyone know where I have gone wrong, or if there is a better approach (faster) then what I have done??
from matplotlib.colorbar import ColorbarBase
import matplotlib.cm as cm
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.basemap import Basemap
import mpl_toolkits.basemap.pyproj as pyproj
llcrnrlon = -30.0
llcrnrlat = 45.0
urcrnrlon = 0.0
urcrnrlat = 65.0
lon_0 = (urcrnrlon + llcrnrlon) / 2.
lat_0 = (urcrnrlat + llcrnrlat) / 2.
lat = 51.58661577 # reference point
lon = -9.18822525
# Generate random back-azimuths.
baz = zeros((20))
for i in xrange(len(baz)):
baz[i] = random.randint(200,230)
####################################################################
## Set up the map background.
m = Basemap(llcrnrlon=llcrnrlon,llcrnrlat=llcrnrlat,urcrnrlon=urcrnrlon,urcrnrlat=urcrnrlat,
resolution='i',projection='lcc',lon_0=lon_0,lat_0=lat_0)
m.drawcoastlines()
m.fillcontinents()
# draw parallels
m.drawparallels(np.arange(10,70,10),labels=[1,0,0,0])
# draw meridians
m.drawmeridians(np.arange(-80, 25, 10),labels=[0,0,0,1])
# Plot station locations.
x, y = m(lon, lat) # array ref points
m.plot(x,y,'ro', ms=5)
####################################################################
## Set up the grids etc.
glons = np.linspace(llcrnrlon, urcrnrlon, 100)
glats = np.linspace(llcrnrlat, urcrnrlat, 100)
# Convert to map coords.
xlons, ylats = m(glons, glats)
# create grid for pcolormesh.
grid_lon, grid_lat = np.meshgrid(xlons, ylats)
# create weights for pcolormesh.
weights = np.zeros(np.shape(grid_lon))
# create grid of lat-lon coords for baz calculation.
gln, glt = np.meshgrid(glons, glats)
####################################################################
## calculate baz from grid_lon, grid_lat to lon, lat. If less
## than error weight grid point.
# method for BAZ calculation via pyproj.
def get_baz(lon1, lat1, lon2, lat2):
g = pyproj.Geod(ellps='WGS84')
az, baz, dist = g.inv(lon1, lat1, lon2, lat2)
return baz
# BAZ calcultion for each point in grid.
ll=0
for mBAZ in baz:
for i in xrange(len(gln)):
for k in xrange(len(gln[i])):
nbaz = get_baz(lon, lat, gln[i][k], glt[i][k])
nbaz += 180
if abs(nbaz - mBAZ) < 2:
weights[i][k] = 1
ll+=1
# plot grid.
m.pcolormesh(grid_lon, grid_lat, weights, cmap=plt.cm.YlOrBr)
plt.colorbar()
plt.show()
Original question below, out of date now.
I am trying to map some data. I have a dataset that gives a range of values (frequencies) for each direction. I want to plot them on a grid so each grid point along a particular azimuth is weighted by the power for a particular frequency.
I have created a map with basemap and plotted a grid over it as follows,
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
from shoot import *
llcrnrlon = -20.0
llcrnrlat = 45.0
urcrnrlon = 10.0
urcrnrlat = 65.0
lon_0 = (urcrnrlon + llcrnrlon) / 2.
lat_0 = (urcrnrlat + llcrnrlat) / 2.
m = Basemap(llcrnrlon=llcrnrlon,llcrnrlat=llcrnrlat,urcrnrlon=urcrnrlon,urcrnrlat=urcrnrlat,
resolution='i',projection='lcc',lon_0=lon_0,lat_0=lat_0)
## Set up the grid.
glons = np.linspace(-20,10,50)
glats = np.linspace(45, 65, 50)
xlons, ylats = m(glons, glats)
grid_lon, grid_lat = np.meshgrid(xlons, ylats)
pwr = np.zeros((50,50))
m.drawcoastlines()
m.fillcontinents()
# draw parallels
m.drawparallels(np.arange(10,70,10),labels=[1,0,0,0])
# draw meridians
m.drawmeridians(np.arange(-80, 25, 10),labels=[0,0,0,1])
lats = [54.8639587, 51.5641564]
lons = [-8.1778180, -9.2754284]
x, y = m(lons, lats) # array ref points
# Plot station locations.
m.plot(x,y,'ro', ms=5)
m.pcolormesh(grid_lon, grid_lat, pwr)
then I shoot out the great circle I want using some functions I found at this nice site
glon1 = lons[0]
glat1 = lats[0]
azimuth = 280.
maxdist = 200.
great(m, glon1, glat1, azimuth, color='orange', lw=2.0)
plt.show()
However, plotting the line is not enough, I want to be able to find the grid points that the great circle crosses so I can assign a value to them. Does anyone know how to go about this??

Can you specify which crossing point do you mean? Running your code returns only one line ...