I'm trying to overlay a Lambert Conformal Conical satellite image onto a Holoviews interactive map. I can map the satellite image just fine, but I can't figure out how to translate this map onto a Holoviews map properly. Below is reproducible code where I grab data using the Unidata Siphon library.
Import Packages
from datetime import datetime
import matplotlib.pyplot as plt
from netCDF4 import Dataset
from siphon.catalog import TDSCatalog
import holoviews as hv
import geoviews as gv
import geoviews.feature as gf
from cartopy import crs
from cartopy import feature as cf
hv.extension('bokeh')
Grab data and create figure
date=datetime.utcnow()
idx=-2
regionstr = 'CONUS'
channelnum = 13
datestr = str(date.year) + "%02d"%date.month + "%02d"%date.day
channelstr = 'Channel' + "%02d"%channelnum
cat = TDSCatalog('http://thredds-test.unidata.ucar.edu/thredds/catalog/satellite/goes16/GOES16/' + regionstr +
'/' + channelstr + '/' + datestr + '/catalog.xml')
ds = cat.datasets[idx].remote_access(service='OPENDAP')
x = ds.variables['x'][:]
y = ds.variables['y'][:]
z = ds.variables['Sectorized_CMI'][:]
proj_var = ds.variables[ds.variables['Sectorized_CMI'].grid_mapping]
# Create a Globe specifying a spherical earth with the correct radius
globe = ccrs.Globe(ellipse='sphere', semimajor_axis=proj_var.semi_major,
semiminor_axis=proj_var.semi_minor)
proj = ccrs.LambertConformal(central_longitude=proj_var.longitude_of_central_meridian,
central_latitude=proj_var.latitude_of_projection_origin,
standard_parallels=[proj_var.standard_parallel],
globe=globe)
fig = plt.figure(figsize=(14, 10))
ax = fig.add_subplot(1, 1, 1, projection=proj)
ax.coastlines(resolution='50m', color='lightblue')
ax.add_feature(cf.STATES, linestyle=':', edgecolor='lightblue')
ax.add_feature(cf.BORDERS, linewidth=1, edgecolor='lightblue')
for im in ax.images:
im.remove()
im = ax.imshow(z, extent=(x.min(), x.max(), y.min(), y.max()), origin='upper', cmap='jet')
plt.colorbar(im)
Now plot an interactive image using Holoviews (which uses Bokeh backend)
goes = hv.Dataset((x, y, z),['Lat', 'Lon'], 'ABI13')
%opts Image (cmap='jet') [width=1000 height=800 xaxis='bottom' yaxis='left' colorbar=True toolbar='above' projection=proj]
goes.to.image()* gf.coastline().options(projection=crs.LambertConformal(central_longitude=proj_var.longitude_of_central_meridian,central_latitude=proj_var.latitude_of_projection_origin,standard_parallels=[proj_var.standard_parallel],globe=globe))
I must not be translating it properly, though I've found documentation on Holoviews with regards to Lambert Conformal Conical projection to be sparse. I'm open to using any other interactive map package. My main desire is to be able to plot relatively quickly, get state/country lines on the image properly, and be able to zoom in. I've tried folium but also fell into projection issues there.
So I think the main thing to understand, which is explained here: is how projections are declared. The elements (e.g. Image, Points etc.) in GeoViews have a parameter called crs which declares the coordinate system the data is in, while the projection plot option declares what to project the data to for display.
In your case I think you want to display the image in the same coordinate system it is already in (Lambert Conformal), so technically you don't have to declare the coordinate system (crs) on the element at all and can just use a hv.Image (which is entirely unaware of projections).
As far as I can tell your code should already work as expected if you are using GeoViews 1.5, but here is what I would do:
# Apply mask
masked = np.ma.filled(z, np.NaN)
# Declare image from data
goes = hv.Image((x, y, masked),['Lat', 'Lon'], 'ABI13')
# Declare some options
options = dict(width=1000, height=800, yaxis='left', colorbar=True,
toolbar='above', cmap='jet', projection=proj)
# Display plot
gf.ocean * gf.land * goes.options(**options) * gf.coastline.options(show_bounds=False)
Note how we declare the projection on the Image but not crs. If you do want to display the data in a different projection it is defined in, you do have to declare the crs and use a gv.Image. In this case I'd recommend using project_image with the fast option enabled (which might introduce some artifacts but is much faster):
# Apply mask
masked = np.ma.filled(z, np.NaN)
# Declare the gv.Image with the crs
goes = gv.Image((x, y, masked), ['Lat', 'Lon'], 'ABI13', crs=proj)
# Now regrid the data and apply the reprojection
projected_goes = gv.operation.project_image(goes, fast=False, projection=ccrs.GOOGLE_MERCATOR)
# Declare some options
options = dict(width=1000, height=800, yaxis='left', colorbar=True,
toolbar='above', cmap='jet')
# Display plot
projected_goes.options(**options) * gv.tile_sources.ESRI.options(show_bounds=False)
Another final tip, when you plot with bokeh all the data you are plotting will be sent to the browser, so when dealing with images any larger than you are already using I'd recommend using the holoviews' regrid operation which uses datashader to dynamically resize the image as you are zooming. To use that simply apply the operation to the image like so:
from holoviews.operation.datashader import regrid
regridded = regrid(goes)
Related
I wanted to plot a Contact Map using the sparse.matrix feature along with colormap function from the matplotlib module of Python. I could generate the plot but the 2-D colormap condenses only to the left corner independent of x-range. How to make the 2-D colormap in full frame ? I used the following commands to generate the plot:-
trajectory_contacts1 = ContactFrequency(t1,switch1,switch2)
sparse_cm1 = trajectory_contacts1.residue_contacts.sparse_matrix
cm1 = sparse_cm1.todense()
with open('CM-pE42-Ab1-42.dat','wb') as f1:
np.savetxt(f1, np.array(cm1), fmt='%.2f1')
# Import data from file
CM1 = np.loadtxt(fname = "CM-pE42-Ab1-42_Cluster-1.dat")
# Contact Map Plot for Ab1-42 & pE-Ab3-42
from matplotlib import cm as cmp
cmap = cmp.get_cmap('Spectral_r',50)
plt.matshow(CM1,cmap=cmap,interpolation='None')
ax = plt.gca()
ax.set_xlabel('Residue Number',fontsize=12)
ax.set_ylabel('Residue Number',fontsize=12)
ax.set_aspect('auto')
tick_params(bottom=True, right=True,labelbo[![enter image description here][1]][1]ttom=True, labelright=True)
plt.savefig("CM-pE42-Ab1-42_Cluster-3.png",dpi=300,format="png",bbox_inches='tight')
Here is the dropbox link for the .dat file:- https://www.dropbox.com/s/zqh4w541s6sohmj/CM-pE42-Ab1-42_Cluster-1.dat?dl=0
Kindly help me with the suggestions to correct the script to get the 2-D plot in full frame.
I have a shapefile of the United states, and I have an m x n array of Cartesian data that represents temperature at each pixel. I am able to load in the shapefile and plot it:
import shapefile as shp
import matplotlib.pyplot as plt
sf = shp.Reader("/path/to/USA.shp")
plt.figure()
for shape in sf.shapeRecords():
for i in range(len(shape.shape.parts)):
i_start = shape.shape.parts[i]
if i==len(shape.shape.parts)-1:
i_end = len(shape.shape.points)
else:
i_end = shape.shape.parts[i+1]
x = [i[0] for i in shape.shape.points[i_start:i_end]]
y = [i[1] for i in shape.shape.points[i_start:i_end]]
plt.plot(x,y, color = 'black')
plt.show()
And I am able to read in my data and plot it:
import pickle
from matplotlib import pyplot as mp
Tfile = '/path/to/file.pkl'
with open(Tfile) as f:
reshapeT = pickle.load(f)
mp.matshow(reshapeT)
The problem is reshapeT has dimensions of 536 x 592, and is a subdomain of the US. However, I do have information about the top-left corner of the reshapeT grid (lat / long) as well as the spacing between each pixel (0.01)
My question is: How do I overlay the reshapeT data ontop of the shapefile domain?
If I understand you correctly you would like to overlay a 536x592 numpy array over a specifc part of a plotted shapefile. I would suggest you use Matplotlib's imwshow() method, with the extent parameter, which allows you to place the image within the plot.
Your way of plotting the shapefile is fine, however, if you have the possibility to use geopandas, it will dramatically simplify things. Plotting the shapefile will reduce to the following lines:
import geopandas as gpd
sf = gpd.read_file("/path/to/USA.shp")
ax1 = sf.plot(edgecolor='black', facecolor='none')
As you have done previously, let's load the array data now:
import pickle
Tfile = '/path/to/file.pkl'
with open(Tfile) as f:
reshapeT = pickle.load(f)
Now in order to be able to plot numpy array in the correct position, we first need to calculate its extent (the area which it will cover expressed in coordinates). You mentioned that you have information about the top-left corner and the resolution (0.01) - that's all we need. In the following I'm assuming that the lat/lon information about the top-left corner is saved in the the top_left_lat and top_left_lon variables. The extent needs to be passed in a tuple with a value for each of the edges (in the order left, right, bottom, top).
Hence, our extent can be calculated as follows:
extent_mat = (top_left_lon, top_left_lon + reshapeT.shape[1] * 0.01, top_left_lat - reshapeT.shape[0] * 0.01, top_left_lat)
Finally, we plot the matrix onto the same axes object, ax1, on which we already plotted the shape file to the calculated extent:
# Let's turn off autoscale first. This prevents
# the view of the plot to be limited to the image
# dimensions (instead of the entire shapefile). If you prefer
# that behaviour, just remove the following line
ax1.autoscale(False)
# Finally, let's plot!
ax1.imshow(reshapeT, extent=extent_mat)
I am trying to create a color wheel in Python, preferably using Matplotlib. The following works OK:
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
xval = np.arange(0, 2*pi, 0.01)
yval = np.ones_like(xval)
colormap = plt.get_cmap('hsv')
norm = mpl.colors.Normalize(0.0, 2*np.pi)
ax = plt.subplot(1, 1, 1, polar=True)
ax.scatter(xval, yval, c=xval, s=300, cmap=colormap, norm=norm, linewidths=0)
ax.set_yticks([])
However, this attempt has two serious drawbacks.
First, when saving the resulting figure as a vector (figure_1.svg), the color wheel consists (as expected) of 621 different shapes, corresponding to the different (x,y) values being plotted. Although the result looks like a circle, it isn't really. I would greatly prefer to use an actual circle, defined by a few path points and Bezier curves between them, as in e.g. matplotlib.patches.Circle. This seems to me the 'proper' way of doing it, and the result would look nicer (no banding, better gradient, better anti-aliasing).
Second (relatedly), the final plotted markers (the last few before 2*pi) overlap the first few. It's very hard to see in the pixel rendering, but if you zoom in on the vector-based rendering you can clearly see the last disc overlap the first few.
I tried using different markers (. or |), but none of them go around the second issue.
Bottom line: can I draw a circle in Python/Matplotlib which is defined in the proper vector/Bezier curve way, and which has an edge color defined according to a colormap (or, failing that, an arbitrary color gradient)?
One way I have found is to produce a colormap and then project it onto a polar axis. Here is a working example - it includes a nasty hack, though (clearly commented). I'm sure there's a way to either adjust limits or (harder) write your own Transform to get around it, but I haven't quite managed that yet. I thought the bounds on the call to Normalize would do that, but apparently not.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm
import matplotlib as mpl
fig = plt.figure()
display_axes = fig.add_axes([0.1,0.1,0.8,0.8], projection='polar')
display_axes._direction = 2*np.pi ## This is a nasty hack - using the hidden field to
## multiply the values such that 1 become 2*pi
## this field is supposed to take values 1 or -1 only!!
norm = mpl.colors.Normalize(0.0, 2*np.pi)
# Plot the colorbar onto the polar axis
# note - use orientation horizontal so that the gradient goes around
# the wheel rather than centre out
quant_steps = 2056
cb = mpl.colorbar.ColorbarBase(display_axes, cmap=cm.get_cmap('hsv',quant_steps),
norm=norm,
orientation='horizontal')
# aesthetics - get rid of border and axis labels
cb.outline.set_visible(False)
display_axes.set_axis_off()
plt.show() # Replace with plt.savefig if you want to save a file
This produces
If you want a ring rather than a wheel, use this before plt.show() or plt.savefig
display_axes.set_rlim([-1,1])
This gives
As per #EelkeSpaak in comments - if you save the graphic as an SVG as per the OP, here is a tip for working with the resulting graphic: The little elements of the resulting SVG image are touching and non-overlapping. This leads to faint grey lines in some renderers (Inkscape, Adobe Reader, probably not in print). A simple solution to this is to apply a small (e.g. 120%) scaling to each of the individual gradient elements, using e.g. Inkscape or Illustrator. Note you'll have to apply the transform to each element separately (the mentioned software provides functionality to do this automatically), rather than to the whole drawing, otherwise it has no effect.
I just needed to make a color wheel and decided to update rsnape's solution to be compatible with matplotlib 2.1. Rather than place a colorbar object on an axis, you can instead plot a polar colored mesh on a polar plot.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm
import matplotlib as mpl
# If displaying in a Jupyter notebook:
# %matplotlib inline
# Generate a figure with a polar projection
fg = plt.figure(figsize=(8,8))
ax = fg.add_axes([0.1,0.1,0.8,0.8], projection='polar')
# Define colormap normalization for 0 to 2*pi
norm = mpl.colors.Normalize(0, 2*np.pi)
# Plot a color mesh on the polar plot
# with the color set by the angle
n = 200 #the number of secants for the mesh
t = np.linspace(0,2*np.pi,n) #theta values
r = np.linspace(.6,1,2) #radius values change 0.6 to 0 for full circle
rg, tg = np.meshgrid(r,t) #create a r,theta meshgrid
c = tg #define color values as theta value
im = ax.pcolormesh(t, r, c.T,norm=norm) #plot the colormesh on axis with colormap
ax.set_yticklabels([]) #turn of radial tick labels (yticks)
ax.tick_params(pad=15,labelsize=24) #cosmetic changes to tick labels
ax.spines['polar'].set_visible(False) #turn off the axis spine.
It gives this:
I want to use imshow (for example) to display some data inside the boundaries of a country (for the purposes of example I chose the USA) The simple example below illustrates what I want:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import RegularPolygon
data = np.arange(100).reshape(10, 10)
fig = plt.figure()
ax = fig.add_subplot(111)
im = ax.imshow(data)
poly = RegularPolygon([ 0.5, 0.5], 6, 0.4, fc='none',
ec='k', transform=ax.transAxes)
im.set_clip_path(poly)
ax.add_patch(poly)
ax.axis('off')
plt.show()
The result is:
Now I want to do this but instead of a simple polygon, I want to use the complex shape of the USA. I have created some example data contained in the array of "Z" as can be seen in the code below. It is this data that I want to display, using a colourmap but only within the boundaries of mainland USA.
So far I have tried the following. I get a shape file from here contained in "nationp010g.shp.tar.gz" and I use the Basemap module in python to plot the USA. Note that this is the only method I have found which gives me the ability get a polygon of the area I need. If there are alternative methods I would also be interested in them. I then create a polygon called "mainpoly" which is almost the polygon I want coloured in blue:
Notice how only one body has been coloured, all other disjoint polygons remain white:
So the area coloured blue is almost what I want, note that there are unwanted borderlines near canada because the border actually goes through some lakes, but that is a minor problem. The real problem is, why doesn't my imshow data display inside the USA? Comparing my first and second example codes I can't see why I don't get a clipped imshow in my second example, the way I do in the first. Any help would be appreciated in understanding what I am missing.
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap as Basemap
from matplotlib.patches import Polygon
# Lambert Conformal map of lower 48 states.
m = Basemap(llcrnrlon=-119,llcrnrlat=22,urcrnrlon=-64,urcrnrlat=49,
projection='lcc',lat_1=33,lat_2=45,lon_0=-95)
shp_info = m.readshapefile('nationp010g/nationp010g', 'borders', drawbounds=True) # draw country boundaries.
for nshape,seg in enumerate(m.borders):
if nshape == 1873: #This nshape denotes the large continental body of the USA, which we want
mainseg = seg
mainpoly = Polygon(mainseg,facecolor='blue',edgecolor='k')
nx, ny = 10, 10
lons, lats = m.makegrid(nx, ny) # get lat/lons of ny by nx evenly space grid.
x, y = m(lons, lats) # compute map proj coordinates.
Z = np.zeros((nx,ny))
Z[:] = np.NAN
for i in np.arange(len(x)):
for j in np.arange(len(y)):
Z[i,j] = x[0,i]
ax = plt.gca()
im = ax.imshow(Z, cmap = plt.get_cmap('coolwarm') )
im.set_clip_path(mainpoly)
ax.add_patch(mainpoly)
plt.show()
Update
I realise that the line
ax.add_patch(mainpoly)
does not even add the polygon shape to a plot. Am I not using it correctly? As far as I know mainpoly was calculated correctly using the Polygon() method. I checked that the coordinate inputs are a sensible:
plt.plot(mainseg[:,0], mainseg[:,1] ,'.')
which gives
I have also considered about this problem for so long.
And I found NCL language has the function to mask the data outside some border.
Here is the example:
http://i5.tietuku.com/bdb1a6c007b82645.png
The contourf plot only show within China border. Click here for the code.
I know python has a package called PyNCL which support all NCL code in Python framework.
But I really want to plot this kind of figure using basemap. If you have figured it out, please post on the internet. I'll learn at the first time.
Thanks!
Add 2016-01-16
In a way, I have figured it out.
This is my idea and code, and it's inspired from this question I have asked today.
My method:
1. Make the shapefile of the interesting area(like U.S) into shapely.polygon.
2. Test each value point within/out of the polygon.
3. If the value point is out of the study area, mask it as np.nan
Intro
* the polygon xxx was a city in China in ESRI shapefile format.
* fiona, shapely package were used here.
# generate the shapely.polygon
shape = fiona.open("xxx.shp")
pol = shape.next()
geom = shape(pol['geometry'])
poly_data = pol["geometry"]["coordinates"][0]
poly = Polygon(poly_data)
It shows like:
http://i4.tietuku.com/2012307faec02634.png
### test the value point
### generate the grid network which represented by the grid midpoints.
lon_med = np.linspace((xi[0:2].mean()),(xi[-2:].mean()),len(x_grid))
lat_med = np.linspace((yi[0:2].mean()),(yi[-2:].mean()),len(y_grid))
value_test_mean = dsu.mean(axis = 0)
value_mask = np.zeros(len(lon_med)*len(lat_med)).reshape(len(lat_med),len(lon_med))
for i in range(0,len(lat_med),1):
for j in range(0,len(lon_med),1):
points = np.array([lon_med[j],lat_med[i]])
mask = np.array([poly.contains(Point(points[0], points[1]))])
if mask == False:
value_mask[i,j] = np.nan
if mask == True:
value_mask[i,j] = value_test_mean[i,j]
# Mask the np.nan value
Z_mask = np.ma.masked_where(np.isnan(so2_mask),so2_mask)
# plot!
fig=plt.figure(figsize=(6,4))
ax=plt.subplot()
map = Basemap(llcrnrlon=x_map1,llcrnrlat=y_map1,urcrnrlon=x_map2,urcrnrlat=y_map2)
map.drawparallels(np.arange(y_map1+0.1035,y_map2,0.2),labels= [1,0,0,1],size=14,linewidth=0,color= '#FFFFFF')
lon_grid = np.linspace(x_map1,x_map2,len(x_grid))
lat_grid = np.linspace(y_map1,y_map2,len(y_grid))
xx,yy = np.meshgrid(lon_grid,lat_grid)
pcol =plt.pcolor(xx,yy,Z_mask,cmap = plt.cm.Spectral_r ,alpha =0.75,zorder =2)
result
http://i4.tietuku.com/c6620c5b6730a5f0.png
http://i4.tietuku.com/a22ad484fee627b9.png
original result
http://i4.tietuku.com/011584fbc36222c9.png
my goal is to drap a texture (i.e. an image) over a surface and visualize it in 3D with python. The application is viewing an orthophotography over a DEM, I thence use gdal for importing my data (both image and DEM). I tried to use plot_surface from matplotlib but it seems that I can't add texture to the surface.
here is the current code:
from osgeo import gdal
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
import numpy as np
ds = gdal.Open('MyDEM.cub')
dem = ds.ReadAsArray()
do = gdal.Open('MyOrtho.cub')
or = do.ReadAsArray()
xres = gt[1]
yres = gt[5]
X = np.arange(gt[0], gt[0] + dem.shape[1]*xres, xres)
Y = np.linspace(gt[3], gt[3] + dem.shape[0]*yres, ds.RasterYSize)
X, Y = np.meshgrid(X, Y)
fig, ax = plt.subplots(figsize=(16,8), subplot_kw={'projection': '3d'})
surf = ax.plot_surface(X,Y,dem,rstride=1, cstride=1,linewidth=0, antialiased=True,cmap=plt.cm.RdYlBu_r)
fig.colorbar(surf, shrink=0.4, aspect=20)
plt.show()
How can I use or array as a texture (e.g., or can have a different resolution/size than dem but I'll manage this later), I want first to have a stupid 3D surface with a texture. This is easy in Matlab, but how to do it with Python? Any idea ?
Yeah Mayavi will do this. You can open the DEM file in gdal and then pull the image into TVTK as a texture. Finally you can wrap it over the surface with the mlab.surf() commands. Here is a link to a good example of this.
Example
From the matplotlib docs:
Axes3D.plot_surface(X, Y, Z, *args, **kwargs)
Create a surface plot.
By default it will be colored in shades of a solid color, but it also supports color mapping by supplying the cmap argument.
It seems, therefore, that matplotlib does not support the use of an arbitrary image on a surface plot. You will need to identify another library that provides this feature (a quick search suggests that mayavi may do what you want).