Given a list of points x_coords,y_coords and a their corresponding values, I want to rasterize it onto a 2D canvas with these specific values. Is there a python library to do this? As an attempt I was using PIL, however, it only allows me to fill with a single value:
draw = ImageDraw.Draw(canvas)
draw.point([*zip(x_coords, y_coords)], fill=1)
# Ideally I want to fill it with specific values:
# draw.point([*zip(x_coords, y_coords)], fill=values)
sounds like exactly what you want, scipy.interpolate.griddata. example code included. basically you need:
# rearrange your coordinates into one array of rows
points = np.stack([x_coords, y_coords]).T
# the j-"notation" gives you 100 by 200 points in the 0..1 interval, or...
grid_x, grid_y = np.mgrid[0:1:100j, 0:1:200j]
# this gives you the specified step
grid_x, grid_y = np.mgrid[0:1:0.01, 0:1:0.01]
# resample point data according to grid
grid_z2 = griddata(points, values, (grid_x, grid_y), method='cubic')
Related
I have a square 2D array data that I would like to add to a larger 2D array frame at some given set of non-integer coordinates coords. The idea is that data will be interpolated onto frame with it's center at the new coordinates.
Some toy data:
# A gaussian to add to the frame
x, y = np.meshgrid(np.linspace(-1,1,10), np.linspace(-1,1,10))
data = 50*np.exp(-np.sqrt(x**2+y**2)**2)
# The frame to add the gaussian to
frame = np.random.normal(size=(100,50))
# The desired (x,y) location of the gaussian center on the new frame
coords = 23.4, 22.6
Here's the idea. I want to add this:
to this:
to get this:
If the coordinates were integers (indexes), of course I could simply add them like this:
frame[23:33,22:32] += data
But I want to be able to specify non-integer coordinates so that data is regridded and added to frame.
I've looked into PIL.Image methods but my use case is just for 2D data, not images. Is there a way to do this with just scipy? Can this be done with interp2d or a similar function? Any guidance would be greatly appreciated!
Scipy's shift function from scipy.ndimage.interpolation is what you are looking for, as long as the grid spacings between data and frame overlap. If not, look to the other answer. The shift function can take floating point numbers as input and will do a spline interpolation. First, I put the data into an array as large as frame, then shift it, and then add it. Make sure to reverse the coordinate list, as x is the rightmost dimension in numpy arrays. One of the nice features of shift is that it sets to zero those values that go out of bounds.
import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage.interpolation import shift
# A gaussian to add to the frame.
x, y = np.meshgrid(np.linspace(-1,1,10), np.linspace(-1,1,10))
data = 50*np.exp(-np.sqrt(x**2+y**2)**2)
# The frame to add the gaussian to
frame = np.random.normal(size=(100,50))
x_frame = np.arange(50)
y_frame = np.arange(100)
# The desired (x,y) location of the gaussian center on the new frame.
coords = np.array([23.4, 22.6])
# First, create a frame as large as the frame.
data_large = np.zeros(frame.shape)
data_large[:data.shape[0], :data.shape[1]] = data[:,:]
# Subtract half the distance as the bottom left is at 0,0 instead of the center.
# The shift of 4.5 is because data is 10 points wide.
# Reverse the coords array as x is the last coordinate.
coords_shift = -4.5
data_large = shift(data_large, coords[::-1] + coords_shift)
frame += data_large
# Plot the result and add lines to indicate to coordinates
plt.figure()
plt.pcolormesh(x_frame, y_frame, frame, cmap=plt.cm.jet)
plt.axhline(coords[1], color='w')
plt.axvline(coords[0], color='w')
plt.colorbar()
plt.gca().invert_yaxis()
plt.show()
The script gives you the following figure, which has the desired coordinates indicated with white dotted lines.
One possible solution is to use scipy.interpolate.RectBivariateSpline. In the code below, x_0 and y_0 are the coordinates of a feature from data (i.e., the position of the center of the Gaussian in your example) that need to be mapped to the coordinates given by coords. There are a couple of advantages to this approach:
If you need to "place" the same object into multiple locations in the output frame, the spline needs to be computed only once (but evaluated multiple times).
In case you actually need to compute integrated flux of the model over a pixel, you can use the integral method of scipy.interpolate.RectBivariateSpline.
Resample using spline interpolation:
from scipy.interpolate import RectBivariateSpline
x = np.arange(data.shape[1], dtype=np.float)
y = np.arange(data.shape[0], dtype=np.float)
kx = 3; ky = 3; # spline degree
spline = RectBivariateSpline(
x, y, data.T, kx=kx, ky=ky, s=0
)
# Define coordinates of a feature in the data array.
# This can be the center of the Gaussian:
x_0 = (data.shape[1] - 1.0) / 2.0
y_0 = (data.shape[0] - 1.0) / 2.0
# create output grid, shifted as necessary:
yg, xg = np.indices(frame.shape, dtype=np.float64)
xg += x_0 - coords[0] # see below how to account for pixel scale change
yg += y_0 - coords[1] # see below how to account for pixel scale change
# resample and fill extrapolated points with 0:
resampled_data = spline.ev(xg, yg)
extrapol = (((xg < -0.5) | (xg >= data.shape[1] - 0.5)) |
((yg < -0.5) | (yg >= data.shape[0] - 0.5)))
resampled_data[extrapol] = 0
Now plot the frame and resampled data:
plt.figure(figsize=(14, 14));
plt.imshow(frame+resampled_data, cmap=plt.cm.jet,
origin='upper', interpolation='none', aspect='equal')
plt.show()
If you also want to allow for scale changes, then replace code for computing xg and yg above with:
coords = 20, 80 # change coords to easily identifiable (in plot) values
zoom_x = 2 # example scale change along X axis
zoom_y = 3 # example scale change along Y axis
yg, xg = np.indices(frame.shape, dtype=np.float64)
xg = (xg - coords[0]) / zoom_x + x_0
yg = (yg - coords[1]) / zoom_y + y_0
Most likely this is what you actually want based on your example. Specifically, the coordinates of pixels in data are "spaced" by 0.222(2) distance units. Therefore it actually seems that for your particular example (whether accidental or intentional), you have a zoom factor of 0.222(2). In that case your data image would shrink to almost 2 pixels in the output frame.
Comparison to #Chiel answer
In the image below, I compare the results from my method (left), #Chiel's method (center) and difference (right panel):
Fundamentally, the two methods are quite similar and possibly even use the same algorithm (I did not look at the code for shift but based on the description - it also uses splines). From comparison image it is visible that the biggest differences are at the edges and, for unknown to me reasons, shift seems to truncate the shifted image slightly too soon.
I think the biggest difference is that my method allows for pixel scale changes and it also allows re-use of the same interpolator to place the original image at different locations in the output frame. #Chiel's method is somewhat simpler but (what I did not like about it is that) it requires creation of a larger array (data_large) into which the original image is placed in the corner.
While the other answers have gone into detail, but here's my lazy solution:
xc,yc = 23.4, 22.6
x, y = np.meshgrid(np.linspace(-1,1,10)-xc%1, np.linspace(-1,1,10)-yc%1)
data = 50*np.exp(-np.sqrt(x**2+y**2)**2)
frame = np.random.normal(size=(100,50))
frame[23:33,22:32] += data
And it's the way you liked it. As you mentioned, the coordinates of both are the same, so the origin of data is somewhere between the indices. Now just simply shift it by the amount you want it to be off a grid point (remainder to one) in the second line and you're good to go (you might need to flip the sign, but I think this is correct).
I have a coarse skymap made up of 128 points, of which I would like to make a smooth healpix map (see attached Figure, LHS). Figures referenced in the text:
I load my data, then make new longitude and latitude arrays of the appropriate pixel length for the final map (with e.g. nside=32).
My input data are:
lats = pi/2 + ths # theta from 0, pi, size 8
lons = phs # phi from 0, 2pi, size 16
data = sky_data[0] # shape (8,16)
New lon/lat array size based on number of pixels from nside:
nside = 32
pixIdx = hp.nside2npix(nside) # number of pixels I can get from this nside
pixIdx = np.arange(pixIdx) # pixel index numbers
I then find the new data values for those pixels by interpolation, and then convert back from angles to pixels.
# new lon/lat
new_lats = hp.pix2ang(nside, pixIdx)[0] # thetas I need to populate with interpolated theta values
new_lons = hp.pix2ang(nside, pixIdx)[1] # phis, same
# interpolation
lut = RectSphereBivariateSpline(lats, lons, data, pole_values=4e-14)
data_interp = lut.ev(new_lats.ravel(), new_lons.ravel()) #interpolate the data
pix = hp.ang2pix(nside, new_lats, new_lons) # convert latitudes and longitudes back to pixels
Then, I construct a healpy map with the interpolated values:
healpix_map = np.zeros(hp.nside2npix(nside), dtype=np.double) # create empty map
healpix_map[pix] = data_interp # assign pixels to new interpolated values
testmap = hp.mollview(healpix_map)
The result of the map is the upper RHS of the attached Figure.
(Forgive the use of jet -- viridis doesn't have a "white" zero, so using that colormap adds a blue background.)
The map doesn't look right: you can see from the coarse map in the Figure that there should be a "hotspot" on the lower RHS, but here it appears in the upper left.
As a sanity check, I used matplotlib to make a scatter plot of the interpolated points in a mollview projection, Figure 2, where I removed the edges of the markers to make it look like a map ;)
ax = plt.subplot(111, projection='astro mollweide')
ax.grid()
colors = data_interp
sky=plt.scatter(new_lons, new_lats-pi/2, c = colors, edgecolors='none', cmap ='jet')
plt.colorbar(sky, orientation = 'horizontal')
You can see that this map, lower RHS of attached Figure, produces exactly what I expect! So the coordinates are ok, and I am completely confused.
Has anyone encountered this before? What can I do? I'd like to use the healpy functions on this and future maps, so just using matplotlib isn't an option.
Thanks!
I figured it out -- I had to add pi/2 to my thetas for the interpolation to work, so in the end need to apply the following transformation for the image to render correctly:
newnew_lats = pi - new_lats
newnew_lons = pi + new_lons
There still seems to be a bit of an issue with the interpolation, although the seem is not so visible now. I may try a different one to compare.
I'm no expert in healpix (actually I've never used it before - I'm a particle physicist), but as far as I can tell it's just a matter of conventions: in a Mollweide projection, healpy places the north pole (positive latitude) at the bottom of the map, for some reason. I'm not sure why it would do that, or whether this is intentional behavior, but it seems pretty clear that's what is happening. If I mask out everything below the equator, i.e. keep only the positive-latitude points
mask = new_lats - pi/2 > 0
pix = hp.ang2pix(nside, new_lats[mask], new_lons[mask])
healpix_map = np.zeros(hp.nside2npix(nside), dtype=np.double)
healpix_map[pix] = data_interp[mask]
testmap = hp.mollview(healpix_map)
it comes up with a plot with no data above the center line:
At least it's easy enough to fix. mollview admits a rot parameter that will effectively rotate the sphere around the viewing axis before projecting it, and a flip parameter which can be set to 'astro' (default) or 'geo' to set whether east is shown at the left or right. A little experimentation shows that you get the coordinate system you want with
hp.mollview(healpix_map, rot=(180, 0, 180), flip='geo')
In the tuple, the first two elements are longitude and latitude of the point to set in the center of the plot, and the third element is the rotation. All are in degrees. With no mask it gives this:
which I believe is just what you're looking for.
I am having trouble contouring some data in matplotlib. I am trying to plot a vertical cross-section of temperature that I sliced from a 3d field of temperature.
My temperature array (T) is of size 50*300 where 300 is the number of horizontal levels which are evenly spaced. However, 50 is the number of vertical levels that are: a) non-uniformly spaced; and b) have a different starting level for each vertical column. As in there are always 50 vertical levels, but sometimes they span from 100 - 15000 m, and sometimes from 300 - 20000 m (due to terrain differences).
I also have a 2d array of height (Z; same shape as T), a 1d array of horizontal location (LAT), and a 1d array of terrain height (TER).
I am trying to get a similar plot to one like here in which you can see the terrain blacked out and the data is contoured around it.
My first attempt to plot this was to create a meshgrid of horizontal distance and height, and then contourf temperature with those arguments as well. However numpy.meshgrid requires 1d inputs, and my height is a 2d variable. Doing something like this only begins contouring upwards from the first column:
ax1 = plt.gca()
z1, x1 = np.meshgrid(LAT, Z[:,0])
plt.contourf(z1, x1, T)
ax1.fill_between(z1[0,:], 0, TER, facecolor='black')
Which produces this. If I use Z[:,-1] in the meshgrid, it contours underground for columns to the left, which obviously I don't want. What I really would like is to use some 2d array for Z in the meshgrid but I'm not sure how to go about that.
I've also looked into the griddata function but that requires 1D inputs as well. Anyone have any ideas on how to approach this? Any help is appreciated!
For what I understand your data is structured. Then you can directly use the contourf or contour option in matplotlib. The code you present have the right idea but you should use
x1, z1 = np.meshgrid(LAT, Z[:,0])
plt.contourf(x1, Z, T)
for the contours. I have an example below
import numpy as np
import matplotlib.pyplot as plt
L, H = np.pi*np.mgrid[-1:1:100j, -1:1:100j]
T = np.cos(L)*np.cos(2*H)
H = np.cos(L) + H
plt.contourf(L, H, T, cmap="hot")
plt.show()
Look that the grid is generated with the original bounding box, but the plot is made with the height that has been transformed and not the initial one. Also, you can use tricontour for nonstructured data (or in general), but then you will need to generate the triangulation (that in your case is straightforward).
Here is my question:
the 2-d numpy array data represent some property of each grid space
the shapefile as the administrative division of the study area(like a city).
For example:
http://i4.tietuku.com/84ea2afa5841517a.png
The whole area has 40x40 grids network, and I want to extract the data inside the purple area. In other words , I want to mask the data outside the administrative
boundary into np.nan.
My early attempt
I label the grid number and select the specific array data into np.nan.
http://i4.tietuku.com/523df4783bea00e2.png
value[0,:] = np.nan
value[1,:] = np.nan
.
.
.
.
Can Someone show me a easier method to achieve the target?
Add
Found an answer here which can plot the raster data into shapefile, but the data itself doesn't change.
Update -2016-01-16
I have already solved this problem inspired by some answers.
Someone which are interested on this target, check this two posts which I have asked:
1. Testing point with in/out of a vector shapefile
2. How to use set clipped path for Basemap polygon
The key step was to test the point within/out of the shapefile which I have already transform into shapely.polygon.
Step 1. Rasterize shapefile
Create a function that can determine whether a point at coordinates (x, y) is or is not in the area. See here for more details on how to rasterize your shapefile into an array of the same dimensions as your target mask
def point_is_in_mask(mask, point):
# this is just pseudocode
return mask.contains(point)
Step 2. Create your mask
mask = np.zeros((height, width))
value = np.zeros((height, width))
for y in range(height):
for x in range(width):
if not point_is_in_mask(mask, (x, y)):
value[y][x] = np.nan
Best is to use matplotlib:
def outline_to_mask(line, x, y):
"""Create mask from outline contour
Parameters
----------
line: array-like (N, 2)
x, y: 1-D grid coordinates (input for meshgrid)
Returns
-------
mask : 2-D boolean array (True inside)
"""
import matplotlib.path as mplp
mpath = mplp.Path(line)
X, Y = np.meshgrid(x, y)
points = np.array((X.flatten(), Y.flatten())).T
mask = mpath.contains_points(points).reshape(X.shape)
return mask
alternatively, you may use shapely contains method as suggested in the above answer. You may speed-up calculations by recursively sub-dividing the space, as indicated in this gist (but matplotlib solution was 1.5 times faster in my tests):
https://gist.github.com/perrette/a78f99b76aed54b6babf3597e0b331f8
I have two 2D numpy arrays (of the same dimensions) that I am plotting using matplotlib. The first array I've plotted as a color map in gray-scale. The second one represents an aperture, but it is an irregular shape (some of the pixels get outlined, and it is a set of horizontal and vertical lines that form the outline). I am not sure how to ask it to plot this second array. The array is composed of three numbers (0, 1, and 3), and I only need the pixels of one value (3) to be outlined, but I need the outline to encompass the region of these pixels, not the pixels individually. I need the interior of all the pixels to remain transparent so that I can see the gray-scale color map through it.
Does anyone know how to accomplish this?
That is an interesting question, if I understood it correctly. In order to make sure what you mean, you would like to draw a line with some color around all contiguous areas where the pixel value is 3.
I do not think there is a ready-made function for that, but let's not let that stop us. We will need to create our own function.
We can start by creating a boolean map of the area which needs to be outlined:
import numpy as np
import matplotlib.pyplot as plt
# our image with the numbers 1-3 is in array maskimg
# create a boolean image map which has trues only where maskimg[x,y] == 3
mapimg = (maskimg == 3)
# a vertical line segment is needed, when the pixels next to each other horizontally
# belong to diffferent groups (one is part of the mask, the other isn't)
# after this ver_seg has two arrays, one for row coordinates, the other for column coordinates
ver_seg = np.where(mapimg[:,1:] != mapimg[:,:-1])
# the same is repeated for horizontal segments
hor_seg = np.where(mapimg[1:,:] != mapimg[:-1,:])
# if we have a horizontal segment at 7,2, it means that it must be drawn between pixels
# (2,7) and (2,8), i.e. from (2,8)..(3,8)
# in order to draw a discountinuous line, we add Nones in between segments
l = []
for p in zip(*hor_seg):
l.append((p[1], p[0]+1))
l.append((p[1]+1, p[0]+1))
l.append((np.nan,np.nan))
# and the same for vertical segments
for p in zip(*ver_seg):
l.append((p[1]+1, p[0]))
l.append((p[1]+1, p[0]+1))
l.append((np.nan, np.nan))
# now we transform the list into a numpy array of Nx2 shape
segments = np.array(l)
# now we need to know something about the image which is shown
# at this point let's assume it has extents (x0, y0)..(x1,y1) on the axis
# drawn with origin='lower'
# with this information we can rescale our points
segments[:,0] = x0 + (x1-x0) * segments[:,0] / mapimg.shape[1]
segments[:,1] = y0 + (y1-y0) * segments[:,1] / mapimg.shape[0]
# and now there isn't anything else to do than plot it
plt.plot(segments[:,0], segments[:,1], color=(1,0,0,.5), linewidth=3)
Let us test this by generating some data and showing it:
image = np.cumsum(np.random.random((20,20))-.5, axis=1)
maskimg = np.zeros(image.shape, dtype='int')
maskimg[image > 0] = 3
x0 = -1.5
x1 = 1.5
y0 = 2.3
y1 = 3.8
plt.figure()
plt.imshow(maskimg, origin='lower', extent=[x0,x1,y0,y1], cmap=plt.cm.gray, interpolation='nearest')
plt.axis('tight')
After that we run the procedure on the top, and get:
The code can be made much denser, if needed, but now comments take a lot of space. With large images it might be wise to optimize the image segment creation by finding continuous paths. That will reduce the number of points to plot by a factor of up to three. However, doing that requires a bit different code, which is not as clear as this one. (If there will appear comments asking for that and an appropriate number of upvotes, I'll add it :)