If I have a nxnxn grid of values, say 32x32x32, and I want to rotate this cube grid of values by some rotation angle in either the x, y, or z axes, and interpolate missing values, what would be the best way to go about doing this without using any existing algorithms from packages (such as Scipy)?
I'm familiar with applying a 3D rotation matrix to a 3D grid of points when it's represented as a [n, 3] matrix, but I'm not sure how to go about applying a rotation when the representation is given in its 3D form as nxnxn.
I found a prior Stack Overflow post about this topic, but it uses three for loops for its approach, which doesn't really scale in terms of speed. Is there a more vectorized approach that can accomplish a similar task?
Thanks in advance!
One way I could think of would look like this:
reshape nxnxn matrix to an array containing n-dimensional points
apply rotation on this array
reshape array back to nxnxn
Here is some code:
import numpy as np
#just a way to create some nxnxn matrix
n = 4
a = np.arange(n)
b = np.array([a]*n)
mat = np.array([b]*n)
#creating an array containg n-dimensional points
flat_mat = mat.reshape((int(mat.size/n),n))
#just a random matrix we will use as a rotation
rot = np.eye(n) + 2
#apply the rotation on each n-dimensional point
result = np.array([rot.dot(x) for x in flat_mat])
#return to original shape
result=result.reshape((n,n,n))
print(result)
I want to regrid georeferenced data to a specific grid with a different resolution on the lat and on the lon dimensions. Before I would use basemap.interp, but this basemap is dead.
I am experimenting with the metpy package, and metpy.interpolate_to_points seems the right candidate. Only, from the documentation I can't work out the format of the parameters I should enter.
It reads:
points (array_like, shape (n, D)) – Coordinates of the data points.
values (array_like, shape (n,)) – Values of the data points. xi
(array_like, shape (M, D)) – Points to interpolate the data onto.
Regarding 'points' I have tried providing them as 1-D arrays, as 2-D meshgrids (obtained with np.meshgrid), and in either lon first or lat first fashion. Same for 'xi'. For example:
from metpy.interpolate import interpolate_to_points
out_lons, out_lats = np.meshgrid(out_lons_1Darray, out_lats_1Darray)
downscaled_array = interpolate_to_points( [in_lons, in_lats], input_array, [out_lons, out_lats] )
From whichever attempt I get ValueError: operands could not be broadcast together with shapes (192,) (288,)
Any suggestion where I am wrong is greatly appreciated.
This function wraps scipy.interpolate.griddata and you can see their documentation here.
Their example shows the following, which works as for the metpy.interpolate.interpolate_to_points function:
def func(x, y):
return x*(1-x)*np.cos(4*np.pi*x) * np.sin(4*np.pi*y**2)**2
grid_x, grid_y = np.mgrid[0:1:100j, 0:1:200j]
points = np.random.rand(1000, 2)
values = func(points[:,0], points[:,1])
grid_out = interpolate_to_points(points, values, (grid_x, grid_y))
I'm trying to find a fast way to fill a Numpy array with rotation symmetric values. Imagine an array of zeros containing a cone shaped area. I have a 1D array of values and want to rotate it 360° around the center of the array. There is no 2D function like z=f(x,y), so I can't calculate the 2D values explicitly. I have something that works, but the for-loop is too slow for big arrays. This should make a circle:
values = np.ones(100)
x = np.arange(values.size)-values.size/2+0.5
y = values.size/2-0.5-np.arange(values.size)
x,y = np.meshgrid(x,y)
grid = np.rint(np.sqrt(x**2+y**2))
arr = np.zeros_like(grid)
for i in np.arange(values.size/2):
arr[grid==i] = values[i+values.size/2]
My 1D array is of course not as simple. Can someone think of a way to get rid of the for-loop?
Update: I want to make a circular filter for convolutional blurring. Before I used np.outer(values,values) which gave me a rectangular filter. David's hint allows me to create a circular filter very fast. See below:
square filter with np.outer()
circular filter with David's answer
You can use fancy indexing to achieve this:
values = np.ones(100)
x = np.arange(values.size)-values.size/2+0.5
y = values.size/2-0.5-np.arange(values.size)
x,y = np.meshgrid(x,y)
grid = np.rint(np.sqrt(x**2+y**2)).astype(np.int)
arr = np.zeros_like(grid)
size_half = values.size // 2
inside = (grid < size_half)
arr[inside] = values[grid[inside] + size_half]
Here, inside select the indices that lie inside the circle, since only these items can be derived from values.
You can do something like that:
x=y=np.arange(-500,501)
r=np.random.randint(0,256,len(x)/np.sqrt(2)+1)
X,Y=np.meshgrid(x,y)
im=(X*X+Y*Y)**(1/2)
circles=r.take(np.int64(im))
plt.imshow(circles)
I have an matrix (ndarray) with real values that I want to scale in a geometrical sense - that is expand the matrix's size while keeping the values as similar as possible. It can be viewed as scaling an image.
But my matrix is NOT an image. I have real values ranging from 8,000 to 50,000. As far as I know these values cannot represent anything from an usual image point of view.
I have searched the web for answers but every answer suggested using PIL or similar image processing libraries, that use standard pixel values that wouldn't accept my matrix.
So is there a way to scale a matrix containing any real numbers in the geometrical (or image) sense?
Is there a python library for that or list comprehension of some kind or someting similar?
Thank you.
What you're describing is 2D interpolation. Scipy provides an implementation in scipy.interpolate.RectBivariateSpline
from scipy.interpolate import RectBivariateSpline
# sample data
data = np.random.rand(8, 4)
width, height = data.shape
xs = np.arange(width)
ys = np.arange(height)
# target size and interpolation locations
new_width, new_height = width*2, height*2
new_xs = np.linspace(0, width-1, new_width)
new_ys = np.linspace(0, height-1, new_height)
# create the spline object, and use it to interpolate
spline = RectBivariateSpline(xs, ys, data) #, kx=1, ky=1) for linear interpolation
spline(new_xs, new_ys)
So, I have three numpy arrays which store latitude, longitude, and some property value on a grid -- that is, I have LAT(y,x), LON(y,x), and, say temperature T(y,x), for some limits of x and y. The grid isn't necessarily regular -- in fact, it's tripolar.
I then want to interpolate these property (temperature) values onto a bunch of different lat/lon points (stored as lat1(t), lon1(t), for about 10,000 t...) which do not fall on the actual grid points. I've tried matplotlib.mlab.griddata, but that takes far too long (it's not really designed for what I'm doing, after all). I've also tried scipy.interpolate.interp2d, but I get a MemoryError (my grids are about 400x400).
Is there any sort of slick, preferably fast way of doing this? I can't help but think the answer is something obvious... Thanks!!
Try the combination of inverse-distance weighting and
scipy.spatial.KDTree
described in SO
inverse-distance-weighted-idw-interpolation-with-python.
Kd-trees
work nicely in 2d 3d ..., inverse-distance weighting is smooth and local,
and the k= number of nearest neighbours can be varied to tradeoff speed / accuracy.
There is a nice inverse distance example by Roger Veciana i Rovira along with some code using GDAL to write to geotiff if you're into that.
This is of coarse to a regular grid, but assuming you project the data first to a pixel grid with pyproj or something, all the while being careful what projection is used for your data.
A copy of his algorithm and example script:
from math import pow
from math import sqrt
import numpy as np
import matplotlib.pyplot as plt
def pointValue(x,y,power,smoothing,xv,yv,values):
nominator=0
denominator=0
for i in range(0,len(values)):
dist = sqrt((x-xv[i])*(x-xv[i])+(y-yv[i])*(y-yv[i])+smoothing*smoothing);
#If the point is really close to one of the data points, return the data point value to avoid singularities
if(dist<0.0000000001):
return values[i]
nominator=nominator+(values[i]/pow(dist,power))
denominator=denominator+(1/pow(dist,power))
#Return NODATA if the denominator is zero
if denominator > 0:
value = nominator/denominator
else:
value = -9999
return value
def invDist(xv,yv,values,xsize=100,ysize=100,power=2,smoothing=0):
valuesGrid = np.zeros((ysize,xsize))
for x in range(0,xsize):
for y in range(0,ysize):
valuesGrid[y][x] = pointValue(x,y,power,smoothing,xv,yv,values)
return valuesGrid
if __name__ == "__main__":
power=1
smoothing=20
#Creating some data, with each coodinate and the values stored in separated lists
xv = [10,60,40,70,10,50,20,70,30,60]
yv = [10,20,30,30,40,50,60,70,80,90]
values = [1,2,2,3,4,6,7,7,8,10]
#Creating the output grid (100x100, in the example)
ti = np.linspace(0, 100, 100)
XI, YI = np.meshgrid(ti, ti)
#Creating the interpolation function and populating the output matrix value
ZI = invDist(xv,yv,values,100,100,power,smoothing)
# Plotting the result
n = plt.normalize(0.0, 100.0)
plt.subplot(1, 1, 1)
plt.pcolor(XI, YI, ZI)
plt.scatter(xv, yv, 100, values)
plt.title('Inv dist interpolation - power: ' + str(power) + ' smoothing: ' + str(smoothing))
plt.xlim(0, 100)
plt.ylim(0, 100)
plt.colorbar()
plt.show()
There's a bunch of options here, which one is best will depend on your data...
However I don't know of an out-of-the-box solution for you
You say your input data is from tripolar data. There are three main cases for how this data could be structured.
Sampled from a 3d grid in tripolar space, projected back to 2d LAT, LON data.
Sampled from a 2d grid in tripolar space, projected into 2d LAT LON data.
Unstructured data in tripolar space projected into 2d LAT LON data
The easiest of these is 2. Instead of interpolating in LAT LON space, "just" transform your point back into the source space and interpolate there.
Another option that works for 1 and 2 is to search for the cells that maps from tripolar space to cover your sample point. (You can use a BSP or grid type structure to speed up this search) Pick one of the cells, and interpolate inside it.
Finally there's a heap of unstructured interpolation options .. but they tend to be slow.
A personal favourite of mine is to use a linear interpolation of the nearest N points, finding those N points can again be done with gridding or a BSP. Another good option is to Delauney triangulate the unstructured points and interpolate on the resulting triangular mesh.
Personally if my mesh was case 1, I'd use an unstructured strategy as I'd be worried about having to handle searching through cells with overlapping projections. Choosing the "right" cell would be difficult.
I suggest you taking a look at GRASS (an open source GIS package) interpolation features (http://grass.ibiblio.org/gdp/html_grass62/v.surf.bspline.html). It's not in python but you can reimplement it or interface with C code.
Am I right in thinking your data grids look something like this (red is the old data, blue is the new interpolated data)?
alt text http://www.geekops.co.uk/photos/0000-00-02%20%28Forum%20images%29/DataSeparation.png
This might be a slightly brute-force-ish approach, but what about rendering your existing data as a bitmap (opengl will do simple interpolation of colours for you with the right options configured and you could render the data as triangles which should be fairly fast). You could then sample pixels at the locations of the new points.
Alternatively, you could sort your first set of points spatially and then find the closest old points surrounding your new point and interpolate based on the distances to those points.
There is a FORTRAN library called BIVAR, which is very suitable for this problem. With a few modifications you can make it usable in python using f2py.
From the description:
BIVAR is a FORTRAN90 library which interpolates scattered bivariate data, by Hiroshi Akima.
BIVAR accepts a set of (X,Y) data points scattered in 2D, with associated Z data values, and is able to construct a smooth interpolation function Z(X,Y), which agrees with the given data, and can be evaluated at other points in the plane.