I have a set of data given here where in the first and second columns there are the sky coordinates (ra,dec), respectively and in the third and forth, the coordinates in a Cartesian system (x,y).
I need to make a two-dimensional interpolation surface using coordinates x and y and another using Ra and Dec. The problem is the existence of masked regions, as shown in the figure above. I can illustrate the missing data just by plotting them (There is non NaN value in the catalogue). That is what I so far tried and didn't give the right answer:
from scipy.interpolate import griddata
import numpy as np
import matplotlib.pyplot as plt
data = np.loadtxt('test.asc')
ra = data[:,0]
dec = data[:,1]
Xpos = data[:,2]
Ypos = data[:,3]
xi = np.linspace(Xpos.min(), Xpos.max(), 1000)
yi = np.linspace(Ypos.min(), Ypos.max(), 1000)
xi, yi = np.meshgrid(xi, yi, copy=False)
ra_int = griddata(data[:,2:4], ra, (xi.flatten(), yi.flatten()),
method='cubic')
dec_int = griddata(data[:,2:4], dec, (xi.flatten(), yi.flatten()),
method='cubic')
Using griddata fails and return just NaN values. Is there any way to do this interpolation in order to estimate the values of Ra and Dec from a given x and y coordinates even in the masked regions (map from x and y to ra and dec)?
If I get it right then it is like this:
just shift the Cartesian coordinate system to middle of the CCD and also the Equatoreal coordinates to middle of CCD. Then compute x,y separately. The only thing you need is to compute focus length f separately for x and y !!!
pos is the cartesian coordinate (x or y)
ang is the equatoreal coordinate (RA or Dec)
get edge point from the database
shift the angles to middle of CCD
compute focus (fx,fy) from it
f = pos/tan(ang)
now you can compute the projection for any entry in dataset
shift the angles to middle of CCD then compute x,y by
pos=f*tan(ang)
shift back from CCD middle to original Cartesian coordinates. You should check few points if is this approach correct
[notes]
x axis is mirrored in your output so just use x=-x at the end before shifting back to original Cartesian coordinates or leave focus f negative.
if your CCD is not axis aligned to equator then you need to compute the rotation (angle between X axis and equator) and apply rotation around Z axis after conversion before shifting back...
Related
I have measurements (PPI arc scans) taken with a doppler wind lidar. The data is stored in a pandas dataframe where rows represent azimuth angle and columns represent radial distance (input shape = 30x197). Link to example scan, (csv). I want to transform this to a cartesian coordinate system, and output a 2d array which is re-gridded into x,y coordinates instead of polar with the values stored in the appropriate grid cell. Interpolation (nearest neighbor) is ok and so is zero or NaN padding of areas where no data exists.
Ideally the X and Y grid should correspond to the actual distances between points, but right now I'm just trying to get this working. This shouldn’t be terribly difficult, but I’m having trouble obtaining the result I want.
So far, I have working code which plots on a polar axis beautifully (example image) but this won't work for the next steps of my analysis.
I have tried many different approaches with scipy.interpolate.griddata, scipy.ndimage.geometric_transform, and scipy.ndimage.map_coordinates but haven't gotten the correct output. Here is an example of my recent attempt (df_polar is the csv file linked):
# Generate polar and cartesian meshgrids
r = df_polar.columns
theta = df_polar.index
theta = np.deg2rad(theta)
# Polar meshgrid
rad_c, theta_c = np.meshgrid(r,theta)
# Cartesian meshgrid
X = rad_c * np.cos(theta_c)
Y = rad_c * np.sin(theta_c)
x,y = np.meshgrid(X,Y)
# Interpolate from polar to cartesian grid
new_grid = scipy.interpolate.griddata(
(rad_c.flatten(), theta_c.flatten()),
np.array(df_polar).flatten(), (x,y), method='nearest')
The result is not correct at all, and from reading the documentation and examples I don't understand why. I would greatly appreciate any tips on where I have gone wrong. Thanks a lot!!
I think you might be feeding griddata the wrong points. It wants cartesian points and if you want the values interpolated over a regular x/y grid you need to create one and provide that too.
Try this and let me know if it produces the expected result. It's hard for me to tell if this is what it should produce:
from scipy.interpolate import griddata
import pandas as pd
import numpy as np
df_polar = pd.read_csv('onescan.txt', index_col=0)
# Generate polar and cartesian meshgrids
r = pd.to_numeric(df_polar.columns)
theta = np.deg2rad(df_polar.index)
# Polar meshgrid
rad_c, theta_c = np.meshgrid(r, theta)
# Cartesian equivalents of polar co-ordinates
X = rad_c*np.cos(theta_c)
Y = rad_c*np.sin(theta_c)
# Cartesian (x/y) meshgrid
grid_spacing = 100.0 # You can change this
nx = (X.max() - X.min())/grid_spacing
ny = (Y.max() - Y.min())/grid_spacing
x = np.arange(X.min(), X.max() + grid_spacing, grid_spacing)
y = np.arange(Y.min(), Y.max() + grid_spacing, grid_spacing)
grid_x, grid_y = np.meshgrid(x, y)
# Interpolate from polar to cartesian grid
new_grid = griddata(
(X.flatten(), Y.flatten()),
df_polar.values.flatten(),
(grid_x, grid_y),
method='nearest'
)
The resulting values look something like this (with grid_spacing = 10 and flipping x and y):
import matplotlib.pyplot as plt
plt.imshow(new_grid.T, cmap='hot')
Clearly interpolate "nearest" needs taming...
I'm trying to find a method of linear interpolation in 2D over a regular grid using python, but each proposed type in scipy seems to have it's disadvantages.
My aim is basically:
Have smooth linearly interpolated data over a regular grid, or as close as possible
The original data can be at arbitrary locations
(optional) Linearly extrapolate to the edges
But all the functions seem to have problems with this:
Functions like griddata, interp2d, LinearNDInterpolator appear to create triangles and interpolate within them, creating a bunch of hard lines/creases that I can't have.
Rbf seems at first to do exactly what I want, but when presented with planes that are flat, it generates an interpolation based on some kind of sphere, creating a curved surface.
If Rbf would simply interpolate a flat plane as a flat plane using the linear setting, as would be expected, it'd be perfect.
Are there any ideas on how to achieve this, or if there's another function that does what I'm after? I've attached a basic example below.
import numpy as np
from scipy import interpolate
import matplotlib.pyplot as plt
#create some objects to store data
x=np.empty((2,2))
y=np.empty((2,2))
f_shape=(100,100,100)
#generate coordinates
x[0, :] = 0
x[-1, :] = f_shape[0]
y[:, 0] = 0
y[:, -1] = f_shape[1]
#--------constant height----------
z=np.full((2,2),50)
#create interpolation function and interpolate across grid
interp=interpolate.Rbf(x,y,z,function='linear')
grid=np.mgrid[0:f_shape[0],0:f_shape[1]]
result=interp(grid[0,:,:],grid[1,:,:])
plt.imshow(result) #incorrect curved surface from constant height!!!
#--------random heights-----------
z=np.random.uniform(25,75,(2,2))
#create interpolation function and interpolate across grid
interp=interpolate.Rbf(x,y,z,function='linear')
grid=np.mgrid[0:f_shape[0],0:f_shape[1]]
result=interp(grid[0,:,:],grid[1,:,:])
plt.imshow(result) #Apparently nice smooth linear-ish interpolation
Incorrect curved surface from constant height:
Apparently nice smooth linear-ish interpolation:
I've managed to write a function that suits my purpose. It interpolates (fills in) a plane from a grid of coordinates by interpolating along the grid lines, then interpolating the plane in the x and y directions, and taking the average of the two.
It should be possible to speed this up a bit by reshaping the coordinates into a 1D vector, interpolating the plane in one go, then reshaping back into 2D. However this code is certainly fast enough already for reasonable plane sizes.
Seems to work ok if the coordinates are outside of the plane too.
Extrapolation also works if the grid is approximately regular. It'll extrapolate regardless but you'll start to see some sharp creases away from the edge as the grid irregularity increases.
Here's the code. An example is provided in the docstring.
def interlin2d(x,y,z,fsize):
"""
Linear 2D interpolation of a plane from arbitrary gridded points.
:param x: 2D array of x coordinates
:param y: 2D array of y coordinates
:param z: 2D array of z coordinates
:param fsize: Tuple of x and y dimensions of plane to be interpolated.
:return: 2D array with interpolated plane.
This function works by interpolating lines along the grid point in both dimensions,
then interpolating the plane area in both the x and y directions, and taking the
average of the two. Result looks like a series of approximately curvilinear quadrilaterals.
Note, the structure of the x,y,z coordinate arrays are such that the index of the coordinates
indicates the relative physical position of the point with respect to the plane to be interpoalted.
Plane is allowed to be a subset of the range of grid coordinates provided.
Extrapolation is accounted for, however sharp creases will start to appear
in the extrapolated region as the grid of coordinates becomes increasingly irregular.
Scipy's interpolation function is used for the grid lines as it allows for proper linear extrapolation.
However Numpy's interpolation function is used for the plane itself as it is robust against gridlines
that overlap (divide by zero distance).
Example:
#set up number of grid lines and size of field to interpolate
nlines=[3,3]
fsize=(100,100,100)
#initialize the coordinate arrays
x=np.empty((nlines[0],nlines[1]))
y=np.empty((nlines[0],nlines[1]))
z=np.random.uniform(0.25*fsize[2],0.75*fsize[2],(nlines[0],nlines[1]))
#set random ordered locations for the interior points
spacings=(fsize[0]/(nlines[0]-2),fsize[1]/(nlines[1]-2))
for k in range(0, nlines[0]):
for l in range(0, nlines[1]):
x[k, l] = round(random.uniform(0, 1) * (spacings[0] - 1) + spacings[0] * (k - 1) + 1)
y[k, l] = round(random.uniform(0, 1) * (spacings[1] - 1) + spacings[1] * (l - 1) + 1)
#fix the edge points to the edge
x[0, :] = 0
x[-1, :] = fsize[1]-1
y[:, 0] = 0
y[:, -1] = fsize[0]-1
field = interlin2d(x,y,z,fsize)
"""
from scipy.interpolate import interp1d
import numpy as np
#number of lines in grid in x and y directions
nsegx=x.shape[0]
nsegy=x.shape[1]
#lines along the grid points to be interpolated, x and y directions
#0 indicates own axis, 1 is height (z axis)
intlinesx=np.empty((2,nsegy,fsize[0]))
intlinesy=np.empty((2,nsegx,fsize[1]))
#account for the first and last points being fixed to the edges
intlinesx[0,0,:]=0
intlinesx[0,-1,:]=fsize[1]-1
intlinesy[0,0,:]=0
intlinesy[0,-1,:]=fsize[0]-1
#temp fields for interpolation in x and y directions
tempx=np.empty((fsize[0],fsize[1]))
tempy=np.empty((fsize[0],fsize[1]))
#interpolate grid lines in the x direction
for k in range(nsegy):
interp = interp1d(x[:,k], y[:,k], kind='linear', copy=False, fill_value='extrapolate')
intlinesx[0,k,:] = np.round(interp(range(fsize[0])))
interp = interp1d(x[:, k], z[:, k], kind='linear', copy=False, fill_value='extrapolate')
intlinesx[1, k, :] = interp(range(fsize[0]))
intlinesx[0,:,:].sort(0)
# interpolate grid lines in the y direction
for k in range(nsegx):
interp = interp1d(y[k, :], x[k, :], kind='linear', copy=False, fill_value='extrapolate')
intlinesy[0, k, :] = np.round(interp(range(fsize[1])))
interp = interp1d(y[k, :], z[k, :], kind='linear', copy=False, fill_value='extrapolate')
intlinesy[1, k, :] = interp(range(fsize[1]))
intlinesy[0,:,:].sort(0)
#interpolate plane in x direction
for k in range(fsize[1]):
tempx[k, :] = np.interp(range(fsize[1]),intlinesx[0,:,k], intlinesx[1,:,k])
#interpolate plane in y direction
for k in range(fsize[1]):
tempy[:, k] = np.interp(range(fsize[0]), intlinesy[0, :, k], intlinesy[1, :, k])
return (tempx+tempy)/2
Example of interpolation based on 9 points (shown as red dots)
Scipy's griddata works just fine:
import numpy as np
from scipy import interpolate as intp
import matplotlib.pyplot as plt
%matplotlib inline
grid_size = G = 100
height = H = 50
points = np.array([
(0, 0),
(G-1, 0),
(0, G-1),
(G-1, G-1)
], dtype=np.float32)
gy, gx = np.mgrid[:G, :G]
result = intp.griddata(points, np.full(points.shape[0], H), (gy, gx))
And the plot:
plt.imshow(result, interpolation='none')
plt.colorbar()
And just to be sure:
>>> np.allclose(result, 50)
True
I have an n x n x n numpy array that contains density values on a cubic grid. I'm trying to align the principal axes of inertia of the density map with the cartesian x,y,z axes of the grid. I have the following so far:
import numpy as np
from scipy import ndimage
def center_rho(rho):
"""Move density map so its center of mass aligns with the center of the grid"""
rhocom = np.array(ndimage.measurements.center_of_mass(rho))
gridcenter = np.array(rho.shape)/2.
shift = gridcenter-rhocom
rho = ndimage.interpolation.shift(rho,shift,order=1,mode='wrap')
return rho
def inertia_tensor(rho,side):
"""Calculate the moment of inertia tensor for the given density map."""
halfside = side/2.
n = rho.shape[0]
x_ = np.linspace(-halfside,halfside,n)
x,y,z = np.meshgrid(x_,x_,x_,indexing='ij')
Ixx = np.sum(rho*(y**2 + z**2))
Iyy = np.sum(rho*(x**2 + z**2))
Izz = np.sum(rho*(x**2 + y**2))
Ixy = -np.sum(rho*x*y)
Iyz = -np.sum(rho*y*z)
Ixz = -np.sum(rho*x*z)
I = np.array([[Ixx, Ixy, Ixz],
[Ixy, Iyy, Iyz],
[Ixz, Iyz, Izz]])
return I
def principal_axes(I):
"""Calculate the principal inertia axes and order them in ascending order."""
w,v = np.linalg.eigh(I)
return w,v
#number of grid points along side
n = 10
#note n <= 3 produces unit eigenvectors, not sure why
#in practice, n typically between 10 and 50
np.random.seed(1)
rho = np.random.random(size=(n,n,n))
side = 1. #physical width of box, set to 1.0 for simplicity
rho = center_rho(rho)
I = inertia_tensor(rho,side)
PAw, PAv = principal_axes(I)
#print magnitude and direction of principal axes
print "Eigenvalues/eigenvectors before rotation:"
for i in range(3):
print PAw[i], PAv[:,i]
#sanity check that I = R * D * R.T
#where R is the rotation matrix and D is the diagonalized matrix of eigenvalues
D = np.eye(3)*PAw
print np.allclose(np.dot(PAv,np.dot(D,PAv.T)),I)
#rotate rho to align principal axes with cartesian axes
newrho = ndimage.interpolation.affine_transform(rho,PAv.T,order=1,mode='wrap')
#recalculate principal axes
newI = inertia_tensor(newrho,side)
newPAw, newPAv = principal_axes(newI)
#print magnitude and direction of new principal axes
print "Eigenvalues/eigenvectors before rotation:"
for i in range(3):
print newPAw[i], newPAv[:,i]
Here I'm assuming that the eigenvectors of the inertia tensor define the rotation matrix (which based on this question and Google results such as this webpage seems correct?) However this doesn't give me the correct result.
I expect the printed matrix to be:
[1 0 0]
[0 1 0]
[0 0 1]
(which could be wrong) but don't even get unit vectors to start with. What I get is:
Eigenvalues/eigenvectors before rotation:
102.405523732 [-0.05954221 -0.8616362 0.5040216 ]
103.177395578 [-0.30020273 0.49699978 0.81416801]
104.175688943 [-0.95201526 -0.10283129 -0.288258 ]
True
Eigenvalues/eigenvectors after rotation:
104.414931478 [ 0.38786 -0.90425086 0.17859172]
104.731536038 [-0.74968553 -0.19676735 0.63186566]
106.151322662 [-0.53622405 -0.37896304 -0.75422197]
I'm not sure if the problem is my code or my assumptions about rotating principal axes, but any help would be appreciated.
Here is the link to the code I developed to do such alignment.
Given a set of scatter points with coordinates (x,y,z), the objective is to match the eigenvector associated to the minimum eigenvalue with the X-axis of a 3D cartesian axis and the eigenvector associated to the median eigenvalue with the Y axis from the same 3D cartesian axis.
For this purpose, I followed the following steps:
Translate the set of points with centroid in (xmn, ymn, zmn) to a new set of points with centroid in (0,0,0) only by doing: (x-xmn, y-ymn, z-zmn).
Calculate the angle THETA (rotation around z) between the xy-projection of the eigenvector associated to the minimum eigenvalue (min_eigen) and the x-axis in a cartesian axis. After obtention of the resulting tetha, rotate the min_eigen the given theta so that it is contained in the xy-plane. Let's call this resulting vector: rotz
Calculate the angle PHI between rotz and x-axis in order to perform a rotation around the y-axis. Once the phi is obtained, a rotation is applied to rotz aound the y axis. With this last rotation, the eigenvector associated to the medium eigenvector (medium_eigen) is then in the yz proyection of the cartesian axis, so we will just need to find the angle between medium_eigen and the y-axis of the cartesian axis.
Calculate the angle ALPHA between the medium_eigen and y-axis. Apply the rotation around the x-axis aaaand: IT'S DONE!
NOTE: After applying steps 1,2,3 to your set of points, you have to recalculate the 3D_SVD (3D_single value decomposition) and from the resulting set of eigenvectors, then implement the 4th step with the new medium_eigen.
I really hope this helps.
The rotations are implemented by means of the rotation matrix defined here: Rotating a Vector in 3D Space
I want to extract given point probes from a surface contour plot. Therefore I create the surface with following code snippet:
def createInterpolatedSurface():
npts=1000
xi = np.linspace(min(x), max(x),npts)
yi = np.linspace(min(y), max(y),npts)
xi,yi=np.meshgrid(xi,yi,indexing='xy')
ui=scipy.interpolate.griddata((x,y),u,(xi,yi),method='linear')
return xi,yi,ui
My next step is to set up an array of points, with initSensorArray as nested Loop function, I am interested in:
So there is my main problem. I want to use the realistic physical coordinates of my points, not the ij indexing coordinates of the griddata interpolation function.
For example: Point(0.5,0.1) in physical space equals Pointg(100,125) in griddata ij indexing,.
How can I map the physical point coordinates to the griddata, extrapolate the points and map them back?
Thanks for help
You can use scipy'` interpolate functions in 2D,
from scipy import interpolate
x,y,u = createInterpolatedSurface()
#Create an interpolation function
f = interpolate.interp2d(x, y, u, kind='cubic')
#Use interpolation function to get results
xpoint = 0.5
ypoint = 0.1
upoint = f(xpoint, ypoint)
I want to draw curves between any two points in 3d space. The curve must be, umm, "vertical". I mean, x,y positions of the points of curve must be on the same line, but z values must change as if you sent a projectile from ground, it traveled in air, and hit the ground again. It does not need to be physically correct, an arc is OK.
This is the starting code:
import numpy as np
p1=np.array([1,1,1]) #x,y,z coordinates of the first point
p2=np.array([3,3,3]) #x,y,z coordinates of the second point
xi=np.linspace(p1[0],p2[0],100) #determine 100 x coordinates between two points
yi=np.linspace(p1[1],p2[1],100) #determine 100 y coordinates between two points
zi= ?? #determine 100 z coordinates between two points.
How can I determine those 100 z coordinates (zi)?
After determining zi it is trivial to draw lines between consecutive points(using mayavi or mplot3d) , giving the visual of a curve.
I ended up using scipy.interpolate to get the curve, and adding it to z coordinates of the line between points. As others said, there are more than one way to do this. This will be enough for my purpose.
### objective: draw an arc between points p1 and p2. z coordinates are raised.
import numpy as np
from scipy import interpolate
from mayavi import mlab
###inputs
p1=np.random.uniform(0,20,(3)) #first point
p2=np.random.uniform(0,20,(3)) #second point
npts = 100 # number of points to sample
y=np.array([0,.5,.75,.75,.5,0]) #describe your shape in 1d like this
amp=5 #curve height factor. bigger means heigher
#get the adder. This will be used to raise the z coords
x=np.arange(y.size)
xnew = np.linspace(x[0],x[-1] , npts) #sample the x coord
tck = interpolate.splrep(x,y,s=0)
adder = interpolate.splev(xnew,tck,der=0)*amp
adder[0]=adder[-1]=0
adder=adder.reshape((-1,1))
#get a line between points
shape3=np.vstack([np.linspace(p1[dim],p2[dim],npts) for dim in xrange(3)]).T
#raise the z coordinate
shape3[:,-1]=shape3[:,-1]+adder[:,-1]
#plot
x,y,z=(shape3[:,dim] for dim in xrange(3))
mlab.points3d(x,y,z,color=(0,0,0))
mlab.plot3d(x,y,z,tube_radius=1)
mlab.outline()
mlab.axes()
mlab.show()
There isn't one right answer to this question because the curvature of the arc isn't constrained. The basis for the math for this problem is projectile motion, which gives you two key equations:
x_2 - x_1 = v_1 cos theta dt
z_2 - z_1 = -1/2 g dt^2 + v_0 sin theta dt
where v_1 is the initial velocity of the projectile, theta is the angle from horizontal that the projectile is shot at, dt is the time it takes for the projectile to go from point 1 to point 2, and g is the gravitational constant. This neglects y for now for simplicity. The problem for you is that this gives you two equations, but you have three unknowns, v_1, theta, and dt.
You can add a constraint, for example, that the higher of p1 and p2 is the peak of the trajectory. If p2 is higher, for example,
v_2 = v_1 - g dt = 0
Solving those three equations gives you v_1, which gives the z coordinate over time:
z = -1/2 g t^2 + v_1 t + z_1
t = np.linspace(0, dt, 100) gives you a numpy vector of times, and you can plug that into your formula for z.