Related
I have 2 lists of points as numpy.ndarray, each row is the coordinate of a point, like:
a = np.array([[1,0,0],[0,1,0],[0,0,1]])
b = np.array([[1,1,0],[0,1,1],[1,0,1]])
Here I want to calculate the euclidean distance between all pairs of points in the 2 lists, for each point p_a in a, I want to calculate the distance between it and every point p_b in b. So the result is
d = np.array([[1,sqrt(3),1],[1,1,sqrt(3)],[sqrt(3),1,1]])
How to use matrix multiplication in numpy to compute the distance matrix?
Using direct numpy broadcasting, you can do this:
dist = np.sqrt(((a[:, None] - b[:, :, None]) ** 2).sum(0))
Alternatively, scipy has a routine that will compute this slightly more efficiently (particularly for large matrices)
from scipy.spatial.distance import cdist
dist = cdist(a, b)
I would avoid solutions that depend on factoring-out matrix products (of the form A^2 + B^2 - 2AB), because they can be numerically unstable due to floating point roundoff errors.
To compute the squared euclidean distance for each pair of elements off them - x and y, we need to find :
(Xik-Yjk)**2 = Xik**2 + Yjk**2 - 2*Xik*Yjk
and then sum along k to get the distance at coressponding point as dist(Xi,Yj).
Using associativity, it reduces to :
dist(Xi,Yj) = sum_k(Xik**2) + sum_k(Yjk**2) - 2*sum_k(Xik*Yjk)
Bringing in matrix-multiplication for the last part, we would have all the distances, like so -
dist = sum_rows(X^2), sum_rows(Y^2), -2*matrix_multiplication(X, Y.T)
Hence, putting into NumPy terms, we would end up with the euclidean distances for our case with a and b as the inputs, like so -
np.sqrt((a**2).sum(1)[:,None] + (b**2).sum(1) - 2*a.dot(b.T))
Leveraging np.einsum, we could replace the first two summation-reductions with -
np.einsum('ij,ij->i',a,a)[:,None] + np.einsum('ij,ij->i',b,b)
More info could be found on eucl_dist package's wiki page (disclaimer: I am its author).
If you have 2 each 1-dimensional arrays, x and y, you can convert the arrays into matrices with repeating columns, transpose, and apply the distance formula. This assumes that x and y are coordinated pairs. The result is a symmetrical distance matrix.
x = [1, 2, 3]
y = [4, 5, 6]
xx = np.repeat(x,3,axis = 0).reshape(3,3)
yy = np.repeat(y,3,axis = 0).reshape(3,3)
dist = np.sqrt((xx-xx.T)**2 + (yy-yy.T)**2)
dist
Out[135]:
array([[0. , 1.41421356, 2.82842712],
[1.41421356, 0. , 1.41421356],
[2.82842712, 1.41421356, 0. ]])
L2 distance = (a^2 + b^2 - 2ab)^0.5
a = np.random.randn(5, 3)
b = np.random.randn(2, 3)
a2 = np.sum(np.square(a), axis = 1)[..., None]
b2 = np.sum(np.square(b), axis = 1)[None, ...]
ab = -2*np.dot(a, b.T)
dist = np.sqrt(a2 + b2 + ab)
I am fitting a 2d polynomial with the numpy function linalg.lstsq:
coeffs = np.array([y*0+1, y, x, x**2, y**2]).T
coeff_r, r, rank, s =np.linalg.lstsq(coeffs, values)
Some points that I am trying to fit are more reliable than others.
Is there a way to weigh the points differently?
Thanks
lstsq is enough for this; the weights can be applied to the equations. That is, if in an overdetermined system
3*a + 2*b = 9
2*a + 3*b = 4
5*a - 4*b = 2
you care about the first equation more than about the others, multiply it by some number greater than 1. For example, by 5:
15*a + 10*b = 45
2*a + 3*b = 4
5*a - 4*b = 2
Mathematically, the system is the same, but the least squares solution will be different because it minimizes the sum of squares of the residuals, and the residual of the 1st equation got multiplied by 5.
Here is an example based on your code (with small adjustments to make it more NumPythonic). First, unweighted fit:
import numpy as np
x, y = np.meshgrid(np.arange(0, 3), np.arange(0, 3))
x = x.ravel()
y = y.ravel()
values = np.sqrt(x+y+2) # some values to fit
functions = np.stack([np.ones_like(y), y, x, x**2, y**2], axis=1)
coeff_r = np.linalg.lstsq(functions, values, rcond=None)[0]
values_r = functions.dot(coeff_r)
print(values_r - values)
This displays the residuals as
[ 0.03885814 -0.00502763 -0.03383051 -0.00502763 0.00097465 0.00405298
-0.03383051 0.00405298 0.02977753]
Now I give the 1st data point greater weight.
weights = np.ones_like(x)
weights[0] = 5
coeff_r = np.linalg.lstsq(functions*weights[:, None], values*weights, rcond=None)[0]
values_r = functions.dot(coeff_r)
print(values_r - values)
Residuals:
[ 0.00271103 -0.01948647 -0.04828936 -0.01948647 0.00820407 0.0112824
-0.04828936 0.0112824 0.03700695]
The first residual is now an order of magnitude smaller, of course at the expense of others residuals.
I am using the least squares method below to calculate coefficients for:
#Estimate coefficients of linear equation y = a + b*x
def calc_coefficients(_x, _y):
x, y = np.mean(_x), np.mean(_y)
xy = np.mean(_x*_y)
x2, y2 = np.mean(_x**2), np.mean(_y**2)
n = len(_x)
b = (xy - x*y) / (x2 - x**2)
a = y - b*x
sig_b = np.sqrt((y2-y**2)/(x2-x**2)-b**2) / np.sqrt(n)
sig_a = sig_b * np.sqrt(x2 - x**2)
return a, b, sig_a, sig_b
example data:
_x= [(0.009412743,0.014965211,0.013263312,0.013529132,0.009989368,0.013932615,0.020849682,0.010953529,0.003608903,0.007220992,0.012750529,0.021608436,0.031742052,0.022482958,0.021137599,0.018703295,0.021633681,0.019866029,0.020260629,0.034433715,0.009241074,0.012027059)]
_y = 0.294158677,0.359935335,0.313484808,0.301917271,0.169190763,0.486254864,0.305846328,0.347077387,0.188928817,0.422194367,0.41157232,0.39281496,0.497935681,0.34763333,0.281712023,0.352045535,0.339958296,0.395932086,0.359905526,0.450004349,0.395200865,0.365162443)]
However, I need a (y-intercept) to be zero. (y = bx).
I have tried using:
np.linalg.lstsq(_x, _y)
but I get this error:
LinAlgError: 1-dimensional array given. Array must be two-dimensional
What is the best method to fit the data for y = bx?
The error is because you pass a 1-dimensional array, which should have been a two-dimensional array of shape (n, 1) - so, a matrix with 1 column. You could just do x.reshape(-1, 1) but here is a way to do least squares fit with an arbitrary set of degrees of x:
import numpy as np
x = np.array([0, 1, 2, 3, 4, 5])
y = np.array([3, 6, 5, 7, 9, 1])
degrees = [1] # list of degrees of x to use
matrix = np.stack([x**d for d in degrees], axis=-1) # stack them like columns
coeff = np.linalg.lstsq(matrix, y)[0] # lstsq returns some additional info we ignore
print("Coefficients", coeff)
fit = np.dot(matrix, coeff)
print("Fitted curve/line", fit)
The matrix you pass to lstsq should have columns of the form f(x) where f runs through the terms you allow in the model. So if it's a general linear model, you'll have x**0 and x**1. With zero intercept forced, it's just x**1. In general these don't have to be powers of x, either.
Output for degrees = [1], model y = bx
Coefficients [ 1.41818182]
Fitted curve/line [ 0. 1.41818182 2.83636364 4.25454545 5.67272727 7.09090909]
Output for degrees = [0, 1], model y = a + bx
Coefficients [ 5.0952381 0.02857143]
Fitted curve/line [ 5.0952381 5.12380952 5.15238095 5.18095238 5.20952381 5.23809524]
I have 2 lists of points as numpy.ndarray, each row is the coordinate of a point, like:
a = np.array([[1,0,0],[0,1,0],[0,0,1]])
b = np.array([[1,1,0],[0,1,1],[1,0,1]])
Here I want to calculate the euclidean distance between all pairs of points in the 2 lists, for each point p_a in a, I want to calculate the distance between it and every point p_b in b. So the result is
d = np.array([[1,sqrt(3),1],[1,1,sqrt(3)],[sqrt(3),1,1]])
How to use matrix multiplication in numpy to compute the distance matrix?
Using direct numpy broadcasting, you can do this:
dist = np.sqrt(((a[:, None] - b[:, :, None]) ** 2).sum(0))
Alternatively, scipy has a routine that will compute this slightly more efficiently (particularly for large matrices)
from scipy.spatial.distance import cdist
dist = cdist(a, b)
I would avoid solutions that depend on factoring-out matrix products (of the form A^2 + B^2 - 2AB), because they can be numerically unstable due to floating point roundoff errors.
To compute the squared euclidean distance for each pair of elements off them - x and y, we need to find :
(Xik-Yjk)**2 = Xik**2 + Yjk**2 - 2*Xik*Yjk
and then sum along k to get the distance at coressponding point as dist(Xi,Yj).
Using associativity, it reduces to :
dist(Xi,Yj) = sum_k(Xik**2) + sum_k(Yjk**2) - 2*sum_k(Xik*Yjk)
Bringing in matrix-multiplication for the last part, we would have all the distances, like so -
dist = sum_rows(X^2), sum_rows(Y^2), -2*matrix_multiplication(X, Y.T)
Hence, putting into NumPy terms, we would end up with the euclidean distances for our case with a and b as the inputs, like so -
np.sqrt((a**2).sum(1)[:,None] + (b**2).sum(1) - 2*a.dot(b.T))
Leveraging np.einsum, we could replace the first two summation-reductions with -
np.einsum('ij,ij->i',a,a)[:,None] + np.einsum('ij,ij->i',b,b)
More info could be found on eucl_dist package's wiki page (disclaimer: I am its author).
If you have 2 each 1-dimensional arrays, x and y, you can convert the arrays into matrices with repeating columns, transpose, and apply the distance formula. This assumes that x and y are coordinated pairs. The result is a symmetrical distance matrix.
x = [1, 2, 3]
y = [4, 5, 6]
xx = np.repeat(x,3,axis = 0).reshape(3,3)
yy = np.repeat(y,3,axis = 0).reshape(3,3)
dist = np.sqrt((xx-xx.T)**2 + (yy-yy.T)**2)
dist
Out[135]:
array([[0. , 1.41421356, 2.82842712],
[1.41421356, 0. , 1.41421356],
[2.82842712, 1.41421356, 0. ]])
L2 distance = (a^2 + b^2 - 2ab)^0.5
a = np.random.randn(5, 3)
b = np.random.randn(2, 3)
a2 = np.sum(np.square(a), axis = 1)[..., None]
b2 = np.sum(np.square(b), axis = 1)[None, ...]
ab = -2*np.dot(a, b.T)
dist = np.sqrt(a2 + b2 + ab)
edit: I'm not looking for you to debug this code. If you are familiar with this well-known algorithm, then you may be able to help. Please note that the algorithm produces the coefficients correctly.
This code for cubic spline interpolation is producing linear splines and I can't seem to figure out why (yet). The algorithm comes from Burden's Numerical Analysis, which is just about identical to the pseudo code here, or you can find that book from a link in the comments (see chapter 3, it's worth having anyway). The code is producing the correct coefficients; I believe that I am misunderstanding the implementation. Any feedback is greatly appreciated. Also, i'm new to programming, so any feedback on how bad my coding is also welcome. I tried uploading pics of the linear system in terms of h, a, and c, but as a new user i can not. If you want a visual of the tridiagonal linear system that the algorithm solves, and which is set up by the var alpha, see the link in the comments for the book, see chap 3. The system is strictly diagonally dominant, so we know there exists a unique solution c0,...,cn. Once we know the ci values, the other coefficients follow.
import matplotlib.pyplot as plt
# need some zero vectors...
def zeroV(m):
z = [0]*m
return(z)
#INPUT: n; x0, x1, ... ,xn; a0 = f(x0), a1 =f(x1), ... , an = f(xn).
def cubic_spline(n, xn, a, xd):
"""function cubic_spline(n,xn, a, xd) interpolates between the knots
specified by lists xn and a. The function computes the coefficients
and outputs the ranges of the piecewise cubic splines."""
h = zeroV(n-1)
# alpha will be values in a system of eq's that will allow us to solve for c
# and then from there we can find b, d through substitution.
alpha = zeroV(n-1)
# l, u, z are used in the method for solving the linear system
l = zeroV(n+1)
u = zeroV(n)
z = zeroV(n+1)
# b, c, d will be the coefficients along with a.
b = zeroV(n)
c = zeroV(n+1)
d = zeroV(n)
for i in range(n-1):
# h[i] is used to satisfy the condition that
# Si+1(xi+l) = Si(xi+l) for each i = 0,..,n-1
# i.e., the values at the knots are "doubled up"
h[i] = xn[i+1]-xn[i]
for i in range(1, n-1):
# Sets up the linear system and allows us to find c. Once we have
# c then b and d follow in terms of it.
alpha[i] = (3./h[i])*(a[i+1]-a[i])-(3./h[i-1])*(a[i] - a[i-1])
# I, II, (part of) III Sets up and solves tridiagonal linear system...
# I
l[0] = 1
u[0] = 0
z[0] = 0
# II
for i in range(1, n-1):
l[i] = 2*(xn[i+1] - xn[i-1]) - h[i-1]*u[i-1]
u[i] = h[i]/l[i]
z[i] = (alpha[i] - h[i-1]*z[i-1])/l[i]
l[n] = 1
z[n] = 0
c[n] = 0
# III... also find b, d in terms of c.
for j in range(n-2, -1, -1):
c[j] = z[j] - u[j]*c[j+1]
b[j] = (a[j+1] - a[j])/h[j] - h[j]*(c[j+1] + 2*c[j])/3.
d[j] = (c[j+1] - c[j])/(3*h[j])
# This is my only addition, which is returning values for Sj(x). The issue I'm having
# is related to this implemention, i suspect.
for j in range(n-1):
#OUTPUT:S(x)=Sj(x)= aj + bj(x - xj) + cj(x - xj)^2 + dj(x - xj)^3; xj <= x <= xj+1)
return(a[j] + b[j]*(xd - xn[j]) + c[j]*((xd - xn[j])**2) + d[j]*((xd - xn[j])**3))
For the bored, or overachieving...
Here is code for testing, the interval is x: [1, 9], y:[0, 19.7750212]. The test function is xln(x), so we start 1 and increase by .1 up to 9.
ln = []
ln_dom = []
cub = []
step = 1.
X=[1., 9.]
FX=[0, 19.7750212]
while step <= 9.:
ln.append(step*log(step))
ln_dom.append(step)
cub.append(cubic_spline(2, x, fx, step))
step += 0.1
...and for plotting:
plt.plot(ln_dom, cub, color='blue')
plt.plot(ln_dom, ln, color='red')
plt.axis([1., 9., 0, 20], 'equal')
plt.axhline(y=0, color='black')
plt.axvline(x=0, color='black')
plt.show()
Ok, got this working. The problem was in my implementation. I got it working with a different approach, where the splines are constructed individually instead of continuously. This is fully functioning cubic spline interpolation by method of first constructing the coefficients of the spline polynomials (which is 99% of the work), then implementing them. Obviously this is not the only way to do it. I may work on a different approach and post that if there is interest. One thing that would clarify the code would be an image of the linear system that is solved, but i can't post pics until my rep gets up to 10. If you want to go deeper into the algorithm, see the text book link in the comments above.
import matplotlib.pyplot as plt
from pylab import arange
from math import e
from math import pi
from math import sin
from math import cos
from numpy import poly1d
# need some zero vectors...
def zeroV(m):
z = [0]*m
return(z)
#INPUT: n; x0, x1, ... ,xn; a0 = f(x0), a1 =f(x1), ... , an = f(xn).
def cubic_spline(n, xn, a):
"""function cubic_spline(n,xn, a, xd) interpolates between the knots
specified by lists xn and a. The function computes the coefficients
and outputs the ranges of the piecewise cubic splines."""
h = zeroV(n-1)
# alpha will be values in a system of eq's that will allow us to solve for c
# and then from there we can find b, d through substitution.
alpha = zeroV(n-1)
# l, u, z are used in the method for solving the linear system
l = zeroV(n+1)
u = zeroV(n)
z = zeroV(n+1)
# b, c, d will be the coefficients along with a.
b = zeroV(n)
c = zeroV(n+1)
d = zeroV(n)
for i in range(n-1):
# h[i] is used to satisfy the condition that
# Si+1(xi+l) = Si(xi+l) for each i = 0,..,n-1
# i.e., the values at the knots are "doubled up"
h[i] = xn[i+1]-xn[i]
for i in range(1, n-1):
# Sets up the linear system and allows us to find c. Once we have
# c then b and d follow in terms of it.
alpha[i] = (3./h[i])*(a[i+1]-a[i])-(3./h[i-1])*(a[i] - a[i-1])
# I, II, (part of) III Sets up and solves tridiagonal linear system...
# I
l[0] = 1
u[0] = 0
z[0] = 0
# II
for i in range(1, n-1):
l[i] = 2*(xn[i+1] - xn[i-1]) - h[i-1]*u[i-1]
u[i] = h[i]/l[i]
z[i] = (alpha[i] - h[i-1]*z[i-1])/l[i]
l[n] = 1
z[n] = 0
c[n] = 0
# III... also find b, d in terms of c.
for j in range(n-2, -1, -1):
c[j] = z[j] - u[j]*c[j+1]
b[j] = (a[j+1] - a[j])/h[j] - h[j]*(c[j+1] + 2*c[j])/3.
d[j] = (c[j+1] - c[j])/(3*h[j])
# Now that we have the coefficients it's just a matter of constructing
# the appropriate polynomials and graphing.
for j in range(n-1):
cub_graph(a[j],b[j],c[j],d[j],xn[j],xn[j+1])
plt.show()
def cub_graph(a,b,c,d, x_i, x_i_1):
"""cub_graph takes the i'th coefficient set along with the x[i] and x[i+1]'th
data pts, and constructs the polynomial spline between the two data pts using
the poly1d python object (which simply returns a polynomial with a given root."""
# notice here that we are just building the cubic polynomial piece by piece
root = poly1d(x_i,True)
poly = 0
poly = d*(root)**3
poly = poly + c*(root)**2
poly = poly + b*root
poly = poly + a
# Set up our domain between data points, and plot the function
pts = arange(x_i,x_i_1, 0.001)
plt.plot(pts, poly(pts), '-')
return
If you want to test it, here's some data you can use to get started, which comes from the
function 1.6e^(-2x)sin(3*pi*x) between 0 and 1:
# These are our data points
x_vals = [0, 1./6, 1./3, 1./2, 7./12, 2./3, 3./4, 5./6, 11./12, 1]
# Set up the domain
x_domain = arange(0,2, 1e-2)
fx = zeroV(10)
# Defines the function so we can get our fx values
def sine_func(x):
return(1.6*e**(-2*x)*sin(3*pi*x))
for i in range(len(x_vals)):
fx[i] = sine_func(x_vals[i])
# Run cubic_spline interpolant.
cubic_spline(10,x_vals,fx)
Comments on your coding style:
Where are your comments and documentation? At the very least, provide function documentation so that people can tell how your function is supposed to be used.
Instead of:
def cubic_spline(xx,yy):
Please write something like:
def cubic_spline(xx, yy):
"""function cubic_spline(xx,yy) interpolates between the knots
specified by lists xx and yy. The function returns the coefficients
and ranges of the piecewise cubic splines."""
You can make lists of repeated elements by using the * operator on a list.
Like this:
>>> [0] * 10
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
So that your zeroV function can be replaced by [0] * m.
Just don't do this with mutable types! (especially lists).
>>> inner_list = [1,2,3]
>>> outer_list = [inner_list] * 3
>>> outer_list
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
>>> inner_list[0] = 999
>>> outer_list
[[999, 2, 3], [999, 2, 3], [999, 2, 3]] # wut
Math should probably be done using numpy or scipy.
Apart from that, you should read Idiomatic Python by David Goodger.