Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
Assuming General formula of hyperbola to be y = 1 / (a*x + b), and we are provided with 100 data points out of which 99 points exactly fits a hyperbola and one data point is doesn't fits in it (unknown), from this information we need to find approximate values of a and b parameters for the hyperbola that will be formed from correct data points which are provided.
The approach which I took was by using scipy.optimize.curve_fit method as "parameters, _ = optimize.curve_fit(test, x_data ,y_data) "
where my "test" function was "def test(x, a, b): return 1 / (a*x + b)" using this method provides me perfect solution is my data points are all in first quadrant but if the data is distributed in more than one quadrant then this approach fails and I get wrong values of a and b.
Code:
import numpy as np
from scipy import optimize
x_data = np.linspace(-5,1, num=99)
y_data = 1 / (5 * x_data + 4) # value of original value of a = 5 & b = 4
np.random.seed(0)
# adding wrong point at 36th position
x_data = np.insert(x_data, 36 , 7)
y_data = np.insert(y_data, 36, 5)
def test(x, a, b):
return 1 / (a*x + b)
parameters, _ = optimize.curve_fit(test, x_data ,y_data)
[a,b] = parameters
a = 146.83956808191303
b = 148.78257639384725
# which is too wrong
Solution for above will certainly be appreciated.
Your problem is easy if the given points "exactly fit the hyperbola," and you need only two data points.
Your equation y = 1 / (ax + b) can be transformed into
(x*y) * a + (y) * b = 1
That is a linear equation in a and b. Use two data points to find the corresponding values of x * y and y and you end up with two linear equations in two variables (though in a and b rather than x and y). Then solve those two linear equations. This can easily be automated. This also does not depend on the quadrants of your two given points.
This does not work if your given points only approximate a hyperbola, of course.
In your last edit, you added the statement that only 99 of the points fit on the hyperbola, and one does not. This can be handled by choosing three pairs of your points (six distinct points), and finding the hyperbola that goes through each pair of points. That means calculating three hyperbolas (equivalently, calculating three values of a and b). If those three pairs of a and b all agree with small precision, the non-matching point was not in the chosen sample of three pairs of points and you have your hyperbola. If only two of them agree within precision, use that hyperbola. If no pair of the hyperbolas agree, some mistake was made and you should raise an exception.
I'm trying to write a function to take the derivative of any general function / array of numbers. Specifically, I am using a Central difference formula. The issue is, I cannot compute the boundary points of the derivative as the central difference formula uses indices that would be out of bound. My code is below
import numpy as np
n = 20000 # number of points in array
xs = np.linspace(start=-2*np.pi, stop=2*np.pi, num=n) # x values
y = np.array([np.sin(i) for i in xs]) # our function, sine
def deriv(f, h):
"""
Calauclate the numerical derivative of any function
:param f: numpy.array(float), the array of numbers we differentiate
:param h: step size
:rtype d: numpy.array(float)
"""
d = np.zeros_like(f)
# this loop misses the first and last points in f
for i in range(1, f.shape[0]-1):
# 2-point formula
d[i] = (f[i+1] - f[i-1])/(2*h)
return d
h = abs(xs[0] - xs[1]) # step size
y1 = deriv(y, h) # first derivative
y2 = deriv(y1, h) # second derivative
y3 = deriv(y2, h) # third derivative
When I plot y,y1,y2,y3 you can see it blows up at the end points
What I have tried to do is set the end points to their nearest neighbors in deriv as below. While this works for low order derivatives (1st and 2nd) it starts to break at higher order derivatives (3rd and greater).
...
d = np.zeros_like(f)
for i in range(1, f.shape[0]-1):
d[i] = (f[i+1] - f[i-1])/(2*h)
d[0] = d[1]
d[-1] = d[-2]
...
The derivative in the middle, away from the boundaries, is calculating fine. The issue is with the boundaries.
How should I treat the boundary conditions here? Would a different numerical differentiation scheme work better than the central difference scheme?
EDIT: I am looking for a general method to solve this, not just a method that can be applied to the sine function or any other periodic function as I have used to illustrate the issue here.
This is more a numerical methods question than a programming question.
Anyways, if your function has periodic boundary conditions (it looks it is a sinusoidal wave so in this case you have periodicity) just create a new array with 2 additional elements: the new array start element will be your last element of the original array and the end element of the new array will be the start element of the original array. Here is a way to do it
f_periodic = np.zeros(f.size+2)
f_periodic[1:-1], f_periodic[0], f_periodic[-1] = f, f[-1], f[0]
You can now differentiate on f_periodic for which d[1] and d[-2] will be the correct derivative value on the boundaries (disregard d[0] and d[-1]) .
Edit after OP's new requirements...
For more general boundary conditions, say a specific value at the boundaries, there are different approaches one can follow:
Use ghost values:
Again extend the function and extrapolate values for the new boundaries. Depending on the order of numerical differentiation more ghost cells will be required. For the current scheme, a simple linear extrapolation will do (only 1 ghost value at each boundary are required):
f_new = np.zeros(f.size+2)
f_new[1:-1] = f
f_new[-1] = f[-2] + (f[-2]-f[-3])/(x[-2]-x[-3])*(x[-1]-x[-2])
f_new[0] = f[1] + (f[1]-f[2])/(x[1]-x[2])*(x[0]-x[1])
Note that you have to also extend x. However, since you have a constant spacing just use h instead of spatial differences, e.g. x[-2]-x[-3]. You can now differentiate f_new and you will get an 1st-order approximation of the derivative on the boundaries (since you used a linear extrapolation to find the ghost value).
Use forwards and backwards schemes on the boundaries
I will not show code here, but basically you need to differentiate using an boundary value and the right (forwards) or left (backwards) value for the left and right boundaries respectively. This is a first-order approximation.
You can use the forward and backward differentiation scheme of order 2 for the boundary points. Essentially, we know that
(f(x+h)-f(x))/h = f'(x) + h/2*f''(x) + O(h²) (I)
and
(f(x+2h) - 2f(x+h) + f(x))/h² = f''(x+h) + O(h²) = f''(x) + O(h) (II)
Use the last to replace the first order term with the second derivative, that is, compute (I)-h/2*(II) to get
(-1/2*f(x+2h) + 2*f(x+h) -3/2*f(x))/h = f'(x) + O(h²)
Note that the O(h²) error in the first derivative will in general lead to an O(h) error in the second iteration of the divided differences and O(1) in the third. One may argue that the error terms cancel suitably, but that will only happen for the inner points, the one-sided derivatives will "spoil" that pattern in increasing distance from the boundary.
In this question I asked for a way to compute the closest projected point to a hyperbolic paraboloid using python.
Thanks to the answer, I was able to use the code below to calculate the closest point to multiple paraboloids.
from scipy.optimize import minimize
# This function calculate the closest projection on a hyperbolic paraboloid
# As Answered by #Jaime https://stackoverflow.com/questions/18858448/speeding-up-a-closest-point-on-a-hyperbolic-paraboloid-algorithm
def fun_single(x, p0, p1, p2, p3, p):
u, v = x
s = u*(p1-p0) + v*(p3-p0) + u*v*(p2-p3-p1+p0) + p0
return np.linalg.norm(p-s)
# Example use case:
# Generate some random data for 3 random hyperbolic paraboloids
# A real life use case will count in the tens of thousands.
import numpy as np
COUNT = 3
p0 = np.random.random_sample((COUNT,3))
p1 = np.random.random_sample((COUNT,3))
p2 = np.random.random_sample((COUNT,3))
p3 = np.random.random_sample((COUNT,3))
p = np.random.random_sample(3)
uv = []
for i in xrange(COUNT):
uv.append(minimize(fun_single, (0.5, 0.5), (p0[i], p1[i], p2[i], p3[i], p)).x)
uv = np.array(uv)
# UV projections for my random data
#[[ 0.34109572 4.39237344]
# [-0.2720813 0.17083423]
# [ 0.48993333 -0.99415568]]
Now that I have a projection for each item it's possible to find more useful info, such as which of the given items is closest to the query point, find its array index and derive more data from it, etc...
The problem with calling minimize for each item is that it becomes very slow when dealing with hundreds of thousands of items. So to try to resolve the issue I took a crack at changing the function to work with many inputs.
from numpy.core.umath_tests import inner1d
# This function calculate the closest projection to many hyperbolic paraboloids
def fun_array(x, p0, p1, p2, p3, p):
u, v = x
s = u*(p1-p0) + v*(p3-p0) + u*v*(p2-p3-p1+p0) + p0
V = p-s
return np.min(np.sqrt(inner1d(V,V)))
# Lets pass all the data to minimize
uv = minimize(fun_array, (0.5, 0.5), (p0, p1, p2, p3, p)).x
# Result: [ 0.25090064, 1.19732181]
# This corresponds to index 2 of my random data,
# which is the closest projection.
Minimizing the function fun_array is much faster than the iterative approach, but it only returns the single closest projection, not all projections.
QUESTION
Is it possible to use minimize to return all projections as with the iterative approach? And if not, is it at least possible to get the index of the "winning" array element?
The strict answer
You have to be tricky but it's not that difficult to trick minimize. The point is that minimize only works for scalar cost functions. But we can get away with summing up all your distances, since they are naturally nonnegative quantities and the global minimum is defined by the configuration where each distance is minimal. So instead of asking for the minimum points of COUNT bivariate scalar functions, instead we ask for the minimum of a single scalar function of COUNT*2 variables. This just happens to be the sum of COUNT bivariate functions. But note that I'm not convinced that this will be faster, because I can imagine higher-dimensional minimum searches to be less stable than a corresponding set of lower-dimensional independent minimum searches.
What you should definitely do is pre-allocate memory for uv and insert values into that, rather than growing a list item by item a lot of times:
uv = np.empty((COUNT,2))
for i in range(COUNT):
uv[i,:] = minimize(fun_single, (0.5, 0.5), (p0[i], p1[i], p2[i], p3[i], p)).x
Anyway, in order to use a single call to minimize we only need to vectorize your function, which is easier than you'd think:
def fun_vect(x, p0, p1, p2, p3, p):
x = x.reshape(-1,2) # dimensions are mangled by minimize() call
u,v = x.T[...,None] # u,v shaped (COUNT,1) for broadcasting
s = u*(p1-p0) + v*(p3-p0) + u*v*(p2-p3-p1+p0) + p0 # shape (COUNT,3)
return np.linalg.norm(p-s, axis=1).sum() # sum up distances for overall cost
x0 = 0.5*np.ones((COUNT,2))
uv_vect = minimize(fun_vect, x0, (p0, p1, p2, p3, p)).x.reshape(-1,2)
This function, as you see, extend the scalar one along columns. Each row corresponds to an independent minimization problem (consistently with your definition of the points). The vectorization is straightforward, the only nontrivial part is that we need to play around with dimensions to make sure that everything broadcasts nicely, and we should take care to reshape x0 on input because minimize has a habit of flattening the array-valued input position. And of course the final result has to be reshaped again. Correspondingly, an array of shape (COUNT,2) has to be provided as x0, this is the only feature from which minimize can deduce the dimensionality of your problem.
Comparison for my random data:
>>> uv
array([[-0.13386872, 0.14324999],
[ 2.42883931, 0.55099395],
[ 1.03084756, 0.35847593],
[ 1.47276203, 0.29337082]])
>>> uv_vect
array([[-0.13386898, 0.1432499 ],
[ 2.42883952, 0.55099405],
[ 1.03085143, 0.35847888],
[ 1.47276244, 0.29337179]])
Note that I changed COUNT to be 4, because I like to keep every dimension distinct when testing. This way I can be sure that I run into an error if I mess up my dimensions. Also note that in general you might want to keep the complete object returned by minimize just to make sure that everything went fine and converged.
A more useful solution
As we discussed in comments, the above solution---while perfectly answers the question---is not particularly feasible, since it takes too long to run, much longer than doing each minimization separately. The problem was interesting enough that it got me thinking. Why not try to solve the problem as exactly as possible?
What you're trying to do (now considering a single hyperboloid and a query point q) is finding the s(u,v) point with the parametrization by Jaime
s(u,v) = p0 + u * (p1 - p0) + v * (p3 - p0) + u * v * (p2 - p3 - p1 + p0)
for which the distance d(s,q) is minimal. Since the distance is a proper metric (in particular, it is non-negative), this is equivalent to minimizing d(s,q)^2. So far so good.
Let's rewrite the parametrized equation of s by introducing a few constant vectors in order to simplify the derivation:
s(u,v) = p0 + u*a + v*b + u*v*c
s - q = p0-q0 + u*a + v*b + u*v*c
= d + u*a + v*b + u*v*c
d(s,q)^2 = (s-q)^2
(In this section ^ will represent the power, because this is linear algebra.) Now, the minimum of the distance function is a stationary point, so in the u_min,v_min point we're looking for the gradient of s(u,v) with respect to u and v is zero. This is equivalent to saying that the derivative of d(s,q)^2 with respect to both u and v has to be simultaneously zero; this gives us two nonlinear equations with the unknowns u and v:
2*(s-q)*ds/du = 0 (1)
2*(s-q)*ds/dv = 0 (2)
Expanding these two equations is a somewhat tedious job. The first equation happens to be linear in u, the second in v. I collected all the terms containing u in the first equation, which gave me the relationship
u(v) = (-v^2*b.c - v*(c.d + a.b) - a.d)/(a + v*c)^2
where . represents the dot product. The above equation tells us that for whatever v we choose, equation (1) will exactly be satisfied if u is chosen thus. So we have to solve equation (2).
What I did was expand all the terms in equation (2), and substitute u(v) into u. The original equation had polynomial terms of 1,u,v,uv,u^2,u^2v, so I can tell you this is not pretty. With some minor assumptions of no divergence (which divergences would probably correspond to the equivalent of vertical lines in the case of a line fitting problem), we can arrive at the following beautiful equation:
(b.d + v*b^2)*f^2 - (c.d + a.b + 2*v*b.c)*e*f + (a.c + v*c^2)*e^2 = 0
with the new scalars defined as
e = v^2*b.c + v*(c.d + a.b) + a.d
f = (a + v*c)^2 = (a^2 + 2*v*a.c + v^2*c^2)
Whatever v solves this equation, the corresponding (u(v),v) point will correspond to a stationary point of the distance. We should first note that this equation considers the root of a fifth-order polynomial if v. There's guaranteed to be at least one real root, and in the worst case there can be as many as 5 real roots. Whether these correspond to minima, maxima, or (in unlikely cases) saddle points is open for discussion.
The real benefit of the above result is that we have a fighting chance of finding all the roots of the equation! This is a huge deal, since nonlinear root searching/minimization will in general give you only one root at a time, without being able to tell you if you've missed any. Enter numpy.polynomial.polynomial.polyroots. Despite all the linear algebra fluff surrounding it, we're only looking for the (at most 5!) root of a polynomial, for which we can test the distances and choose the global minimum (if necessary). If there's only one root, we can be sure that it's the minimum based on geometrical considerations.
Note that I haven't mentioned a caveat yet: the polynomial library can only work with one polynomial at a time. We will still have to loop over each hyperboloid manually. But here's the deal: we will be able to guarantee that we're finding the exact minimum, rather than unknowingly accepting local distance minima. And it might even be faster than minimize. Let's see:
import numpy as np
# generate dummy inputs
COUNT = 100
p0 = np.random.random_sample((COUNT,3))
p1 = np.random.random_sample((COUNT,3))
p2 = np.random.random_sample((COUNT,3))
p3 = np.random.random_sample((COUNT,3))
p = np.random.random_sample(3)
def mydot(v1,v2):
"""generalized dot product for multidimensional arrays: (...,N,3)x(...,N,3) -> (...,N,1)"""
# (used in u_from_v for vectorized dot product)
return np.einsum('...j,...j->...',v1,v2)[...,None]
def u_from_v(v, a, b, c, d):
"""return u(v) corresponding to zero of gradient"""
# use mydot() instead of dot to enable array-valued v input
res = (- v**2*mydot(b,c) - v*(mydot(c,d)+mydot(a,b)) - mydot(a,d))/np.linalg.norm(a+v*c, axis=-1, keepdims=True)**2
return res.squeeze()
def check_distance(uv, p0, p1, p2, p3, p):
"""compute the distance from optimization results to query point"""
u,v = uv.T[...,None]
s = u*(p1-p0) + v*(p3-p0) + u*v*(p2-p3-p1+p0) + p0
return np.linalg.norm(p-s, axis=-1)
def poly_for_v(a, b, c, d):
"""return polynomial representation of derivative of d(s,p)^2 for the parametrized s(u(v),v) point"""
# only works with a scalar problem:( one polynomial at a time
# v is scalar, a-b-c-d are 3-dimensional vectors (for a given paraboloid)
# precompute scalar products appearing multiple times in the formula
ab = a.dot(b)
ac = a.dot(c)
cc = c.dot(c)
cd = c.dot(d)
bc = b.dot(c)
Poly = np.polynomial.polynomial.Polynomial
e = Poly([a.dot(d), cd+ab, bc])
f = Poly([a.dot(a), 2*ac, cc])
res = Poly([b.dot(d), b.dot(b)])*f**2 - Poly([cd+ab,2*bc])*e*f + Poly([ac,cc])*e**2
return res
def minimize_manually(p0, p1, p2, p3, p):
"""numpy polynomial version for the minimization problem"""
# auxiliary arrays, shape (COUNT,3)
a = p1 - p0
b = p3 - p0
c = p2 - p3 - p1 + p0
d = p0 - p
# preallocate for collected result
uv_min = np.empty((COUNT,2))
for k in range(COUNT):
# collect length-3 vectors needed for a given surface
aa,bb,cc,dd = (x[k,:] for x in (a,b,c,d))
# compute 5 complex roots of the derivative distance
roots = poly_for_v(aa, bb, cc, dd).roots()
# keep exactly real roots
vroots = roots[roots.imag==0].real
if vroots.size == 1:
# we're done here
vval, = vroots
uval = u_from_v(vval, aa, bb, cc, dd)
uv_min[k,:] = uval,vval
else:
# need to find the root with minimal distance
uvals = u_from_v(vroots[:,None], aa, bb, cc, dd)
uvtmp = np.stack((uvals,vroots),axis=-1)
dists = check_distance(uvtmp, p0[k,:], p1[k,:], p2[k,:], p3[k,:], p)
winner = np.argmin(dists) # index of (u,v) pair of minimum
uv_min[k,:] = uvtmp[winner,:]
return uv_min
uv_min = minimize_manually(p0, p1, p2, p3, p)
# for comparison with the minimize-based approaches:
# distances = check_distance(uv_manual,p0,p1,p2,p3,p))
The above example has COUNT of 100, but if you start with COUNT=1 and keep running both the minimize version and the above exact version, you'll see roughly once in every 10-20 runs that the minimize-based approach misses the real minimum. So the above is safer, it's guaranteed to find the proper minima.
I also did some timing checks with COUNT=100, around 100 ms for the polynomial-based solution, around 200 ms for the minimize-based looping version. With COUNT=1000: 1 second for the polynomial, 2 seconds for the looping minimize-based. Considering how even for larger problems the above is both more precise and more efficient, I see no reason why not to use this instead.
I have a set of experimentally determined (x, y, z) points which correspond to a parabola. Unfortunately, the data is not aligned along any particular axis, and hence corresponds to a rotated parabola.
I have the following general surface:
Ax^2 + By^2 + Cz^2 + Dxy + Gyz + Hzx + Ix + Jy + Kz + L = 0
I need to produce a model that can represent the parabola accurately using (I'm assuming) least squares fitting. I cannot seem to figure out how this works. I have though of rotating the parabola until its central axis lines up with z-axis but I do not know what this axis is. Matlab's cftool only seems to fit equations of the form z = f(x, y) and I am not aware of anything in python that can solve this.
I also tried solving for the parameters numerically. When I tried making this into a matrix equation and solving by least squares, the matrix turned out to be invertible and hence my parameters were just all zero. I also am stuck on this and any help would be appreciated. I don't really mind the method as I am familiar with matlab, python and linear algebra if need be.
Thanks
Dont use any toolboxes, GUIs or special functions for this problem. Your problem is very common and the equation you provided may be solved in a very straight-forward manner. The solution to the linear least squares problem can be outlined as:
The basis of the vector space is x^2, y^2, z^2, xy, yz, zx, x, y, z, 1. Therefore your vector has 10 dimensions.
Your problem may be expressed as Ap=b, where p = [A B C D E F G H I J K L]^T is the vector containing your parameters. The right hand side b should be all zeros, but will contain some residual due to model errors, uncertainty in the data or for numerical reasons. This residual has to be minimized.
The matrix A has a dimension of N by 10, where N denotes the number of known points on surface of the parabola.
A = [x(1)^2 y(1)^2 ... y(1) z(1) 1
...
x(N)^2 y(N)^2 ... y(N) z(N) 1]
Solve the overdetermined system of linear equations by computing p = A\b.
Do you have enough data points to fit all 10 parameters - you will need at least 10?
I also suspect that 10 parameters are to many to describe a general paraboloid, meaning that some of the parameters are dependent. My fealing is that a translated and rotated paraboloid needs 7 parameters (although I'm not really sure)
I've been trying to work out how to solve the following problem using python:
We have points a, b, c, d which form a rigid body
Some unknown 3D translation and rotation is applied to the rigid body
We now know the coordinates for a, b, c
We want to calculate coordinates for d
What I know so far:
Trying to do this with "straightforward" Euler angle calculations seems like a bad idea due to gimbal lock etc.
Step 4 will therefore involve a transformation matrix, and once you know the rotation and translation matrix it looks like this step is easy using one of these:
http://www.lfd.uci.edu/~gohlke/code/transformations.py.html
https://pypi.python.org/pypi/euclid/0.01
What I can't work out is how I can calculate the rotation and translation matrices given the "new" coordinates of a, b, c.
I can see that in the general case (non-rigid body) the rotation part of this is Wahba's problem, but I think that for rigid bodies there should be some faster way of calculating it directly by working out a set of orthogonal unit vectors using the points.
For a set of corresponding points that you're trying to match (with possible perturbation) I've used SVD (singular value decomposition), which appears to exist in numpy.
An example of this technique (in Python even) can be found here, but I haven't evaluated it for correctness.
What you're going for is a "basis transform" or "change of basis" which will be represented as a transformation matrix. Assuming your 3 known points are not collinear, you can create your initial basis by:
Computing the vectors: x=(b-a) and y=(c-a)
Normalize x (x = x / magnitude(x))
Project y onto x (proj_y = x DOT y * x)
Subtract the projection from y (y = y - proj_y)
Normalize y
Compute z = x CROSS y
That gives you an initial x,y,z coordinate basis A. Do the same for your new points, and you get a second basis B. Now you want to find transform T which will take a point in A and convert it to B (change of basis). That part is easy. You can invert A to transform the points back to the Normal basis, then use B to transform into the second one. Since A is orthonormal, you can just transpose A to get the inverse. So the "new d" is equal to d * inverse(A) * B. (Though depending on your representation, you may need to use B * inverse(A) * d.)
You need to have some familiarity with matrices to get all that. Your representation of vectors and matrices will inform you as to which order to multiply the matrices to get T (T is either inverse(A)*B or B*inverse(A)).
To compute your basis matrix from your vectors x=(x1,x2,x3), y=(y1,y2,y3), z=(z1,z2,z3) you populate it as:
| x1 y1 z1 |
| x2 y2 z2 |
| x3 y3 z3 |