I have a set of points p and I need to transform them so that the they align with another given set of points q (find the transform T from source to target).
So far it is an easy problem. My problem is that I do have some freedom aligning these points i.e, I only have to keep the alignment error below some given threshold (alpha) and not minimize the distance. I want to exploit this alignment freedom to minimize distances between p and a different set of points r. I marked the vectors to be optimized E = Tp - r
So basically I want to use the first alignment as a hard constraint and try to minimize another set of correspondences (I attached a picture). I want to minimize |E| (the green distances) under the constraint that the black points are within the red circles (alpha) after applying the transformation T.
I tried some heuristic solutions like calculating the maximum allowed rotation around the centroid and only then taking the maximum allowed translation but none of these solutions guarantee the optimal solution.
Have you heard about Lagrange optimization?
Here's the corresponding article.
You minimize a cost function (in your case E) under certain inequality
constraints and equality constraints (in your case no equality constraints).
This may be an approach for your solution?
Step 1:
Build augmented cost function: E - L * (Tp - q - alpha)
Step 2:
Find partial derivatives w.r.t T and L
Step 3:
Solve for zeros in partial derivatives
Related
I'm trying to solve the following boundary value problem on [0,\infty]:
f''=-f'/r+f/r^2+m^2*f+2 \lambda *f^3
f(0)=0 \ ; f(\infty)=\sqrt{-m^2/(2\lambda)}
for some constants m^2<0, \lambda>0. There is no closed form but we should have f monotonically increasing from 0 to sqrt{-m^2/(2\lambda). There is a removable singularity at r=0. This problem is just Bessel's equation plus a term in f^3.
I'm trying to solve this with Scipy's integrate.solve_bvp which can solve multi-boundary problems with a singularity at one boundary, defining y=[f,rf'] so that
y'=[0,r(m^2f+2\lambda f^3)]+(1/r)*[[0,1],[1,0]]*y
I impose the boundary condition at infinity at some large value max_x. Unfortunately my code, following the structure of the example at https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_bvp.html, gives the wrong solution:
import scipy.integrate
import numpy as np
import matplotlib.pyplot as plt
m_squared=-1
Lambda=1
asymptote=np.sqrt(-m_squared/(2*Lambda))
#evaluate infinity b.c here
max_x=100
def fun(r,v):
z=(m_squared*v[0]+(2*Lambda)*(v[0]**3) )*r
return np.vstack((z-z, z))
#boundary condition
def bc(ya,yb):
return np.array([ya[0], yb[0]-asymptote])
# to treat singularity
S=np.array([[0,1],[1,0]])
x=np.linspace(0,max_x,5000)
# guess for vector y at points x
y=np.zeros((2, len(x)))
y[0,-1]=asymptote
print(y)
#solve
res=scipy.integrate.solve_bvp(fun, bc, x, y, p=None, S=S)
x_plot=np.linspace(0,max_x,1000)
y_plot=res.sol(x_plot)[0]
plt.plot(x_plot,y_plot,label="numerical")
plt.axhline(asymptote,linestyle="--",label="asymptote")
plt.xlabel("r")
plt.ylabel("f")
plt.legend()
I checked that modifying the above code to solve e.g $f''=f-1$ with $f(0)=0, f(\infty)=1$ works fine. There are no singularity in this case, so it suffices to modify fun and set S=None.
Is there an issue with my code or should I use a different boundary value solver?
I read the equation from your source as
f''(r)+f'(r)/r-f(r)/r^2 = 2*lambda*eta^2*f(r)*(f(r)^2-1)
with the proposed parameters lambda=0.2, eta=2.
In the long-term limit, the left side reduces to the second derivative and the equation to a conservative system with a center at f=0 and two saddle points at f=+-1. The task is to find a solution curve that converges to a saddle point. In more practical terms, this is similar to the task to push a rigid pendulum in such a way that it ends up in the upright position, or moves ever closer to that position.
Writing f=1-g(r) for a solution approaching the saddle point at f=1, the equation is approximately
g''(r) = a^2*g(r), a^2=4*lambda*eta^2=3.2
This again characterizes this equilibrium as a saddle point, the solutions converging toward it satisfy the reduced ODE g'(r)=-a*g(r). This can be used as upper boundary condition. Translated into the state vector this gives
def bc(ya,yb):
return np.array([ya[0], yb[1]+a*max_x*(yb[0]-1)])
(replace the equilibrium constant 1 with asymptote if you want to stay with your version).
I got good results from that, with the parameters in the paper as well as with your parameters.
However, the singular mode of the solver seems broken, it inserts nodes up to the allowed max_nodes close to zero where the solution should be simply linear. I set the initial guess to
x=np.logspace(-5,np.log10(max_x),10)
x[0]=0
# guess for vector y at points x
y=[np.tanh(a*x),a*x/np.cosh(a*x)**2]
so that this maximal node number is not violated from the start.
Not using the singular mechanism, transferring the singular terms back into the ODE function and using that the finite solutions are almost linear in the initial segment, one can use f(r)=r*f'(r) as initial condition, ya[0]-ya[1] == 0. The interval start is then some small positive number. This results in reasonable node numbers in the solution, 25 for the default tolerance 1e-3 and 100 for tolerance 1e-6.
I have been trying to find an answer to this problem for a couple of hours now, but i can't find anything so far...
So I have two vectors let's call them b and x, of which i know all values. They add up to be the same amount, so sum(b) = sum(x).
I also have a Matrix, let's call it A, of which i know what values are 0, all the other values are unknown (but are different from 0).
Furthermore, the the elements of each column of A has the sum of 1 (I think that's called it's a left stochastic matrix)
Generally the Equation can be written in the form A*x = b.
Now I'm trying to find the missing values of A.
I have found one answer to the general problem here: https://math.stackexchange.com/questions/1170843/solving-ax-b-when-x-and-b-are-given
Furthermore i looked at the documentation of numpy.linalg
:https://docs.scipy.org/doc/numpy/reference/routines.linalg.html, but i just can't figure out how to do it.
It looks similar to a multi linear regression problem, but also on sklearn, i couldn't find anything: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression
Not a complete answer, but a bit of a more formal statement of the problem.
I think this can be solved as just a system of linear equations. Let
NZ = {(i,j)|a(i,j) is not fixed to zero}
Then write:
sum( j | (i,j) ∈ NZ, a(i,j) * x(j) ) = b(i) ∀i
sum( i | (i,j) ∈ NZ, a(i,j)) = 1 ∀j
This is just a system of linear equations in a(i,j). It may be under- (or over-) determined and it may be sparse. I think it depends a bit on this how to solve it. It may possible to think about these as constraints in a linear (or quadratic) programming problem. That would allow you to add an objective (in case of an underdetermined system or overdetermined -- in that case minimize sum of squared deviations, or 1-norm of deviations). In addition we can add bounds on a(i,j) (e.g. lower bounds of zero and upper bounds of one). So a linear programming approach may be what you are looking for.
This problem looks a bit like matrix balancing. This is used a lot for economic data sets that come from different sources and where we want to reconcile the data to get a consistent data set usable for subsequent modeling.
My question is how can I put a weighted least squares problem into a python solver. I'm trying to implement the approaches in the paper found here (PDF warning). There is an overview of the problem at the bottom of the post.
Specifically I want to start with the following minimization equation (19 in the paper):
latex formula can be found here:
\frac{min}{\Theta \epsilon M} \sum_{j=1}^{n} \sum_{i=1}^{m}(w(i,j))\left | \Psi(i,j)*\Theta (i,j) - I(i,j) \right |^{2}
It is represented as a weighted least squares problem.
w, psi, and I are my knowns, and I am trying to solve for theta.
I tried at first creating a function that takes a theta and returns the sum of this equation exactly as it's expressed above. Then I passed it to scipy.optimize.least_squares, but the theta values always remained the same after optimization. I tried implementing a jacobian, but the resulting sum explodes to huge negative values. It also takes ages as I'm attempting to run this on images (I is the pixel value for a pixel j with light i).
I then realized I'm almost certainly misunderstanding how to solve this problem and could use some help approaching it. My current code is below:
def theta_solver(self, theta):
imshape = self.images.shape
sm = 0
for j in j_array:
for i in i_array:
w = self.get_w(i, j, theta)
psi = self.non_diff_smoothing(self.get_psi(i, j))
diff = psi*(theta[i, j]) - self.I[i, j]
res = w*(diff)
sm += res
return sm
def solve_theta(self, theta_guess):
res = scipy.optimize.least_squares(self.theta_solver, theta_guess)
Something tells me I'm way off base for how I'm approaching this problem, and I could use a finger in the right direction. Thanks for your time.
Problem overview:
This particular vision approach is called photometric stereo. By taking several images of a scene with different light sources, we can create a 3D reconstruction of that scene.
One issue is the 1/r^2 decay in lighting is dependent on distance from the light source, which means this can't be solved by normal linear solutions.
The approach documented in the paper is a nonlinear approach for solving near light photometric stereo. It does two things:
it solves the surface Z, and
the albedos/intensities at each pixel represented by theta, by alternating the solvers.
In this question I'm only trying to solve the theta element of the equation, which can be solved via weighted least squares.
Turns out I was heavily overthinking the problem. This can be decomposed to a simple linear solution of the form Ax = b. When looking at an error equation, in this case:
argmin(THETA) sum(W * ||PSI * THETA - I||^2)
we can just distribute the weight through the parts within the root mean square. Our equation ends up being:
W * PSI * THETA = W * I
Which we can solve using your favorite linear solver (i.e. conjugate gradient descent)
I am using the Exemplar-Based algorithm by Criminisi. In section 3 of his paper it describes the algorithm. The target region that needs to be inpainted is denoted as Ω (omega). The border or the contour of Ω where it meets the rest of the image (denoted as Φ(phi)), is δΩ (delta omega).
Now on page four of the paper, it states that np(n subscript p) is the normal to the contour of δΩ. and ▽Ip (also includes orthogonal superscript) is the isophote at point p, which is the gradient turned 90 degrees.
My multivariable calculus is rusty, but how do we go about computing np and ▽Ip with python libraries? Also isn't np different for each point p on δΩ?
There are different ways of computing those variables, all depending in your numeric description of that boundary. n_p is the normal direction of the contour.
Generally, if your contour is described with an analytic equation, or if you can write an analytic equation that approximates the contour (e.g. a spline curve that fits 5 points (2 in each side of the point you want), you can derive that spline, compute the tangent line in the point you want using
Then, get a unit vector among that line and get the orthonormal vector to that one. All this is very easy to do (ask if you don't understand).
Then you have the isophone. It looks like its a vector orthonormal of the gradient with its modulus. Computing the directional gradient on an image is very very commonly used technique in image processing. You can get the X and Y derivatives of the image easily (hint: numpy.gradient, or SO python gradient). Then, the total gradient of the image is described as:
So just create a vector with the x and y gradients (taken from numpy.gradient). Then get the orthogonal vector to that one.
NOTE: How to get an orthogonal vector in 2D
[v2x v2y] = [v1y, -v1x]
I've generated a bunch of data for the (x,y,z) coordinates of a planet as it orbits around the Sun. Now I want to fit an ellipse through this data.
What I tried to do:
I created a dummy ellipse based on five parameters: The semi-major axis & eccentricity that defines the size & shape and the three euler angles that rotate the ellipse around. Since my data is not always centered at origin I also need to translate the ellipse requiring additional three variables (dx,dy,dz).
Once I initialise this function with these eight variables I get back N number of points that lie on this ellipse. (N = number of data points I am plotting the ellipse through)
I calculate the deviation of these dummy points from the actual data and then I minimise this deviation using some minimisation method to find the best fitting values for these eight variables.
My problem is with the very last part: minimising the deviation and finding the variables' values.
To minimise the deviation I use scipy.optimize.minimize to try and approximate the best fitting variables but it just doesn't do good enough of a job:
Here is an image of what one of my best fits looks like and that's with a very generously accurate initial guess. (blue = data, red = fit)
Here is the entire code. (No data required, it generates its own phony data)
In short, I use this scipy function:
initial_guess = [0.3,0.2,0.1,0.7,3,0.0,-0.1,0.0]
bnds = ((0.2, 0.5), (0.1, 0.3), (0, 2*np.pi), (0, 2*np.pi), (0, 2*np.pi), (-0.5,0.5), (-0.5,0.5), (-0.3,0.3)) #reasonable bounds for the variables
result = optimize.minimize(deviation, initial_guess, args=(data,), method='L-BFGS-B', bounds=bnds, tol=1e-8) #perform minimalisation
semi_major,eccentricity,inclination,periapsis,longitude,dx,dy,dz = result["x"]
To minimize this error (or deviation) function:
def deviation(variables, data):
"""
This function calculates the cumulative seperation between the ellipse fit points and data points and returns it
"""
num_pts = len(data[:,0])
semi_major,eccentricity,inclination,periapsis,longitude,dx,dy,dz = variables
dummy_ellipse = generate_ellipse(num_pts,semi_major,eccentricity,inclination,periapsis,longitude,dz,dy,dz)
deviations = np.zeros(len(data[:,0]))
pair_deviations = np.zeros(len(data[:,0]))
# Calculate separation between each pair of points
for j in range(len(data[:,0])):
for i in range(len(data[:,0])):
pair_deviations[i] = np.sqrt((data[j,0]-dummy_ellipse[i,0])**2 + (data[j,1]-dummy_ellipse[i,1])**2 + (data[j,2]-dummy_ellipse[i,2])**2)
deviations[j] = min(pair_deviations) # only pick the closest point to the data point j.
total_deviation = sum(deviations)
return total_deviation
(My code may be a bit messy & inefficient, I'm new to this)
I may be making some logical error in my coding but I think it comes down to the scipy.minimize.optimize function. I don't know enough about how it works and what to expect of it. I was also recommended to try Markov chain Monte Carlo when dealing with this many variables. I did take a look at the emcee, but it's a little above my head right now.
First, you have a typo in your objective function that prevents optimization of one of the variables:
dummy_ellipse = generate_ellipse(...,dz,dy,dz)
should be
dummy_ellipse = generate_ellipse(...,dx,dy,dz)
Also, taking sqrt out and minimizing the sum of squared euclidean distances makes it numerically somewhat easier for the optimizer.
Your objective function is also not everywhere differentiable because of the min(), as assumed by the BFGS solver, so its performance will be suboptimal.
Also, approaching the problem from analytical geometry perspective may help: an ellipse in 3d is defined as a solution of two equations
f1(x,y,z,p) = 0
f2(x,y,z,p) = 0
Where p are the parameters of the ellipse. Now, to fit the parameters to a data set, you could try to minimize
F(p) = sum_{j=1}^N [f1(x_j,y_j,z_j,p)**2 + f2(x_j,y_j,z_j,p)**2]
where the sum goes over data points.
Even better, in this problem formulation you could use optimize.leastsq, which may be more efficient in least squares problems.