Related
numpy version 1.20.1
Bob lawblaw's Law Blog, I need more details, this post has tooooo much code.
def quadratic_formula(a, b, c):
"""just like the song
Args:
a (numpy.ndarray): shape(N,1)
b (numpy.ndarray): shape(N,1)
c (numpy.ndarray): shape(N,1)
Returns:
numpy.ndarray: shape(N,2)
* [soln_a, soln_b]
* [np.NaN, np.NaN] when discriminant is negative
* [soln_a, soln_a] single soln when discriminate == 0]
Notes:
.. math::
ax^2 + bx + c = 0
solns = \\frac{-b \\pm \\sqrt{b^2 -4ac}}{2a}
"""
# TODO: raise value error if any a == 0
a = np.array(a)
b = np.array(b)
c = np.array(c)
det = b ** 2 - 4 * a * c
res_a = np.where(
det >= 0,
(-b + np.sqrt(det)) / (2 * a),
np.NaN,
)
res_b = np.where(
det >= 0,
(-b - np.sqrt(det)) / (2 * a),
np.NaN,
)
res = np.array([res_a, res_b]).T
return res
a = [1,2]
b = [1,0]
c = [0,1]
res = quadratic_formula(a,b,c)
print(res)
>>> [[0, -1],
[NaN, NaN]]
works, but raise RuntimeWarning: invalid value encountered in sqrt.
Why is the square root even evaluated for a negative discriminant?
Any suggestions for implementation?
Note that you are still computing np.sqrt(det) for all values of det hence the warning. The where filters the x and y arrays after they have been computed.
The implementation can be fixed by simply casting the a,b and c arrays to complex.
a = np.array(a).astype(complex)
b = np.array(b).astype(complex)
c = np.array(c).astype(complex)
That way numpy knows to use the complex version of sqrt. Once you are there you can completely omit the np.where and check after the fact if your solutions are real, if that is what you are interested on only.
I have the following code for a symbolic matrix calculation
from sympy import *
A = MatrixSymbol( 'A', 3, 3 )
B = MatrixSymbol( 'B', 3, 3 )
C = MatrixSymbol( 'C', 3, 3 )
Z = A * ( A + B ) * C.inverse() * ( A + B ).transpose()
Z.expand()
This gives me the following result
A( A + B ) C^-1 ( A^T + B^T )
But I want the expanded result
A * A * C^-1 * A^T + A * B C^-1 * A^T + A * A C^-1 * B^T + A * B C^-1 * B^T
Could I ask you guys that how to do it ? Thank you very much
Currently, there is no way to expand matrix operations in sympy. The expression from the result you have above is the only way to view the result. Expansion functions available in sympy include expand(), expand_trig(), expand_power_exp(), expand_power_base(), expand_log(), expand_func() and hyperexpand(), none of which seem to work with your matrix expression.
The simplify function doesn't seem to work either if we wanted to evaluate your desired output to compare to the sympy output of Z
simplify(A*A*C.inverse()*A.transpose() + A*A*C.inverse()*B.transpose() + A*B*C.inverse()*A.transpose() + A*B*C.inverse()*B.transpose())
>>> A*A*C^-1*A.T + A*A*C^-1*B.T + A*B*C^-1*A.T + A*B*C^-1*B.T
which clearly isn't in the same form as the sympy expression for Z.
Even if you were to just test your desired output against the output of sympy, you still get a False result
Z == A * A * C.inverse() * A.transpose() + A * B*C.inverse() * A.transpose() + A * A*C.inverse() * B.transpose() + A * B*C.inverse() * B.transpose()
>>> False
We cannot define matrices A, B and C with known values for each index and then evaluate the expressions you have above to compare their outputs because MutableDenseMatrix objects do not have an inverse attribute.
If you want to actually compare the two expressions to see if they are equal, you could use numpy and compare both expressions with known matrix values. This is quite convoluted and wont work for very complex expressions, but it does work for simple expressions.
e.g.
import numpy as np
# define 3 numpy arrays
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
B = np.array([[0,0.1,-4],[19,0.67,6],[-1,99,5]])
C = np.array([[1,2,-1],[4,5,6],[-2,2,0]])
# sympy expression, compute iteratively
Z1 = np.dot(A, A + B)
Z1 = np.dot(Z1, np.linalg.inv(C))
Z1 = np.dot(Z1, A.T + B.T)
print(Z1)
>>>[[ 143.22088889 -672.5076 -100.24 ]
[-239.47685185 4558.45416667 3597.5 ]
[ 65.1457037 -94.81393333 519.56 ]]
# desired expression
p1 = np.dot(A, A)
p1 = np.dot(p1, np.linalg.inv(C))
p1 = np.dot(p1, A.T)
p2 = np.dot(A, B)
p2 = np.dot(p2, np.linalg.inv(C))
p2 = np.dot(p2, A.T)
p3 = np.dot(A, A)
p3 = np.dot(p3, np.linalg.inv(C))
p3 = np.dot(p3, B.T)
p4 = np.dot(A, B)
p4 = np.dot(p4, np.linalg.inv(C))
p4 = np.dot(p4, B.T)
Z2 = p1 + p2 + p3 + p4
print(Z2)
>>>[[ 143.22088889 -672.5076 -100.24 ]
[-239.47685185 4558.45416667 3597.5 ]
[ 65.1457037 -94.81393333 519.56 ]]
np.isclose(Z1,Z2)
>>>[[ True, True, True],
[ True, True, True],
[ True, True, True]
So we can see that the sympy expression Z1 and your desired expression Z2 are in fact the same, given the matrices above.
Starting with:
a,b=np.ogrid[0:n+1:1,0:n+1:1]
B=np.exp(1j*(np.pi/3)*np.abs(a-b))
B[z,b] = np.exp(1j * (np.pi/3) * np.abs(z - b +x))
B[a,z] = np.exp(1j * (np.pi/3) * np.abs(a - z +x))
B[diag,diag]=1-1j/np.sqrt(3)
this produces an n*n grid that acts as a matrix.
n is just a number chosen to represent the indices, i.e. an a*b matrix where a and b both go up to n.
Where z is a constant I choose to replace a row and column with the B[z,b] and B[a,z] formulas. (Essentially the same formula but with a small number added to the np.abs(a-b))
The diagonal of the matrix is given by the bottom line:
B[diag,diag]=1-1j/np.sqrt(3)
where,
diag=np.arange(n+1)
I would like to repeat this code 50 times where the only thing that changes is x so I will end up with 50 versions of the B np.ogrid. x is a randomly generated number between -0.8 and 0.8 each time.
x=np.random.uniform(-0.8,0.8)
I want to generate 50 versions of B with random values of x each time and take a geometric average of the 50 versions of B using the definition:
def geo_mean(y):
y = np.asarray(y)
return np.prod(y ** (1.0 / y.shape[0]), axis=-1)
I have tried to set B as a function of some index and then use a for _ in range(): loop, this doesn't work. Aside from copy and pasting the block 50 times and denoting each one as B1, B2, B3 etc; I can't think of another way of working this out.
EDIT:
I'm now using part of a given solution in order to show clearly what I am looking for:
#A matrix with 50 random values between -0.8 and 0.8 to be used in the loop
X=np.random.uniform(-0.8,0.8, (50,1))
#constructing the base array before modification by random x values in position z
a,b = np.ogrid[0:n+1:1,0:n+1:1]
B = np.exp(1j * ( np.pi / 3) * np.abs( a - b ))
B[diag,diag] = 1 - 1j / np.sqrt(3)
#list to store all modified arrays
randomarrays = []
for i in range( 0,50 ):
#copy array and modify it
Bnew = np.copy( B )
Bnew[z, b] = np.exp( 1j * ( np.pi / 3 ) * np.abs(z - b + X[i]))
Bnew[a, z] = np.exp( 1j * ( np.pi / 3 ) * np.abs(a - z + X[i]))
randomarrays.append(Bnew)
Bstack = np.dstack(randomarrays)
#calculate the geometric mean value along the axis that was the row in 2D arrays
B0 = geo_mean(Bstack)
From this example, every iteration of i uses the same value of X, I can't seem to get a way to get each new loop of i to use the next value in the matrix X. I am unsure of the ++ action in python, I know it does not work in python, I just don't know how to use the python equivalent. I want a loop to use a value of X, then the next loop to use the next value and so on and so forth so I can dstack all the matrices at the end and find a geo_mean for each element in the stacked matrices.
One pedestrian way would be to use a list comprehension or generator expression:
>>> def f(n, z, x):
... diag = np.arange(n+1)
... a,b=np.ogrid[0:n+1:1,0:n+1:1]
... B=np.exp(1j*(np.pi/3)*np.abs(a-b))
... B[z,b] = np.exp(1j * (np.pi/3) * np.abs(z - b +x))
... B[a,z] = np.exp(1j * (np.pi/3) * np.abs(a - z +x))
... B[diag,diag]=1-1j/np.sqrt(3)
... return B
...
>>> X = np.random.uniform(-0.8, 0.8, (10,))
>>> np.prod((*map(np.power, map(f, 10*(4,), 10*(2,), X), 10 * (1/10,)),), axis=0)
But in your concrete example we can do much better than that;
using the identity exp(a) x exp(b) = exp(a + b) we can convert the geometric mean after exponentiation to an arithmetic mean before exponentition. A bit of care is required because of the multivaluedness of the complex n-th root which occurs in the geometric mean. In the code below we normalize the angles occurring to range -pi, pi so as to always hit the same branch as the n-th root.
Please also note that the geo_mean function you provide is definitely wrong. It fails the basic sanity check that taking the average of copies of the same thing should return the same thing. I've provided a better version. It is still not perfect, but I think there actually is no perfect solution, because of the nonuniqueness of the complex root.
Because of this I recommend taking the average before exponentiating. As long as your random spread is less than pi this allows a well-defined averaging procedure with an average that is actually close to the samples
import numpy as np
def f(n, z, X, do_it_pps_way=True):
X = np.asanyarray(X)
diag = np.arange(n+1)
a,b=np.ogrid[0:n+1:1,0:n+1:1]
B=np.exp(1j*(np.pi/3)*np.abs(a-b))
X = X.reshape(-1,1,1)
if do_it_pps_way:
zbx = np.mean(np.abs(z-b+X), axis=0)
azx = np.mean(np.abs(a-z+X), axis=0)
else:
zbx = np.mean((np.abs(z-b+X)+3) % 6 - 3, axis=0)
azx = np.mean((np.abs(a-z+X)+3) % 6 - 3, axis=0)
B[z,b] = np.exp(1j * (np.pi/3) * zbx)
B[a,z] = np.exp(1j * (np.pi/3) * azx)
B[diag,diag]=1-1j/np.sqrt(3)
return B
def geo_mean(y):
y = np.asarray(y)
dim = len(y.shape)
y = np.atleast_2d(y)
v = np.prod(y, axis=0) ** (1.0 / y.shape[0])
return v[0] if dim == 1 else v
def geo_mean_correct(y):
y = np.asarray(y)
return np.prod(y ** (1.0 / y.shape[0]), axis=0)
# demo that orig geo_mean is wrong
B = np.exp(1j * np.random.random((5, 5)))
# the mean of four times the same thing should be the same thing:
if not np.allclose(B, geo_mean([B, B, B, B])):
print('geo_mean failed')
if np.allclose(B, geo_mean_correct([B, B, B, B])):
print('but geo_mean_correct works')
n, z, m = 10, 3, 50
X = np.random.uniform(-0.8, 0.8, (m,))
B0 = f(n, z, X, do_it_pps_way=False)
B1 = np.prod((*map(np.power, map(f, m*(n,), m*(z,), X), m * (1/m,)),), axis=0)
B2 = geo_mean_correct([f(n, z, x) for x in X])
# This is the recommended way:
B_recommended = f(n, z, X, do_it_pps_way=True)
print()
print(np.allclose(B1, B0))
print(np.allclose(B2, B1))
I think you should rely more on numpy functionality, when approaching your problem. Not a numpy expert myself, so there is surely room for improvement:
from scipy.stats import gmean
n = 2
z = 1
a = np.arange(n + 1).reshape(1, n + 1)
#constructing the base array before modification by random x values in position z
B = np.exp(1j * (np.pi / 3) * np.abs(a - a.T))
B[a, a] = 1 - 1j / np.sqrt(3)
#list to store all modified arrays
random_arrays = []
for _ in range(50):
#generate random x value
x=np.random.uniform(-0.8, 0.8)
#copy array and modify it
B_new = np.copy(B)
B_new[z, a] = np.exp(1j * (np.pi / 3) * np.abs(z - a + x))
B_new[a, z] = np.exp(1j * (np.pi / 3) * np.abs(a - z + x))
random_arrays.append(B_new)
#store all B arrays as a 3D array
B_stack = np.stack(random_arrays)
#calculate the geometric mean value along the axis that was the row in 2D arrays
geom_mean_for_rows = gmean(B_stack, axis = 2)
It uses the geometric mean function from scipy.stats module to have a vectorised approach for this calculation.
I have a 3D numpy array A of shape (2133, 3, 3). Basically this is a list of 2133 lists with three 3D points. Furthermore I have a function which takes three 3D points and returns one 3D point, x = f(a, b, c), with a, b, c, x numpy arrays of length 3. Now I want to apply f to A, so that the output is an array of shape (2133, 3). So something like numpy.array([f(*A[0]),...,f(*A[2132])).
I tried numpy.apply_along_axis and numpy.vectorize without success.
To be more precise the function f I consider is given by:
def f(a, b, c, r1, r2=None, r3=None):
a = np.asarray(a)
b = np.asarray(b)
c = np.asarray(c)
if np.linalg.matrix_rank(np.matrix([a, b, c])) != 3:
# raise ValueError('The points are not collinear.')
return None
a, b, c, = sort_triple(a, b, c)
if any(r is None for r in (r2, r3)):
r2, r3 = (r1, r1)
ex = (b - a) / (np.linalg.norm(b - a))
i = np.dot(ex, c - a)
ey = (c - a - i*ex) / (np.linalg.norm(c - a - i*ex))
ez = np.cross(ex, ey)
d = np.linalg.norm(b - a)
j = np.dot(ey, c - a)
x = (pow(r1, 2) - pow(r2, 2) + pow(d, 2)) / (2 * d)
y = ((pow(r1, 2) - pow(r3, 2) + pow(i, 2) + pow(j, 2)) / (2*j)) - ((i/j)*x)
z_square = pow(r1, 2) - pow(x, 2) - pow(y, 2)
if z_square >= 0:
z = np.sqrt(z_square)
intersection = a + x * ex + y*ey + z*ez
return intersection
A = np.array([[[131.83, 25.2, 0.52], [131.51, 22.54, 0.52],[133.65, 23.65, 0.52]], [[13.02, 86.98, 0.52], [61.02, 87.12, 0.52],[129.05, 87.32, 0.52]]])
r1 = 1.7115
Thanks to the great help of #jdehesa I was able to produce an alternative solution to the one given by #hpaulj. I am not sure if this solution is the most elegant one but it worked so far. Comments are appreciated.
def sort_triple(a, b, c):
pts = np.stack((a, b, c), axis=1)
xSorted = pts[np.arange(pts.shape[0])[:, None], np.argsort(pts[:, :, 0])]
orientation = np.cross(xSorted[:, 1] - xSorted[:, 0], xSorted[:, 2] -
xSorted[:, 0])[:, 2] >= 0
xSorted_flipped = np.stack((xSorted[:, 0], xSorted[:, 2], xSorted[:, 1]),
axis=1)
xSorted = np.where(orientation[:, np.newaxis, np.newaxis], xSorted,
xSorted_flipped)
return map(np.squeeze, np.split(xSorted, 3, axis=1))
def f(A, r1, r2=None, r3=None):
a, b, c = map(np.squeeze, np.split(A, 3, axis=1))
a, b, c = sort_triple(a, b, c)
if any(r is None for r in (r2, r3)):
r2, r3 = (r1, r1)
ex = (b - a) / (np.linalg.norm(b - a, axis=1))[:, np.newaxis]
i = inner1d(ex, (c - a))
ey = ((c - a - i[:, np.newaxis]*ex) /
(np.linalg.norm(c - a - i[:, np.newaxis]*ex, axis=1))[:, np.newaxis])
ez = np.cross(ex, ey)
d = np.linalg.norm(b - a, axis=1)
j = inner1d(ey, c - a)
x = (np.square(r1) - np.square(r2) + np.square(d)) / (2 * d)
y = ((np.square(r1) - np.square(r3) + np.square(i) + np.square(j)) / (2*j) -
i/j*x)
z_square = np.square(r1) - np.square(x) - np.square(y)
mask = z_square < 0
z_square[mask] *= 0
z = np.sqrt(z_square)
z[mask] = np.nan
intersection = (a + x[:, np.newaxis] * ex + y[:, np.newaxis] * ey +
z[:, np.newaxis] * ez)
return intersection
Probably the map parts in each function could be done better. Maybe also the excessive use of np.newaxis.
This works fine (after commenting out sort_triple):
res = [f(*row,r1) for row in A]
print(res)
producing:
[array([ 132.21182324, 23.80481826, 1.43482849]), None]
That looks like one row produced a (3,) array, the other had some sort of problem and produced None. I don't know if that None was due to removing the sort or not. But in any case, turning a mix of arrays and None back into an array would be a problem. If all items of res were matching arrays, we could stack them back into a 2d array.
There are ways of getting modest speed improvements (compared to this list comprehension). But with a complex function like this, the time spent in the function (called 2000 times) dominates the time spent by the iteration mechanism.
And since you are iterating on the 1st dimension, and passing the other 2 (as 3 arrays), this explicit loop is a lot easier to use than vectorize, frompyfunc or apply_along/over...
To get significant time savings you have to write f() to work with the 3d array directly.
I'm using Python+Numpy (can maybe also use Scipy) and have three 2D points
(P1, P2, P3);
I am trying to get the distance from P3 perpendicular to a line drawn between P1 and P2. Let P1=(x1,y1), P2=(x2,y2) and P3=(x3,y3)
In vector notation this would be pretty easy, but I'm fairly new to python/numpy and can't get anythng that works (or even close).
Any tips appreciated, thanks!
Try using the norm function from numpy.linalg
d = norm(np.cross(p2-p1, p1-p3))/norm(p2-p1)
np.cross returns the z-coordinate of the cross product only for 2D vectors. So the first norm in the accepted answer is not needed, and is actually dangerous if p3 is an array of vectors rather than a single vector. Best just to use
d=np.cross(p2-p1,p3-p1)/norm(p2-p1)
which for an array of points p3 will give you an array of distances from the line.
For the above-mentioned answers to work, the points need to be numpy arrays, here's a working example:
import numpy as np
p1=np.array([0,0])
p2=np.array([10,10])
p3=np.array([5,7])
d=np.cross(p2-p1,p3-p1)/np.linalg.norm(p2-p1)
To find distance to line from point if you have slope and intercept you can use formula from wiki
https://en.wikipedia.org/wiki/Distance_from_a_point_to_a_line
Python:
def distance(point,coef):
return abs((coef[0]*point[0])-point[1]+coef[1])/math.sqrt((coef[0]*coef[0])+1)
coef is a tuple with slope and intercept
Based on the accepted answer
Test with below line equation -
Find the perpendicular distance from the point (5, 6) to the line −2x + 3y + 4 = 0
x-intercept p1 = [0, -4/3]
y-intercept p2 = [2, 0]
shortest distance from p3 = [5, 6] = 3.328
import numpy as np
norm = np.linalg.norm
p1 = np.array([0,-4/3])
p2 = np.array([2, 0])
p3 = np.array([5, 6])
d = np.abs(norm(np.cross(p2-p1, p1-p3)))/norm(p2-p1)
# output d = 3.328201177351375
abs((x2-x1)*(y1-y0) - (x1-x0)*(y2-y1)) / np.sqrt(np.square(x2-x1) + np.square(y2-y1))
Can be used directly through the formula, just have to plug in the values and boom it will work.
Shortest Distance from Point to a Line
This is the code I got from https://www.geeksforgeeks.org:
import math
# Function to find distance
def shortest_distance(x1, y1, a, b, c):
d = abs((a * x1 + b * y1 + c)) / (math.sqrt(a * a + b * b))
print("Perpendicular distance is", d)
Now you have to find A, B, C, x, and y.
import numpy as np
closest = []
x = (x ,y)
y = (x, y)
coef = np.polyfit(x, y, 1)
A = coef[0]
B = coef[1]
C = A*x[0] + B*x[1]
Now you can plug in the values:
shortest_dis = shortest_distance(x, y, A, B, C)
The full code may look like this:
import math
import numpy as np
def shortest_distance(x1, y1, a, b, c):
d = abs((a * x1 + b * y1 + c)) / (math.sqrt(a * a + b * b))
print("Perpendicular distance is", d)
closest = []
x = (x ,y)
y = (x, y)
coef = np.polyfit(x, y, 1)
A = coef[0]
B = coef[1]
C = A*x[0] + B*x[1]
shortest_dis = shortest_distance(x, y, A, B, C)
Please let me know if any of this is unclear.
Cross products are helpful for the 2D case, but they do not generalize well to other dimensions. Dot products do however. The dot product of two orthogonal vectors is zero in any space, which you can use to come up with a simple solution.
Let's say you have P4 on the same line as P1-P2. You could parametrize it with parameter t such that
P4 = P1 + t * (P2 - P1)
The goal is to find P4 such that
(P3 - P4) . (P2 - P1) == 0
Expanding P4 in terms of t and simplifying:
(P3 - P1 - t * (P2 - P1)) . (P2 - P1) == 0
(P3 - P1) . (P2 - P1) == t * ||P2 - P1||^2
t = (P3 - P1) . (P2 - P1) / ||P2 - P1||^2
You therefore have
D = ||P3 - P4|| = ||P3 - (P3 - P1) . (P2 - P1) / (||P2 - P1||^2)||
I've written a function in my library of utility routines called haggis. You can use haggis.math.segment_distance to compute the distance to the entire line (not just the bounded line segment) like this:
d = haggis.math.segment_distance(P3, P1, P2, segment=False)
3D distance should use np.dot
def threeD_corres(points_3_d,pre_points_3_d,points_camera):
for j in range (0,len(pre_points_3_d)):
vec1 = list(map(lambda x:x[0]- x[1],zip(pre_points_3_d[j], points_camera)))
vec2 = list(map(lambda x:x[0]- x[1],zip(pre_points_3_d[j], points_3_d[j])))
vec3 = list(map(lambda x:x[0]- x[1],zip(points_3_d[j], points_camera)))
distance = np.abs(np.dot(vec1_1,vec2_2))/np.linalg.norm(vec3)
print("#########distance:\n",distance)
return distance