Part of my code inverts a matrix (really an ndarray) using numpy.linalg.inv. However, this frequently errors out as follows:
numpy.linalg.linalg.LinAlgError: Singular matrix
That would be fine if the matrix was actually singular. But that doesn't seem to be the case.
For example, I'm printing the matrix before trying to invert it. So right before the error it prints this:
[[ 0.76400334 0.22660491]
[ 0.22660491 0.06721147]]
... and then returns the above singularity error when it tries to invert that matrix. But from what I can tell this matrix is invertible. Numpy even seems to agree when asked later.
>>> numpy.linalg.inv([[0.76400334, 0.22660491], [0.22660491, 0.06721147]])
array([[ 2.88436275e+07, -9.72469076e+07],
[ -9.72469076e+07, 3.27870046e+08]])
Here's the exact code snippet:
print np.dot(np.transpose(X), X)
print np.linalg.inv(np.dot(np.transpose(X),X))
The first line prints the matrix above; the second line fails.
So what distinguishes the two actions above? Why does the stand-alone code work even though it errors out in my script?
EDIT: Per Colonel Beauvel's request, if I do
try:
print np.dot(np.transpose(X), X)
z = np.linalg.inv(np.dot(np.transpose(X), X))
except:
z = "whoops"
print z
it outputs
[[ 0.01328185 0.1092696 ]
[ 0.1092696 0.89895982]]
whoops
but trying this on its own I get
>>> numpy.linalg.inv([[0.01328185, 0.1092696], [0.1092696, 0.89895982]])
array([[ 2.24677775e+08, -2.73098420e+07],
[ -2.73098420e+07, 3.31954382e+06]])
It's a matter of printing precision. The IEEE 754 doubles, that you're most likely using, have about 16 decimal digits of precision and you need to write out 17 to preserve the binary value.
Here's a small example. First create a singlular matrix:
In [1]: import numpy as np
In [2]: np.random.seed(0)
In [3]: a, b, c = np.random.rand(3)
In [4]: d = b*c / a
In [5]: X = np.array([[a, b],[c, d]])
Print and try to invert it:
In [6]: X
Out[6]:
array([[ 0.5488135 , 0.71518937],
[ 0.60276338, 0.78549444]])
In [7]: np.linalg.inv(X)
LinAlgError: Singular matrix
Try to invert the printed matrix:
In [8]: Y = np.array([[ 0.5488135 , 0.71518937],
...: [ 0.60276338, 0.78549444]])
In [9]: np.linalg.inv(Y)
Out[9]:
array([[-85805775.2940297 , 78125795.99532071],
[ 65844615.19517545, -59951242.76033063]])
Succes!
Increase printing precision and try again:
In [10]: np.set_printoptions(precision=17)
In [11]: X
Out[11]:
array([[ 0.54881350392732475, 0.71518936637241948],
[ 0.60276337607164387, 0.78549444195576024]])
In [12]: Z = np.array([[ 0.54881350392732475, 0.71518936637241948],
...: [ 0.60276337607164387, 0.78549444195576024]])
In [13]: np.linalg.inv(Z)
LinAlgError: Singular matrix
I just compute the determinant:
In [130]: m = np.array([[ 0.76400334, 0.22660491],[ 0.22660491,0.06721147]])
In [131]: np.linalg.det(m)
Out[131]: 2.3302017068132921e-09
# which is in fact for a 2D matrix 0.76400334*0.06721147 - 0.22660491*0.22660491
Which is already quit close to 0.
If a matrix m can be inverted, mathematically you can compute the adjoint and divide by the determinant to get the inverted matrix.
Numerically if the determinant is too small, this can entail the kind of error you have ...
Related
I was trying to implement the matrix exponential function as in scipy.linalg.expm. I gained inspiration from kaityo256's github repository. I thus wrote down the following.
from scipy.linalg import expm
from scipy.linalg import eigh
from scipy.linalg import inv
from math import exp as math_exp
from numpy import array, zeros
from numpy.random import random_sample
from numpy.testing import assert_allclose
def diag2sqr(x):
'''Makes an square matrix from a diagonal one.
Takes a 1d matrix. Determines its data type.
Finds out the shape of the 1d matrix.
Makes an empty square matrix with both
dimensions equal to largest (nonzero) dimension of
the 1d matrix. It then fills the elements of the
1d matrix into diagonal slots of the empty
square one.
Parameters
----------
x : ndarray
ndarray of be coverted to a square ndarray
Returns
-------
xsqr : ndarray
ndarray with diagonals sameas those of x
all other elements are zero
dtype same as that of x
'''
x_flat = x.ravel()
xsqr = zeros((x_flat.shape[0], x_flat.shape[0]), dtype=x.dtype)
# Making the empty matrix
for i in range(x_flat.shape[0]):
xsqr[i, i] = x_flat[i]
# filling up the ith element
print('xsqr', xsqr)
return xsqr
def kaityo_expm(x, ):
'''Exponentiates an ndarray (kaityo).
Exponentiates a ndarray in the most naive way.
Parameters
----------
x : ndarray
The ndarray to be exponentiated
Returns
-------
kexpm : ndarray
x after exponentiating
'''
rx, ux = eigh(x)
# Find eigenvalues and eigenvectors
# eigenvectors composed to form a unitary
ux_inv = inv(ux)
# Inverse of the unitary
# tx = diag([array([math_exp(i) for i in rx]).ravel()])
# tx = array([math_exp(i) for i in rx])
tx = diag2sqr(array([math_exp(i) for i in rx]))
# Constructing the diagonal matrix
kexpm1 = tx#ux_inv
kexpm = ux#kexpm1
return kexpm
Afterwards, I tried to test the above code versus scipy.linalg.expm.
x = random_sample((10, 10))
assert_allclose(expm(x), kaityo_expm(x))
This leads to the following output.
AssertionError:
Not equal to tolerance rtol=1e-07, atol=0
Mismatch: 100%
Max absolute difference: 7.04655733
Max relative difference: 0.59875635
x: array([[18.032424, 16.224408, 12.432163, 16.614248, 12.85653 , 13.705387,
15.096966, 10.577946, 18.399573, 17.938062],
[16.352809, 17.525898, 12.79079 , 16.295562, 13.512996, 14.407979,...
y: array([[18.649103, 13.157682, 11.264763, 16.099163, 15.2293 , 17.854499,
11.691586, 13.412066, 15.023189, 15.598455],
[13.157682, 13.612502, 9.628261, 12.659313, 13.559437, 13.382417,..
Obviously, both the implementations differ.
The questions are as follows:
Is it acceptable for them to differ?
Is my implementation wrong?
If my implementation is wrong, how do I fix it?
If my implementation is correct, when is it safe to use scipy.linalg.expm?
I have seen the following questions:
Matrix exponentiation with scipy: expm, expm2 and expm3
from a mathematical approach the definition of exponential of a matrix is made using the Taylor series of the exponential, so:
let A be a diagonal square matrix:
the problem arise when A is a generic square matrix, so before doing the exponential you will need do diagonalize it using eigenvalue and eigenvectors:
with U the matrix of eigenvectors and Lambda the matrix with the eigenvalues on the diagonal.
at this point we are close to finding what is an exponential of a matrix:
now lets implement this result in a simple script:
>>> import numpy as np
>>> import scipy.linalg as ln
>>> A = [[2/3, -4/3, 2],
[5/6, 4/3, -2],
[5/6, -2/3, 0]]
>>> A = np.matrix(A)
>>> print(A)
[[ 0.66666667 -1.33333333 2. ]
[ 0.83333333 1.33333333 -2. ]
[ 0.83333333 -0.66666667 0. ]]
>>> eigvalue, eigvectors = np.linalg.eig(A)
>>> print("eigvalue: ", eigvalue)
>>> print("eigvectors:")
>>> print(eigvectors)
eigvalue: [ 1. -1. 2.]
eigvectors:
[[ 0.81649658 0.27216553 0.87287156]
[ 0.40824829 -0.68041382 -0.21821789]
[ 0.40824829 -0.68041382 0.43643578]]
>>> e_Lambda = np.eye(np.size(A, 0))*(np.exp(eigvalue))
>>> print(e_Lambda)
[[2.71828183 0. 0. ]
[0. 0.36787944 0. ]
[0. 0. 7.3890561 ]]
>>> e_A = eigvectors*e_Lambda*eigvectors.I
>>> print(e_A)
[[ 2.3265481 -6.22769903 7.01116649]
[ 0.97933433 4.27520659 -3.51559341]
[ 0.97933433 -3.11384951 3.87346269]]
>>> e_A2 = ln.expm(A)
>>> print(e_A2)
[[ 2.3265481 -6.22769903 7.01116649]
[ 0.97933433 4.27520659 -3.51559341]
[ 0.97933433 -3.11384951 3.87346269]]
>>> np.testing.assert_allclose(e_A, e_A2)
>>> print(e_A - e_A2)
[[-1.77635684e-15 1.77635684e-15 -8.88178420e-16]
[ 4.44089210e-16 -1.77635684e-15 8.88178420e-16]
[-2.22044605e-16 0.00000000e+00 4.44089210e-16]]
we see that the result is basically the same, so i think it's safe to use scipy.linalg.expm for matrix exponentiation.
i created a repo with the notebook for further testing.
i want to solve this linear equation in python
import numpy as np
x2=264
x1=266
x3=294
y2=270
y1=240
y3=227
fract=(x2-x1)*(y3-y1)-(y2-y1)*(x3-x1)
A = np.matrix([[fract-(y3-y1)*(x3-x1)+(y2-y1)*(x2-x1),((x3-x1)**2)-(x2-x1)**2],[((y2-y1)**2)-(y3-y1)**2,fract+(y3-y1)*(x3-x1)-(y2-y1)*(x2-x1)]])
B = np.matrix([[(fract+(y3-y1)*(x3-x1)-(y2-y1)*(x2-x1))], [y1*fract+(y2-y1)*(x1*y2-y1*x2)+(y3-y1)*(x3*y1-y3*x1)]])
A_inverse = np.linalg.inv(A)
X = A_inverse * B
print (X)
LinAlgError: Singular matrix
This is explained simply by printing A:
[[ -510 780]
[ 731 -1118]]
Both cofactors are 570180, so the determinant is 0.
As the error message tells you, the matrix is singular, which means there is no unique solution: either none or infinite, depending on the constants applied.
Using numpy, given a square matrix, A and column vector x, use np.linalg.solve to compute A^(−1)x.
The documentation provides a simple example
>>>a = np.array([[3,1], [1,2]])
>>>b = np.array([9,8])
>>>x = np.linalg.solve(a, b)
>>>x
array([ 2., 3.])
But I do not see how this problem relates and can be applied to the given problem to solve?
I have a large matrix A of shape (n, n, 3, 3) with n is about 5000. Now I want find the inverse and transpose of matrix A:
import numpy as np
A = np.random.rand(1000, 1000, 3, 3)
identity = np.identity(3, dtype=A.dtype)
Ainv = np.zeros_like(A)
Atrans = np.zeros_like(A)
for i in range(1000):
for j in range(1000):
Ainv[i, j] = np.linalg.solve(A[i, j], identity)
Atrans[i, j] = np.transpose(A[i, j])
Is there a faster, more efficient way to do this?
This is taken from a project of mine, where I also do vectorized linear algebra on many 3x3 matrices.
Note that there is only a loop over 3; not a loop over n, so the code is vectorized in the important dimensions. I don't want to vouch for how this compares to a C/numba extension to do the same thing though, performance wise. This is likely to be substantially faster still, but at least this blows the loops over n out of the water.
def adjoint(A):
"""compute inverse without division by det; ...xv3xc3 input, or array of matrices assumed"""
AI = np.empty_like(A)
for i in xrange(3):
AI[...,i,:] = np.cross(A[...,i-2,:], A[...,i-1,:])
return AI
def inverse_transpose(A):
"""
efficiently compute the inverse-transpose for stack of 3x3 matrices
"""
I = adjoint(A)
det = dot(I, A).mean(axis=-1)
return I / det[...,None,None]
def inverse(A):
"""inverse of a stack of 3x3 matrices"""
return np.swapaxes( inverse_transpose(A), -1,-2)
def dot(A, B):
"""dot arrays of vecs; contract over last indices"""
return np.einsum('...i,...i->...', A, B)
A = np.random.rand(2,2,3,3)
I = inverse(A)
print np.einsum('...ij,...jk',A,I)
for the transpose:
testing a bit in ipython showed:
In [1]: import numpy
In [2]: x = numpy.ones((5,6,3,4))
In [3]: numpy.transpose(x,(0,1,3,2)).shape
Out[3]: (5, 6, 4, 3)
so you can just do
Atrans = numpy.transpose(A,(0,1,3,2))
to transpose the second and third dimensions (while leaving dimension 0 and 1 the same)
for the inversion:
the last example of http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.inv.html#numpy.linalg.inv
Inverses of several matrices can be computed at once:
from numpy.linalg import inv
a = np.array([[[1., 2.], [3., 4.]], [[1, 3], [3, 5]]])
>>> inv(a)
array([[[-2. , 1. ],
[ 1.5, -0.5]],
[[-5. , 2. ],
[ 3. , -1. ]]])
So i guess in your case, the inversion can be done with just
Ainv = inv(A)
and it will know that the last two dimensions are the ones it is supposed to invert over, and that the first dimensions are just how you stacked your data. This should be much faster
speed difference
for the transpose: your method needs 3.77557015419 sec, and mine needs 2.86102294922e-06 sec (which is a speedup of over 1 million times)
for the inversion: i guess my numpy version is not high enough to try that numpy.linalg.inv trick with (n,n,3,3) shape, to see the speedup there (my version is 1.6.2, and the docs i based my solution on are for 1.8, but it should work on 1.8, if someone else can test that?)
Numpy has the array.T properties which is a shortcut for transpose.
For inversions, you use np.linalg.inv(A).
As posted by wim A.I also works on matrix. e.g.
print (A.I)
for numpy-matrix object, use matrix.getI.
e.g.
A=numpy.matrix('1 3;5 6')
print (A.getI())
I have two one-dimensional numpy matrices:
[[ 0.69 0.41]] and [[ 0.81818182 0.18181818]]
I want to multiply these two to get the result
[[0.883, 0.117]] (the result is normalized)
If I use np.dot I get ValueError: matrices are not aligned
Does anybody have an idea what I am doing wrong?
EDIT
I solved it in a kind of hacky way, but it worked for me, regardless of if there is a better solution or not.
new_matrix = np.matrix([ a[0,0] * b[0,0], a[0,1] * b[0,1] ])
It seems you want to do element-wise math. Numpy arrays do this by default.
In [1]: import numpy as np
In [2]: a = np.matrix([.69,.41])
In [3]: b = np.matrix([ 0.81818182, 0.18181818])
In [4]: np.asarray(a) * np.asarray(b)
Out[4]: array([[ 0.56454546, 0.07454545]])
In [5]: np.matrix(_)
Out[5]: matrix([[ 0.56454546, 0.07454545]])