I try to solve eigenvalue and eigenvector by QR iteration, the code is super simple. But the answer by QR iteration always has some opposite or incorrect values compare to the answer by linalg.eigs.
import numpy as np
import scipy.linalg as linalg
def qr_iteration(A):
for i in range(100):
Q, R = linalg.qr(A)
A = np.dot(R, Q)
return np.diag(R), Q
a, b = linalg.eig(A)
c, d = qr_iteration(A)
print(a) # [ 1.61168440e+01+0.j -1.11684397e+00+0.j -1.30367773e-15+0.j]
print(c) # [-1.61168440e+01 1.11684397e+00 -1.33381856e-15]
Some of the values are opposite to the correct answer
Which part of my code is wrong?
Thanks for all the answers.
The final eigenvalues should be found as the diagonal elements of A instead of R (change the return statement to np.diag(A)). Moreover, the order of the eigenvalues appearing in the diagonal may differ from other algorithms.
Are you dealing with real symmetric matrices? If not, the eigenvalues may be complex and QR algorithm should not be applied. If it has complex eigenvalues, those eigenvalues come in pairs having the same magnitude and the algorithm will not converge. You will never get imaginary numbers using your procedure.
To get the eigenvectors, you need to multiply all the Q's, i.e. $Q1 Q2 Q3...$; the column vectors are the corresponding eigenvectors (don't know how to type latex here)
Related
Let us say I have a block matrix that is 28 x 28 in dimensionality. Like so:
B = np.zeros((14,14))
C = np.zeros((14,14))
D = np.zeros((14,14))
Ct = C.T
A = np.block([[B,C], [Ct, D]])
I proceed to fill up this matrix, which is irrelevant to the question I am asking so just imagine that we have random values inside of the matrix instead. I want to obtain the eigenvalues and eigenvectors of A so I use numpy.linalg for that
vals, vecs = np.linalg.eig(A)
The documentation of np.linalg.eig says that the eigenvalues are "not necessarily ordered" which I find really ambiguous.
If I use numpy's np.linalg.eig to diagonalize the matrix, I get back the respective eigenvalues and eigenvectors, but "not necessarily ordered".
I don't know what order np.linalg.eig chooses to store the vectors and values but if I want the resulting eigenvalues to be sorted in terms of "where" they come from within the block matrix, how can I achieve this?
I am using NumPy's linalg.eig on square matrices. My square matrices are a function of a 2D domain, and I am looking at its eigenvectors' complex angles along a parameterized circle on this domain. As long as the path I am considering is smooth, I expect the complex angles of each eigenvector's components to be smooth. However, for some cases, this is not the case with Python (although it is with other programming languages). For the parameter M=0 (some argument in my matrix that appears on its diagonal), I have components that look like:
when they should ideally look like (M=0.1):
What I have tried:
I verified that the matrices are Hermitian in both cases.
When I use linalg.eigh, M=0.1 becomes discontinuous while M=0 sometimes becomes continuous.
Using np.unwrap did nothing.
The difference between component phases (i.e. np.angle(v1-v2) for eigenvector v=[[v1],[v2]]) is smooth/continuous, but this is not what I want.
Fixing the NumPy seed before solving did nothing for different values of the seed. For example: np.random.seed(1).
What else can I do? I am trying to use Sympy's eigenvects just because I am running out of options, and I asked another question asking about another potential approach here: How do I force first component of NumPy eigenvectors to be real? . But, I do not know what else I can try.
Here is a minimal working example that works nicely in a Jupyter notebook:
import numpy as np
from numpy import linalg as LA
import matplotlib.pyplot as plt
M = 0.01; # nonzero M is okay
M = 0.0; # M=0 causes problems
def matrix_generator(kx,ky,M):
a = 2.46; t = 1; k = np.array((kx,ky));
d1 = (a/2)*np.array((1,np.sqrt(3)));d2 = (a/2)*np.array((1,-np.sqrt(3)));d3 = -a*np.array((1,0));
sx = np.matrix([[0,1],[1,0]]);sy = np.matrix([[0,-1j],[1j,0]]);sz = np.matrix([[1,0],[0,-1]]);
hx = np.cos(k#d1)+np.cos(k#d2)+np.cos(k#d3);hy = np.sin(k#d1)+np.sin(k#d2)+np.sin(k#d3);
return -t*(hx*sx - hy*sy + M*sz)
n_segs = 200; #number of segments in (kx,ky) loop
evecs_along_loop = np.zeros((n_segs,2,2),dtype=float)
# parameterize circular loop
kx0 = 0.5; ky0 = 1; r1=0.2; r2=0.2;
a = np.linspace(0.0, 2*np.pi, num=n_segs+2)
kloop=np.zeros((n_segs+2,2))
for i in range(n_segs+2):
kloop[i,:]=np.array([kx0 + r1*np.cos(a[i]), ky0 + r2*np.sin(a[i])])
# assign eigenvector complex angles
for j in np.arange(n_segs):
np.random.seed(2)
H = matrix_generator(kloop[j][0],kloop[j][1],M)
eval0, psi0 = LA.eig(H)
evecs_along_loop[j,:,:] = np.angle(psi0)
# plot eigenvector complex angles
for p in np.arange(2):
for q in np.arange(2):
print(f"Phase for eigenvector element {p},{q}:")
fig = plt.figure()
ax = plt.axes()
ax.plot((evecs_along_loop[:,p,q]))
plt.show()
Clarification for anon01's comment:
For M=0, a sample matrix at some value of (kx,ky) would look like:
a = np.matrix([[0.+0.j, 0.99286437+1.03026667j],
[0.99286437-1.03026667j, 0.+0.j]])
For M =/= 0, the diagonal will be non-zero (but real).
I think that in general this is a tough problem. The fundamental issue is that eigenvectors (unlike eigenvalues) are not unambiguously defined. An eigenvector v of M with eigenvalue c is any non-zero vector for which
M*v = c*v
In particular for any non zero scalar s, multiplying an eigenvector by s yields an eigenvector, and even if you demand (as usual) that eigenvectors have length 1, we are still free to multiply by any scalar of absolute value 1. Even worse, if v1,..vd are orthogonal eigenvectors for c, then any non-zero linear combination of the v's is also an eigenvector for c.
Different eigendecomposition routines might well, therefore, come up with very different eigenvectors and still be doing their job. Moreover some routines might produce eigenvectors that are far apart for matrices that are close together.
A simple tractable case is where you know that all your eigenvalues are non-degenerate (i.e. each eigenspace is of dimension 1) and you happen to know that for a particular i, the i'th component of each eigenvector will be non zero. Then you could multiply the eigenvector v by a scalar, of absolute value 1, chosen so that after the multiplication v[i] is a positive real number. In C
s = conj(v[i])/cabs(v[i])
where
conj(z) is the complex conjugate of the complex number z,
and cabs(z) is the absolute value of the complex number z
Note that the above supposes that we are using the same index for every eigenvector, though the factor s varies from eigenvector to eigenvector.
This would impose a uniqueness on the eigenvectors, and, one would hope, mean that they varied continuously with the parameters of your matrix.
I am trying to find the top k leading eigenvalues of a numpy matrix (using python dot product notation)
L#L + a*Y#Y.T, where L and Y are a symmetric nxn and an nxd matrix, respectively.
According to the below text from this paper, I should be able to calculate these leading eigenvalues with L#(L#v) + a*X#(X.T#v), where I guess v is an arbitrary vector. The Lanczos paper they cite is here.
I'm not quite sure where to start. I know that scipy has scipy.sparse.linalg.eigsh here, and from the notes it looks like it uses the Lanczos algorithm - but I am at a loss as to whether it's possible to use sparse.linalg.eigsh for my specific use case. I googled around and didn't find a Python implementation for this very quickly -- does anybody know if I can use sparse.linalg.eigsh to calculate this somehow? I definitely don't want to write this algorithm myself.
I also wasn't sure whether to post this in math.stackexchange or here, since it's a question about the Python implementation of a very mathy thing.
You could check scipy.sparse.linalg.eigsh.
import numpy as np;
from scipy.sparse.linalg import eigsh;
from numpy.linalg import eigh
a = 1.4
n = 20;
d = 7;
# random symmetric n x n matrix
L = np.random.randn(n, n)
L = L + L.T
# random n x d matrix
Y = np.random.randn(n, d)
A = L # L.T + a * Y # Y.T # your equation
A must be positive-definite to use eigsh, this is guaranteed to be true if a>0.
You could check the four eigenvalues as follows
eigsh(La, 4)[0]
For reference you can compare based on numpy.linalg.eigh that compute all the eigenvalues. Sort them, and take the last four elements of the sorted array, the results should be close.
np.sort(eigh(La)[0])[-4:]
I am a newbie when it comes to using python libraries for numerical tasks. I was reading a paper on LexRank and wanted to know how to compute eigenvectors of a transition matrix. I used the eigval function and got a result that I have a hard time interpreting:
a = numpy.zeros(shape=(4,4))
a[0,0]=0.333
a[0,1]=0.333
a[0,2]=0
a[0,3]=0.333
a[1,0]=0.25
a[1,1]=0.25
a[1,2]=0.25
a[1,3]=0.25
a[2,0]=0.5
a[2,1]=0.0
a[2,2]=0.0
a[2,3]=0.5
a[3,0]=0.0
a[3,1]=0.333
a[3,2]=0.333
a[3,3]=0.333
print LA.eigval(a)
and the eigenvalue is:
[ 0.99943032+0.j
-0.13278637+0.24189178j
-0.13278637-0.24189178j
0.18214242+0.j ]
Can anyone please explain what j is doing here? Isn't the eigenvalue supposed to be a scalar quantity? How can I interpret this result broadly?
j is the imaginary number, the square root of minus one. In math it is often denoted by i, in engineering, and in Python, it is denoted by j.
A single eigenvalue is a scalar quantity, but an (m, m) matrix will have m eigenvalues (and m eigenvectors). The Wiki page on eigenvalues and eigenvectors has some examples that might help you to get your head around the concepts.
As #unutbu mentions, j denotes the imaginary number in Python. In general, a matrix may have complex eigenvalues (i.e. with real and imaginary components) even if it contains only real values (see here, for example). Symmetric real-valued matrices are an exception, in that they are guaranteed to have only real eigenvalues.
In my attempt to perform cholesky decomposition on a variance-covariance matrix for a 2D array of periodic boundary condition, under certain parameter combinations, I always get LinAlgError: Matrix is not positive definite - Cholesky decomposition cannot be computed. Not sure if it's a numpy.linalg or implementation issue, as the script is straightforward:
sigma = 3.
U = 4
def FromListToGrid(l_):
i = np.floor(l_/U)
j = l_ - i*U
return np.array((i,j))
Ulist = range(U**2)
Cov = []
for l in Ulist:
di = np.array([np.abs(FromListToGrid(l)[0]-FromListToGrid(i)[0]) for i, x in enumerate(Ulist)])
di = np.minimum(di, U-di)
dj = np.array([np.abs(FromListToGrid(l)[1]-FromListToGrid(i)[1]) for i, x in enumerate(Ulist)])
dj = np.minimum(dj, U-dj)
d = np.sqrt(di**2+dj**2)
Cov.append(np.exp(-d/sigma))
Cov = np.vstack(Cov)
W = np.linalg.cholesky(Cov)
Attempts to remove potential singularies also failed to resolve the problem. Any help is much appreciated.
Digging a bit deeper in problem, I tried printing the Eigenvalues of the Cov matrix.
print np.linalg.eigvalsh(Cov)
And the answer turns out to be this
[-0.0801339 -0.0801339 0.12653595 0.12653595 0.12653595 0.12653595 0.14847999 0.36269785 0.36269785 0.36269785 0.36269785 1.09439988 1.09439988 1.09439988 1.09439988 9.6772531 ]
Aha! Notice the first two negative eigenvalues? Now, a matrix is positive definite if and only if all its eigenvalues are positive. So, the problem with the matrix is not that it's close to 'zero', but that it's 'negative'. To extend #duffymo analogy, this is linear algebra equivalent of trying to take square root of negative number.
Now, let's try to perform same operation, but this time with scipy.
scipy.linalg.cholesky(Cov, lower=True)
And that fails saying something more
numpy.linalg.linalg.LinAlgError: 12-th leading minor not positive definite
That's telling something more, (though I couldn't really understand why it's complaining about 12-th minor).
Bottom line, the matrix is not quite close to 'zero' but is more like 'negative'
The problem is the data you're feeding to it. The matrix is singular, according to the solver. That means a zero or near-zero diagonal element, so inversion is impossible.
It'd be easier to diagnose if you could provide a small version of the matrix.
Zero diagonals aren't the only way to create a singularity. If two rows are proportional to each other then you don't need both in the solution; they're redundant. It's more complex than just looking for zeroes on the diagonal.
If your matrix is correct, you have a non-empty null space. You'll need to change algorithms to something like SVD.
See my comment below.