I have problem with understanding this piece of code which based on the output, I guess it computes the eigenvector of the matrix.
def simplexProj(y):
"""
Given y, computes its projection x* onto the simplex
Delta = { x | x >= 0 and sum(x) <= 1 },
that is, x* = argmin_x ||x-y||_2 such that x in Delta.
x = SimplexProj(y)
****** Input ******
y : input vector.
****** Output ******
x : projection of y onto Delta.
"""
if len(y.shape) == 1: # Reshape to (1,-1) if y is a vector.
y = y.reshape(1, -1) # row vector
x = y.copy()
x[x < 0] = 0 #element within the matrix that is negative will be replaced with 0, python2 feature
K = np.flatnonzero(np.sum(x, 0) > 1) #return indices that are non-zero in the flattened version of a ; sum of each column
# K gives the column index for column that has colum sum>1, True = 1, False = 0
x[:, K] = blockSimplexProj(y[:, K])
return x
def blockSimplexProj(y):
""" Same as function SimplexProj except that sum(max(Y,0)) > 1. """
r, c = y.shape
ys = -np.sort(-y, axis=0) #sort each column of the matrix with biggest entry on the first row
mu = np.zeros(c, dtype=float)
S = np.zeros((r, c), dtype=float)
for i in range(1, r): #1st to r-1th row
S[i, :] = np.sum(ys[:i, :] - ys[i, :], 0)
print(S)
colInd_ge1 = np.flatnonzero(S[i, :] >= 1)
colInd_lt1 = np.flatnonzero(S[i, :] < 1)
if len(colInd_ge1) > 0:
mu[colInd_ge1] = (1 - S[i - 1, colInd_ge1]) / i - ys[i - 1, colInd_ge1]
if i == r:
mu[colInd_lt1] = (1 - S[r, colInd_lt1]) / (r + 1) - ys[r, colInd_lt1]
x = y + mu
x[x < 0] = 0
return x
I'm a bit puzzle by the step computing the matrix S because according to the code, the row of first row of S should be all 0. Take for example the matrix A = np.array([[25,70,39,10,80],[12,45,32,89,43],[67,24,84,39,21],[0.1,0.2,0.3,0.035,0.06]]) The 3 iterations (i=1,2,3) are computed as expected but then there is an extra step which seemingly gives back S as basis of eigenvectors. It would be great if somebody can help me with understanding this problem. Also I#m not sure what's the name of this algorithm (how S is computed)
Related
I am trying to numerically compute in python integrals of the form
To that aim, I first define two discrete sets of x and t values, let's say
x_samples = np.linspace(-10, 10, 100)
t_samples = np.linspace(0, 1, 100)
dx = x_samples[1]-x_samples[0]
dt = t_samples[1]-t_samples[0]
declare symbolically that the function g(x,t) is equal to 0 if t<0 and discretise the two functions to integrate as
discretG = g(x_samples[None, :], t_samples[:, None])
discretH = h(x_samples[None, :], t_samples[:, None])
I have then tried to run
discretF = signal.fftconvolve(discretG, discretH, mode='full') * dx * dt
Yet, on basic test functions such as
g(x,t) = lambda x,t: np.exp(-np.abs(x))+t
h(x,t) = lambda x,t: np.exp(-np.abs(x))-t
I don't find an agreement between the the numerical integration and the convolution using scipy and I would like to have a fairly fast way of computing these integrals, especially when I only have access to discretised representations of the functions rather than their symbolic one.
According to your code, I assume you want to conduct convolution on two function g and h that are non-zero only on [a, b]*[m,n].
Of course you can use signal.fftconvolve to compute the convolution. The key is don't forget the transformation between the indices inside discretF and the real coordinates. Here I use interpolation to compute for arbitrary (x,t).
import numpy as np
from scipy import signal, interpolate
a = -1
b = 2
m = -10
n = 15
samples_num = 1000
x_eval_index = 200
t_eval_index = 300
x_samples = np.linspace(a, b, samples_num)
t_samples = np.linspace(m, n, samples_num)
dx = x_samples[1]-x_samples[0]
dt = t_samples[1]-t_samples[0]
g = lambda x,t: np.exp(-np.abs(x))+t
h = lambda x,t: np.exp(-np.abs(x))-t
discretG = g(x_samples[None, :], t_samples[:, None])
discretH = h(x_samples[None, :], t_samples[:, None])
discretF = signal.fftconvolve(discretG, discretH, mode='full')
def compute_f(x, t):
if x < 2*a or x > 2*b or t < 2*m or t > 2*n:
return 0
# use interpolation t get data on new point
x_samples_for_conv = np.linspace(2*a, 2*b, 2*samples_num-1)
t_samples_for_conv = np.linspace(2*m, 2*n, 2*samples_num-1)
f = interpolate.RectBivariateSpline(x_samples_for_conv, t_samples_for_conv, discretF.T)
return f(x, t)[0, 0] * dx * dt
Note: you can extend my codes to compute convolution on a meshgrid defined by x and y, where x and y are 1D array. (In my code, x and y are float now)
You can use the following code to explore the "agreement" between "the numerical integration" and "the convolution using scipy" (and also, the correctness of compute_f function above)：
# how the convolve work
# for 1D f[i]=sigma_{j} g[j]h[i-j]
sum = 0
for y_idx, y in enumerate(x_samples[0:]):
for s_idx, s in enumerate(t_samples[0:]):
if x_eval_index - y_idx < 0 or t_eval_index - s_idx < 0:
continue
if t_eval_index - s_idx >= len(x_samples[0:]) or x_eval_index - y_idx >= len(t_samples[0:]):
continue
sum += discretG[t_eval_index - s_idx, x_eval_index - y_idx] * discretH[s_idx, y_idx] * dx * dt
print("Do discrete convolution manually, I get: %f" % sum)
print("Do discrete convolution using scipy, I get: %f" % (discretF[t_eval_index, x_eval_index] * dx * dt))
# numerical integral
# the x_val and t_val
# take 1D convolution as example, function defined on [a, b], and index of your samples range from [0, samples_num-1]
# after convolution, function defined on [2a, 2b], index of your samples range from [0, 2*samples_num-2]
dx_prime = (b-a) / (samples_num-1)
dt_prime = (n-m) / (samples_num-1)
x_eval = 2*a + x_eval_index * dx_prime
t_eval = 2*m + t_eval_index * dt_prime
sum = 0
for y in x_samples[:]:
for s in t_samples[:]:
if x_eval - y < a or x_eval - y > b:
continue
if t_eval - s < m or t_eval - s > n:
continue
if y < a or y >= b:
continue
if s < m or s >= n:
continue
sum += g(x_eval - y, t_eval - s) * h(y, s) * dx * dt
print("Do numerical integration, I get: %f" % sum)
print("The convolution result of 'compute_f' is: %f" % compute_f(x_eval, t_eval))
Which gives:
Do discrete convolution manually, I get: -154.771369
Do discrete convolution using scipy, I get: -154.771369
Do numerical integration, I get: -154.771369
The convolution result of 'compute_f' is: -154.771369
I have this function to get determinant of matrix
def determinant(self) -> int:
"""
Calculates the Determinant of matrix objects.
Parameters
----------
self
Returns
-------
int
Example
-------
>>> _matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> _matrix = Matrix(_matrix)
>>> _matrix.determinant()
0
"""
if self.row != self.column:
raise ValueError('Cannot get determinant of this matrix! Must be a square Matrix')
else:
def det(matrix):
row = len(matrix)
col = len(matrix[0])
if (row, col) == (1, 1):
return matrix[0][0]
# hard coding for 2x2
elif (row, col) == (2, 2):
return matrix[0][0] * matrix[1][1] - matrix[0][1] * matrix[1][0]
# using sarrus method to solve for 3x3, it's a little faster.
elif (row, col) == (3, 3):
matrix1 = matrix[:]
# Extending matrix to use Sarrus Rule.
for i in range(row - 1):
_col = []
for j in range(col):
_col.append(matrix1[i][j])
matrix1.append(_col)
# Calculating Determinant
# Adding part
add_pointers = [(i, i) for i in range(row)]
result = 0
for pointer in range(row):
temp = 1
for tup in add_pointers:
i, j = tup
temp *= matrix1[i + pointer][j]
result += temp
# Subtracting part
sub_pointers = [((row - 1) - i, 0 + i) for i in range(row)]
for pointers in range(row):
temp = 1
for tup in sub_pointers:
i, j = tup
temp *= matrix1[i + pointers][j]
result -= temp
return result
else:
sign = -1
result = 0
row1 = [matrix[0][i] * (sign ** i) for i in range(col)]
for x, y in enumerate(row1):
mat = matrix[:][1:]
sub_matrix = [[mat[i][j] for j in range(col) if j != x] for i in range(row - 1)]
result += y * det(sub_matrix)
return result
return det(self.matrix)
i have hard-coded determinant of 2x2 and 3x3 matrix, then im recusing through the rest
as u can see its using recursion of nxn matrix(s)... i'm sure there is a faster way, this is extremely slow
A python implementation of the method would be recommended, thank you
The most common best ways would be either list comprehension or the numpy module.
Reason: The for loops will almost certainly be slower than a numpy array simply because of the contiguous and homogeneous nature of a numpy array. In simple terms numpy is basically one memory block all of the same type, where as a list points to different memory blocks and can contain any type.
Here is the numpy example (for 2d):
import numpy as np
a = np.array([[1, 2], [3, 4]])
result = np.linalg.det(a)
print(result)
One of the comments already (correctly) points to this:
https://numpy.org/doc/stable/reference/generated/numpy.linalg.det.html
For more general larger m*n matricies, the advantages would be significant.
Find determinant for 3x3 matrix using the first row:
"""
M:
M11 M12 M13
M21 M22 M23
M31 M32 M33
detM:
M11 * det2D([ [M22, M23], [M32, M33] ]) -
M12 * det2D([ [M21, M23], [M31, M33] ]) +
M13 * det2D([ [M21, M22], [M31, M32] ])
"""
import numpy as np
def det3D(M):
a = M[0][0] * det2D(np.array([ [ M[1][1],M[1][2] ], [ M[2][1],M[2][2] ] ]))
b = M[0][1] * det2D(np.array([ [ M[1][0],M[1][2] ], [ M[2][0],M[2][2] ] ]))
c = M[0][2] * det2D(np.array([ [ M[1][0],M[1][1] ], [ M[2][0],M[2][1] ] ]))
return a - b + c
def det2D(M):
return M[0][0]*M[1,1] - M[0][1] * M[1][0]
M = [ [1,0,0], [0,2,2], [0,2,4] ]
A = det3D(M)
B = round(np.linalg.det(M))
print(A)
print(B)
print(A == B)
Output:
4
4
True
Find determinant of NxN Matrix using recursion:
Note: there are two methods for finding determinants, smartDetNxN run >35X faster than detNxN in the best case on a large matrix.
import numpy as np
# compute partial determinant terms
def terms(M, col = 1, row = 1):
return [x[:col-1] + x[col:] for x in M[0:row-1] + M[row:]]
# compute determinant using first row
def detNxN(M):
N = len(M[0])
# Recursion Base: 2x2 determenant
if (N == 2):
M = np.array(M)
return M[0][0] * M[1,1] - M[0][1] * M[1][0]
# Recursion Loop
else:
rowValues = M[:1][0]
colsSigns = [1 if (col % 2 == 0) else -1 for col in range(N)]
colsDets = [detNxN(terms(M, col + 1)) for col in range(N)]
return sum([rowValues[col] * colsSigns[col] * colsDets[col] for col in range(N)])
# compute determinant using optimum row while skipping zero value columns
def smartDetNxN(M):
N = len(M[0])
# Recursion Base: 2x2 determenant
if (N == 2):
M = np.array(M)
return M[0][0] * M[1,1] - M[0][1] * M[1][0]
# Recursion Loop
else:
# find optimun row
flatM = [len(np.flatnonzero(x)) for x in M]
row = flatM.index(min(flatM))
rowSign = 1 if (row % 2 == 0) else -1
rowValues = M[row]
# compute partial determinants
colsSigns = [1 if (col % 2 == 0) else -1 for col in range(N)]
colsDets = [smartDetNxN(terms(M, col + 1, row + 1)) if (rowValues[col] != 0) else 0 for col in range(N)]
return sum([rowValues[col] * rowSign * colsSigns[col] * colsDets[col] for col in range(N)])
# test case for matrix
def testCase(M):
print()
N1 = len(M[0])
N2 = len(M[0])
A = smartDetNxN(M)
B = round(np.linalg.det(M))
print("Matrix %ix%i:" % (N1, N2))
print("Actual detM = %d, Expected detM = %d " % (A, B))
print("Test Pass:", A == B)
# main
def main():
# Matrix 2 x 2
M1 = [[1,2,],[0,1]]
testCase(M1)
# Matrix 3 x 3
M2 = [[1,2,3],[2,1,2],[3,2,1]]
testCase(M2)
# Matrix 4 x 4
M3 = [[1,2,3,4], [2,1,0,3], [3,0,1,2], [4,0,0,1]]
testCase(M3)
# Matrix 10 x 10
M4 = [
[0,1,2,3,4,5,6,7,8,9],
[1,1,0,0,0,0,0,0,0,8],
[2,0,1,0,0,0,0,0,0,7],
[3,0,0,1,0,0,0,0,0,6],
[4,0,0,0,1,0,0,0,0,5],
[5,0,0,0,0,1,0,0,0,4],
[6,0,0,0,0,0,1,0,0,3],
[7,0,0,0,0,0,0,1,0,2],
[8,0,0,0,0,0,0,0,1,1],
[9,0,0,0,0,0,0,0,0,0],
]
testCase(M4)
main()
Output:
Matrix 2x2:
Actual detM = 1, Expected detM = 1
Test Pass: True
Matrix 3x3:
Actual detM = 8, Expected detM = 8
Test Pass: True
Matrix 4x4:
Actual detM = 20, Expected detM = 20
Test Pass: True
Matrix 10x10:
Actual detM = 999, Expected detM = 999
Test Pass: True
I'm trying to get my head around the example code on the wikipedia page for Laplacian matricies.
It is written in MatLab and I only have access to open source tools.
I think most of it is fairly straight forward but I'm having a bit of trouble on this one line.
C0V = V'*C0; % Transform the initial condition into the coordinate system
% of the eigenvectors
Now my maths isn't up to scratch to really understand what the comment means. Judging by the matlab website this appears to be a transposed matrix multiplied (inner product) by another matrix.
Where the left matrix (after transposing) is ( m x p ) and the right is ( p x n ).
The matrix V is produced by a call to eig a few lines above, which from this answer I am substituting scipy.linalg.eig.
The problem is that C0 is clearly defined as an N x N matrix (ndarray), but in my code V is an N**2 x N**2 matrix. I have no way of knowing what shape V is in the original code.
For reference the wikipedia code is below, and below that is my attempt to rewrite it in python (using scipy and numpy), followed by the error attributed to the line described above.
Matlab code from wikipedia
N = 20; % The number of pixels along a dimension of the image
A = zeros(N, N); % The image
Adj = zeros(N * N, N * N); % The adjacency matrix
% Use 8 neighbors, and fill in the adjacency matrix
dx = [- 1, 0, 1, - 1, 1, - 1, 0, 1];
dy = [- 1, - 1, - 1, 0, 0, 1, 1, 1];
for x = 1:N
for y = 1:N
index = (x - 1) * N + y;
for ne = 1:length(dx)
newx = x + dx(ne);
newy = y + dy(ne);
if newx > 0 && newx <= N && newy > 0 && newy <= N
index2 = (newx - 1) * N + newy;
Adj(index, index2) = 1;
end
end
end
end
% BELOW IS THE KEY CODE THAT COMPUTES THE SOLUTION TO THE DIFFERENTIAL EQUATION
Deg = diag(sum(Adj, 2)); % Compute the degree matrix
L = Deg - Adj; % Compute the laplacian matrix in terms of the degree and adjacency matrices
[V, D] = eig(L); % Compute the eigenvalues/vectors of the laplacian matrix
D = diag(D);
% Initial condition (place a few large positive values around and
% make everything else zero)
C0 = zeros(N, N);
C0(2:5, 2:5) = 5;
C0(10:15, 10:15) = 10;
C0(2:5, 8:13) = 7;
C0 = C0(:);
C0V = V'*C0; % Transform the initial condition into the coordinate system
% of the eigenvectors
for t = 0:0.05:5
% Loop through times and decay each initial component
Phi = C0V .* exp(- D * t); % Exponential decay for each component
Phi = V * Phi; % Transform from eigenvector coordinate system to original coordinate system
Phi = reshape(Phi, N, N);
% Display the results and write to GIF file
imagesc(Phi);
caxis([0, 10]);
title(sprintf('Diffusion t = %3f', t));
frame = getframe(1);
im = frame2im(frame);
[imind, cm] = rgb2ind(im, 256);
if t == 0
imwrite(imind, cm, 'out.gif', 'gif', 'Loopcount', inf, 'DelayTime', 0.1);
else
imwrite(imind, cm, 'out.gif', 'gif', 'WriteMode', 'append', 'DelayTime', 0.1);
end
end
My attempted translation
import numpy as np
import matplotlib.pyplot as plt
import scipy.linalg as la
N = 20 # The number of pixels along a dimension of the image
A = np.zeros((N, N)) # The image
Adj = np.zeros((N**2, N**2)) # The adjacency matrix
# Use 8 neighbors, and fill in the adjacency matrix
dx = [- 1, 0, 1, - 1, 1, - 1, 0, 1]
dy = [- 1, - 1, - 1, 0, 0, 1, 1, 1]
for x in range(N):
for y in range(N):
index = x * N + y
for ne in range(len(dx)):
newx = x + dx[ne]
newy = y + dy[ne]
if (newx >= 0 and newx < N
and newy >= 0 and newy < N):
index2 = newx * N + newy;
Adj[index, index2] = 1
# BELOW IS THE KEY CODE THAT COMPUTES THE SOLUTION TO THE DIFFERENTIAL EQUATION
Deg = np.diag(np.sum(Adj, 1)) # Compute the degree matrix
L = Deg - Adj # Compute the laplacian matrix in terms of the degree and adjacency matrices
D, V = la.eig(L) # Compute the eigenvalues/vectors of the laplacian matrix
D = np.diag(D)
# Initial condition (place a few large positive values around and
# make everything else zero)
C0 = np.zeros((N, N))
C0[1:4, 1:4] = 5
C0[9:14, 9:14] = 10
C0[1:5, 7:12] = 7
#C0 = C0(:) #This doesn't seem to do anything?
# matlab:C0V = V'*C0; % Transform the initial condition into the coordinate system
# of the eigenvectors
C0V = V.T * C0 # ???
Error
----------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-10-c3014dc90c9f> in <module>
2 # of the eigenvectors
3
----> 4 C0V = V.T * C0
ValueError: operands could not be broadcast together with shapes (400,400) (20,20)
Edit
It appears #hpaulj has identified my problem. The line I ommited is a ravel operation ie C0 = C0(:) takes a 20, 20 matrix to a 400, 1 vector. So I need
C0 = C0.ravel()
I'm basically trying to sum a gradient here, see the screenshots I have attached for a better idea. Rho is an nx1 input vector, the screenshot I have attached shows the idea for a 3x1 rho vector but it really has an undefined length.
enter image description here
enter image description here
# JACOBIAN
def derivative(rho, a, A, tilde_k, x, y, vecinc, chi):
n = rho.shape[0]
result1 = np.array([n,1],complex)
result2 = np.array([n,1],complex)
result = np.array([n,1],complex)
u = np.zeros((n, 3))
W_tilde = np.array([3,3],complex)
loop1 = 0
loop2 = 0
for i in range(n):
for j in range(n):
u[i] = x[i] - y[j] # n x 3
W_tilde = A_matrix * chi.imag * A_matrix * G(u[i],k) * A_matrix # 3 x 3
ei_block = np.exp(1j * np.vdot(x[i], tilde_k)) * vecinc # 3 x 1
ej_block = np.exp(1j * np.vdot(x[j], tilde_k)) * vecinc # 3 x 1
eiT_block = np.matrix.getH(ei_block) # 1 x 3
mm = np.matmul(W_tilde, ej_block) # (3 x 3)(3 x 1) = 3 x 1
alpha_tilde = np.dot(eiT_block, mm) # (1 x 3)(3 x 1) = 1 x 1 = scalar
loop1 = loop1 + (2 * rho[i] * alpha_tilde * rho[j]) # scalar
if (i != j):
loop2 = loop2 + ((rho[j]**2) * alpha_tilde) # scalar
result1[i] = loop1
result2[i] = loop2
result = result1 + result2 # (n x 1) + (n x 1) = n x 1 vector
return result
I am getting "IndexError: index 2 is out of bounds for axis 0 with size 2" for the line, result1[i] = loop1. Pls help :(
That error means that you are attempting to access the third element (index 2) of an array with only two elements (size 2).
It looks like you're defining your arrays in a funny way; np.array([n,1],complex) creates an array of length 2, not n. What you want is probably np.zeros(n,complex), which will create an n-length array filled with 0s.
I implemented Conjugate Gradient in python by looking into the Wikipedia reference - https://en.wikipedia.org/wiki/Conjugate_gradient_method
The implementation should solve for
ax = b
my application inputs goes as below,
a = <400x400 sparse matrix of type '<class 'numpy.float64'>'
with 1920 stored elements in Compressed Sparse Row format>
b = vector of shape (400, ) and dtype = float64
x = vector of random numbers of shape (400, )
Here is my implementation -
def ConjGrad(a, b, x):
r = (b - np.dot(np.array(a), x));
p = r;
rsold = np.dot(r.T, r);
for i in range(len(b)):
a_p = np.dot(a, p);
alpha = rsold / np.dot(p.T, a_p);
x = x + (alpha * p);
r = r - (alpha * a_p);
rsnew = np.dot(r.T, r);
if (np.sqrt(rsnew) < (10 ** -5)):
break;
p = r + ((rsnew / rsold) * p);
rsold = rsnew;
return p
When i call the above CG function, i get an error within the function for the line -
r = (b - np.dot(np.array(a), x));
The error goes like this -
NotImplementedError: subtracting a sparse matrix from a nonzero scalar is
not supported
At run time, below are the properties of variables within the CG function -
np.dot(np.array(a), x).shape
(400,)
b.shape
(400,)
I wonder why the subtraction is not happenning???
I tested the same function with the sample input arguments below and it worked fine.
a = np.array([[3, 2, -1], [2, -1, 1], [-1, 1, -1]]) # 3X3 symmetric matrix
b = (np.array([1, -2, 0])[np.newaxis]).T # 3X1 matrix
x = (np.array([0, 1, 2])[np.newaxis]).T
Can someone please tell me why its not working for a sparse matrix?
When multiplying a sparsa matrix by a array you should not use: np.dot(np.array(a), x)) but a.dot(x). See the documentation below:
https://docs.scipy.org/doc/scipy/reference/sparse.html
Follows a correct routine:
def conjGrad(A,x,b,tol,N):
r = b - A.dot(x)
p = r.copy()
for i in range(N):
Ap = A.dot(p)
alpha = np.dot(p,r)/np.dot(p,Ap)
x = x + alpha*p
r = b - A.dot(x)
if np.sqrt(np.sum((r**2))) < tol:
print('Itr:', i)
break
else:
beta = -np.dot(r,Ap)/np.dot(p,Ap)
p = r + beta*p
return x