I'm working on a project and i'm doing additions on diagonal submatrixes of a square matrix of big size (180x180) with a "kernel" square matrix of size around 40*40.
size1 = matrix1.shape[0] #square matrix shape 180x180
sizeKernel = kernelMatrix.shape[0] # square matrix shape 40x40
newMatrix1 = matrix1.copy()
np.fill_diagonal(newMatrix1, 0)
for i in range(size1):
newMatrix1[i:i+sizeKernel,i:i+sizeKernel] += matrix1[i][i] * kernelMatrix[:size1-i, :size1-i]
The goal is to distribute diagonal coefficients with the kernel matrix to other coefficients (a kind of partial convolution).
As an example, let's say the matrix is size 4x4 and the kernel is size 2x2:
a = [[1,2,1,0],
[3,-1,0,0],
[0,1,1,1],
[0,2,0,2]]
kern = [[1,2],
[0,1]]
we create the matrix res by copying every coef of a but the diagonal coefs:
res = [[0,2,1,0],
[3,0,0,0],
[0,1,0,1],
[0,2,0,0]]
we distribute every coefficient at diagonal to the bottom right submatrix of same size as kernel (if not possible we just take the remaining and cut the kernel matrix)
For the first coefficient of diagonal it gives:
res[0:2, 0:2] = res[0:2,0:2] + a[0,0]*kern
res[0:2, 0:2] = [[0,2] + 1 * [[1,2],
[3,0]] [0,1]]
res = [[1,4,1,0],
[3,1,0,0],
[0,1,0,1],
[0,2,0,0]]
Then we do the same idea for the other coefs. For the last coef of diagonal, instead of taking the whole kernel, we take apply the same thing extending the elements of the matrix with 0s.
As it is not vectorized, it takes quite a long time to compute this.
Another way to compute this would be to use directly the scipy method fftconvolve or convolve2d only on the diagonal matrix extracted, and add the result to the matrix with 0s on the diagonal:
newMatrix = matrix1.copy()
matrix1Size = matrix1.shape[0]
np.fill_diagonal(newMatrix , 0)
newMatrix += scipy.signal.fftconvolve(matrix1, kernelMatrix)[:matrix1Size,:matrixSize]
but it takes approximately as much time as the first method (of course their are more calculation if we do the full convolution of a whole matrix)
Is there any method to vectorize this calculation on submatrixes?
Thank you in advance!
Related
I try to check the equality or the inequality between 2 Fisher matrices.
The goal is too see if the projection (with Jacobian) and marginalisation (inversion of matrix and remove a row/column and reinversion) commute.
Each of these 2 matrices is computed slightly differently. These 2 matrices are Fisher matrices.
Actually, this is the computation of changing parameters between initial parameters for each row/colum and final parameters for the final matrix. That's why in both computations, I am using the Jacobian J formulating the derivatives between initial and final parameters :
The formula is : F_final = J^T F_initial J
The first matrix has size 5x5 and the second one has 4x4 size. They are identical except the 4th row/column.
First : I inverse the 5x5 matrix (which gives a 5x5 covariance matrix). Then, I "marginalise", that is to say, I remove the 4th row/column of this covariance matrix. Then, I inverse again to get a 4x4 matrix.
Finally, I perform a projection with a Jacobian 4x4 with formula : F_final = J^T F_initial J : so I get at the end a 4x4 matrix
For the second matrix to build : I am doing directly projection on 5x5 second matrix (which I recall is identical to the 5x5 except for the 4th row/column).
I perform this projection with the Jacobian 5x5. Then I get the second projected matrix 5x5. Finally, I remove the 4th row/column on this 5x5 matrix to get a 4x4 matrix new projected matrix.
I wonder under which conditions I could have equality between the 2 matrices 4x4. I don't know if my method is correct.
To show you a practical example, I put below a small Matlab script that tries to follow all the reasoning explained above :
clear;
clc;
% Big_31 Fisher :
FISH_Big_1_SYM = sym('sp_', [4,4], 'real');
% Force symmetry for Big_31
FISH_Big_1_SYM = tril(FISH_Big_1_SYM.') + triu(FISH_Big_1_SYM,1);
% Big_32 Fisher :
FISH_Big_2_SYM = sym('sp_', [5,5], 'real');
% Force symmetry for Big_32
FISH_Big_2_SYM = tril(FISH_Big_2_SYM.') + triu(FISH_Big_2_SYM,1);
% Jacobian 1
J_1_SYM = sym('j_', [4,4], 'real');
% Jacobian 2
J_2_SYM = sym('j_', [5,5], 'real');
% Remove 4th row/column
J_2_SYM(4,:) = [];
J_2_SYM(:,4) = [];
% Projection
FISH_proj_1 = J_1_SYM'*FISH_Big_1_SYM*J_1_SYM;
size(FISH_proj_1)
% Invert Fisher_2
COV_Big_2_SYM = inv(FISH_Big_2_SYM);
% Remove 4th row/column
COV_Big_2_SYM(4,:) = [];
COV_Big_2_SYM(:,4) = [];
% Re-inverse
FISH_Big_2_SYM_new = inv(COV_Big_2_SYM);
% Projection 2x2
FISH_proj_2 = J_2_SYM'*FISH_Big_2_SYM_new*J_2_SYM;
size(FISH_proj_2)
% Test equality between 2 matrices
isequal(FISH_proj_1,FISH_proj_2)
The problem with this script is even I have small matrices (4x4 or 5x5), the code takes a little bit long runtime but the result is that matrices are different.
Update
I gave some feedback from persons. An important point is at this portion of Matlab code :
When I do :
% Remove 4th row/column
J_2_SYM(4,:) = [];
J_2_SYM(:,4) = [];
I don't remove elements line j_5_1, j_5_2, j_5_3 of Jacobian J, these terms won't disappear when I do the projection. On the other side, these terms will remain in the other method, in the sense that I take into account of them.
So is it a lost battle ?
If yes, which modifications or assumptions could lead to have an equality ? i.e to have both operations do commute.
To do np.dot last dimension of first matrix must be the same as first dimension of second one. They are not, so you are getting ValueError, that shapes are not aligned.
Everything seems to be fine as you printed, but then you forgot about lines:
j_temp = np.copy(J_2_SYM)
# Add row/col into J_2_SYM
j_temp = np.insert(j_temp, 2, J_NEU_row_SYM[0,:], axis=0)
j_temp = np.insert(j_temp, 2, J_NEU_col_SYM[:,0], axis=1)
# Copy built J_2_SYM
J_2_SYM = np.copy(j_temp)
So that's where you change size of J_2_SYM, and after all it is (33, 16), so you cannot do dot product with (32, 32) array.
I have a 2D means matrix in size n*m, where n is number of samples and m is the dimension of the data.
I have as well n matrices of m*m, namely sigma is my variance matrix in shape n*m*m.
I wish to sample n samples from a the distributions above, such that x_i~N(mean[i], sigma[i]).
Any way to do that in numpy or any other standard lib w/o running with a for loop?
The only option I thought was using np.random.multivariate_normal() by flatting the means matrix to one vector, and flatten the 3D sigma to a 2D blocks-diagonal matrix. And of course reshape afterwards. But that means we are going the sample with sigma in shape (n*m)*(n*m) which can easily be ridiculously huge, and only computing and allocating that matrix (if possible) can take longer than running in a for loop.
In my specific task, right now Sigma is the same matrix for all the samples, means I can express Sigma in m*m, and it is the same one for all n points. But I am interested in a general solution.
Appreciate your help.
Difficult to tell without testable code, but this should be close:
A = numpy.linalg.cholesky(sigma) # => shape (n, m, m), same as sigma
Z = np.random.normal(size = (n, m)) # shape (n, m)
X = np.einsum('ijk, ik -> ij', A, Z) + mean # shape (n, m)
What's going on:
We're manually sampling multivariate normal distributions according to the standard Cholesky decomposition method outlined here. A is built such that A#A.T = sigma. Then X (the multivariate normal) can be formed by the dot product of A and a univariate normal N(0, 1) vector Z, plus the mean.
You keep the extraneous dimension throughout the calculation in the first (index = 0, 'i' in the einsum) axis, while contracting the last ('k') axis, forming the dot product.
I am trying to implement a multivariate Gaussian Mixture Model and am trying to calculate the probability distribution function using tensors. There are n data points, k clusters, and d dimensions. So far, I have two tensors. One is a (n,k,d) tensor of centered data points and the other is a kxdxd tensor of covariance matricies. I can compute an nxk matrix of probabilities by doing
centered = np.repeat(points[:,np.newaxis,:],K,axis=1) - mu[np.newaxis,:] # KxNxD
prob = np.zeros(n,k)
constant = 1/2/np.pow(np.pi, d/2)
for n in range(centered.shape[1]):
for k in range(centered.shape[0]):
p = centered[n,k,:][np.newaxis] # 1xN
power = -1/2*(p # np.linalg.inv(sigma[k,:,:]) # p.T)
prob[n,k] = constant * np.linalg.det(sigma[k,:,:]) * np.exp(power)
where sigma is the triangularized kxdxd matrix of covariances and centered are mypoints. What is a more pythonic way of doing this using numpy's tensor capabilites?
Just a couple of quick observations:
I don't see you using p in the loop; is this a mistake? Using n instead?
The T in centered[n,k,:].T does nothing; with that index the array is 1d
I'm not sure if np.linal.inv can handle batches of arrays, allowing np.linalg.inv(sigma).
# allows batches, just so long as the last 2 dim are the ones entering into the dot (with the usual last of A, 2nd to the last of B rule; einsum can also be used.
again does np.linalg.det handle batches?
I'm trying to get a homography matrix that describes the transformation from one image to another. I tried doing this by using an eigendecomposition and taking the smallest eigenvector. Apparently, I have to reshape it into a 3x3 matrix, but the numpy linalg function returns an eigenvalue of shape (9,9). (trying to compute it from 4 points)
A is a 8x9 matrix. pts1 and pts2 are arrays of 4 points in the source image and the target image respectively.
Code starts from symmetry matrix for homography calculation (size 8x9)
A_t = A.transpose()
sym_mat = np.dot(A_t,A)
eig_val,eig_vec = np.linalg.eig(sym_mat)
#sort according to value
idx = np.argsort(eig_val)
eig_val = eig_val[idx]
eig_vec = eig_vec[:,idx]
# Return the eigenvector corresponding to the smallest eigenvalue, reshaped
# as a 3x3 matrix.
H = np.reshape(smallest,(3,3))
`
I'm wondering if there is a simple way to multiply a numpy matrix by a scalar. Essentially I want all values to be multiplied by the constant 40. This would be an nxn matrix with 40's on the diagonal, but I'm wondering if there is a simpler function to use to scale this matrix. Or how would I go about making a matrix with the same shape as my other matrix and fill in its diagonal?
Sorry if this seems a bit basic, but for some reason I couldn't find this in the doc.
If you want a matrix with 40 on the diagonal and zeros everywhere else, you can use NumPy's function fill_diagonal() on a matrix of zeros. You can thus directly do:
N = 100; value = 40
b = np.zeros((N, N))
np.fill_diagonal(b, value)
This involves only setting elements to a certain value, and is therefore likely to be faster than code involving multiplying all the elements of a matrix by a constant. This approach also has the advantage of showing explicitly that you fill the diagonal with a specific value.
If you want the diagonal matrix b to be of the same size as another matrix a, you can use the following shortcut (no need for an explicit size N):
b = np.zeros_like(a)
np.fill_diagonal(b, value)
Easy:
N = 100
a = np.eye(N) # Diagonal Identity 100x100 array
b = 40*a # Multiply by a scalar
If you actually want a numpy matrix vs an array, you can do a = np.asmatrix(np.eye(N)) instead. But in general * is element-wise multiplication in numpy.