Develop Reference

Image-processing convolution kernels are calculated dynamically

Using standard numpy and cv2.filter2D solutions I can apply static convolutions to an image:
import numpy as np
convolution_kernel = np.array([[-2, -1, 0],
[-1, 1, 1],
[0, 1, 2]])
import cv2
image = cv2.imread('1.png') result = cv2.filter2D(image, -1, convolution_kernel)
(example from https://stackoverflow.com/a/58383803/3310334)
Every pixel at [i, j] in the output image has a value calculated by centering a 3x3 "window" onto [i, j] in the input image, and then multiplying each value in the window by the corresponding value in the convolution kernel (Hadamard product) and finally summing the 9 products to get the value for [i, j] in the output image (for each color channel).
(image from: https://github.com/ashushekar/image-convolution-from-scratch#convolution)
In my case, the function to perform to calculate for each output pixel is not as simple as sum of Hadamard product. It is for each pixel calculated from operations performed on known-size windows into two input matrices centered around that pixel.
I have two input matrixes ("images"), like
A = [[179, 97, 77, 118, 144, 105],
[ 68, 56, 184, 210, 141, 230],
[178, 166, 218, 47, 106, 172],
[ 38, 183, 50, 185, 48, 87],
[ 60, 200, 228, 232, 6, 190],
[253, 75, 231, 166, 117, 134]]
B = [[116, 95, 94, 220, 80, 223],
[135, 9, 166, 78, 5, 129],
[102, 167, 120, 81, 141, 29],
[ 83, 117, 81, 129, 255, 48],
[130, 231, 165, 7, 187, 169],
[ 44, 137, 16, 50, 229, 202]]
And in the output matrix, each [i, j] pixel should be calculated as the sum of all of A[u,v] ** 2 - B[u,v] ** 2 values for [u, v] coordinates within 3x3 "windows" onto the two (same-sized) input matrixes.
How can I calculate this output matrix quickly in Python?
Using numpy, it seems to be the 3x3 sums of A * A - B * B, but how to do those sums? Or is there another "2d map" process I could be using?
I've written a loop-based solution to calculate the expected output for these two examples:
W = 3 # size of kernel is WxW
out = np.zeros(A.shape)
difference_of_squares = A * A - B * B
for i, j in np.ndindex(out.shape):
starti = max(i - W//2, 0) # use smaller kernels at input's boundaries, output will have same dimension as input
stopi = min(i - W//2 + W, np.shape(out)[0]) # I'm not worried at this point about what happens at boundaries
startj = max(j - W//2, 0) # standard convolution solutions are often just reducing output size or padding input with zeroes
stopj = min(j - W//2 + W, np.shape(out)[1])
out[i, j] = np.sum(difference_of_squares[starti:stopi, startj:stopj])
print(out)
[[ 8423. 11816. 10372. 41125. 35287. 31747.]
[ 29370. 65887. 38811. 61252. 51033. 51845.]
[ 24756. 60119. 109133. 35101. 70005. 18757.]
[ 8641. 62463. 126935. 14530. 2255. -64752.]
[ 36623. 110426. 163513. 33812. -50035. -146450.]
[ 22268. 100132. 130190. 83010. -10163. -88994.]]

You can use scipy.signal.convolve2d:
from scipy.signal import convolve2d
# Same shape as original (6x6)
>>> convolve2d(A**2-B**2, np.ones((3, 3), dtype=int), mode='same')
array([[ 8423, 11816, 10372, 41125, 35287, 31747],
[ 29370, 65887, 38811, 61252, 51033, 51845],
[ 24756, 60119, 109133, 35101, 70005, 18757],
[ 8641, 62463, 126935, 14530, 2255, -64752],
[ 36623, 110426, 163513, 33812, -50035, -146450],
[ 22268, 100132, 130190, 83010, -10163, -88994]])
# Shape reduce by 1 (5x5)
>>> convolve2d(A**2-B**2, np.ones((3, 3), dtype=int), mode='valid')
array([[ 65887, 38811, 61252, 51033],
[ 60119, 109133, 35101, 70005],
[ 62463, 126935, 14530, 2255],
[110426, 163513, 33812, -50035]])
Note: You have to play around with the "mode" and "limit" parameters until you get what you want.
Update
If the border is not a problem at this point, you can use sliding_window_view:
from numpy.lib.stride_tricks import sliding_window_view
>>> np.sum(sliding_window_view(A**2-B**2, (3, 3)), axis=(2, 3))
array([[ 65887, 38811, 61252, 51033],
[ 60119, 109133, 35101, 70005],
[ 62463, 126935, 14530, 2255],
[110426, 163513, 33812, -50035]])

Python - How one can vectorize two matrices twice with a dot product in a stochatic process?

I have a stochastic process with Mmax trajectories. For each trajectory, I have to take the dot product of two matrices, A and B.
With a loop, it works great
A=np.zeros((2,Mmax),dtype=np.complex64)
B=np.zeros((2,2,Mmax),dtype=np.complex64)
C=np.zeros((2,Mmax),dtype=np.complex64)
for m in range(Mmax):
C[:,m]=B[:,:,m].dot(A[:,m])
(here are just 2x2 matrices to simplify it, when in reality they are much larger)
However, this loop is slow for a large number of trajectories. I want to optimize it by vectorizing it, and I have some problems when I try to implement it
B[:,:,:].dot(A[:,:])
It gives me the error 'shapes (2,2,10) and (2,10) not aligned: 10 (dim 2) != 2 (dim 0)', which makes sense. However, I would really need to vectorize this process, or at least optimize it as much as possible.
Is there any way to get this?

If speed is your concern, there is a way to have that multiplication non-vectorised and yet extremely fast - usually even significantly faster than that. It needs numba though:
import numpy as np
import numba as nb
#nb.njit
def mat_mul(A, B):
n, Mmax = A.shape
C = np.zeros((n, Mmax))
for m in range(Mmax):
for j in range(n):
for i in range(n):
C[j, m] += B[j, i, m]*A[i, m]
return C
Mmax = 100
A = np.ones((2, Mmax))
B = np.ones((2, 2, Mmax))
C = mat_mul(A, B)

Define sample arrays that aren't all zeros. We want to verify values as well as shapes.
In [82]: m = 5
In [83]: A = np.arange(2*m).reshape(2,m)
In [84]: B = np.arange(2*2*m).reshape(2,2,m)
Your iteration:
In [85]: C = np.zeros((2,m))
In [86]: for i in range(m):
...: C[:,i]=B[:,:,i].dot(A[:,i])
...:
In [87]: C
Out[87]:
array([[ 25., 37., 53., 73., 97.],
[ 75., 107., 143., 183., 227.]])
It's fairly easy to express that in einsum:
In [88]: np.einsum('ijk,jk->ik',B,A)
Out[88]:
array([[ 25, 37, 53, 73, 97],
[ 75, 107, 143, 183, 227]])
matmul/# is a variation on np.dot that handles 'batches' nicely. But the batch dimension has to be first (of 3). Your batch dimension, m, is last, so we have do some transposing to get the same result:
In [90]: (np.matmul(B.transpose(2,0,1),A.transpose(1,0)[...,None])[...,0]).T
Out[90]:
array([[ 25, 37, 53, 73, 97],
[ 75, 107, 143, 183, 227]])

How to multiply two lists to matrices to a tensor?

I have the two lists of arrays
splocations = [array([1,2,3]),array([4,5,6]),array([7,8,9])]
eviddisp = [array([10,11,12]), array([13,14,15])]
which I would like to multiply with each other such that I multiply each list element (which is an array) with each other list element. Here I would get a 3x2 matrix where each element is a vector. So the matrix element [0,0] would be
array([10, 22, 36]) = array([1,2,3]) * array([10,11,12])
So this matrix would be in fact a tensor of shape 3x2x3. How can I get this tensor/matrix?
I get that I need to use array(splocations) and array(eviddisp) somehow. By I realised, I am looking for a solution with numpy's tensordot, but I don't get it right. How to I proceed?

I think this is what you want, taking automatic broadcasting into account:
from numpy import array
splocations = [array([1,2,3]),array([4,5,6]),array([7,8,9])]
eviddisp = [array([10,11,12]), array([13,14,15])]
splocations = array(splocations)
viddisp = array(eviddisp)
result = splocations[:, None, :]*eviddisp
result
array([[[ 10, 22, 36],
[ 13, 28, 45]],
[[ 40, 55, 72],
[ 52, 70, 90]],
[[ 70, 88, 108],
[ 91, 112, 135]]])

numpy with python: convert 3d array to 2d

Say that I have a color image, and naturally this will be represented by a 3-dimensional array in python, say of shape (n x m x 3) and call it img.
I want a new 2-d array, call it "narray" to have a shape (3,nxm), such that each row of this array contains the "flattened" version of R,G,and B channel respectively. Moreover, it should have the property that I can easily reconstruct back any of the original channel by something like
narray[0,].reshape(img.shape[0:2]) #so this should reconstruct back the R channel.
The question is how can I construct the "narray" from "img"? The simple img.reshape(3,-1) does not work as the order of the elements are not desirable for me.
Thanks

You need to use np.transpose to rearrange dimensions. Now, n x m x 3 is to be converted to 3 x (n*m), so send the last axis to the front and shift right the order of the remaining axes (0,1). Finally , reshape to have 3 rows. Thus, the implementation would be -
img.transpose(2,0,1).reshape(3,-1)
Sample run -
In [16]: img
Out[16]:
array([[[155, 33, 129],
[161, 218, 6]],
[[215, 142, 235],
[143, 249, 164]],
[[221, 71, 229],
[ 56, 91, 120]],
[[236, 4, 177],
[171, 105, 40]]])
In [17]: img.transpose(2,0,1).reshape(3,-1)
Out[17]:
array([[155, 161, 215, 143, 221, 56, 236, 171],
[ 33, 218, 142, 249, 71, 91, 4, 105],
[129, 6, 235, 164, 229, 120, 177, 40]])

[ORIGINAL ANSWER]
Let's say we have an array img of size m x n x 3 to transform into an array new_img of size 3 x (m*n)
Initial Solution:
new_img = img.reshape((img.shape[0]*img.shape[1]), img.shape[2])
new_img = new_img.transpose()
[EDITED ANSWER]
Flaw: The reshape starts from the first dimension and reshapes the remainder, this solution has the potential to mix the values from the third dimension. Which in the case of images could be semantically incorrect.
Adapted Solution:
# Dimensions: [m, n, 3]
new_img = new_img.transpose()
# Dimensions: [3, n, m]
new_img = img.reshape(img.shape[0], (img.shape[1]*img.shape[2]))
Strict Solution:
# Dimensions: [m, n, 3]
new_img = new_img.transpose((2, 0, 1))
# Dimensions: [3, m, n]
new_img = img.reshape(img.shape[0], (img.shape[1]*img.shape[2]))
The strict is a better way forward to account for the order of dimensions, while the results from the Adapted and Strict will be identical in terms of the values (set(new_img[0,...])), however with the order shuffled.

If you have the scikit module installed, then you can use the rgb2grey (or rgb2gray) to make a photo from color to gray (from 3D to 2D)
from skimage import io, color
lina_color = io.imread(path+img)
lina_gray = color.rgb2gray(lina_color)
In [33]: lina_color.shape
Out[33]: (1920, 1280, 3)
In [34]: lina_gray.shape
Out[34]: (1920, 1280)

Python: Why are eigenvectors not the same as first PCA weights?

Let's generate an array:
import numpy as np
data = np.arange(30).reshape(10,3)
data=data*data
array([[ 0, 1, 4],
[ 9, 16, 25],
[ 36, 49, 64],
[ 81, 100, 121],
[144, 169, 196],
[225, 256, 289],
[324, 361, 400],
[441, 484, 529],
[576, 625, 676],
[729, 784, 841]])
Then find the eigenvalues of the covariance matrix:
mn = np.mean(data, axis=0)
data -= mn
C = np.cov(data.T)
evals, evecs = la.eig(C)
idx = np.argsort(evals)[::-1]
evecs = evecs[:,idx]
print evecs
array([[-0.53926461, -0.73656433, 0.40824829],
[-0.5765472 , -0.03044111, -0.81649658],
[-0.61382979, 0.67568211, 0.40824829]])
Now let's run the matplotlib.mlab.PCA function on the data:
import matplotlib.mlab as mlab
mpca=mlab.PCA(data)
print mpca.Wt
[[ 0.57731894 0.57740574 0.57732612]
[ 0.72184459 -0.03044628 -0.69138514]
[ 0.38163232 -0.81588947 0.43437443]]
Why are the two matrices different? I thought that in finding the PCA, first one had to find the eigenvectors of the covariance matrix, and that this would be exactly equal to the weights.

You need to normalize your data, not just center it, and the output of np.linalg.eig has to be transposed to match that of mlab.PCA:
>>> n_data = (data - data.mean(axis=0)) / data.std(axis=0)
>>> evals, evecs = np.linalg.eig(np.cov(n_data.T))
>>> evecs = evecs[:, np.argsort(evals)[::-1]].T
>>> mlab.PCA(data).Wt
array([[ 0.57731905, 0.57740556, 0.5773262 ],
[ 0.72182079, -0.03039546, -0.69141222],
[ 0.38167716, -0.8158915 , 0.43433121]])
>>> evecs
array([[-0.57731905, -0.57740556, -0.5773262 ],
[-0.72182079, 0.03039546, 0.69141222],
[ 0.38167716, -0.8158915 , 0.43433121]])