Develop Reference

Matrix-vector-multiplication with tensors in numpy

I have a numpy.array A with shape (l,l) and another numpy.array B with shape (l,m,n). Usually, the second and third dimension in B correspond to spatial cells and the first to something else.
I want to compute
l,m,n = 2,3,4 # dummy dimensions
A = np.random.rand(l,l) # dummy data
B = np.random.rand(l,m,n) # dummy data
C = np.zeros((l,m,n))
for i in range(m):
for j in range(n):
C[:,i,j] = A#B[:,i,j]
i.e., in every spatial cell, I want to perform a matrix-vector-multiplication.
Since I have to do this frequently, I would like to know, if there's a more compact way to write this with numpy. (Especially, because there are several situations in which the tensor has shape (l,m,n,o,p).)
Thank you in advance!

I found the answer using np.einsum:
np.einsum('ij,jkl->ikl', A,B)
Explanation:
Einstein notation implies that we sum over matching subscripts.
np.einsum('ij,jkl->ikl', A,B)
= rewritten in math terms
A_{i,j} B_{j,k,l}
= Einstein notation implies summation
sum_j A_{i,j} B_{j,k,l}

Python - matrix multiplication

i have an array y with shape (n,), I want to compute the inner product matrix, which is a n * n matrix
However, when I tried to do it in Python
np.dot(y , y)
I got the answer n, this is not what I am looking for
I have also tried:
np.dot(np.transpose(y),y)
np.dot(y, np.transpose(y))
I always get the same answer n

I think you are looking for:
np.multiply.outer(y,y)
or equally:
y = y[None,:]
y.T#y
example:
y = np.array([1,2,3])[None,:]
output:
#[[1 2 3]
# [2 4 6]
# [3 6 9]]

You can try to reshape y from shape (70,) to (70,1) before multiplying the 2 matrices.
# Reshape
y = y.reshape(70,1)
# Either below code would work
y*y.T
np.matmul(y,y.T)

One-liner?
np.dot(a[:, None], a[None, :])
transpose doesn't work on 1-D arrays, because you need atleast two axes to 'swap' them. This solution adds a new axis to the array; in the first argument, it looks like a column vector and has two axes; in the second argument it still looks like a row vector but has two axes.

Looks like what you need is the # matrix multiplication operator. dot method is only to compute dot product between vectors, what you want is matrix multiplication.
>>> a = np.random.rand(70, 1)
>>> (a # a.T).shape
(70, 70)
UPDATE:
Above answer is incorrect. dot does the same things if the array is 2D. See the docs here.
np.dot computes the dot product of two arrays. Specifically,
If both a and b are 1-D arrays, it is inner product of vectors (without complex conjugation).
If both a and b are 2-D arrays, it is matrix multiplication, but using matmul or a # b is preferred.
Simplest way to do what you want is to convert the vector to a matrix first using np.matrix and then using the #. Although, dot can also be used # is better because conventionally dot is used for vectors and # for matrices.
>>> a = np.random.rand(70)
(70,)
>>> a.shape
>>> a = np.matrix(a).T
>>> a.shape
(70, 1)
>>> (a # a.T).shape
(70, 70)

Python: vectorise function over two multi-dimensional arrays simultaneously

In python, it is simple to vectorise a function f(x) of a scalar x over a single array a1: just use f(a1). But suppose I have two (or in principle, multiple) arrays a1, a2 having the same shape Nx3, and I want to vectorise a function, g(x,y) with x,y scalars, simultaneously over both. Something like g(a1,a2), which will return an object again with the common dimension N.
EDIT:
If a1, a2 are both 1-dimensional, this becomes trivial. We use simple broadcasting as noted below. However, for multi-dimensional arrays, the answer is not evident to me. So, how do I this, preferably using numpy?
Example (EDITED):
a1 = np.array of size 20x3 # so that each row is a 3-vector
a2 = np.array of size 20x3 # ditto
def f(x, y): # acts on each element
... complicated function, using other global variables ...
return ... (scalar)
Without vectorisation, I need to loop f individually over all 20 rows, and get an output length 20 vector:
result = []
for i, elem in a1:
result.append(f(elem, a2[i]))
result = np.array(result)
However, I want to eliminate the for loop, and have a single statement, using numpy vectorisation. The reason is to be able to use the numpy wrapper of jax (https://github.com/google/jax) then to speed this up on a GPU. Something naive like
result = f(a1, a2)
does not work. So what is the correct syntax?

use numpy's vectorize.
A simple example of np.vectorize using a simple lambda function:
import numpy as np
f = np.vectorize(lambda x: 2*x)
f([[2,3],[3,4],[1,1]])
# output:
array([[4, 6],
[6, 8],
[2, 2]])

It may depend on the operation you need to execute, if it was a simple sum than the following will work:
import numpy as np
a = np.arange(3*2*20).reshape((20,3,2))
b = np.arange(2*20).reshape((20,2))
res = (a.transpose((1,2,0))+b.transpose((1,0))).transpose((2,0,1))
print(a[0],b[0])
[[0 1]
[2 3]
[4 5]] [0 1]
print(res[0])
[[0 2]
[2 4]
[4 6]]
First the input data is transposed so that the correct dimensions will be involved in the broadcasted operation. After summation, the output is tranposed back.

I have also been trying to do something similar over the past few days. I finally managed to do it with np.vectorize, using function signatures. Try with the code snippet below:
fn_vectorized = np.vectorize(interpolate.interp1d,
signature='(n),(n)->()')
interp_fn_array = fn_vectorized(x[np.newaxis, :, :], y)
Here, I was doing the vectorization of the interp1d function. x and y are arrays of shape (m x n). The objective was to generate an array of interpolation functions, for row i of x and row i of y. The array 'interp_fn_array' contains the interpolation functions (shape is (1 x m).

Normalize 2d arrays

Consider a square matrix containing positive numbers, given as a 2d numpy array A of shape ((m,m)). I would like to build a new array B that has the same shape with entries
B[i,j] = A[i,j] / (np.sqrt(A[i,i]) * np.sqrt(A[j,j]))
An obvious solution is to loop over all (i,j) but I'm wondering if there is a faster way.

Two approaches leveraging broadcasting could be suggested.
Approach #1 :
d = np.sqrt(np.diag(A))
B = A/d[:,None]
B /= d
Approach #2 :
B = A/(d[:,None]*d) # d same as used in Approach #1
Approach #1 has lesser memory overhead and as such I think would be faster.

You can normalize each row of your array by the main diagonal leveraging broadcasting using
b = np.sqrt(np.diag(a))
a / b[:, None]
Also, you can normalize each column using
a / b[None, :]
To do both, as your question seems to ask, using
a / (b[:, None] * b[None, :])
If you want to prevent the creation of intermediate arrays and do the operation in place, you can use
a /= b[:, None]
a /= b[None, :]

3D array multiplication

I have arrays A and B both of dimension MxNxH.
I would like to define a binary operator, to "multiply", such that the result is MxN dimensions.
The equivalent operation would be:
C = A[:,:,0] * B[:,:,0] + A[:,:,1] * B[:,:,1] + .... + A[:,:,H] * B[:,:,H]
Is there a way to do this operation in a more efficient way?
For example, using a built in function in numpy?
I have tried tensordot, but this gives a different result.

The easiest is:
C = numpy.sum(A * B, -1)
I think this might work too:
C = numpy.einsum("...i,...i->...", A, B)

try this: numpy.sum( A*B, axis=2 )
this is similar to the other suggestion but perhaps clearer (axes are numbered from 0, so axis=2 is the 3rd axis or H out of MxNxH)