Develop Reference

whether to use numpy's dot or matmul function

I need to do the following two operations:
solve Ax=b by inverting the n-by-n matrix A, and
solve r=Ar using power iteration (i.e. by repeated multiplying current vector r by A) such as one would do for the PageRank algorithm.
My question is: When computing the matrix-vector product A^{-1}b or the matrix-vector product Ar, is it better to use numpy.dot or numpy.matmul? (I understand there might be differences in higher dimensions, but my question is only for the case where A is a 2D array and b, r are vectors.)

From the numpy doc for np.dot:
Dot product of two arrays. Specifically, If both a and b are 2-D arrays, it is matrix multiplication, but using matmul or a # b is
preferred.
So basically for your case, it does not matter, although matmul is preferred according to the doc.
Also since one of your arrays is 1-D, from docs for np.matmul:
If the second argument is 1-D, it is promoted to a matrix by appending
a 1 to its dimensions. After matrix multiplication the appended 1 is
removed.
And:
matmul differs from dot in two important ways:
Multiplication by scalars is not allowed, use * instead. Stacks of matrices are
broadcast together as if the matrices were elements, respecting the
signature
Therefore, they would work the same in your case, but I would go with numpy doc's recommendation on using matmul.

Calculating a vectors with its transposed vector

I'm working on a calculation for a within matrix scatter where i have a 50x20 vector and something that occured to me is that multiplying transposed vectors by the original vector, gives me a dimensional error, saying the following:
operands could not be broadcast together with shapes (50,20) (20,50)
What i tried is: array = my_array * my_array_transposed and got the aforementioned error.
The alternative was to do, then:
new_array = np.dot(my_array, np.transpose(my_array))
In Octave for instance, this would've been a lot easier, but due to the size of the vector, it's kinda hard for me to confirm for ground truth if this is the way to do the following calculation:
Because as far as i know, there is something related as to whether the multiplication is element wise.
My question is, am i applying that formula the right way? If not, whats the right way to multiply a transposed vector by the non-tranposed vector?

Yes, the np.dot formula is the correct one. If you write array = my_array * my_array_transposed you are asking Python to perform component-wise multiplication. Instead you need a row-by-column multiplication which is achieved in numpy with np.dot.

Finding eigen values of numpy array of arrays

So I have a 3 numpy arrays which has the following dimensions,
a.shape = (704, 528)
b.shape = (704, 528)
c.shape = (704, 528)
And I have a square matrix that looks like this,
mat = np.array([[a, b], [b, c]])
I need to find the eigen values of this. I'm aware that it's going to be a matrix of eigen values. But when I use numpy.linalg.eig(), it gives me an error: numpy.linalg.LinAlgError: Last 2 dimensions of the array must be square.
I haven't found many resources as to how to do this, could someone guide me to any sources or give me a solution? Thank you!

Eigenvalues are only defined for square matrices.
Your matrix has 2*704 = 1408 rows and 2*528 = 1056 columns, hence you get an error as numpy.linalg.eig() is expecting a square matrix as input.
Depending on your goal of wanting to compute eigenvalues, you might want to consider SVD which is defined for non-square matrix as well. You might also want to examine if the matrix that you constructed is indeed the matrix that you intend to construct.

Solve broadcasting error without for loop, speed up code

I may be misunderstanding how broadcasting works in Python, but I am still running into errors.
scipy offers a number of "special functions" which take in two arguments, in particular the eval_XX(n, x[,out]) functions.
See http://docs.scipy.org/doc/scipy/reference/special.html
My program uses many orthogonal polynomials, so I must evaluate these polynomials at distinct points. Let's take the concrete example scipy.special.eval_hermite(n, x, out=None).
I would like the x argument to be a matrix shape (50, 50). Then, I would like to evaluate each entry of this matrix at a number of points. Let's define n to be an a numpy array narr = np.arange(10) (where we have imported numpy as np, i.e. import numpy as np).
So, calling
scipy.special.eval_hermite(narr, matrix)
should return Hermitian polynomials H_0(matrix), H_1(matrix), H_2(matrix), etc. Each H_X(matrix) is of the shape (50,50), the shape of the original input matrix.
Then, I would like to sum these values. So, I call
matrix1 = np.sum( [scipy.eval_hermite(narr, matrix)], axis=0 )
but I get a broadcasting error!
ValueError: operands could not be broadcast together with shapes (10,) (50,50)
I can solve this with a for loop, i.e.
matrix2 = np.sum( [scipy.eval_hermite(i, matrix) for i in narr], axis=0)
This gives me the correct answer, and the output matrix2.shape = (50,50). But using this for loop slows down my code, big time. Remember, we are working with entries of matrices.
Is there a way to do this without a for loop?

eval_hermite broadcasts n with x, then evaluates Hn(x) at each point. Thus, the output shape will be the result of broadcasting n with x. So, if you want to make this work, you'll have to make n and x have compatible shapes:
import scipy.special as ss
import numpy as np
matrix = np.ones([100,100]) # example
narr = np.arange(10) # example
ss.eval_hermite(narr[:,None,None], matrix).shape # => (10, 100, 100)
But note that this might actually be faster:
out = np.zeros_like(matrix)
for n in narr:
out += ss.eval_hermite(n, matrix)
In testing, it appears to be between 5-10% faster than np.sum(...) of above.

The documentation for these functions is skimpy, and a lot of the code is compiled, so this is just based on experimentation:
special.eval_hermite(n, x, out=None)
n apparently is a scalar or array of integers. x can be an array of floats.
special.eval_hermite(np.ones(5,int)[:,None],np.ones(6)) gives me a (5,6) result. This is the same shape as what I'd get from np.ones(5,int)[:,None] * np.ones(6).
The np.ones(5,int)[:,None] is a (5,1) array, np.ones(6) a (6,), which for this purpose is equivalent of (1,6). Both can be expanded to (5,6).
So as best I can tell, broadcasting rules in these special functions is the same as for operators like *.
Since special.eval_hermite(nar[:,None,None], x) produces a (10,50,50), you just apply sum to axis 0 of that to produce the (50,50).
special.eval_hermite(nar[:,Nar,Nar], x).sum(axis=0)
Like I wrote before, the same broadcasting (and summing) rules apply for this hermite as they do for a basic operation like *.

Numpy: Multiplying a matrix with a 3d tensor -- Suggestion

I have a matrix P with shape MxN and a 3d tensor T with shape KxNxR. I want to multiply P with every NxR matrix in T, resulting in a KxMxR 3d tensor.
P.dot(T).transpose(1,0,2) gives the desired result. Is there a nicer solution (i.e. getting rid of transpose) to this problem? This must be quite a common operation, so I assume, others have found different approaches, e.g. using tensordot (which I tried but failed to get the desired result). Opinions/Views would be highly appreciated!

scipy.tensordot(P, T, axes=[1,1]).swapaxes(0,1)

You could also use Einstein summation notation:
P = numpy.random.randint(1,10,(5,3))
P.shape
T = numpy.random.randint(1,10,(2,3,4))
T.shape
numpy.einsum('ij,kjl->kil',P,T)
which should give you the same results as:
P.dot(T).transpose(1,0,2)