Develop Reference

Applying matrix functions like scipy.linalg.eigh to higher dimensional arrays

I am new to numpy but have been using python for quite a while as an engineer.
I am writing a program that currently stores stress tensors as 3x3 numpy arrays within another NxM array which represents values through time and through the thickness of a wall, so overall it is an NxMx3x3 numpy array. I want to efficiently calculate the eigenvals and vectors of each 3x3 array within this larger array. So far I have tried to using "fromiter" but this doesn't seem to work because the functions returns 2 arrays. I have also tried apply_along_axis which also doesn't work because it says the inner 3x3 is not a square matrix? I can do it with list comprehension, but this doesn't seem ideal to resort to using lists.
Example just calculating eigenvals using list comprehension
import numpy as np
from scipy import linalg
a=np.random.random((2,2,3,3))
f=linalg.eigvalsh
ans=np.asarray([f(x) for x in a.reshape((4,3,3))])
ans.shape=(2,2,3)
I thought something like this would work but I have played around with it and can't get it working:
np.apply_along_axis(f,0,a)
BTW the 2x2 bit could be up to 5000x100 and this code is repeated ~50x50x200 times hence the need for efficiency. Any help would be greatly appreciated?

You can use numpy.linalg.eigh. It accepts an array like your example a.
Here's an example. First, create an array of 3x3 symmetric arrays:
In [96]: a = np.random.random((2, 2, 3, 3))
In [97]: a = a + np.transpose(a, axes=(0, 1, 3, 2))
In [98]: a[0, 0]
Out[98]:
array([[0.61145048, 0.85209618, 0.03909677],
[0.85209618, 1.79309413, 1.61209077],
[0.03909677, 1.61209077, 1.55432465]])
Compute the eigenvalues and eigenvectors of all the 3x3 arrays:
In [99]: evals, evecs = np.linalg.eigh(a)
In [100]: evals.shape
Out[100]: (2, 2, 3)
In [101]: evecs.shape
Out[101]: (2, 2, 3, 3)
Take a look at the result for a[0, 0]:
In [102]: evals[0, 0]
Out[102]: array([-0.31729364, 0.83148477, 3.44467813])
In [103]: evecs[0, 0]
Out[103]:
array([[-0.55911658, 0.79634401, 0.23070516],
[ 0.63392772, 0.23128064, 0.73800062],
[-0.53434473, -0.55887877, 0.63413738]])
Verify that it is the same as computing the eigenvalues and eigenvectors for a[0, 0] separately:
In [104]: np.linalg.eigh(a[0, 0])
Out[104]:
(array([-0.31729364, 0.83148477, 3.44467813]),
array([[-0.55911658, 0.79634401, 0.23070516],
[ 0.63392772, 0.23128064, 0.73800062],
[-0.53434473, -0.55887877, 0.63413738]]))

Scipy CSR Matrix Element-wise Addition

Unlike numpy arrays/matrices, CSR matrix seems not to allow automatic broadcasting. There are methods in the CSR implementation for element-wise multiplication, but not addition. How to add to a CSR Sparse matrix by a scalar efficiently?

Here we want to add a scalar to the non-zero entries and leave alone the matrix
sparseness, i.e. do not touch the zero entries.
From the fine Scipy docs (** emphasis ** is mine):
Attributes
nnz Get the count of explicitly-stored values (nonzeros)
has_sorted_indices Determine whether the matrix has sorted indices
dtype (dtype) Data type of the matrix
shape (2-tuple) Shape of the matrix
ndim (int) Number of dimensions (this is always 2)
**data CSR format data array of the matrix**
indices CSR format index array of the matrix
indptr CSR format index pointer array of the matrix
So I tried (the first part is "stolen" from the referenced documentation)
In [18]: from scipy import *
In [19]: from scipy.sparse import *
In [20]: row = array([0,0,1,2,2,2])
...: col = array([0,2,2,0,1,2])
...: data =array([1,2,3,4,5,6])
...: a = csr_matrix( (data,(row,col)), shape=(3,3))
...:
In [21]: a.todense()
Out[21]:
matrix([[1, 0, 2],
[0, 0, 3],
[4, 5, 6]], dtype=int64)
In [22]: a.data += 10
In [23]: a.todense()
Out[23]:
matrix([[11, 0, 12],
[ 0, 0, 13],
[14, 15, 16]], dtype=int64)
In [24]:
It works. Should you save the original matrix you can use the constructor
using a modified data array.
Disclaimer
This answer addresses this interpretation of the question
I have a sparse matrix, I want to add a scalar to the non zero entries, preserving the sparseness both of the matrix and of its programmatic representation.
My reasoning for choosing this interpretation is that adding a scalar to all entries turns the sparse matrix in a VERY dense matrix...
If this is the correct interpretation, I don't know: on one hand the OP approved my answer (at least today 2017-07-13) on the other hand in the comments beneath their question it seems they has of a different opinion.
The answer is however useful in the use case that the sparse matrix represents, e.g., sparse measurements and you want to correct a measurement bias, subtract a mean value, etc. so I'm going to leave it here, even if it can be judged controversial.

Differences between array class and matrix class in numpy for matrix operation

I was trying to do matrix dot product and transpose with Numpy, and I found array can do many things matrix can do, such as dot product, point wise product, and transpose.
When I have to create a matrix, I have to create an array first.
example:
import numpy as np
array = np.ones([3,1])
matrix = np.matrix(array)
Since I can do matrix transpose and dot product in array type, I don't have to convert array into matrix to do matrix operations.
For example, the following line is valid, where A is an ndarray :
dot_product = np.dot(A.T, A )
The previous matrix operation can be expressed with matrix class variable A
dot_product = A.T * A
The operator * is exactly the same as point-wise product for ndarray. Therefore, it makes ndarray and matrix almost indistinguishable and causes confusions.
The confusion is a serious problem, as said in REP465
Writing code using numpy.matrix also works fine. But trouble begins as
soon as we try to integrate these two pieces of code together. Code
that expects an ndarray and gets a matrix, or vice-versa, may crash or
return incorrect results. Keeping track of which functions expect
which types as inputs, and return which types as outputs, and then
converting back and forth all the time, is incredibly cumbersome and
impossible to get right at any scale.
It will be very tempting if we stick to ndarray and deprecate matrix and support ndarray with matrix operation methods such as .inverse(), .hermitian(), outerproduct(), etc, in the future.
The major reason I still have to use matrix class is that it handles 1d array as 2d array, so I can transpose it.
It is very inconvenient so far how I transpose 1d array, since 1d array of size n has shape (n,) instead of (1,n). For example, if I have to do the inner product of two arrays :
A = [[1,1,1],[2,2,2].[3,3,3]]
B = [[1,2,3],[1,2,3],[1,2,3]]
np.dot(A,B) works fine, but if
B = [1,1,1]
,its transpose is still a row vector.
I have to handle this exception when the dimensions of input variable is unknown.
I hope this help some people with the same trouble, and hope to know if there is any better way to handle matrix operation like in Matlab, especially with 1d array. Thanks.

Your first example is a column vector:
In [258]: x = np.arange(3).reshape(3,1)
In [259]: x
Out[259]:
array([[0],
[1],
[2]])
In [260]: xm = np.matrix(x)
dot produces the inner product, and dimensions operate as: (1,2),(2,1)=>(1,1)
In [261]: np.dot(x.T, x)
Out[261]: array([[5]])
the matrix product does the same thing:
In [262]: xm.T * xm
Out[262]: matrix([[5]])
(The same thing with 1d arrays produces a scalar value, np.dot([0,1,2],[0,1,2]) # 5)
element multiplication of the arrays produces the outer product (so does np.outer(x, x) and np.dot(x,x.T))
In [263]: x.T * x
Out[263]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
For ndarray, * IS element wise multiplication (the .* of MATLAB, but with broadcasting added). For element multiplication of matrix use np.multiply(xm,xm). (scipy sparse matrices have a multiply method, X.multiply(other))
You quote from the PEP that added the # operator (matmul). This, as well as np.tensordot and np.einsum can handle larger dimensional arrays, and other mixes of products. Those don't make sense with np.matrix since that's restricted to 2d.
With your 3x3 A and B
In [273]: np.dot(A,B)
Out[273]:
array([[ 3, 6, 9],
[ 6, 12, 18],
[ 9, 18, 27]])
In [274]: C=np.array([1,1,1])
In [281]: np.dot(A,np.array([1,1,1]))
Out[281]: array([3, 6, 9])
Effectively this sums each row. np.dot(A,np.array([1,1,1])[:,None]) does the same thing, but returns a (3,1) array.
np.matrix was created years ago to make numpy (actually one of its predecessors) feel more like MATLAB. A key feature is that it is restricted to 2d. That's what MATLAB was like back in the 1990s. np.matrix and MATLAB don't have 1d arrays; instead they have single column or single row matrices.
If the fact that ndarrays can be 1d (or even 0d) is a problem there are many ways of adding that 2nd dimension. I prefer the [None,:] kind of syntax, but reshape is also useful. ndmin=2, np.atleast_2d, np.expand_dims also work.
np.sum and other operations that reduced dimensions have a keepdims=True parameter to counter that. The new # gives an operator syntax for matrix multiplication. As far as I know, np.matrix class does not have any compiled code of its own.
============
The method that implements * for np.matrix uses np.dot:
def __mul__(self, other):
if isinstance(other, (N.ndarray, list, tuple)) :
# This promotes 1-D vectors to row vectors
return N.dot(self, asmatrix(other))
if isscalar(other) or not hasattr(other, '__rmul__') :
return N.dot(self, other)
return NotImplemented

Numpy matrix multiplication of 2d matrix to give 3d matrix

I have two numpy arrays, like
A: = array([[0, 1],
[2, 3],
[4, 5]])
B = array([[ 6, 7],
[ 8, 9],
[10, 11]])
For each row of A and B, say Ra and Rb respectively, I want to calculate transpose(Ra)*Rb. So for given value of A and B, i want following answer:
array([[[ 0, 0],
[ 6, 7]],
[[ 16, 18],
[ 24, 27]],
[[ 40, 44],
[ 50, 55]]])
I have written the following code to do so:
x = np.outer(np.transpose(A[0]), B[0])
for i in range(1,len(A)):
x = np.append(x,np.outer(np.transpose(A[i]), B[i]),axis=0)
Is there any better way to do this task.

You can use extend dimensions of A and B with np.newaxis/None to bring in broadcasting for a vectorized solution like so -
A[...,None]*B[:,None,:]
Explanation : np.outer(np.transpose(A[i]), B[i]) basically does elementwise multiplications between a columnar version of A[i] and B[i]. You are repeating this for all rows in A against corresoinding rows in B. Please note that the np.transpose() doesn't seem to make any impact as np.outer takes care of the intended elementwise multiplications.
I would describe these steps in a vectorized language and thus implement, like so -
Extend dimensions of A and B to form 3D shapes for both of them such that we keep axis=0 aligned and keep as axis=0 in both of those extended versions too. Thus, we are left with deciding the last two axes.
To bring in the elementwise multiplications, push axis=1 of A in its original 2D version to axis=1 in its 3D version, thus creating a singleton dimension at axis=2 for extended version of A.
This last singleton dimension of 3D version of A has to align with the elements from axis=1 in original 2D version of B to let broadcasting happen. Thus, extended version of B would have the elements from axis=1 in its 2D version being pushed to axis=2 in its 3D version, thereby creating a singleton dimension for axis=1.
Finally, the extended versions would be : A[...,None] & B[:,None,:], multiplying whom would give us the desired output.

What are the differences between numpy arrays and matrices? Which one should I use?

What are the advantages and disadvantages of each?
From what I've seen, either one can work as a replacement for the other if need be, so should I bother using both or should I stick to just one of them?
Will the style of the program influence my choice? I am doing some machine learning using numpy, so there are indeed lots of matrices, but also lots of vectors (arrays).

Numpy matrices are strictly 2-dimensional, while numpy arrays (ndarrays) are
N-dimensional. Matrix objects are a subclass of ndarray, so they inherit all
the attributes and methods of ndarrays.
The main advantage of numpy matrices is that they provide a convenient notation
for matrix multiplication: if a and b are matrices, then a*b is their matrix
product.
import numpy as np
a = np.mat('4 3; 2 1')
b = np.mat('1 2; 3 4')
print(a)
# [[4 3]
# [2 1]]
print(b)
# [[1 2]
# [3 4]]
print(a*b)
# [[13 20]
# [ 5 8]]
On the other hand, as of Python 3.5, NumPy supports infix matrix multiplication using the # operator, so you can achieve the same convenience of matrix multiplication with ndarrays in Python >= 3.5.
import numpy as np
a = np.array([[4, 3], [2, 1]])
b = np.array([[1, 2], [3, 4]])
print(a#b)
# [[13 20]
# [ 5 8]]
Both matrix objects and ndarrays have .T to return the transpose, but matrix
objects also have .H for the conjugate transpose, and .I for the inverse.
In contrast, numpy arrays consistently abide by the rule that operations are
applied element-wise (except for the new # operator). Thus, if a and b are numpy arrays, then a*b is the array
formed by multiplying the components element-wise:
c = np.array([[4, 3], [2, 1]])
d = np.array([[1, 2], [3, 4]])
print(c*d)
# [[4 6]
# [6 4]]
To obtain the result of matrix multiplication, you use np.dot (or # in Python >= 3.5, as shown above):
print(np.dot(c,d))
# [[13 20]
# [ 5 8]]
The ** operator also behaves differently:
print(a**2)
# [[22 15]
# [10 7]]
print(c**2)
# [[16 9]
# [ 4 1]]
Since a is a matrix, a**2 returns the matrix product a*a.
Since c is an ndarray, c**2 returns an ndarray with each component squared
element-wise.
There are other technical differences between matrix objects and ndarrays
(having to do with np.ravel, item selection and sequence behavior).
The main advantage of numpy arrays is that they are more general than
2-dimensional matrices. What happens when you want a 3-dimensional array? Then
you have to use an ndarray, not a matrix object. Thus, learning to use matrix
objects is more work -- you have to learn matrix object operations, and
ndarray operations.
Writing a program that mixes both matrices and arrays makes your life difficult
because you have to keep track of what type of object your variables are, lest
multiplication return something you don't expect.
In contrast, if you stick solely with ndarrays, then you can do everything
matrix objects can do, and more, except with slightly different
functions/notation.
If you are willing to give up the visual appeal of NumPy matrix product
notation (which can be achieved almost as elegantly with ndarrays in Python >= 3.5), then I think NumPy arrays are definitely the way to go.
PS. Of course, you really don't have to choose one at the expense of the other,
since np.asmatrix and np.asarray allow you to convert one to the other (as
long as the array is 2-dimensional).
There is a synopsis of the differences between NumPy arrays vs NumPy matrixes here.

Scipy.org recommends that you use arrays:
*'array' or 'matrix'? Which should I use? - Short answer
Use arrays.
They support multidimensional array algebra that is supported in
MATLAB
They are the standard vector/matrix/tensor type of NumPy. Many
NumPy functions return arrays, not matrices.
There is a clear
distinction between element-wise operations and linear algebra
operations.
You can have standard vectors or row/column vectors if you
like.
Until Python 3.5 the only disadvantage of using the array type
was that you had to use dot instead of * to multiply (reduce) two
tensors (scalar product, matrix vector multiplication etc.). Since
Python 3.5 you can use the matrix multiplication # operator.
Given the above, we intend to deprecate matrix eventually.

Just to add one case to unutbu's list.
One of the biggest practical differences for me of numpy ndarrays compared to numpy matrices or matrix languages like matlab, is that the dimension is not preserved in reduce operations. Matrices are always 2d, while the mean of an array, for example, has one dimension less.
For example demean rows of a matrix or array:
with matrix
>>> m = np.mat([[1,2],[2,3]])
>>> m
matrix([[1, 2],
[2, 3]])
>>> mm = m.mean(1)
>>> mm
matrix([[ 1.5],
[ 2.5]])
>>> mm.shape
(2, 1)
>>> m - mm
matrix([[-0.5, 0.5],
[-0.5, 0.5]])
with array
>>> a = np.array([[1,2],[2,3]])
>>> a
array([[1, 2],
[2, 3]])
>>> am = a.mean(1)
>>> am.shape
(2,)
>>> am
array([ 1.5, 2.5])
>>> a - am #wrong
array([[-0.5, -0.5],
[ 0.5, 0.5]])
>>> a - am[:, np.newaxis] #right
array([[-0.5, 0.5],
[-0.5, 0.5]])
I also think that mixing arrays and matrices gives rise to many "happy" debugging hours.
However, scipy.sparse matrices are always matrices in terms of operators like multiplication.

As per the official documents, it's not anymore advisable to use matrix class since it will be removed in the future.
https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
As other answers already state that you can achieve all the operations with NumPy arrays.

As others have mentioned, perhaps the main advantage of matrix was that it provided a convenient notation for matrix multiplication.
However, in Python 3.5 there is finally a dedicated infix operator for matrix multiplication: #.
With recent NumPy versions, it can be used with ndarrays:
A = numpy.ones((1, 3))
B = numpy.ones((3, 3))
A # B
So nowadays, even more, when in doubt, you should stick to ndarray.

Matrix Operations with Numpy Arrays:
I would like to keep updating this answer
about matrix operations with numpy arrays if some users are interested looking for information about matrices and numpy.
As the accepted answer, and the numpy-ref.pdf said:
class numpy.matrix will be removed in the future.
So now matrix algebra operations has to be done
with Numpy Arrays.
a = np.array([[1,3],[-2,4]])
b = np.array([[3,-2],[5,6]])
Matrix Multiplication (infix matrix multiplication)
a#b
array([[18, 16],
[14, 28]])
Transpose:
ab = a#b
ab.T
array([[18, 14],
[16, 28]])
Inverse of a matrix:
np.linalg.inv(ab)
array([[ 0.1 , -0.05714286],
[-0.05 , 0.06428571]])
ab_i=np.linalg.inv(ab)
ab#ab_i # proof of inverse
array([[1., 0.],
[0., 1.]]) # identity matrix
Determinant of a matrix.
np.linalg.det(ab)
279.9999999999999
Solving a Linear System:
1. x + y = 3,
x + 2y = -8
b = np.array([3,-8])
a = np.array([[1,1], [1,2]])
x = np.linalg.solve(a,b)
x
array([ 14., -11.])
# Solution x=14, y=-11
Eigenvalues and Eigenvectors:
a = np.array([[10,-18], [6,-11]])
np.linalg.eig(a)
(array([ 1., -2.]), array([[0.89442719, 0.83205029],
[0.4472136 , 0.5547002 ]])

An advantage of using matrices is for easier instantiation through text rather than nested square brackets.
With matrices you can do
np.matrix("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
and get the desired output directly:
matrix([[1.+0.j, 1.+1.j, 0.+0.j],
[0.+0.j, 0.+1.j, 0.+0.j],
[0.+0.j, 0.+0.j, 1.+0.j]])
If you use arrays, this does not work:
np.array("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
output:
array('1, 1+1j, 0; 0, 1j, 0; 0, 0, 1', dtype='<U29')