Related
Let's suppose I have two arrays that represent pixels in pictures.
I want to build an array of tensordot products of pixels of a smaller picture with a bigger picture as it "scans" the latter. By "scanning" I mean iteration over rows and columns while creating overlays with the original picture.
For instance, a 2x2 picture can be overlaid on top of 3x3 in four different ways, so I want to produce a four-element array that contains tensordot products of matching pixels.
Tensordot is calculated by multiplying a[i,j] with b[i,j] element-wise and summing the terms.
Please examine this code:
import numpy as np
a = np.array([[0,1,2],
[3,4,5],
[6,7,8]])
b = np.array([[0,1],
[2,3]])
shape_diff = (a.shape[0] - b.shape[0] + 1,
a.shape[1] - b.shape[1] + 1)
def compute_pixel(x,y):
sub_matrix = a[x : x + b.shape[0],
y : y + b.shape[1]]
return np.tensordot(sub_matrix, b, axes=2)
def process():
arr = np.zeros(shape_diff)
for i in range(shape_diff[0]):
for j in range(shape_diff[1]):
arr[i,j]=compute_pixel(i,j)
return arr
print(process())
Computing a single pixel is very easy, all I need is the starting location coordinates within a. From there I match the size of the b and do a tensordot product.
However, because I need to do this all over again for each x and y location as I'm iterating over rows and columns I've had to use a loop, which is of course suboptimal.
In the next piece of code I have tried to utilize a handy feature of tensordot, which also accepts tensors as arguments. In order words I can feed an array of arrays for different combinations of a, while keeping the b the same.
Although in order to create an array of said combination, I couldn't think of anything better than using another loop, which kind of sounds silly in this case.
def try_vector():
tensor = np.zeros(shape_diff + b.shape)
for i in range(shape_diff[0]):
for j in range(shape_diff[1]):
tensor[i,j]=a[i: i + b.shape[0],
j: j + b.shape[1]]
return np.tensordot(tensor, b, axes=2)
print(try_vector())
Note: tensor size is the sum of two tuples, which in this case gives (2, 2, 2, 2)
Yet regardless, even if I produced such array, it would be prohibitively large in size to be of any practical use. For doing this for a 1000x1000 picture, could probably consume all the available memory.
So, is there any other ways to avoid loops in this problem?
In [111]: process()
Out[111]:
array([[19., 25.],
[37., 43.]])
tensordot with 2 is the same as element multiply and sum:
In [116]: np.tensordot(a[0:2,0:2],b, axes=2)
Out[116]: array(19)
In [126]: (a[0:2,0:2]*b).sum()
Out[126]: 19
A lower-memory way of generating your tensor is:
In [121]: np.lib.stride_tricks.sliding_window_view(a,(2,2))
Out[121]:
array([[[[0, 1],
[3, 4]],
[[1, 2],
[4, 5]]],
[[[3, 4],
[6, 7]],
[[4, 5],
[7, 8]]]])
We can do a broadcasted multiply, and sum on the last 2 axes:
In [129]: (Out[121]*b).sum((2,3))
Out[129]:
array([[19, 25],
[37, 43]])
Given a numpy array, say
x = np.array(
[[0],
[1],
[2]]
)
I would like to find the matrix containing a-b for every possible pair a,b in x. I.e.
[[0-0, 0-1, 0-2]
[1-0, 1-1, 1-2]
[2-0, 2-1, 2-2]]
==
[[ 0, -1, -2]
[+1, 0, -1]
[+2, +1, 0]]
I am avoiding using a for loop for the sake of efficiency.
As Michael wrote, numpy broadcasting can help you with this. If you try to perform an operation on a vector a with shape (3,1) with vector b with shape (1,3), numpy under the treats it as if the rows of a were repeated across the columns and columns of b where repeated across the rows result in the operations you described.
That's why Michael told you to subtract the transpose of the first vector with itself to recover the result you asked for. x-x.T
Broadcasting is good because it acheives this with striding, and most of the time uses less memory.
In the more general case, a and b do not have the same length for this to work. There are more details here:
https://numpy.org/doc/stable/user/basics.broadcasting.html
What are the advantages and disadvantages of each?
From what I've seen, either one can work as a replacement for the other if need be, so should I bother using both or should I stick to just one of them?
Will the style of the program influence my choice? I am doing some machine learning using numpy, so there are indeed lots of matrices, but also lots of vectors (arrays).
Numpy matrices are strictly 2-dimensional, while numpy arrays (ndarrays) are
N-dimensional. Matrix objects are a subclass of ndarray, so they inherit all
the attributes and methods of ndarrays.
The main advantage of numpy matrices is that they provide a convenient notation
for matrix multiplication: if a and b are matrices, then a*b is their matrix
product.
import numpy as np
a = np.mat('4 3; 2 1')
b = np.mat('1 2; 3 4')
print(a)
# [[4 3]
# [2 1]]
print(b)
# [[1 2]
# [3 4]]
print(a*b)
# [[13 20]
# [ 5 8]]
On the other hand, as of Python 3.5, NumPy supports infix matrix multiplication using the # operator, so you can achieve the same convenience of matrix multiplication with ndarrays in Python >= 3.5.
import numpy as np
a = np.array([[4, 3], [2, 1]])
b = np.array([[1, 2], [3, 4]])
print(a#b)
# [[13 20]
# [ 5 8]]
Both matrix objects and ndarrays have .T to return the transpose, but matrix
objects also have .H for the conjugate transpose, and .I for the inverse.
In contrast, numpy arrays consistently abide by the rule that operations are
applied element-wise (except for the new # operator). Thus, if a and b are numpy arrays, then a*b is the array
formed by multiplying the components element-wise:
c = np.array([[4, 3], [2, 1]])
d = np.array([[1, 2], [3, 4]])
print(c*d)
# [[4 6]
# [6 4]]
To obtain the result of matrix multiplication, you use np.dot (or # in Python >= 3.5, as shown above):
print(np.dot(c,d))
# [[13 20]
# [ 5 8]]
The ** operator also behaves differently:
print(a**2)
# [[22 15]
# [10 7]]
print(c**2)
# [[16 9]
# [ 4 1]]
Since a is a matrix, a**2 returns the matrix product a*a.
Since c is an ndarray, c**2 returns an ndarray with each component squared
element-wise.
There are other technical differences between matrix objects and ndarrays
(having to do with np.ravel, item selection and sequence behavior).
The main advantage of numpy arrays is that they are more general than
2-dimensional matrices. What happens when you want a 3-dimensional array? Then
you have to use an ndarray, not a matrix object. Thus, learning to use matrix
objects is more work -- you have to learn matrix object operations, and
ndarray operations.
Writing a program that mixes both matrices and arrays makes your life difficult
because you have to keep track of what type of object your variables are, lest
multiplication return something you don't expect.
In contrast, if you stick solely with ndarrays, then you can do everything
matrix objects can do, and more, except with slightly different
functions/notation.
If you are willing to give up the visual appeal of NumPy matrix product
notation (which can be achieved almost as elegantly with ndarrays in Python >= 3.5), then I think NumPy arrays are definitely the way to go.
PS. Of course, you really don't have to choose one at the expense of the other,
since np.asmatrix and np.asarray allow you to convert one to the other (as
long as the array is 2-dimensional).
There is a synopsis of the differences between NumPy arrays vs NumPy matrixes here.
Scipy.org recommends that you use arrays:
*'array' or 'matrix'? Which should I use? - Short answer
Use arrays.
They support multidimensional array algebra that is supported in
MATLAB
They are the standard vector/matrix/tensor type of NumPy. Many
NumPy functions return arrays, not matrices.
There is a clear
distinction between element-wise operations and linear algebra
operations.
You can have standard vectors or row/column vectors if you
like.
Until Python 3.5 the only disadvantage of using the array type
was that you had to use dot instead of * to multiply (reduce) two
tensors (scalar product, matrix vector multiplication etc.). Since
Python 3.5 you can use the matrix multiplication # operator.
Given the above, we intend to deprecate matrix eventually.
Just to add one case to unutbu's list.
One of the biggest practical differences for me of numpy ndarrays compared to numpy matrices or matrix languages like matlab, is that the dimension is not preserved in reduce operations. Matrices are always 2d, while the mean of an array, for example, has one dimension less.
For example demean rows of a matrix or array:
with matrix
>>> m = np.mat([[1,2],[2,3]])
>>> m
matrix([[1, 2],
[2, 3]])
>>> mm = m.mean(1)
>>> mm
matrix([[ 1.5],
[ 2.5]])
>>> mm.shape
(2, 1)
>>> m - mm
matrix([[-0.5, 0.5],
[-0.5, 0.5]])
with array
>>> a = np.array([[1,2],[2,3]])
>>> a
array([[1, 2],
[2, 3]])
>>> am = a.mean(1)
>>> am.shape
(2,)
>>> am
array([ 1.5, 2.5])
>>> a - am #wrong
array([[-0.5, -0.5],
[ 0.5, 0.5]])
>>> a - am[:, np.newaxis] #right
array([[-0.5, 0.5],
[-0.5, 0.5]])
I also think that mixing arrays and matrices gives rise to many "happy" debugging hours.
However, scipy.sparse matrices are always matrices in terms of operators like multiplication.
As per the official documents, it's not anymore advisable to use matrix class since it will be removed in the future.
https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
As other answers already state that you can achieve all the operations with NumPy arrays.
As others have mentioned, perhaps the main advantage of matrix was that it provided a convenient notation for matrix multiplication.
However, in Python 3.5 there is finally a dedicated infix operator for matrix multiplication: #.
With recent NumPy versions, it can be used with ndarrays:
A = numpy.ones((1, 3))
B = numpy.ones((3, 3))
A # B
So nowadays, even more, when in doubt, you should stick to ndarray.
Matrix Operations with Numpy Arrays:
I would like to keep updating this answer
about matrix operations with numpy arrays if some users are interested looking for information about matrices and numpy.
As the accepted answer, and the numpy-ref.pdf said:
class numpy.matrix will be removed in the future.
So now matrix algebra operations has to be done
with Numpy Arrays.
a = np.array([[1,3],[-2,4]])
b = np.array([[3,-2],[5,6]])
Matrix Multiplication (infix matrix multiplication)
a#b
array([[18, 16],
[14, 28]])
Transpose:
ab = a#b
ab.T
array([[18, 14],
[16, 28]])
Inverse of a matrix:
np.linalg.inv(ab)
array([[ 0.1 , -0.05714286],
[-0.05 , 0.06428571]])
ab_i=np.linalg.inv(ab)
ab#ab_i # proof of inverse
array([[1., 0.],
[0., 1.]]) # identity matrix
Determinant of a matrix.
np.linalg.det(ab)
279.9999999999999
Solving a Linear System:
1. x + y = 3,
x + 2y = -8
b = np.array([3,-8])
a = np.array([[1,1], [1,2]])
x = np.linalg.solve(a,b)
x
array([ 14., -11.])
# Solution x=14, y=-11
Eigenvalues and Eigenvectors:
a = np.array([[10,-18], [6,-11]])
np.linalg.eig(a)
(array([ 1., -2.]), array([[0.89442719, 0.83205029],
[0.4472136 , 0.5547002 ]])
An advantage of using matrices is for easier instantiation through text rather than nested square brackets.
With matrices you can do
np.matrix("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
and get the desired output directly:
matrix([[1.+0.j, 1.+1.j, 0.+0.j],
[0.+0.j, 0.+1.j, 0.+0.j],
[0.+0.j, 0.+0.j, 1.+0.j]])
If you use arrays, this does not work:
np.array("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
output:
array('1, 1+1j, 0; 0, 1j, 0; 0, 0, 1', dtype='<U29')
I have a large 2D numpy matrix that needs to be made smaller (ex: convert from 100x100 to 10x10).
My goal is essentially: break the nxn matrix into smaller mxm matrices, average the cells in these mxm slices, and then construct a new (smaller) matrix out of these mxm slices.
I'm thinking about using something like matrix[a::b, c::d] to extract the smaller matrices, and then averaging those values, but this seems overly complex. Is there a better way to accomplish this?
You could split your array into blocks with the view_as_blocks function (in scikit-image).
For a 2D array, this returns a 4D array with the blocks ordered row-wise:
>>> import skimage.util as ski
>>> import numpy as np
>>> a = np.arange(16).reshape(4,4) # 4x4 array
>>> ski.view_as_blocks(a, (2,2))
array([[[[ 0, 1],
[ 4, 5]],
[[ 2, 3],
[ 6, 7]]],
[[[ 8, 9],
[12, 13]],
[[10, 11],
[14, 15]]]])
Taking the mean along the last two axes returns a 2D array with the mean in each block:
>>> ski.view_as_blocks(a, (2,2)).mean(axis=(2,3))
array([[ 2.5, 4.5],
[ 10.5, 12.5]])
Note: view_as_blocks returns a view of the array by modifying the strides (it also works with arrays with more than two dimensions). It is implemented purely in NumPy using as_strided, so if you don't have access to the scikit-image library you can copy the code from here.
Without ski-learn, you can simply reshape, and take the appropriate mean.
M=np.arange(10000).reshape(100,100)
M1=M.reshape(10,10,10,10)
M2=M1.mean(axis=(1,3))
quick check to see if I got the right axes
In [127]: M2[0,0]
Out[127]: 454.5
In [128]: M[:10,:10].mean()
Out[128]: 454.5
In [131]: M[-10:,-10:].mean()
Out[131]: 9544.5
In [132]: M2[-1,-1]
Out[132]: 9544.5
Adding .transpose([0,2,1,3]) puts the 2 averaging dimensions at the end, as view_as_blocks does.
For this (100,100) case, the reshape approach is 2x faster than the as_strided approach, but both are quite fast.
However the direct strided solution isn't much slower than reshaping.
as_strided(M,shape=(10,10,10,10),strides=(8000,80,800,8)).mean((2,3))
as_strided(M,shape=(10,10,10,10),strides=(8000,800,80,8)).mean((1,3))
I'm coming in late but I'd recommend scipy.ndimage.zoom() as an off-the-shelf solution for this. It does down-sizing (or upsizing) using spline interpolations of arbitrary order from 0 to 5. Sounds like order 0 would be sufficient for you based on your question.
from scipy import ndimage as ndi
import numpy as np
M=np.arange(1000000).reshape(1000,1000)
shrinkby=10
Mfilt = ndi.filters.uniform_filter(input=M, size=shrinkby)
Msmall = ndi.interpolation.zoom(input=Mfilt, zoom=1./shrinkby, order=0)
That's all you need. It's perhaps slightly less convenient to specify a zoom rather than a desired output size, but at least for order=0 this method is very fast.
The output size is 10% of the input in each dimension, i.e.
print M.shape, Msmall.shape
gives (1000, 1000) (100, 100) and the speed you can get from
%timeit Mfilt = ndi.filters.uniform_filter(input=M, size=shrinkby)
%timeit Msmall = ndi.interpolation.zoom(input=Mfilt, zoom=1./shrinkby, order=0)
which on my machine gave 10 loops, best of 3: 20.5 ms per loop for the uniform_filter call and 1000 loops, best of 3: 1.67 ms per loop for the zoom call.
What are the advantages and disadvantages of each?
From what I've seen, either one can work as a replacement for the other if need be, so should I bother using both or should I stick to just one of them?
Will the style of the program influence my choice? I am doing some machine learning using numpy, so there are indeed lots of matrices, but also lots of vectors (arrays).
Numpy matrices are strictly 2-dimensional, while numpy arrays (ndarrays) are
N-dimensional. Matrix objects are a subclass of ndarray, so they inherit all
the attributes and methods of ndarrays.
The main advantage of numpy matrices is that they provide a convenient notation
for matrix multiplication: if a and b are matrices, then a*b is their matrix
product.
import numpy as np
a = np.mat('4 3; 2 1')
b = np.mat('1 2; 3 4')
print(a)
# [[4 3]
# [2 1]]
print(b)
# [[1 2]
# [3 4]]
print(a*b)
# [[13 20]
# [ 5 8]]
On the other hand, as of Python 3.5, NumPy supports infix matrix multiplication using the # operator, so you can achieve the same convenience of matrix multiplication with ndarrays in Python >= 3.5.
import numpy as np
a = np.array([[4, 3], [2, 1]])
b = np.array([[1, 2], [3, 4]])
print(a#b)
# [[13 20]
# [ 5 8]]
Both matrix objects and ndarrays have .T to return the transpose, but matrix
objects also have .H for the conjugate transpose, and .I for the inverse.
In contrast, numpy arrays consistently abide by the rule that operations are
applied element-wise (except for the new # operator). Thus, if a and b are numpy arrays, then a*b is the array
formed by multiplying the components element-wise:
c = np.array([[4, 3], [2, 1]])
d = np.array([[1, 2], [3, 4]])
print(c*d)
# [[4 6]
# [6 4]]
To obtain the result of matrix multiplication, you use np.dot (or # in Python >= 3.5, as shown above):
print(np.dot(c,d))
# [[13 20]
# [ 5 8]]
The ** operator also behaves differently:
print(a**2)
# [[22 15]
# [10 7]]
print(c**2)
# [[16 9]
# [ 4 1]]
Since a is a matrix, a**2 returns the matrix product a*a.
Since c is an ndarray, c**2 returns an ndarray with each component squared
element-wise.
There are other technical differences between matrix objects and ndarrays
(having to do with np.ravel, item selection and sequence behavior).
The main advantage of numpy arrays is that they are more general than
2-dimensional matrices. What happens when you want a 3-dimensional array? Then
you have to use an ndarray, not a matrix object. Thus, learning to use matrix
objects is more work -- you have to learn matrix object operations, and
ndarray operations.
Writing a program that mixes both matrices and arrays makes your life difficult
because you have to keep track of what type of object your variables are, lest
multiplication return something you don't expect.
In contrast, if you stick solely with ndarrays, then you can do everything
matrix objects can do, and more, except with slightly different
functions/notation.
If you are willing to give up the visual appeal of NumPy matrix product
notation (which can be achieved almost as elegantly with ndarrays in Python >= 3.5), then I think NumPy arrays are definitely the way to go.
PS. Of course, you really don't have to choose one at the expense of the other,
since np.asmatrix and np.asarray allow you to convert one to the other (as
long as the array is 2-dimensional).
There is a synopsis of the differences between NumPy arrays vs NumPy matrixes here.
Scipy.org recommends that you use arrays:
*'array' or 'matrix'? Which should I use? - Short answer
Use arrays.
They support multidimensional array algebra that is supported in
MATLAB
They are the standard vector/matrix/tensor type of NumPy. Many
NumPy functions return arrays, not matrices.
There is a clear
distinction between element-wise operations and linear algebra
operations.
You can have standard vectors or row/column vectors if you
like.
Until Python 3.5 the only disadvantage of using the array type
was that you had to use dot instead of * to multiply (reduce) two
tensors (scalar product, matrix vector multiplication etc.). Since
Python 3.5 you can use the matrix multiplication # operator.
Given the above, we intend to deprecate matrix eventually.
Just to add one case to unutbu's list.
One of the biggest practical differences for me of numpy ndarrays compared to numpy matrices or matrix languages like matlab, is that the dimension is not preserved in reduce operations. Matrices are always 2d, while the mean of an array, for example, has one dimension less.
For example demean rows of a matrix or array:
with matrix
>>> m = np.mat([[1,2],[2,3]])
>>> m
matrix([[1, 2],
[2, 3]])
>>> mm = m.mean(1)
>>> mm
matrix([[ 1.5],
[ 2.5]])
>>> mm.shape
(2, 1)
>>> m - mm
matrix([[-0.5, 0.5],
[-0.5, 0.5]])
with array
>>> a = np.array([[1,2],[2,3]])
>>> a
array([[1, 2],
[2, 3]])
>>> am = a.mean(1)
>>> am.shape
(2,)
>>> am
array([ 1.5, 2.5])
>>> a - am #wrong
array([[-0.5, -0.5],
[ 0.5, 0.5]])
>>> a - am[:, np.newaxis] #right
array([[-0.5, 0.5],
[-0.5, 0.5]])
I also think that mixing arrays and matrices gives rise to many "happy" debugging hours.
However, scipy.sparse matrices are always matrices in terms of operators like multiplication.
As per the official documents, it's not anymore advisable to use matrix class since it will be removed in the future.
https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
As other answers already state that you can achieve all the operations with NumPy arrays.
As others have mentioned, perhaps the main advantage of matrix was that it provided a convenient notation for matrix multiplication.
However, in Python 3.5 there is finally a dedicated infix operator for matrix multiplication: #.
With recent NumPy versions, it can be used with ndarrays:
A = numpy.ones((1, 3))
B = numpy.ones((3, 3))
A # B
So nowadays, even more, when in doubt, you should stick to ndarray.
Matrix Operations with Numpy Arrays:
I would like to keep updating this answer
about matrix operations with numpy arrays if some users are interested looking for information about matrices and numpy.
As the accepted answer, and the numpy-ref.pdf said:
class numpy.matrix will be removed in the future.
So now matrix algebra operations has to be done
with Numpy Arrays.
a = np.array([[1,3],[-2,4]])
b = np.array([[3,-2],[5,6]])
Matrix Multiplication (infix matrix multiplication)
a#b
array([[18, 16],
[14, 28]])
Transpose:
ab = a#b
ab.T
array([[18, 14],
[16, 28]])
Inverse of a matrix:
np.linalg.inv(ab)
array([[ 0.1 , -0.05714286],
[-0.05 , 0.06428571]])
ab_i=np.linalg.inv(ab)
ab#ab_i # proof of inverse
array([[1., 0.],
[0., 1.]]) # identity matrix
Determinant of a matrix.
np.linalg.det(ab)
279.9999999999999
Solving a Linear System:
1. x + y = 3,
x + 2y = -8
b = np.array([3,-8])
a = np.array([[1,1], [1,2]])
x = np.linalg.solve(a,b)
x
array([ 14., -11.])
# Solution x=14, y=-11
Eigenvalues and Eigenvectors:
a = np.array([[10,-18], [6,-11]])
np.linalg.eig(a)
(array([ 1., -2.]), array([[0.89442719, 0.83205029],
[0.4472136 , 0.5547002 ]])
An advantage of using matrices is for easier instantiation through text rather than nested square brackets.
With matrices you can do
np.matrix("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
and get the desired output directly:
matrix([[1.+0.j, 1.+1.j, 0.+0.j],
[0.+0.j, 0.+1.j, 0.+0.j],
[0.+0.j, 0.+0.j, 1.+0.j]])
If you use arrays, this does not work:
np.array("1, 1+1j, 0; 0, 1j, 0; 0, 0, 1")
output:
array('1, 1+1j, 0; 0, 1j, 0; 0, 0, 1', dtype='<U29')