I have an N*M matrix A and a N-length vector V. I want to do A + V where each element in a row i of A is summed with the element i in V. How to do that?
e.g.:
A = np.random.rand(3,2)
V = np.array([1,2,3])
A + V
ValueError: operands could not be broadcast together with shapes (3,2) (3)
I want to do the same with multiplication and division.
Solution:
V = V.reshape(-1,1)
A + V
now this works
The concept to look up here is broadcasting. It enables you to have pointwise interaction between any two matrices as long as their shapes correspond or, on the axes without correspondence, at least one of the sides is degenerate, i.e. of size 1. In your case, you need to add an axis to V. One can do this elegantly as follows
A + V[:, np.newaxis]
You need to tell numpy it has to extend the dimensions of the vector V by one, and you can do it using the special index np.newaxis. It would look like this:
import numpy as np
A = np.array([[10,20],[100,200],[1000,2000]])
V = np.array([1,2,3])
A + V[:,np.newaxis]
array([[ 11, 21],
[ 102, 202],
[1003, 2003]])
From the slicing docs:
Each newaxis object in the selection tuple serves to expand the dimensions of the resulting selection by one unit-length dimension. The added dimension is the position of the newaxis object in the selection tuple.
Related
i have an array y with shape (n,), I want to compute the inner product matrix, which is a n * n matrix
However, when I tried to do it in Python
np.dot(y , y)
I got the answer n, this is not what I am looking for
I have also tried:
np.dot(np.transpose(y),y)
np.dot(y, np.transpose(y))
I always get the same answer n
I think you are looking for:
np.multiply.outer(y,y)
or equally:
y = y[None,:]
y.T#y
example:
y = np.array([1,2,3])[None,:]
output:
#[[1 2 3]
# [2 4 6]
# [3 6 9]]
You can try to reshape y from shape (70,) to (70,1) before multiplying the 2 matrices.
# Reshape
y = y.reshape(70,1)
# Either below code would work
y*y.T
np.matmul(y,y.T)
One-liner?
np.dot(a[:, None], a[None, :])
transpose doesn't work on 1-D arrays, because you need atleast two axes to 'swap' them. This solution adds a new axis to the array; in the first argument, it looks like a column vector and has two axes; in the second argument it still looks like a row vector but has two axes.
Looks like what you need is the # matrix multiplication operator. dot method is only to compute dot product between vectors, what you want is matrix multiplication.
>>> a = np.random.rand(70, 1)
>>> (a # a.T).shape
(70, 70)
UPDATE:
Above answer is incorrect. dot does the same things if the array is 2D. See the docs here.
np.dot computes the dot product of two arrays. Specifically,
If both a and b are 1-D arrays, it is inner product of vectors (without complex conjugation).
If both a and b are 2-D arrays, it is matrix multiplication, but using matmul or a # b is preferred.
Simplest way to do what you want is to convert the vector to a matrix first using np.matrix and then using the #. Although, dot can also be used # is better because conventionally dot is used for vectors and # for matrices.
>>> a = np.random.rand(70)
(70,)
>>> a.shape
>>> a = np.matrix(a).T
>>> a.shape
(70, 1)
>>> (a # a.T).shape
(70, 70)
Let's say I have a 4d array A with shape (D0, D1, D2, D3). I have a 1d array B with shape (D0,), which includes the indices I need at axis 2.
The trivial way to implement what I need:
output_lis = []
for i in range(D0):
output_lis.append(A[i, :, B[i], :])
#output = np.concatenate(output_lis, axis=0) #it is wrong to use concatenate. Thanks to #Mad Physicist. Instead, using stack.
output = np.stack(output_lis, axis=0) #shape: [D0, D1, D3]
So, my question is how to implement it with numpy API in a fast way?
Use fancy indexing to step along two dimensions in lockstep. In this case, arange provides the sequence i, while B provides the sequence B[i]:
A[np.arange(D0), :, B, :]
The shape of this array is indeed (D0, D1, D3), unlike the shape of your for loop result.
To get the same result from your example, use stack (which adds a new axis), rather than concatenate (which uses an existing axis):
output = np.stack(output_lis, axis=0)
I sometimes work with 1D arrays:
A = np.array([1, 2, 3, 4])
or 2D arrays (mono or stereo signals read with scipy.io.wavfile):
A = np.array([[1, 2], [3, 4], [5, 6], [7,8]])
Without having to distinguish these 2 cases with an if A.ndim == 2:..., is there a simple one-line solution to multiply this array A by an 1D array B = np.linspace(0., 1., 4)?
If A is 1D, then it's just A * B, and if A is 2D, what I mean is multiply each line of A by each element of B.
Note: this question arises naturally when working with both mono and stereo sounds read with scipy.io.wavfile.
Approach #1
We can use einsum to cover generic ndarrays -
np.einsum('i...,i->i...',A,B)
The trick that works here is the ellipsis that broadcasts for the trailing dimensions after the first axis as they are in A and keeps them in the output, while keeping the first axes for the two inputs aligned, which is the intended multiplication here. With A as 1D array, there's no broadcasting and that essentially reduces to : np.einsum('i,i->i',A,B) under the hoods.
Schematically put :
A : i x ....
B : i
out : i x ....
Hence, covers for A with any number of dimensions.
More info on the use of ellipsis from the docs :
To enable and control broadcasting, use an ellipsis. Default
NumPy-style broadcasting is done by adding an ellipsis to the left of
each term, like np.einsum('...ii->...i', a). To take the trace along
the first and last axes, you can do np.einsum('i...i', a), or to do a
matrix-matrix product with the left-most indices instead of rightmost,
you can do np.einsum('ij...,jk...->ik...', a, b).
Approach #2
Using the fact that we are trying to align the first axis of A with the only axis of 1D array B, we can simply transpose A, multiply with B and finally transpose back -
(A.T*B).T
I'm trying to understand how numpy works when you try to call the dot product of two row vectors.
I have this code:
X = np.array([[1,2,3]])
THETA = np.array([[1,2,3]])
print X.dot(THETA)
This gives me the error:
ValueError: shapes (1,3) and (1,3) not aligned: 3 (dim 1) != 1 (dim 0)
I thought that you could take the dot product of two row vectors however to get:
x1*theta1 + x2*theta2 + x3*theta3
And this would also transfer to the dot product of two column vectors.
The weird part is, I have to take the transpose of the second matrix in order to actually use the dot product:
print X.dot(THETA.T)
array([[14]])
However, I didn't think this would actually work, and why it would work instead of just doing a row dot row operation. Can anyone help me understand what's going on? Is it some rule in linear algebra that I forgot from long ago?
dot for 2D input is matrix multiplication, not a dot product. What you're seeing is just the result of the normal rules of matrix multiplication. If you want a vector dot product, the easiest way is to use 1D vectors, with no superfluous second dimension:
X = np.array([1, 2, 3])
THETA = np.array([1, 2, 3])
print X.dot(THETA)
dot-ting two 1D arrays takes a dot product and produces a scalar result.
If you want to use row and column vectors, then by the standard rules of matrix multiplication, you need to multiply a 1-by-N array (a row vector) by an N-by-1 array (a column vector) to get a 1-by-1 result, and NumPy will give you a 1-by-1 array rather than a scalar.
The alignment error you're seeing is because you're trying to represent a 1D vector as a 2D array.
In [1]: import numpy as np
In [2]: X = np.array([1,2,3])
In [3]: THETA = np.array([1,2,3])
In [4]: print X.dot(THETA)
14
In [5]: print X.dot(THETA.T)
14
And:
x1*theta1 + x2*theta2 + x3*theta3 =
1*1 + 2*2 + 3*3 =
14
I don't understand broadcasting. The documentation explains the rules of broadcasting but doesn't seem to define it in English. My guess is that broadcasting is when NumPy fills a smaller dimensional array with dummy data in order to perform an operation. But this doesn't work:
>>> x = np.array([1,3,5])
>>> y = np.array([2,4])
>>> x+y
*** ValueError: operands could not be broadcast together with shapes (3,) (2,)
The error message hints that I'm on the right track, though. Can someone define broadcasting and then provide some simple examples of when it works and when it doesn't?
The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations.
It's basically a way numpy can expand the domain of operations over arrays.
The only requirement for broadcasting is a way aligning array dimensions such that either:
Aligned dimensions are equal.
One of the aligned dimensions is 1.
So, for example if:
x = np.ndarray(shape=(4,1,3))
y = np.ndarray(shape=(3,3))
You could not align x and y like so:
4 x 1 x 3
3 x 3
But you could like so:
4 x 1 x 3
3 x 3
How would an operation like this result?
Suppose we have:
x = np.ndarray(shape=(1,3), buffer=np.array([1,2,3]),dtype='int')
array([[1, 2, 3]])
y = np.ndarray(shape=(3,3), buffer=np.array([1,1,1,1,1,1,1,1,1]),dtype='int')
array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]])
The operation x + y would result in:
array([[2, 3, 4],
[2, 3, 4],
[2, 3, 4]])
I hope you caught the drift. If you did not, you can always check the official documentation here.
Cheers!
1.What is Broadcasting?
Broadcasting is a Tensor operation. Helpful in Neural Network (ML, AI)
2.What is the use of Broadcasting?
Without Broadcasting addition of only identical Dimension(shape) Tensors is supported.
Broadcasting Provide us the Flexibility to add two Tensors of Different Dimension.
for Example: adding a 2D Tensor with a 1D Tensor is not possible without broadcasting see the image explaining Broadcasting pictorially
Run the Python example code understand the concept
x = np.array([1,3,5,6,7,8])
y = np.array([2,4,5])
X=x.reshape(2,3)
x is reshaped to get a 2D Tensor X of shape (2,3), and adding this 2D Tensor X with 1D Tensor y of shape(1,3) to get a 2D Tensor z of shape(2,3)
print("X =",X)
print("\n y =",y)
z=X+y
print("X + y =",z)
You are almost correct about smaller Tensor, no ambiguity, the smaller tensor will be broadcasted to match the shape of the larger tensor.(Small vector is repeated but not filled with Dummy Data or Zeros to Match the Shape of larger).
3. How broadcasting happens?
Broadcasting consists of two steps:
1 Broadcast axes are added to the smaller tensor to match the ndim of
the larger tensor.
2 The smaller tensor is repeated alongside these new axes to match the full shape
of the larger tensor.
4. Why Broadcasting not happening in your code?
your code is working but Broadcasting can not happen here because both Tensors are different in shape but Identical in Dimensional(1D).
Broadcasting occurs when dimensions are nonidentical.
what you need to do is change Dimension of one of the Tensor, you will experience Broadcasting.
5. Going in Depth.
Broadcasting(repetition of smaller Tensor) occurs along broadcast axes but since both the Tensors are 1 Dimensional there is no broadcast Axis.
Don't Confuse Tensor Dimension with the shape of tensor,
Tensor Dimensions are not same as Matrices Dimension.
Broadcasting is numpy trying to be smart when you tell it to perform an operation on arrays that aren't the same dimension. For example:
2 + np.array([1,3,5]) == np.array([3, 5, 7])
Here it decided you wanted to apply the operation using the lower dimensional array (0-D) on each item in the higher-dimensional array (1-D).
You can also add a 0-D array (scalar) or 1-D array to a 2-D array. In the first case, you just add the scalar to all items in the 2-D array, as before. In the second case, numpy will add row-wise:
In [34]: np.array([1,2]) + np.array([[3,4],[5,6]])
Out[34]:
array([[4, 6],
[6, 8]])
There are ways to tell numpy to apply the operation along a different axis as well. This can be taken even further with applying an operation between a 3-D array and a 1-D, 2-D, or 0-D array.
>>> x = np.array([1,3,5])
>>> y = np.array([2,4])
>>> x+y
*** ValueError: operands could not be broadcast together with shapes (3,) (2,)
Broadcasting is how numpy do math operations with array of different shapes. Shapes are the format the array has, for example the array you used, x , has 3 elements of 1 dimension; y has 2 elements and 1 dimension.
To perform broadcasting there are 2 rules:
1) Array have the same dimensions(shape) or
2)The dimension that doesn't match equals one.
for example x has shape(2,3) [or 2 lines and 3 columns];
y has shape(2,1) [or 2 lines and 1 column]
Can you add them? x + y?
Answer: Yes, because the mismatched dimension is equal to 1 (the column in y). If y had shape(2,4) broadcasting would not be possible, because the mismatched dimension is not 1.
In the case you posted:
operands could not be broadcast together with shapes (3,) (2,);
it is because 3 and 2 mismatched altough both have 1 line.
I would like to suggest to try the np.broadcast_arrays, run some demos may give intuitive ideas. Official Document is also helpful. From my current understanding, numpy will compare the dimension from tail to head. If one dim is 1, it will broadcast in the dimension, if one array has more axes, such (256*256*3) multiply (1,), you can view (1) as (1,1,1). And broadcast will make (256,256,3).