I've got two 2D numpy arrays called A and B, where A is M x N and B is M x n. My problem is that I wish to multiply each element of each row of B with corresponding row of A and create a 3D matrix C which is of size M x n x N, without using for-loops.
As an example, if A is:
A = np.array([[1, 2, 3],
[4, 5, 6]])
and B is
B = np.array([[1, 2],
[3, 4]])
Then the resulting multiplication C = A x B would look something like
C = [
[[1, 2],
[12, 16]],
[[2, 4],
[15, 20]],
[[3, 6],
[18, 24]]
]
Is it clear what I'm trying to achieve, and is it possible doing without any for-loops? Best, tingis
C=np.einsum('ij,ik->jik',A,B)
It is possible by creating a new axis in each array and transposing the modified A:
A[np.newaxis,...].T * B[np.newaxis,...]
giving:
array([[[ 1, 2],
[12, 16]],
[[ 2, 4],
[15, 20]],
[[ 3, 6],
[18, 24]]])
Related
I want to add one numpy array two another so it will look like this:
a = [3, 4]
b = [[6, 5], [2, 1]]
output:
[[3, 4], [[6, 5], [2, 1]]]
It should look like the output above and not like [[3,4],[6,5],[2,1]].
How do I do that with numpy arrays?
Work with pure python lists, and not numpy arrays.
It doesn't make sense to have a numpy array holding two list objects. There's literally no gain in doing that.
If you directly instantiate an array like so:
np.array([[3, 4], [[6, 5], [2, 1]]])
You get
array([[3, 4],
[list([6, 5]), list([2, 1])]], dtype=object)
which is an array with dtype=object. Most of numpy's power is lost in this case. For more information on examples and why, take a look at this thread.
If you work with pure python lists, then you can easily achieve what you want:
>>> a + b
[[3, 4], [[6, 5], [2, 1]]]
Numpy as built-in stack commands that are (in my opinion) slightly easier to use:
>>> a = np.array([3, 4])
>>> b = np.array([[6, 5], [2, 1]])
>>> np.row_stack([b, a])
array([[3, 4],
[6, 5],
[2, 1]])
There's also a column stack.
Ref: https://numpy.org/doc/stable/reference/generated/numpy.ma.row_stack.html
You can't stack arrays of different shapes in one array the way you want (or you have to fill in gaps with NaNs or zeroes), so if you want to iterate over them, consider using list.
a = np.array([3, 4])
b = np.array([[6, 5], [2, 1]])
c = [a, b]
for arr in c:
...
If you still want a numpy array, you can try this:
>>> a = np.array([3, 4])
>>> b = np.array([[6, 5], [2, 1]])
>>> a.resize(b.shape,refcheck=False)
>>> c = np.array([a, b])
array([[[3, 4],
[0, 0]],
[[6, 5],
[2, 1]]])
A and B matrices will be different when i run the program
A = np.array([[1, 1, 1], [2, 2, 2]])
B = np.array([[1, 1, 1], [2, 2, 2], [3, 3, 3]])
The output matrix (C) should be the same dimension as matrix A.
As title says, I'm trying to multiply each row from one matrix (A) to every rows to another matrix (B) and would like to sum them.
For example,
Dimension of C = (2,3)
C = [[A(0)*B(0) + A(1)*B(0)], [A(0)*B(1) + A(1)*B(1)],[A(0)*B(1) + A(1)*B(1)]]
I would like to know if there is a numpy function does that.
Use numpy broadcasting:
C = (A * B[:, None]).sum(axis=1)
Output:
>>> C
array([[3, 3, 3],
[6, 6, 6],
[9, 9, 9]])
Suppose I have an array a of shape (2, 2, 2):
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
and an array b that is the max of a: b=a.max(-1) (row-wise):
b = np.array([[9, 19],
[24, 18]])
I'd like to obtain the index of elements in b using index in flattened a, i.e. a.reshape(-1):
array([ 7, 9, 19, 18, 24, 5, 18, 11])
The result should be an array that is the same shape with b with indices of b in flattened a:
array([[1, 2],
[4, 6]])
Basically this is the result of maxpool2d when return_indices= True in pytorch, but I'm looking for an implementation in numpy. I've used where but it seems doesn't work, also is it possible to combine finding max and indices in one go, to be more efficient? Thanks for any help!
I have a solution similar to that of Andras based on np.argmax and np.arange. Instead of "indexing the index" I propose to add a piecewise offset to the result of np.argmax:
import numpy as np
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
off = np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
>>> off
array([[0, 2],
[4, 6]])
This results in:
>>> a.argmax(-1) + off
array([[1, 2],
[4, 6]])
Or as a one-liner:
>>> a.argmax(-1) + np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
array([[1, 2],
[4, 6]])
The only solution I could think of right now is generating a 2d (or 3d, see below) range that indexes your flat array, and indexing into that with the maximum indices that define b (i.e. a.argmax(-1)):
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
b_shape = a.shape[:-1]
b_size = np.prod(b_shape)
flat_inds = np.arange(a.size).reshape(b_size, -1)
flat_max_inds = flat_inds[range(b_size), multi_inds.ravel()]
max_inds = flat_max_inds.reshape(b_shape)
I separated the steps with some meaningful variable names, which should hopefully explain what's going on.
multi_inds tells you which "column" to choose in each "row" in a to get the maximum:
>>> multi_inds
array([[1, 0],
[0, 0]])
flat_inds is a list of indices, from which one value is to be chosen in each row:
>>> flat_inds
array([[0, 1],
[2, 3],
[4, 5],
[6, 7]])
This is indexed into exactly according to the maximum indices in each row. flat_max_inds are the values you're looking for, but in a flat array:
>>> flat_max_inds
array([1, 2, 4, 6])
So we need to reshape that back to match b.shape:
>>> max_inds
array([[1, 2],
[4, 6]])
A slightly more obscure but also more elegant solution is to use a 3d index array and use broadcasted indexing into it:
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
i, j = np.indices(a.shape[:-1])
max_inds = np.arange(a.size).reshape(a.shape)[i, j, multi_inds]
This does the same thing without an intermediate flattening into 2d.
The last part is also how you can get b from multi_inds, i.e. without having to call a *max function a second time:
b = a[i, j, multi_inds]
This is a long one-liner
new = np.array([np.where(a.reshape(-1)==x)[0][0] for x in a.max(-1).reshape(-1)]).reshape(2,2)
print(new)
array([[1, 2],
[4, 3]])
However number = 18 is repeated twice; So which index is the target.
I am new to using numpy so sorry if this sounds obvious, I did try to search through stackoverflow before I post this though..
I have two "list of lists" numpy arrays of length n (n = 3 in the example below)
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[2, 2], [3, 3], [4, 4]])
I want to get a 1d array with the dot product of the lists at each corresponding index, i.e.
[(1*2 + 2*2), (3*3 + 4*3), (5*4 + 6*4)]
[6, 21, 44]
how should I go about doing it? thanks in advance!
You can do this
np.sum(a*b,axis=1)
The sum method in the other answer is the most straight forward method:
In [19]: a = np.array([[1, 2], [3, 4], [5, 6]])
...: b = np.array([[2, 2], [3, 3], [4, 4]])
In [20]: a*b
Out[20]:
array([[ 2, 4],
[ 9, 12],
[20, 24]])
In [21]: _.sum(1)
Out[21]: array([ 6, 21, 44])
With dot we have think a bit outside the box. einsum is easiest way of specifying a dot like action with less-than-obvious dimension combinations:
In [22]: np.einsum('ij,ij->i',a,b)
Out[22]: array([ 6, 21, 44])
Note that the i dimension is carried through. dot does ij,jk->ik, which would require extracting the diagonal (throwing away extra terms). In matmul/# terms, the i dimension is a 'batch' one, that doesn't actually participate in the sum-of-products. To use that:
In [23]: a[:,None,:]#b[:,:,None]
Out[23]:
array([[[ 6]],
[[21]],
[[44]]])
and then remove the extra size 1 dimensions:
In [24]: _.squeeze()
Out[24]: array([ 6, 21, 44])
In einsum terms this is i1j,ij1->i11
Is there any efficient numpy way to do the following:
Assume I have some matix M of size R X C. Now assume I have another matrix
E which is of shape R X a (where a is just some constant a < C), which contains row indices of
M (and -1 for padding, i.e., every element of E is in {-1, 0, .., R-1}). For example,
M=array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
E = array([[ 0, 1],
[ 2, -1],
[-1, 0]])
Now, given those matrices, I want to generate a third matrix P, where the i'th row of P will
contain the sum of the following rows of M : E[i,:]. In the example, P will be,
P[0,:] = M[0,:] + M[1,:]
P[1,:] = M[2,:]
P[2,:] = M[0,:]
Yes, doing it with a loop is pretty straight forward and easy, I was wondering if there is
any fancy numpy way to make it more efficient (assuming that I want to do it with large matrices,
e.g., 200 X 200.
Thanks!
One way would be to sum with indexed on original array and then subtract out the summations caused by the last indexed ones by -1s -
out = M[E].sum(1) - M[-1]*(E==-1).sum(1)[:,None]
Another way would be pad zeros at the end of M, so that those -1 would index into those zeros and hence have no effect on the final sum after indexing -
M1 = np.vstack((M, np.zeros((1,M.shape[1]), dtype=M.dtype)))
out = M1[E].sum(1)
If there is exactly one or lesser -1 per row in E, we can optimize further -
out = M[E].sum(1)
m = (E==-1).any(1)
out[m] -= M[-1]
Another based on tensor-multiplication -
np.einsum('ij,kli->kj',M, (E[...,None]==np.arange(M.shape[1])))
You could index M with E, and np.sum where the actual indices in E are greater or equal to 0. For that we have the where parameter:
np.sum(M[E], where=(E>=0)[...,None], axis=1)
array([[5, 7, 9],
[7, 8, 9],
[1, 2, 3]])
Where we have that:
M[E]
array([[[1, 2, 3],
[4, 5, 6]],
[[7, 8, 9],
[7, 8, 9]],
[[7, 8, 9],
[1, 2, 3]]])
Is added on the rows:
(E>=0)[...,None]
array([[[ True],
[ True]],
[[ True],
[False]],
[[False],
[ True]]])
Probably not the fastest but maybe educational: The operation you are describing can be thought of as matrix multiplication with a certain adjacency matrix:
from scipy import sparse
# construct adjacency matrix
indices = E[E!=-1]
indptr = np.concatenate([[0],np.count_nonzero(E!=-1,axis=1).cumsum()])
data = np.ones_like(indptr)
aux = sparse.csr_matrix((data,indices,indptr))
# multiply
aux*M
# array([[5, 7, 9],
# [7, 8, 9],
# [1, 2, 3]], dtype=int64)