Suppose I have an array a of shape (2, 2, 2):
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
and an array b that is the max of a: b=a.max(-1) (row-wise):
b = np.array([[9, 19],
[24, 18]])
I'd like to obtain the index of elements in b using index in flattened a, i.e. a.reshape(-1):
array([ 7, 9, 19, 18, 24, 5, 18, 11])
The result should be an array that is the same shape with b with indices of b in flattened a:
array([[1, 2],
[4, 6]])
Basically this is the result of maxpool2d when return_indices= True in pytorch, but I'm looking for an implementation in numpy. I've used where but it seems doesn't work, also is it possible to combine finding max and indices in one go, to be more efficient? Thanks for any help!
I have a solution similar to that of Andras based on np.argmax and np.arange. Instead of "indexing the index" I propose to add a piecewise offset to the result of np.argmax:
import numpy as np
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
off = np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
>>> off
array([[0, 2],
[4, 6]])
This results in:
>>> a.argmax(-1) + off
array([[1, 2],
[4, 6]])
Or as a one-liner:
>>> a.argmax(-1) + np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
array([[1, 2],
[4, 6]])
The only solution I could think of right now is generating a 2d (or 3d, see below) range that indexes your flat array, and indexing into that with the maximum indices that define b (i.e. a.argmax(-1)):
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
b_shape = a.shape[:-1]
b_size = np.prod(b_shape)
flat_inds = np.arange(a.size).reshape(b_size, -1)
flat_max_inds = flat_inds[range(b_size), multi_inds.ravel()]
max_inds = flat_max_inds.reshape(b_shape)
I separated the steps with some meaningful variable names, which should hopefully explain what's going on.
multi_inds tells you which "column" to choose in each "row" in a to get the maximum:
>>> multi_inds
array([[1, 0],
[0, 0]])
flat_inds is a list of indices, from which one value is to be chosen in each row:
>>> flat_inds
array([[0, 1],
[2, 3],
[4, 5],
[6, 7]])
This is indexed into exactly according to the maximum indices in each row. flat_max_inds are the values you're looking for, but in a flat array:
>>> flat_max_inds
array([1, 2, 4, 6])
So we need to reshape that back to match b.shape:
>>> max_inds
array([[1, 2],
[4, 6]])
A slightly more obscure but also more elegant solution is to use a 3d index array and use broadcasted indexing into it:
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
i, j = np.indices(a.shape[:-1])
max_inds = np.arange(a.size).reshape(a.shape)[i, j, multi_inds]
This does the same thing without an intermediate flattening into 2d.
The last part is also how you can get b from multi_inds, i.e. without having to call a *max function a second time:
b = a[i, j, multi_inds]
This is a long one-liner
new = np.array([np.where(a.reshape(-1)==x)[0][0] for x in a.max(-1).reshape(-1)]).reshape(2,2)
print(new)
array([[1, 2],
[4, 3]])
However number = 18 is repeated twice; So which index is the target.
Related
z = np.arange(15).reshape(3,5)
indexx = [0,2]
indexy = [1,2,3,4]
zz = []
for i in indexx:
for j in indexy:
zz.append(z[i][j])
Output:
zz >> [1, 2, 3, 4, 11, 12, 13, 14]
This essentially flattens the array but only keeping the elements that have indicies present in the two indices list.
This works, but it is very slow for larger arrays/list of indicies. Is there a way to speed this up using numpy?
Thanks.
Edited to show desired output.
A list of integers can be used to access the entries of interest for numpy arrays.
z[indexx][:,indexy].flatten()
x = {"apple", "banana", "cherry"}
y = {"google", "microsoft", "apple"}
z = x.intersection(y)
print(z)
z => apples
If I understand you correctly, just use Python set. And then cast it to list.
Indexing in several dimensions at once requires broadcasting the indices against each other. np.ix_ is a handy tool for doing this:
In [127]: z
Out[127]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
In [128]: z[np.ix_(indexx, indexy)]
Out[128]:
array([[ 1, 2, 3, 4],
[11, 12, 13, 14]])
Converting that to 1d is a trivial ravel() task.
Look at the ix_ produces, here it's a (2,1) and (1,4) array. You can construct such arrays 'from-scratch':
In [129]: np.ix_(indexx, indexy)
Out[129]:
(array([[0],
[2]]),
array([[1, 2, 3, 4]]))
I am new to using numpy so sorry if this sounds obvious, I did try to search through stackoverflow before I post this though..
I have two "list of lists" numpy arrays of length n (n = 3 in the example below)
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[2, 2], [3, 3], [4, 4]])
I want to get a 1d array with the dot product of the lists at each corresponding index, i.e.
[(1*2 + 2*2), (3*3 + 4*3), (5*4 + 6*4)]
[6, 21, 44]
how should I go about doing it? thanks in advance!
You can do this
np.sum(a*b,axis=1)
The sum method in the other answer is the most straight forward method:
In [19]: a = np.array([[1, 2], [3, 4], [5, 6]])
...: b = np.array([[2, 2], [3, 3], [4, 4]])
In [20]: a*b
Out[20]:
array([[ 2, 4],
[ 9, 12],
[20, 24]])
In [21]: _.sum(1)
Out[21]: array([ 6, 21, 44])
With dot we have think a bit outside the box. einsum is easiest way of specifying a dot like action with less-than-obvious dimension combinations:
In [22]: np.einsum('ij,ij->i',a,b)
Out[22]: array([ 6, 21, 44])
Note that the i dimension is carried through. dot does ij,jk->ik, which would require extracting the diagonal (throwing away extra terms). In matmul/# terms, the i dimension is a 'batch' one, that doesn't actually participate in the sum-of-products. To use that:
In [23]: a[:,None,:]#b[:,:,None]
Out[23]:
array([[[ 6]],
[[21]],
[[44]]])
and then remove the extra size 1 dimensions:
In [24]: _.squeeze()
Out[24]: array([ 6, 21, 44])
In einsum terms this is i1j,ij1->i11
Suppose you have a matrix:
a = np.arange(9).reshape(3,3)
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
and I want get or set over the values 1, 5, and 6, how would I do that.
For example I thought doing
# getting
b = a[:, np.array([1,2,0])]
# want b = [1,5,6]
# setting
a[:, np.array([1,2,0])] = np.array([9, 10, 11])
# want:
# a = array([[0, 9, 2],
# [3, 4, 10],
# [11, 7, 8]])
would do it, but that is not the case. Any thoughts on this?
Only a small tweak makes this work:
import numpy as np
a = np.arange(9).reshape(3,3)
# getting
b = a[range(a.shape[0]), np.array([1,2,0])]
# setting
a[range(a.shape[0]), np.array([1,2,0])] = np.array([9, 10, 11])
The reason why your code didn't work as expected is because you were indexing the x-axis with slices instead of indices. Slices mean take all rows, but specifying the index directly will get you the row you want for each index value.
I have a 3 dimensional numpy array. The dimension can go up to 128 x 64 x 8192. What I want to do is to change the order in the first dimension by interchanging pairwise.
The only idea I had so far is to create a list of the indices in the correct order.
order = [1,0,3,2...127,126]
data_new = data[order]
I fear, that this is not very efficient but I have no better idea so far
You could reshape to split the first axis into two axes, such that latter of those axes is of length 2 and then flip the array along that axis with [::-1] and finally reshape back to original shape.
Thus, we would have an implementation like so -
a.reshape(-1,2,*a.shape[1:])[:,::-1].reshape(a.shape)
Sample run -
In [170]: a = np.random.randint(0,9,(6,3))
In [171]: order = [1,0,3,2,5,4]
In [172]: a[order]
Out[172]:
array([[0, 8, 5],
[4, 5, 6],
[0, 0, 2],
[7, 3, 8],
[1, 6, 3],
[2, 4, 4]])
In [173]: a.reshape(-1,2,*a.shape[1:])[:,::-1].reshape(a.shape)
Out[173]:
array([[0, 8, 5],
[4, 5, 6],
[0, 0, 2],
[7, 3, 8],
[1, 6, 3],
[2, 4, 4]])
Alternatively, if you are looking to efficiently create those constantly flipping indices order, we could do something like this -
order = np.arange(data.shape[0]).reshape(-1,2)[:,::-1].ravel()
I've got two 2D numpy arrays called A and B, where A is M x N and B is M x n. My problem is that I wish to multiply each element of each row of B with corresponding row of A and create a 3D matrix C which is of size M x n x N, without using for-loops.
As an example, if A is:
A = np.array([[1, 2, 3],
[4, 5, 6]])
and B is
B = np.array([[1, 2],
[3, 4]])
Then the resulting multiplication C = A x B would look something like
C = [
[[1, 2],
[12, 16]],
[[2, 4],
[15, 20]],
[[3, 6],
[18, 24]]
]
Is it clear what I'm trying to achieve, and is it possible doing without any for-loops? Best, tingis
C=np.einsum('ij,ik->jik',A,B)
It is possible by creating a new axis in each array and transposing the modified A:
A[np.newaxis,...].T * B[np.newaxis,...]
giving:
array([[[ 1, 2],
[12, 16]],
[[ 2, 4],
[15, 20]],
[[ 3, 6],
[18, 24]]])