This question already has answers here:
Indexing one array by another in numpy
(4 answers)
Closed 3 years ago.
Say that I have a numpy array a with the shape: [z, y, x], and another array b with the shape: [y, x].
Array b contains indices along z that I would like to extract from a for each y and x.
So far I have the following inelegant way of doing this:
from_a = np.full_like(b, np.nan)
for i in range(y):
for j in range(x):
des_ind = b[i,j]
from_a[i,j] = a[des_ind,i,j]
Is there a nice neat pythonic way of doing this?
You could use the array b directly as index for a:
import numpy as np
z, y, x = 3, 4, 5
a = np.random.randint(10, (z, y, x))
# array([[[2, 1, 8, 3, 0], | [[9, 3, 6, 8, 7], | [[6, 6, 2, 1, 4],
# [4, 0, 1, 2, 9], | [5, 9, 0, 7, 1], | [3, 9, 4, 4, 8],
# [0, 0, 7, 2, 6], | [7, 9, 8, 4, 9], | [9, 7, 0, 3, 0],
# [3, 5, 2, 9, 3]], | [6, 8, 9, 5, 9]], | [4, 3, 0, 1, 1]]])
b = np.random.randint(z, size=(y, x))
# array([[1, 1, 1, 0, 2],
# [1, 2, 1, 1, 2],
# [2, 0, 0, 0, 2],
# [0, 0, 1, 0, 0]])
from_a = a[b,
np.arange(y)[:, np.newaxis],
np.arange(x)[np.newaxis, :]]
# array([[9, 3, 6, 3, 4],
# [5, 9, 0, 7, 8],
# [9, 0, 7, 2, 0],
# [3, 5, 9, 9, 3]])
See also advanced indexing for numpy arrays.
Related
I have a mask with a size of torch.Size([20, 1, 199]) and a tensor, reconstruct_output and inputs both with a size of torch.Size([20, 1, 161, 199]).
I want to set reconstruct_output to inputs where the mask is 0. I tried:
reconstruct_output[mask == 0] = inputs[mask == 0]
But I get an error:
IndexError: The shape of the mask [20, 1, 199] at index 2 does not match the shape of the indexed tensor [20, 1, 161, 199] at index 2
We can use advanced indexing here. To obtain the indexing arrays which we want to use to index both reconstruct_output and inputs, we need the indices along its axes where m==0. For that we can use np.where, and use the resulting indices to update reconstruct_output as:
m = mask == 0
i, _, l = np.where(m)
reconstruct_output[i, ..., l] = inputs[i, ..., l]
Here's a small example which I've checked with:
mask = np.random.randint(0,3, (2, 1, 4))
reconstruct_output = np.random.randint(0,10, (2, 1, 3, 4))
inputs = np.random.randint(0,10, (2, 1, 3, 4))
Giving for instance:
print(reconstruct_output)
array([[[[8, 9, 7, 2],
[5, 4, 6, 1],
[1, 4, 0, 3]]],
[[[4, 3, 3, 4],
[0, 9, 9, 7],
[3, 4, 9, 3]]]])
print(inputs)
array([[[[7, 3, 9, 8],
[3, 1, 0, 8],
[0, 5, 4, 8]]],
[[[3, 7, 5, 8],
[2, 5, 3, 8],
[3, 6, 7, 5]]]])
And the mask:
print(mask)
array([[[0, 1, 2, 1]],
[[1, 0, 1, 0]]])
By using np.where to find the indices where there are zeroes in mask we get:
m = mask == 0
i, _, l = np.where(m)
i
# array([0, 1, 1])
l
# array([0, 1, 3])
Hence we'll be replacing the 0th column from the first 2D array and the 1st and 3rd from the second 2D array.
We can now use these arrays to replace along the corresponding axes indexing as:
reconstruct_output[i, ..., l] = inputs[i, ..., l]
Getting:
reconstruct_output
array([[[[7, 9, 7, 2],
[3, 4, 6, 1],
[0, 4, 0, 3]]],
[[[4, 7, 3, 8],
[0, 5, 9, 8],
[3, 6, 9, 5]]]])
Suppose I have a matrix A with some arbitrary values:
array([[ 2, 4, 5, 3],
[ 1, 6, 8, 9],
[ 8, 7, 0, 2]])
And a matrix B which contains indices of elements in A:
array([[0, 0, 1, 2],
[0, 3, 2, 1],
[3, 2, 1, 0]])
How do I select values from A pointed by B, i.e.:
A[B] = [[2, 2, 4, 5],
[1, 9, 8, 6],
[2, 0, 7, 8]]
EDIT: np.take_along_axis is a builtin function for this use case implemented since numpy 1.15. See #hpaulj 's answer below for how to use it.
You can use NumPy's advanced indexing -
A[np.arange(A.shape[0])[:,None],B]
One can also use linear indexing -
m,n = A.shape
out = np.take(A,B + n*np.arange(m)[:,None])
Sample run -
In [40]: A
Out[40]:
array([[2, 4, 5, 3],
[1, 6, 8, 9],
[8, 7, 0, 2]])
In [41]: B
Out[41]:
array([[0, 0, 1, 2],
[0, 3, 2, 1],
[3, 2, 1, 0]])
In [42]: A[np.arange(A.shape[0])[:,None],B]
Out[42]:
array([[2, 2, 4, 5],
[1, 9, 8, 6],
[2, 0, 7, 8]])
In [43]: m,n = A.shape
In [44]: np.take(A,B + n*np.arange(m)[:,None])
Out[44]:
array([[2, 2, 4, 5],
[1, 9, 8, 6],
[2, 0, 7, 8]])
More recent versions have added a take_along_axis function that does the job:
A = np.array([[ 2, 4, 5, 3],
[ 1, 6, 8, 9],
[ 8, 7, 0, 2]])
B = np.array([[0, 0, 1, 2],
[0, 3, 2, 1],
[3, 2, 1, 0]])
np.take_along_axis(A, B, 1)
Out[]:
array([[2, 2, 4, 5],
[1, 9, 8, 6],
[2, 0, 7, 8]])
There's also a put_along_axis.
I know this is an old question, but another way of doing it using indices is:
A[np.indices(B.shape)[0], B]
output:
[[2 2 4 5]
[1 9 8 6]
[2 0 7 8]]
Following is the solution using for loop:
outlist = []
for i in range(len(B)):
lst = []
for j in range(len(B[i])):
lst.append(A[i][B[i][j]])
outlist.append(lst)
outarray = np.asarray(outlist)
print(outarray)
Above can also be written in more succinct list comprehension form:
outlist = [ [A[i][B[i][j]] for j in range(len(B[i]))]
for i in range(len(B)) ]
outarray = np.asarray(outlist)
print(outarray)
Output:
[[2 2 4 5]
[1 9 8 6]
[2 0 7 8]]
I have a numpy array. lets say
x=([8, 9, 0, 1, 2, 3, 4, 5, 6, 7,12,13])
x2 = np.reshape(x, (2,6))
now
x2= [[ 8 9 0 1 2 3]
[ 4 5 6 7 12 13]]
I need to shift x2 in such a way that the final result be
X3=[[2 3 0 1 8 9]
[12 13 6 7 4 5]]
A fancy index and swap
x2[:, 0:2], x2[:, -2:] = x2[:, -2:].copy(), x2[:, 0:2].copy()
Out[117]:
array([[ 2, 3, 0, 1, 8, 9],
[12, 13, 6, 7, 4, 5]])
You don't need to copy anything; just slice once and pass both lists of indices.
import numpy as np
x = np.array([8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 12, 13])
x = x.reshape(x, [2, 6])
x = x[:, [[0, -2], [1, -1]]] = x[:, [[-2, 0], [-1, 1]]]
x
# array([
# [ 2, 3, 0, 1, 8, 9],
# [12, 13, 6, 7, 4, 5],
# ])
I noticed you had a tensorflow tag on your question. This is a bit more involved in tensorflow.
import tensorflow as tf
x = tf.constant([
[8, 9, 0, 1, 2, 3],
[4, 5, 6, 7, 12, 13],
])
idx = tf.constant([
[[0, 4], [0, 5], [0, 2], [0, 3], [0, 0], [0, 1]],
[[1, 4], [1, 5], [1, 2], [1, 3], [1, 0], [1, 1]],
])
shp = tf.constant([2, 6])
swapped = tf.scatter_nd(indices=idx, updates=x, shape=shp)
with tf.Session() as sess:
print(swapped.eval(session=sess))
# [[ 2 3 0 1 8 9]
# [12 13 6 7 4 5]]
I am trying to arrange each row of A into a matrix and then compute the eigenvalues.
I need to help to vectorize this operation.
A= np.array([[5, 5, 7, 0, 1, 6],
[4, 0, 9, 3, 4, 0],
[3, 1, 2, 0, 1, 1],
[7, 6, 4, 4, 1, 8],
[3, 1, 9, 8, 0, 1],
[8, 6, 1, 4, 3, 6],
[6, 9, 5, 9, 6, 1],
[5, 9, 6, 8, 3, 3]])
S1 = A[:,0]
S2 = A[:,1]
S3 = A[:,2]
S4 = A[:,3]
S5 = A[:,4]
S6 = A[:,5]
SS=[(S1,S4,S5),(S4,S2,S6),(S5,S6,S3)]
SS=np.array(SS)
reqval=np.zeros([len(A),1])
for i in range(len(A)):
eva = np.linalg.eigvals(SS[:,:,i])
reqval[i] = max(eva)
Permute axes with transpose/rollaxis/moveaxis, such that we bring the first two axes as the last two ones and that lets us use np.linalg.eigvals with a single call, like so -
reqval = np.linalg.eigvals(SS.transpose(2,0,1)).max(1)
To use rollaxis and moveaxis, use : np.rollaxis(SS,2,0) and np.moveaxis(SS,2,0) respectively in place of SS.transpose(2,0,1).
I have a 2D Numpy ndarray, x, that I need to split in square subregions of size s. For each subregion, I want to get the greatest element (which I do), and its position within that subregion (which I can't figure out).
Here is a minimal example:
>>> x = np.random.randint(0, 10, (6,8))
>>> x
array([[9, 4, 8, 9, 5, 7, 3, 3],
[3, 1, 8, 0, 7, 7, 5, 1],
[7, 7, 3, 6, 0, 2, 1, 0],
[7, 3, 9, 8, 1, 6, 7, 7],
[1, 6, 0, 7, 5, 1, 2, 0],
[8, 7, 9, 5, 8, 3, 6, 0]])
>>> h, w = x.shape
>>> s = 2
>>> f = x.reshape(h//s, s, w//s, s)
>>> mx = np.max(f, axis=(1, 3))
>>> mx
array([[9, 9, 7, 5],
[7, 9, 6, 7],
[8, 9, 8, 6]])
For example, the 8 in the lower left corner of mx is the greatest element from subregion [[1,6], [8, 7]] in the lower left corner of x.
What I want is to get an array similar to mx, that keeps the indices of the largest elements, like this:
[[0, 1, 1, 2],
[0, 2, 3, 2],
[2, 2, 2, 2]]
where, for example, the 2 in the lower left corner is the index of 8 in the linear representation of [[1, 6], [8, 7]].
I could do it like this: np.argmax(f[i, :, j, :]) and iterate over i and j, but the speed difference is enormous for large amounts of computation. To give you an idea, I'm trying to use (only) Numpy for max pooling. Basically, I'm asking if there is a faster alternative than what I'm using.
Here's one approach -
# Get shape of output array
m,n = np.array(x.shape)//s
# Reshape and permute axes to bring the block as rows
x1 = x.reshape(h//s, s, w//s, s).swapaxes(1,2).reshape(-1,s**2)
# Use argmax along each row and reshape to output shape
out = x1.argmax(1).reshape(m,n)
Sample input, output -
In [362]: x
Out[362]:
array([[9, 4, 8, 9, 5, 7, 3, 3],
[3, 1, 8, 0, 7, 7, 5, 1],
[7, 7, 3, 6, 0, 2, 1, 0],
[7, 3, 9, 8, 1, 6, 7, 7],
[1, 6, 0, 7, 5, 1, 2, 0],
[8, 7, 9, 5, 8, 3, 6, 0]])
In [363]: out
Out[363]:
array([[0, 1, 1, 2],
[0, 2, 3, 2],
[2, 2, 2, 2]])
Alternatively, to simplify things, we could use scikit-image that does the heavy work of reshaping and permuting axes for us -
In [372]: from skimage.util import view_as_blocks as viewB
In [373]: viewB(x, (s,s)).reshape(-1,s**2).argmax(1).reshape(m,n)
Out[373]:
array([[0, 1, 1, 2],
[0, 2, 3, 2],
[2, 2, 2, 2]])