I want to apply tf.gather() to all the rows of a given parameters tensor and an indices tensor.
I can apply tf.gather() on two 1D tensors to extract a 1D tensor:
# params == array([3, 8, 9, 7, 6])
# inds == array([1, 2, 3])
>>> tf.gather(params, inds).eval()
array([8, 9, 7])
Now what if I have two 2D tensors, and want to apply tf.gather() on them row-wise? I want something like this:
# params == array([[3, 8, 9, 7, 6],
# [6, 1, 7, 0, 7],
# [7, 4, 4, 5, 8]])
# inds == array([[1, 2, 3],
# [2, 3, 4],
# [0, 1, 2]])
>>> row_wise_gather(params, inds)
array([[8, 9, 7],
[7, 0, 7],
[7, 4, 4]]
The closest I've come so far is using tf.gather() with axis=1, which yields a 3D tensor, and then index the result with gather_nd():
>>> gathered3d = tf.gather(params, inds, axis=1)
# gathered3d == array([[[8, 9, 7],
# [9, 7, 6],
# [3, 8, 9]],
#
# [[1, 7, 0],
# [7, 0, 7],
# [6, 1, 7]],
#
# [[4, 4, 5],
# [4, 5, 8],
# [7, 4, 4]]])
>>> tf.gather_nd(gathered3d, [[0, 0], [1, 1], [2, 2]]).eval()
array([[8, 9, 7],
[7, 0, 7],
[7, 4, 4]])
(I'd call other functions instead of giving literal values, but that's beside the point and not an issue)
This is very clumsy. Is there a more efficient way to do this?
By the way, the indices I use are always values increasing one by one; each row just has a different start and end value. That might make the problem easier.
Related
Doing np.roll(a, 1, axis = 1) on:
a = np.array([
[6, 3, 9, 2, 3],
[1, 7, 8, 1, 2],
[5, 4, 2, 2, 4],
[3, 9, 7, 6, 5],
])
results in the correct:
array([
[3, 6, 3, 9, 2],
[2, 1, 7, 8, 1],
[4, 5, 4, 2, 2],
[5, 3, 9, 7, 6]
])
The documentation says:
If a tuple, then axis must be a tuple of the same size, and each of the given axes is shifted by the corresponding number.
Now I like to roll rows of a by different values, like [1,2,1,3] meaning, first row will be rolled by 1, second by 2, third by 1 and forth by 3. But np.roll(a, [1,2,1,3], axis=(1,1,1,1)) doesn't seem to do it. What would be the correct interpretation of the sentence in the docs?
By specifying a tuple in np.roll you can roll an array along various axes. For example, np.roll(a, (3,2), axis=(0,1)) will shift each element of a by 3 places along axis 0, and it will also shift each element by 2 places along axis 1. np.roll does not have an option to roll each row by a different amount. You can do it though for example as follows:
import numpy as np
a = np.array([
[6, 3, 9, 2, 3],
[1, 7, 8, 1, 2],
[5, 4, 2, 2, 4],
[3, 9, 7, 6, 5],
])
shifts = np.c_[[1,2,1,3]]
a[np.c_[:a.shape[0]], (np.r_[:a.shape[1]] - shifts) % a.shape[1]]
It gives:
array([[3, 6, 3, 9, 2],
[1, 2, 1, 7, 8],
[4, 5, 4, 2, 2],
[7, 6, 5, 3, 9]])
I have two matrices of the same size, A, B. I want to use the columns of B to acsses the columns of A, on a per column basis. For example,
A = np.array([[1, 4, 7],
[2, 5, 8],
[3, 6, 9]])
and
B = np.array([[0, 0, 2],
[1, 2, 1],
[2, 1, 0]])
I want something like:
A[B] = [[1, 4, 9],
[2, 6, 8],
[3, 5, 7]]
I.e., I've used the j'th column of B as indices to the j'th column of A.
Is there any effiecnt way of doing so?
Thanks!
You can use advanced indexing:
A[B, np.arange(A.shape[0])]
array([[1, 4, 9],
[2, 6, 8],
[3, 5, 7]])
Or with np.take_along_axis:
np.take_along_axis(A, B, axis=0)
array([[1, 4, 9],
[2, 6, 8],
[3, 5, 7]])
I want to use this code on very huge array. this code take long time to execute and it is not efficient.
is there any way to remove loop and convert this code to optimum way?
>>> import numpy as np
>>> x=np.random.randint(10, size=(4,5,3))
>>> x
array([[[3, 2, 6],
[4, 6, 6],
[3, 7, 9],
[6, 4, 2],
[9, 0, 1]],
[[9, 0, 4],
[1, 8, 9],
[6, 8, 1],
[9, 4, 5],
[1, 5, 2]],
[[6, 1, 6],
[1, 8, 8],
[3, 8, 3],
[7, 1, 0],
[7, 7, 0]],
[[5, 6, 6],
[8, 3, 1],
[0, 5, 4],
[6, 1, 2],
[5, 6, 1]]])
>>> y=[]
>>> for i in range(x.shape[1]):
for j in range(x.shape[2]):
y.append(x[:, i, j].tolist())
>>> y
[[3, 9, 6, 5], [2, 0, 1, 6], [6, 4, 6, 6], [4, 1, 1, 8], [6, 8, 8, 3], [6, 9, 8, 1], [3, 6, 3, 0], [7, 8, 8, 5], [9, 1, 3, 4], [6, 9, 7, 6], [4, 4, 1, 1], [2, 5, 0, 2], [9, 1, 7, 5], [0, 5, 7, 6], [1, 2, 0, 1]]
You could permute axes with np.transpose and then reshape to 2D -
y = x.transpose(1,2,0).reshape(-1,x.shape[0])
Append with .tolist() for list output.
yes, either use np.reshape(x, shape) or try it with np.ndarray.flatten(x, order='F') (F for Fortran style, column first, according to your example).
read the documentation to find out which parameters fit the best. IMHO, I think ndarray.flatten is the better and more elegant option for you here. However, depending on your exact wanted solution, you might have to reshape the array first.
I'm trying to find a way to fill an array with rows of values. It's much easier to express my desired output with an example. Given the input of an N x M matrix, array1,
array1 = np.array([[2, 3, 4],
[4, 8, 3],
[7, 6, 3]])
I would like to output an array of arrays in which each row is an N x N consisting of the values from the respective row. The output would be
[[[2, 3, 4],
[2, 3, 4],
[2, 3, 4]],
[[4, 8, 3],
[4, 8, 3],
[4, 8, 3]],
[[7, 6, 3],
[7, 6, 3],
[7, 6, 3]]]
You can reshape the array from 2d to 3d, then use numpy.repeat() along the desired axis:
np.repeat(array1[:, None, :], 3, axis=1)
#array([[[2, 3, 4],
# [2, 3, 4],
# [2, 3, 4]],
# [[4, 8, 3],
# [4, 8, 3],
# [4, 8, 3]],
# [[7, 6, 3],
# [7, 6, 3],
# [7, 6, 3]]])
Or equivalently you can use numpy.tile:
np.tile(array1[:, None, :], (1,3,1))
Another solution which is sometimes useful is the following
out = np.empty((3,3,3), dtype=array1.dtype)
out[...] = array1[:, None, :]
I'd like to obtain a 1D array of indexes from a 3D matrix.
For instance given x = np.random.randint(10, size=(10,3,3)), I'd like to do something like np.argmax(x, axis=(1,2)) just like you can do with np.max, that is, obtain a 1D array of length 10 containing the indexes (0 to 8) of the maximums of each submatrix of size (3,3).
I have not found anything helpful so far and I want to avoid looping on the first dimension (and use np.argmax(x)) as it is quite big.
Cheers!
Reshape to merge those last two axes and then use np.argmax -
idx = x.reshape(x.shape[0],-1).argmax(-1)
out = np.unravel_index(idx, x.shape[-2:])
Sample run -
In [263]: x = np.random.randint(10, size=(4,3,3))
In [264]: x
Out[264]:
array([[[0, 9, 2],
[7, 7, 8],
[2, 5, 9]],
[[1, 7, 2],
[8, 9, 0],
[2, 8, 3]],
[[7, 5, 0],
[7, 1, 6],
[5, 1, 1]],
[[0, 7, 3],
[5, 4, 1],
[9, 8, 9]]])
In [265]: idx = x.reshape(x.shape[0],-1).argmax(-1)
In [266]: np.unravel_index(idx, x.shape[-2:])
Out[266]: (array([0, 1, 0, 2]), array([1, 1, 0, 0]))
If you meant getting the merged index, then its simpler -
x.reshape(x.shape[0],-1).argmax(1)
Sample run -
In [283]: x
Out[283]:
array([[[2, 3, 7],
[8, 1, 0],
[3, 6, 9]],
[[8, 0, 5],
[2, 2, 9],
[9, 0, 9]],
[[1, 9, 2],
[5, 0, 3],
[7, 2, 1]],
[[1, 6, 5],
[2, 3, 7],
[7, 4, 6]]])
In [284]: x.reshape(x.shape[0],-1).argmax(1)
Out[284]: array([8, 5, 1, 5])