Is there an easy way to index a numpy multidimensional array along the last dimension, using an array of indices? For example, take an array a of shape (10, 10, 20). Let's assume I have an array of indices b, of shape (10, 10) so that the result would be c[i, j] = a[i, j, b[i, j]].
I've tried the following example:
a = np.ones((10, 10, 20))
b = np.tile(np.arange(10) + 10, (10, 1))
c = a[b]
However, this doesn't work because it then tries to index like a[b[i, j], b[i, j]], which is not the same as a[i, j, b[i, j]]. And so on. Is there an easy way to do this without resorting to a loop?
There are several ways to do this. Let's first generate some test data:
In [1]: a = np.random.rand(10, 10, 20)
In [2]: b = np.random.randint(20, size=(10,10)) # random integers in range 0..19
One way to solve the question would be to create two index vectors, where one is a row vector and the other a column vector of 0..9 using meshgrid:
In [3]: i1, i0 = np.meshgrid(range(10), range(10), sparse=True)
In [4]: c = a[i0, i1, b]
This works because i0, i1 and b will all be broadcasted to 10x10 matrices. Quick test for correctness:
In [5]: all(c[i, j] == a[i, j, b[i, j]] for i in range(10) for j in range(10))
Out[5]: True
Another way would be to use choose and rollaxis:
# choose needs a sequence of length 20, so move last axis to front
In [22]: aa = np.rollaxis(a, -1)
In [23]: c = np.choose(b, aa)
In [24]: all(c[i, j] == a[i, j, b[i, j]] for i in range(10) for j in range(10))
Out[24]: True
Related
I have a multidimensional numpy array of dtype object, which was filled with other arrays.
As an example, here is a code reproducing that behavior:
arr = np.empty((3,4,2,1), dtype=object)
for i in range(arr.shape[0]):
for j in range(arr.shape[1]):
for k in range(arr.shape[2]):
for l in range(arr.shape[3]):
arr[i, j, k, l] = np.random.random(10)
Since all the inside arrays have the same size, I would like in this example to "incorporate" the last level into the array and make it an array of size (3,4,2,1,10).
I cannot really change the above code, so what I am looking for is a clean way (few lines, possibly without for loops) to generate this new array once created.
Thank you.
If I understood well your problem you could use random.random_sample() which should give the same result:
arr = np.random.random_sample((3, 4, 2, 1, 10))
After edit the solution is arr = np.array(arr.tolist())
Just by adding a new for loop :
arr = np.empty((3,4,2,1,10), dtype=object)
for i in range(arr.shape[0]):
for j in range(arr.shape[1]):
for k in range(arr.shape[2]):
for l in range(arr.shape[3]):
for m in range(arr.shape[4]):
arr[i, j, k, l, m] = np.random.randint(10)
However, you can one line this code with an optimized numpy function, every random function from numpy has a size parameter to build a array with random number with a particular shape :
arr = np.random.random((3,4,2,1,10))
EDIT :
You can flatten the array, replace every single number by a 1D array of length 10 and then reshape it to your desired shape :
import numpy as np
arr = np.empty((3,4,2,1), dtype=object)
for i in range(arr.shape[0]):
for j in range(arr.shape[1]):
for k in range(arr.shape[2]):
for l in range(arr.shape[3]):
arr[i, j, k, l] = np.random.randint(10)
flat_arr = arr.flatten()
for i in range(len(flat_arr)):
flat_arr[i] = np.random.randint(0, high=10, size=(10))
res_arr = flat_arr.reshape((3,4,2,1))
I am working on a project for cryptography course working with Charm jhuisi link .
I have two numpy matrix: V(2,3) belonging to ZR and M(3x2) belonging to G1. I want to bring V to G1, so I can exponentiate M^V. To perform this operation, in Charm I cannot simply use M**V, but I have to do it element by element.
from charm.toolbox.pairinggroup import PairingGroup,ZR,G1,G2,GT,pair
import numpy as np
M = np.array([[group.random(G1) for i in range(2)] for j in range(3)])
V_t = np.transpose(np.array([[group.random(ZR) for i in range(2)] for j in range(3)]))
matrix = np.array([[M[i][j] ** V[j][i] for j in range(3)] for i in range(2)]
but it returns me an error "IndexError: index 2 is out of bounds for axis 0 with size 2"
Can someone who has used Charm before help me, please?
Your code is a confused mix of python lists and numpy arrays. Lets first do the calculation with lists, and paying particular attention to keeping indices right.
Make 2 lists:
In [358]: M = [[1,2,3],[4,5,6]]
In [359]: V = [[1,2],[3,4],[5,6]]
Starting with an empty list, fill it with a new list:
In [360]: res = []
In [361]: for i in range(2):
...: res1 = []
...: for j in range(3):
...: res1.append(M[i][j]**V[j][i])
...: res.append(res1)
...:
In [362]: res
Out[362]: [[1, 8, 243], [16, 625, 46656]]
Note how i range is 2, j range is 3, matching the lengths of the lists.
The same calculation using numpy arrays:
In [363]: np.array(M)**np.array(V).T
Out[363]:
array([[ 1, 8, 243],
[ 16, 625, 46656]])
np.array(M) is (2,3) shape; np.array(V) is (3,2). To perform the elementwise power, V has to be transposed to (2,3).
The nested loop can be written as a comprehension - again with the same care over indices:
In [364]: [[M[i][j]**V[j][i] for j in range(3)] for i in range(2)]
Out[364]: [[1, 8, 243], [16, 625, 46656]]
V_t
What is V_t? I see
In [365]: from random import random
In [366]: V = np.transpose(np.array([[random() for i in range(2)] for j in rang
...: e(3)]))
In [367]: V
Out[367]:
array([[0.8748556 , 0.10373381, 0.23399403],
[0.95388354, 0.24060715, 0.38468676]])
In [368]: V.shape
Out[368]: (2, 3)
Have you done some undocumented transpose to produce a (3,2)? If so then you need to use V_t[i][j]. Are your problems just the result of a sloppy use of the transpose?
Aren't your indices just the wrong way around?
import numpy as np
from numpy.random import random
M = np.array([[random() for i in range(2)] for j in range(3)])
V = np.transpose(np.array([[random() for i in range(2)] for j in range(3)]))
matrix = [[M[i][j] ** V[j][i] for j in range(2)] for i in range(3)]
Edit
Here's a wild idea. Try:
import numpy as np
from god_knows_what import random
M = np.array([[random() for i in range(2)] for j in range(3)], dtype=object)
V = np.transpose(np.array([[random() for i in range(2)] for j in range(3)], dtype=object))
matrix = np.array([[M[i][j] ** V[j][i] for j in range(2)] for i in range(3)], dtype=object)
If the last line fails, try
matrix = np.array([[M[i][j] ** V[j][i] for j in range(3)] for i in range(2)], dtype=object)
I have for example
import numpy as np
a = np.ones((100, 5, 5))
And I want
d = np.vector_diagonal(a)
assert d.shape == (100, 5)
Where d[i, j] corresponds to a[i, j, j]
How to do this with numpy?
np.diagonal(a, axis1=1, axis2=2)
Just need to select which axes are "the matrix" and which "vectorize the matrices"
The reduction will be done on the selected axes.
I have a matrix A of the shape (N, N, T). Then I have a vector of V shape (N,). I want to perform the following operation A[i, j, ...] = A[i, j, ...]*V[i]/V[j]. I'm doing this with the following loop, but sure there is a way to do it with broadcast.
A = np.random.randint(0, 5, (2, 2, 3))
V = np.array([2, 3])
for i in range(2):
for j in range(2):
A[i, j, ...] *= V[i]
A[i, j, ...] /= V[j]
I've thought about doing it with element-wise multiplication and broadcast of numpy, and I try approaches like A * V[:, None, None] but always got an error.
Is there a more efficient way to do it?
Here's one way to do it -
(A*V[:,None,None])/V[:,None]
Alternatively, in two steps -
A *= V[:,None,None]
A /= V[:,None]
Leverage multi-cores with numexpr -
import numexpr as ne
ne.evaluate('A*V3D/V2D',{'V3D':V[:,None,None],'V2D':V[:,None]})
Note that you might be getting error because you might be doing edits into an int array with float results. So, either convert to float array at the start or write to a new array with the one-step approaches.
I have two arrays: A such that A.shape = (N, M, K) and index such that index.shape = (N, M). As a result I want to get such array B that B[i, j] = A[i, j, index[i, j]]. How to achieve that in the simplest way?
One solution that comes to my mind is that we can do it like this:
B = A.reshape(-1, K)[np.arange(N * M), index.reshape(-1)].reshape(N, M). I wonder if there are any other solutions?
You could use 2D extended range array for the first axis that aligns with the shape of the indexing array index and hence use advanced-indexing, like so -
B = A[np.arange(N)[:,None], np.arange(M), index]