I have a 3D matrix X which contains vectors as rows into the 3rd dimension. I would like to extract each such vector X(:, x, y) and save it as a 2D matrix such that X(:, 0, 0) is the first row of the 2D matrix, X(:, 0, 1) the second, and so on. The following crude graphic might help illustrate this:
I know that I can create my new 2D matrix and then iterate over the original X to add the vectors, but does somebody have some input on how to do this quick and efficiently?
Example: Given
>>> a = np.arange(9*3).reshape(3,3,3)
>>> a
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
I would like to get the following as rows, though the order of the rows does not matter:
array([[ 0, 9, 18],
[ 1, 10, 19]],
...)
Use np.transpose and then reshape like so -
X.transpose(1,2,0).reshape(-1,X.shape[0])
Explanation -
1) You want to get rows formed off X[:, 0, 0], X[:, 0, 1], etc., i.e., we have to "push" the axis=0 elements to the last axis of such a 2D array output. Next up, we have to decided the order of rows, which would be formed out of axes=1,2 from it. Now, going back to the desired 2D array output, between the first and second rows, i.e. between X[:, 0, 0] and X[:, 0, 1], axis=1 stays the same. So, in the 2D array output, the second axis (axis=1) would have precedence over the third axis (axis=2). So, in X we push axis=1 to axis=0 and axis=2 to axis=1. Since, as stated earlier axis=0 in X had to be moved to the last axis, so that would be axis=2. All of this could be done with X.transpose(1,2,0). Let's call it Y .
2) Finally, we have to reshape Y to a 2D array such that the number of elements in each row is same as X.shape[0], which is achieved through Y.reshape(-1,X.shape[0]). Thus, the final solution becomes -
X.transpose(1,2,0).reshape(-1,X.shape[0])
Sample run -
In [25]: X
Out[25]:
array([[[ 0.19508052, 0.02481975],
[ 0.88915956, 0.95974095]],
[[ 0.23271151, 0.14730822],
[ 0.56763563, 0.30607283]],
[[ 0.33259228, 0.42552102],
[ 0.28950926, 0.47782175]]])
In [26]: X[:, 0, 0]
Out[26]: array([ 0.19508052, 0.23271151, 0.33259228])
In [27]: X[:, 0, 1]
Out[27]: array([ 0.02481975, 0.14730822, 0.42552102])
In [28]: X[:, 1, 0]
Out[28]: array([ 0.88915956, 0.56763563, 0.28950926])
In [29]: X[:, 1, 1]
Out[29]: array([ 0.95974095, 0.30607283, 0.47782175])
In [30]: X.transpose(1,2,0).reshape(-1,X.shape[0])
Out[30]:
array([[ 0.19508052, 0.23271151, 0.33259228],
[ 0.02481975, 0.14730822, 0.42552102],
[ 0.88915956, 0.56763563, 0.28950926],
[ 0.95974095, 0.30607283, 0.47782175]])
Related
I want to compute the element-wise tensor product of 2 tensors of the shape (1144,3) meaning I want to compute the tensordot along the second axis if I understood it correctly.
I'd expect my result to be of the shape (1144,3,3).
I am currently trying to achieve this using numpys tensordot() function, but I can't figure out the correct axes to use to get a shape of (1144,3,3).
You can use numpy.einsum for this.
In [30]: a
Out[30]:
array([[0, 1, 2],
[3, 4, 5]])
In [31]: np.einsum('ij,ik->ijk', a, a)
Out[31]:
array([[[ 0, 0, 0],
[ 0, 1, 2],
[ 0, 2, 4]],
[[ 9, 12, 15],
[12, 16, 20],
[15, 20, 25]]])
As numpy.tensordot support only 2 element axes this means there is no way to imitate the
->...-like behavior. So I don't see how this can be done with numpy.tensordot.
So, I'm new to python and learning about the NumPy module.
Here is my array
c = np.array([[[ 0, 1, 2],
[ 10, 12, 13]],
[[100, 101, 102],
[110, 112, 113]]])
in the above array if I try to access it through
c[:1,0:]
it produces expected output that
# expected because print from initial to row 1,0 excluding row 1,0
array([[[ 0, 1, 2],
[10, 12, 13]]])
but now when I try to access it through
c[:1,1:]
it produces output that
array([[[10, 12, 13]]])
why???
This is a 3D array. You can check it with
print(c.shape)
that yields
(2, 2, 3)
Is 3D array really what you wish to do ?
If so, if you slice it with two indices instead of three, that means that the third is implicitly :. So c[1, 1] is equivalent to c[1, 1, :] which is equivalent to c[1, 1, 0:3].
And your query c[:1,1:] is equivalent to c[0, 1, 0:3]: that is the correct result.
Now as per your comment I guess you wish to reshape, filter and reshape:
c.reshape(4, -1)[:3,:].reshape(1, 3, -1)
yields
array([[[ 0, 1, 2],
[ 10, 12, 13],
[100, 101, 102]]])
There are a few questions I've found that are close to what I am asking but they are different enough that they don't seem to solve my problem. I am trying to grab a 1d slice along one axis for an ndarray. As an example for a 3d array
[[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9,10,11],
[12,13,14],
[15,16,17]],
[[18,19,20],
[21,22,23],
[24,25,26]]]
I want the following 1d slices
[0,1,2]
...
[24,25,26]
[0,3,6]
...
[20,23,26]
[0,9,18]
...
[8,17,26]
which effectively equates to the following (for a 3d arrays):
ary[i,j,:]
ary[i,:,k]
ary[:,j,k]
I want this to generalize to an array of n dimensions
(for a 2d array we would get ary[i,:] and ary[:,j], etc.)
Is there a numpy function that lets me do this?
EDIT: Corrected the 2nd dimension indexing
We could permute axes by selecting each one of the axes one at a time pushing it at the end and reshape. We would make use of ndarray.ndim to generalize to generic n-dim ndarrays. Also, np.transpose would be useful here to permute axes and np.roll to get rolled axes order. The implementation would be quite simple and is listed below -
# a is input ndarray
R = np.arange(a.ndim)
out = [np.transpose(a,np.roll(R,i)).reshape(-1,a.shape[i]) for i in R]
Sample run -
In [403]: a = np.arange(27).reshape(3,3,3)
In [325]: R = np.arange(a.ndim)
In [326]: out = [np.transpose(a,np.roll(R,i)).reshape(-1,a.shape[i]) for i in R]
In [327]: out[0]
Out[327]:
array([[ 0, 1, 2],
[ 3, 4, 5],
...
[24, 25, 26]])
In [328]: out[1]
Out[328]:
array([[ 0, 3, 6],
[ 9, 12, 15],
....
[20, 23, 26]])
In [329]: out[2]
Out[329]:
array([[ 0, 9, 18],
[ 1, 10, 19],
....
[ 8, 17, 26]])
I am learning numpy framework.This piece of code I don't understand.
import numpy as np
a =np.array([[0,1,2],[3,4,5],[6,7,8],[9,10,11]])
print(a)
row = np.array([[0,0],[3,3]])
col = np.array([[0,2],[0,2]])
b = a[row,col]
print("This is b array:",b)
This b array returns the corner values of a array, that is, b equals [[0,2],[9,11]].
When indexing is done using an array or "array-like", to access/modify the elements of an array, then it's called advanced indexing.
In [37]: a
Out[37]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
In [38]: row
Out[38]:
array([[0, 0],
[3, 3]])
In [39]: col
Out[39]:
array([[0, 2],
[0, 2]])
In [40]: a[row, col]
Out[40]:
array([[ 0, 2],
[ 9, 11]])
That's what you got. Below is an explanation:
Indices of
`a[row, col]` row column
|| || || ||
VV VV VV VV
a[0, 0] a[0, 2]
a[3, 0] a[3, 2]
|__________| |
row-idx array |
|__________|
column-idx array
You're indexing a using two equally shaped 2d-arrays, hence you're output array will also have the same shape as col and row. To better understand how array indexing works you can check the docs, where as shown, indexing with 1d-arrays over the existing axis' of a given array works as follows:
result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M],
..., ind_N[i_1, ..., i_M]]
Where the same logic applies in the case of indexing with 2d-arrays over each axis, but instead you'd have a result array with up to i_N_M indices.
So going back to your example you are essentially selecting from the rows of a based on row, and from those rows you are selecting some columns col. You might find it more intuitive to translate the row and column indices into (x,y) coordinates:
(0,0), (0,2)
(3,0), (3,2)
Which, by accordingly selecting from a, results in the output array:
print(a[row,col])
array([[ 0, 2],
[ 9, 11]])
You can understand it by making more tries, to see more examples.
If you have one dimensional index:
In [58]: np.arange(10)[np.array([1,3,4,6])]
Out[58]: array([1, 3, 4, 6])
In case of two dimensional index:
In [57]: np.arange(10)[np.array([[1,3],[4,6]])]
Out[57]:
array([[1, 3],
[4, 6]])
If you use 3 dimensional index:
In [59]: np.arange(10)[np.array([[[1],[3]],[[4],[6]]])]
Out[59]:
array([[[1],
[3]],
[[4],
[6]]])
As you can see, if you make hierarchy in indexing, you will get it in the output as well.
Proceeding by steps:
import numpy as np
a = np.array([[0,1,2],[3,4,5],[6,7,8],[9,10,11]])
print(a)
gives 2d array a:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
Then:
row = np.array([[0,0],[3,3]])
assigns to 2d array row values [0,0] and [3,3]:
array([[0, 0],
[3, 3]])
Then:
col = np.array([[0,2],[0,2]])
assigns to 2d array col values [0,2] and [0,2]:
array([[0, 2],
[0, 2]])
Finally:
b = a[row,col]
assigns to b values given by a[0,0], a[0,2] for the first row, a[3,0], a[3,2] for the second row, that is:
array([[ 0, 2],
[ 9, 11]])
Where does b[0,0] <-- a[0,0] come from? It comes from the combination of row[0,0] which is 0 and col[0,0] which is 0.
What about b[0,1] <-- a[0,2]? It comes from the combination of row[0,1] which is 0 and col[0,1] which is 2.
And so forth.
I am trying to improve my understanding of numpy functions. I understand the behaviour of numpy.dot. I'd like to understand the behaviour of numpy.outer in terms of numpy.dot.
Based on this Wikipedia article https://en.wikipedia.org/wiki/Outer_product I'd expect for array_equal to return True in the following code. However it does not.
X = np.matrix([
[1,5],
[5,9],
[4,1]
])
r1 = np.outer(X,X)
r2 = np.dot(X, X.T)
np.array_equal(r1, r2)
How can I assign r2 so that np.array_equal returns True? Also, why does numpy's implementation of np.outer not match the definition of outer multiplication on Wikipedia?
Using numpy 1.9.2
In [303]: X=np.array([[1,5],[5,9],[4,1]])
In [304]: X
Out[304]:
array([[1, 5],
[5, 9],
[4, 1]])
In [305]: np.inner(X,X)
Out[305]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [306]: np.dot(X,X.T)
Out[306]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
The Wiki outer link mostly talks about vectors, 1d arrays. Your X is 2d.
In [310]: x=np.arange(3)
In [311]: np.outer(x,x)
Out[311]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [312]: np.inner(x,x)
Out[312]: 5
In [313]: np.dot(x,x) # same as inner
Out[313]: 5
In [314]: x[:,None]*x[None,:] # same as outer
Out[314]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
Notice that the Wiki outer does not involve summation. Inner does, in this example 5 is the sum of the 3 diagonal values of the outer.
dot also involves summation - all the products followed summation along a specific axis.
Some of the wiki outer equations use explicit indices. The einsum function can implement these calculations.
In [325]: np.einsum('ij,kj->ik',X,X)
Out[325]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [326]: np.einsum('ij,jk->ik',X,X.T)
Out[326]:
array([[ 26, 50, 9],
[ 50, 106, 29],
[ 9, 29, 17]])
In [327]: np.einsum('i,j->ij',x,x)
Out[327]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
In [328]: np.einsum('i,i->',x,x)
Out[328]: 5
As mentioned in the comment, np.outer uses ravel, e.g.
return a.ravel()[:, newaxis]*b.ravel()[newaxis,:]
This the same broadcasted multiplication that I demonstrated earlier for x.
numpy.outer only works for 1-d vectors, not matrices. But for the case of 1-d vectors, there is a relation.
If
import numpy as np
A = np.array([1.0,2.0,3.0])
then this
np.matrix(A).T.dot(np.matrix(A))
should be the same as this
np.outer(A,A)
Another (clunky) version similar to a[:,None] * a[None,:]
a.reshape(a.size, 1) * a.reshape(1, a.size)