I have difficulty understanding how numpy.transpose actually works.
For example
a_value = array([[[0, 1],
[2, 3]],
[[4, 5],
[6, 7]]])
and when I do
np.transpose(a_value, (2, 1, 0))
I get
array([[[0, 4],
[2, 6]],
[[1, 5],
[3, 7]]])
How can I derive this transpose manually? I need to understand the formula or the steps intuitively in the above case so I can generalize it for higher dimensions.
As given in the documentation -
numpy.transpose(a, axes=None)
axes : list of ints, optional
By default, reverse the dimensions, otherwise permute the axes according to the values given.
The second argument is the axes using which the values are permuted. That is for example if the index of initial element is (x,y,z) (where x is 0th axes, y is 1st axes, and z is 2nd axes) , the position of that element in the resulting array becomes (z,y,x) (that is 2nd axes first, then 1st axes, and last 0th axes) , based on the argument you provided for axes .
Since you are transposing an array of shape (2,2,2) , the transposed shape is also (2,2,2) , and the positions would change as -
(0,0,0) -> (0,0,0)
(1,0,0) -> (0,0,1)
(0,1,0) -> (0,1,0)
(1,1,0) -> (0,1,1)
...
Since the axes you choose are trivial, lets explain this for another axes. Example -
In [54]: A = np.arange(30).reshape((2, 3, 5))
In [55]: A
Out[55]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]],
[[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]])
In [56]: np.transpose(A,(1,2,0))
Out[56]:
array([[[ 0, 15],
[ 1, 16],
[ 2, 17],
[ 3, 18],
[ 4, 19]],
[[ 5, 20],
[ 6, 21],
[ 7, 22],
[ 8, 23],
[ 9, 24]],
[[10, 25],
[11, 26],
[12, 27],
[13, 28],
[14, 29]]])
Here, the first element (0,0,0) becomes the (0,0,0) element in the result.
The second element (0,0,1) becomes the (0,1,0) element in the result. And so on -
(0,0,0) -> (0,0,0)
(0,0,1) -> (0,1,0)
(0,0,2) -> (0,2,0)
...
(2,3,4) -> (3,4,2)
...
Here's a little more clarification:
Don't confuse the parameters of np.reshape(z, y, x) with those of np.transpose(0, 1, 2).
np.reshape() uses the dimensions of our matrix, think (sheets, rows, columns), to specify its layout.
np.transpose() uses the integers 0, 1, and 2 to represent the axes we want to swap, and correspond to z, y, and x, respectively.
For example, if we have data in a matrix of 2 sheets, 3 rows, and 5 columns...
We can take the next step and think in terms of lists. So, the z, y, x or sheets, rows, columns representation of a 2x3x5 matrix is...
[[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]],
[[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]]
...but the module we're feeding this data into requires a layout such that sheet 1 contains the first row of each of our sheets and sheet 2 contains the second row and so on. Then, we'll need to transpose our data with np.transpose(1, 0, 2). This swaps the z and the y axes and transposes the data.
[[[ 0, 1, 2, 3, 4],
[15, 16, 17, 18, 19]],
[[ 5, 6, 7, 8, 9],
[20, 21, 22, 23, 24]],
[[10, 11, 12, 13, 14],
[25, 26, 27, 28, 29]]]
Note the difference from using np.reshape(3, 2, 5) as this doesn't transpose the data--only re-arranges it.
[[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9]],
[[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]]
The axes= parameter is how you want the starting axes to be arranged in the result.
In my case I have an array of shape (2, 1997, 4, 4), and I just want to transpose the (4, 4) sub-arrays.
axes=() needs 4 parameters here, one for each axis. axes=(0, 1, 2, 3) yields no change, as the requested result ordering is the same as the start ordering.
To transpose just the (4,4) arrays as I want, I use axes=(0, 1, 3, 2) -- just swap the last two axes, please.
Related
How do I transpose the first 2 dimensions of a 3D array 'matrix?
matrix = np.random.rand(2,3,4)
In the third dimensions I want to swap 'rows' with 'columns', preferably without a loop.
You can use the .transpose() function.
matrix = matrix.transpose(1, 0, 2)
means swap the first and the second axis.
You can use swapaxes:
matrix2 = matrix.swapaxes(0,1)
example:
# input
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
# output
array([[[ 0, 1, 2, 3],
[12, 13, 14, 15]],
[[ 4, 5, 6, 7],
[16, 17, 18, 19]],
[[ 8, 9, 10, 11],
[20, 21, 22, 23]]])
I have an ndarray of shape (10, 3) and an index list of length 10:
import numpy as np
arr = np.arange(10* 3).reshape((10, 3))
idxs = np.array([0, 1, 1, 1, 2, 0, 2, 2, 1 , 0])
I want to use numpy delete (or a numpy function that is suited better for the task) to delete the values in arr as indicated by idxs for each row. So in the zeroth row of arr I want to delete the 0th entry, in the first the first, in the second the first, and so on.
I tried something like
np.delete(arr, idxs, axis=1)
but it won't work. Then I tried building an index list like this:
idlist = [np.arange(len(idxs)), idxs]
np.delete(arr, idlist)
but this doesn't give me the results I want either.
#Quang's answer is good, but may benefit from some explanation.
np.delete works with whole rows or columns, not selected elements from each.
In [30]: arr = np.arange(10* 3).reshape((10, 3))
...: idxs = np.array([0, 1, 1, 1, 2, 0, 2, 2, 1 , 0])
Selecting items from the array is easy:
In [31]: arr[np.arange(10), idxs]
Out[31]: array([ 0, 4, 7, 10, 14, 15, 20, 23, 25, 27])
Selecting everything but these, takes a bit more work. np.delete is complex general code that does different things depending on the delete specification. But one thing it can do is create a True mask, and set the delete items to False.
For your 2d case we can:
In [33]: mask = np.ones(arr.shape, bool)
In [34]: mask[np.arange(10), idxs] = False
In [35]: arr[mask]
Out[35]:
array([ 1, 2, 3, 5, 6, 8, 9, 11, 12, 13, 16, 17, 18, 19, 21, 22, 24,
26, 28, 29])
boolean indexing produces a flat array, so we need to reshape to get 2d:
In [36]: arr[mask].reshape(10,2)
Out[36]:
array([[ 1, 2],
[ 3, 5],
[ 6, 8],
[ 9, 11],
[12, 13],
[16, 17],
[18, 19],
[21, 22],
[24, 26],
[28, 29]])
The Quand's answer creates the mask in another way:
In [37]: arr[np.arange(arr.shape[1]) != idxs[:,None]]
Out[37]:
array([ 1, 2, 3, 5, 6, 8, 9, 11, 12, 13, 16, 17, 18, 19, 21, 22, 24,
26, 28, 29])
Let's try extracting the other items by masking, then reshape:
arr[np.arange(arr.shape[1]) != idxs[:,None]].reshape(len(arr),-1)
Thanks for your question and the answers from Quang, and hpaulj.
I just want to add a second senario, where one wants to do the deletion from the other axis.
The index now has only 3 elements because there are only 3 columns in arr, for example:
idxs2 = np.array([1,2,3])
To delete the elements of each column according to the index in idxs2, one can do this
arr.T[np.array(np.arange(arr.shape[0]) != idxs2[:,None])].reshape(len(idxs2),-1).T
And the result becomes:
array([[ 0, 1, 2],
[ 6, 4, 5],
[ 9, 10, 8],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23],
[24, 25, 26],
[27, 28, 29]])
I'm using Python 3.7.7.
I have a three channels image as a Numpy array with this shape: (200, 200, 3).
Because it is so huge, I have tried to guess what I have to do with this example:
import numpy as np
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print(a[1]) # Output [4, 5, 6]
But I'm not sure if I'm doing right.
What do I have to do to get the second channel from the image array (output shape (200, 200, 1))?
If you are asking about a usage convention, in image processing for machine learning, I usually see each image flattened so that each image is one long row, in row-major order followed by channel order. Numpy has obj.flatten() command to make this easy. Then to retrieve the middle channel, Numpy slicing or indexing can be used. Each processed batch has many images (rows), and each image is one very long flattened row.
Example:
b = a.flatten()
print(b)
# output array([1, 2, 3, 4, 5, 6, 7, 8, 9])
channel2 = b[3:6]
print(channel2)
# output array([4, 5, 6])
For other use cases, there may be a different convention.
Longer example using a 3x3 image array with 3 channels.
Note numerical values are in row-major order followed by channel order.
img_a = np.arange(0, 27).reshape(3, 3, 3)
''' output
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
'''
# Flatten into one long row
row_a = img_a.flatten()
# output array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
# 17, 18, 19, 20, 21, 22, 23, 24, 25, 26])
# Select middle channel using Numpy slicing
channel_mid = row_a[9:18]
# output array([ 9, 10, 11, 12, 13, 14, 15, 16, 17])
# Convert middle channel back into a matrix shape (if needed).
matrix_mid = channel_mid.reshape(3, 3)
''' output
array([[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]])
'''
I have an array that looks like:
test = np.zeros (7110, 514)
I need to "unpack" the first 90 values (rows) into the first value of the second dimension, the second 90 values (rows) into the second value of the second dimension, etc, so that the desired output will have shape:
desired_output = np.zeros(90, 79, 514)
I have tried something like:
a = np.split(test, 90, axis=1)
test1 = np.reshape(a, (79,90, 514))
but it just dragged me down a rabbit whole... Thanks for any help!
I don't know if I understand the question, do you have 7110 rows of 514 elements each and want to "group" the 7110 rows into 90 x 79 rows?
Because then you could do something like this:
>>> np.array(range(24)).reshape((6, 4))
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]])
These are 6 rows of 4 elements each.
>>> np.array(range(24)).reshape((6, 4)).reshape(3, 2, 4)
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
We keep the rows as they are, but instead of 6 rows, we get 3x2 rows.
So the code you would need is just:
desired_output = a.reshape(90, 79, 514)
I have an array like so:
([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]],
[[12, 13, 14],
[15, 16, 17]]])
If i want to slice the numbers 12 to 17 i would use:
arr[2, 0:2, 0:3]
but how would i go about slicing the array to get 12 to 16?
You'll need to "flatten" the last two dimensions first. Only then will you be able to extract the elements you want:
xf = x.view(x.size(0), -1) # flatten the last dimensions
xf[2, 0:5]
Out[87]: tensor([12, 13, 14, 15, 16])
Another way would be to simply index into the tensor and slice what is needed as in:
# input tensor
t = tensor([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]],
[[12, 13, 14],
[15, 16, 17]]])
# slice the last `block`, then flatten it and
# finally slice all elements but the last one
In [10]: t[-1].view(-1)[:-1]
Out[10]: tensor([12, 13, 14, 15, 16])
Please note that since this is a basic slicing, it returns a view. Thus making any changes to the sliced part would affect the original tensor as well. For example:
# assign it to some variable name
In [11]: sliced = t[-1].view(-1)[:-1]
In [12]: sliced
Out[12]: tensor([12, 13, 14, 15, 16])
# modify one element
In [13]: sliced[-1] = 23
In [14]: sliced
Out[14]: tensor([12, 13, 14, 15, 23])
# now, the original tensor is also updated
In [15]: t
Out[15]:
tensor([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]],
[[12, 13, 14],
[15, 23, 17]]])