I have a 4d numpy array which represents a dataset with 3d instances.
Lets say that the shape of the array is (32, 32, 3, 73257).
How can i change the shape of the array to (73257, 32, 32, 3)?
--- Question update
It seems that both rollaxis and transpose do the trick.
Thanx for replying!
The np.transpose function does exactly what you want, you can pass an axis argument which controls which axis you want to swap:
a = np.empty((32, 32, 3, 73257))
b = np.transpose(a, (3, 0, 1, 2))
The axis of b are permuted versions of the ones of a: the axis 0 of b is the 3-rd axis of a, the axis 1 of b is the 0-th axis of a, etc...
That way, you can specify which of the axis of size 32 you want in second or in third place:
b = np.transpose(a, (3, 1, 0, 2))
Also gives an array of the desired shape, but is different from the previous one.
It looks like np.rollaxis(arr, axis=-1) will do what you want. Example:
>>> arr = np.empty(32, 32, 3, 73257)
>>> arr2 = np.rollaxis(arr, axis=-1)
>>> arr2.shape
(73257, 32, 32, 3)
This will make arr[i,j,k,l] == arr2[l,i,j,k] for all ijkl
Related
I'm working on a problem where I've to reshape a (63,16,3) array's each element to an array (4,4,3), and I'm stuck there.
I generated an array of (63,16,3) using the random function of NumPy. Please help me how to reshape that array's each element into a (4,4,3) and store those outputs into an array.
import numpy as np
a = np.random.rand(63, 16, 3)
return an array b whose each element is (4,4,3)
I have successfully converted the array (63, 16, 3) into (4, 4, 3) but elementwise. What I mean can be cleared using the below snippet of code.
a_resize_0th_element = a[0].reshape(4,4,3)
But I'm looking for a method where this element-wise operation of transforming a (16, 3) array into the shape of (4, 4, 3) and can be done for all the 63 elements of array a and store everything into array b.
You just need reshape(). The size of the array is 63 * 16 * 3 = 3,024 elements. If you want to divide that into 4x4x3 arrays, that's 3,024 / (4 * 4 * 3) = 63 elements.
So:
b = np.reshape(a, (63, 4, 4, 3))
print(b[0].shape)
Result:
(4, 4, 3)
So, b is an array with 63 shape (4, 4, 3) arrays.
Note: obviously, 4 * 4 = 16 here, but generally this works. However, if you don't want to do the math yourself, you can also just use this:
b = np.reshape(a, (-1, 4, 4, 3))
The -1 will cause numpy to figure it out itself and it will give you the same result.
I have 6 files with shape (6042,) or 1 column. I used dstack to stack the 6 files in hopes of getting a shape (6042, 1, 6). But after I stack it I get shape (1, 6042, 6). Then I tried to change the order using
new_train = np.reshape(train_x,(train_x[1],1,train_x[2]))
error appears:
IndexError: index 1 is out of bounds for axis 0 with size 1
This is my dstack code:
train_x = dstack([train_data['gx'],train_data['gy'], train_data['gz'], train_data['ax'],train_data['ay'], train_data['az']])
error is because
train_x[1]
tries looking 2nd row of train_x but it has only 1 row as you said shape 1, 6042, 6). So you need to look shape and index it
new_train = np.reshape(train_x, (train_x.shape[1], 1, train_x.shape[2]))
but this can be also doable with transpose
new_train = train_x.transpose(1, 0, 2)
so this changes axes 0 and 1's positions.
Other solution is fixing dstack's way. It gives "wrong" shape because your datas shape not (6042, 1) but (6042,) as you say. So if you reshape the datas before dstack it should also work:
datas = [train_data['gx'],train_data['gy'], train_data['gz'],
train_data['ax'],train_data['ay'], train_data['az']]
#this list comprehension makes all shape (6042, 1) now
new_datas = [td[:, np.newaxis] for td in datas]
new_train = dstack(new_datas)
You can use np.moveaxis(X, 0, -2), where X is your (1,6042,6) array.
This function swaps the axis. 0 for your source axis and -2 is your destination axis.
np.dstack uses:
arrs = atleast_3d(*tup)
to convert the list of arrays to a list of 3d arrays.
In [51]: alist = [np.ones(3,int),np.zeros(3,int)]
In [52]: alist
Out[52]: [array([1, 1, 1]), array([0, 0, 0])]
In [53]: np.atleast_3d(*alist)
Out[53]:
[array([[[1],
[1],
[1]]]),
array([[[0],
[0],
[0]]])]
In [54]: _[0].shape
Out[54]: (1, 3, 1)
Concatenating those on the last dimension produces the (1,n,6) kind of result.
With expand_dims we can adjust the shape of all arrays to (n,1,1), and then do the concatenate:
In [62]: np.expand_dims(alist[0],[1,2]).shape
Out[62]: (3, 1, 1)
In [63]: np.concatenate([np.expand_dims(a,[1,2]) for a in alist], axis=2)
Out[63]:
array([[[1, 0]],
[[1, 0]],
[[1, 0]]])
In [64]: _.shape
Out[64]: (3, 1, 2)
direct reshape or newaxis would work just as well:
In [65]: np.concatenate([a[:,None,None] for a in alist], axis=2).shape
Out[65]: (3, 1, 2)
stack is another cover that adjusts shapes before concatenate:
In [67]: np.stack(alist,1).shape
Out[67]: (3, 2)
In [68]: np.stack(alist,1)[:,None].shape
Out[68]: (3, 1, 2)
So there are lots of ways to get what you want, whether it means adjusting shapes before the concatenate, or after.
I currently have a 5D numpy array of dimensions 40 x 3 x 3 x 5 x 1000 where the dimensions are labelled by a x b x c x d x e respectively.
I have another 2D numpy array of dimensions 3 x 1000 where the dimensions are labelled by b x e respectively.
I wish to subtract the 5D array from the 2D array.
One way I was thinking of was to expand the 2D into a 5D array (since the 2D array does not change for all combinations of the other 3 dimensions). I am not sure what array method/numpy function I can use to do this.
I tend to start getting lost with nD array manipulations. Thank you for assisting.
In [217]: a,b,c,d,e = 2,3,4,5,6
In [218]: A = np.ones((a,b,c,d,e),int); B = np.ones((b,e),int)
In [219]: A.shape
Out[219]: (2, 3, 4, 5, 6)
In [220]: B.shape
Out[220]: (3, 6)
In [221]: B[None,:,None,None,:].shape # could also use reshape()
Out[221]: (1, 3, 1, 1, 6)
In [222]: C = B[None,:,None,None,:]-A
In [223]: C.shape
Out[223]: (2, 3, 4, 5, 6)
The first None isn't essential; numpy will add it as needed, but as a human it might help to see it.
IIUC, suppose your arrays are a and b:
np.swapaxes(np.swapaxes(a, 1, 3) - b, 1, 3)
If I have a batch of uniform 3D grids of coordinate locations, with shape (for example), [1, 32, 32, 32, 3], what is the best way for me to split this up into multiple even chunks, so I could end up with something such as [1, 4096, 2, 2, 2, 3]? In other words, I’m splitting up that one big 32 x 32 x 32 cube where each point is an x, y, z coordinate location into 4096 smaller 2 x 2 x 2 cubes? Does a simple view operation make sense here, or would it throw off the coordinate values? I was looking into operations like torch.chunk, but they require a specific dimension to split along, which I’m not sure applies here.
My use case for this is that I have a smaller [1, 16, 16, 16, 3] cube, so I’m trying to match up points from this smaller shape into the corresponding cubes in the upsampled [1, 32, 32, 32, 3] shape (since a single coordinate point in the 16^3 shape corresponds to 8 points in the 32^3 shape).
For additional context, this is how I generate my 3D grid right now:
pxs = torch.linspace(-1, 1, 32)
pys = torch.linspace(-1, 1, 32)
pzs = torch.linspace(-1, 1, 32)
pxs = pxs.view(-1, 1, 1).expand(*shape).contiguous().view(size)
pys = pys.view(1, -1, 1).expand(*shape).contiguous().view(size)
pzs = pzs.view(1, 1, -1).expand(*shape).contiguous().view(size)
points = torch.stack([pxs, pys, pzs], dim=1)
grid_3d = torch.reshape(points, (32, 32, 32, 3))
Just reshape it with the dimensions you'd like:
In [29]: lin = np.linspace(-1, 1, 32)
...: cube = np.stack(np.meshgrid(lin, lin, lin), axis=-1)
...: cube.shape
Out[29]: (32, 32, 32, 3)
In [30]: new_cube = cube.reshape((1, 4096, 2, 2, 2, 3))
...: new_cube.shape
Out[30]: (1, 4096, 2, 2, 2, 3)
Reading Dynamic Graph CNN for Learning on Point Clouds code, I came across this snippet:
idx_ = tf.range(batch_size) * num_points
idx_ = tf.reshape(idx_, [batch_size, 1, 1])
point_cloud_flat = tf.reshape(point_cloud, [-1, num_dims])
point_cloud_neighbors = tf.gather(point_cloud_flat, nn_idx+idx_) <--- what happens here?
point_cloud_central = tf.expand_dims(point_cloud_central, axis=-2)
debugging the line I made sure that the dims are
point_cloud_flat:(32768,3) nn_idx:(32,1024,20), idx_:(32,1,1)
// indices are (32,1024,20) after broadcasting
Reading the tf.gather doc I couldn't understand what the function does with dimensions higher that the input dimensions
An equivalent function in numpy is np.take, a simple example:
import numpy as np
params = np.array([4, 3, 5, 7, 6, 8])
# Scalar indices; (output is rank(params) - 1), i.e. 0 here.
indices = 0
print(params[indices])
# Vector indices; (output is rank(params)), i.e. 1 here.
indices = [0, 1, 4]
print(params[indices]) # [4 3 6]
# Vector indices; (output is rank(params)), i.e. 1 here.
indices = [2, 3, 4]
print(params[indices]) # [5 7 6]
# Higher rank indices; (output is rank(params) + rank(indices) - 1), i.e. 2 here
indices = np.array([[0, 1, 4], [2, 3, 4]])
print(params[indices]) # equivalent to np.take(params, indices, axis=0)
# [[4 3 6]
# [5 7 6]]
In your case, the rank of indices is higher than params, so output is rank(params) + rank(indices) - 1 (i.e. 2 + 3 - 1 = 4, i.e. (32, 1024, 20, 3)). The - 1 is because the tf.gather(axis=0) and axis must be rank 0 (so a scalar) at this moment. So the indices takes the elements of the first dimension (axis=0) in a "fancy" indexing way.
EDITED:
In brief, in your case, (if I didn't misunderstand the code)
point_cloud is (32, 1024, 3), 32 batches 1024 points which have 3
coordinates.
nn_idx is (32, 1024, 20), indices of 20 neighbors of
32 batches 1024 points. The indices are for indexing in point_cloud.
nn_idx+idx_ (32, 1024, 20), indices of 20 neighbors of
32 batches 1024 points. The indices are for indexing in point_cloud_flat.
point_cloud_neighbors finally is (32, 1024,
20, 3), the same as nn_idx+idx_ except that point_cloud_neighbors are their 3 coordinates while nn_idx+idx_ are just their indices.