I am generating multidimensional array of different sizes, though they'll all have an even number of columns.
>> import numpy as np
>> x = np.arange(24).reshape((3,8))
Which results in:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
I am able to slice with numpy and get the first column in an array:
>> newarr = x[0:,0:2]
array([[ 0, 1],
[ 8, 9],
[16, 17]])
However, I want to have one array that is just a list of the columns where column 1 and 2 are together, 3 and 4 are together, and so on.. For example:
array([[[ 0, 1],
[ 8, 9],
[16, 17]],
[[ 2, 3],
[10, 11],
[18, 19]],
etc....]
)
This code below works but it's clunky and my arrays are not all the same. Some arrays have 16 columns, some have 34, some have 50, etc.
>> newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]])]
There's got to be a better way to do this than
newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]+...+[x[0:,n:n+2]]
Help!
My idea is adding a for loop:
slice_len = 2
x_list = [x[0:, slice_len*i:slice_len*(i+1)] for i in range(x.shape[1] // slice_len)]
Output:
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]]), array([[ 4, 5],
[12, 13],
[20, 21]]), array([[ 6, 7],
[14, 15],
[22, 23]])]
Related
I want to create a torch tensor sequence out of a multi-dimensional numpy array. I've achieved it with 1-d arrays, but can't find the proper way with more dimensions...
This is a 1-d vector example:
import numpy as np
import torch
n = np.arange(10)
t = torch.tensor([n[i: i + 3] for i in range(7)])
Being the output:
tensor([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8]])
Let's say that instead of a 1-d vector I have a 2-d one.
import numpy as np
import torch
n = np.array([np.arange(10), np.arange(10, 20)])
t = torch.tensor([n[..., i: i + 3] for i in range(7)]).view(2, -1, 3)
The output is:
tensor([[[ 0, 1, 2],
[10, 11, 12],
[ 1, 2, 3],
[11, 12, 13],
[ 2, 3, 4],
[12, 13, 14],
[ 3, 4, 5]],
[[13, 14, 15],
[ 4, 5, 6],
[14, 15, 16],
[ 5, 6, 7],
[15, 16, 17],
[ 6, 7, 8],
[16, 17, 18]]])
And what I’m looking for is:
tensor([[[ 0, 1, 2],
[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8]],
[[10, 11, 12],
[11, 12, 13],
[12, 13, 14],
[13, 14, 15],
[14, 15, 16],
[15, 16, 17],
[16, 17, 18]]])
As you can see the rows are alternated... Is there any way of achieving it?
P.D: In case there is a more elegant solution to solve the problem, I will be very grateful too! I've tried with methods as torch.repeat_interleave but couldn't achieve anything...
Thanks a lot!
It is not very clear what formula you want to implement, but looks like it should be t[i, j, k] = i*10 + j + k. This formula is nothing but the outer sum of the three index ranges. The most straightforward way to obtain it is
t = torch.tensor(np.add.outer(np.arange(2)*10, np.add.outer(np.arange(7), np.arange(3))))
Which gives
tensor([[[ 0, 1, 2],
[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8]],
[[10, 11, 12],
[11, 12, 13],
[12, 13, 14],
[13, 14, 15],
[14, 15, 16],
[15, 16, 17],
[16, 17, 18]]], dtype=torch.int32)
You can achieve this by concatenating the sub-tensors along the appropriate axis:
n = np.array([np.arange(10), np.arange(10, 20)])
t = torch.tensor([n[..., i: i + 3] for i in range(7)])
t = torch.cat([_ for _ in t.unsqueeze(-2)], dim=1)
How can I set values in the first 3 channels of a 4 channel numpy array based on values in the 4th channel? Is it possible to do so with a numpy slice as a l-value?
Given a 3 by 2 pixel numpy array with 4 channels
a = np.arange(24).reshape(3,2,4)
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
I can select slices where the 4th channel is modulo 3.
px = np.where(0==a[:,:,3]%3)
(array([0, 1], dtype=int64), array([0, 1], dtype=int64))
a[px]
array([[ 0, 1, 2, 3],
[12, 13, 14, 15]])
Now I want to set the first 3 channels in those rows in a to 0 such that the results looks like:
a
array([[[ 0, 0, 0, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 0, 0, 0, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
I tried
a[px][:,0:3] = 0
but that leaves the array unchanged.
I read Setting values in a numpy arrays indexed by a slice and two boolean arrays and do not understand how to use a Boolean index to set only the first 3 channels.
Here is one way:
>>> px0, px1 = np.where(0==a[:,:,3]%3)
>>> a[px0, px1, :3] = 0
>>> a
array([[[ 0, 0, 0, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 0, 0, 0, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
or
>>> px = np.where(0==a[:,:,3]%3)
>>> a[..., :3][px] = 0
>>> a
array([[[ 0, 0, 0, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 0, 0, 0, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
or
>>> a[(*px, np.s_[:3])] = 0
>>> a
array([[[ 0, 0, 0, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 0, 0, 0, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
I have three 2D arrays a1, a2, and a3
In [165]: a1
Out[165]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
In [166]: a2
Out[166]:
array([[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20]])
In [167]: a3
Out[167]:
array([[6, 7, 8],
[4, 5, 5]])
And I stacked these arrays into a single array:
In [168]: stacked = np.vstack((a1, a2, a3))
In [170]: stacked
Out[170]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 6, 7, 8],
[ 4, 5, 5]])
Now, I want to get rid of the duplicate rows. So, numpy.unique does the job.
In [169]: np.unique(stacked, axis=0)
Out[169]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 4, 5, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20]])
However, there is one issue here. The original order is lost when taking the unique rows. How can I retain the original ordering and still take the unique rows?
So, the expected output should be:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 4, 5, 5]])
Using return_index
_,idx=np.unique(stacked, axis=0,return_index=True)
stacked[np.sort(idx)]
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 4, 5, 5]])
after get the stacked array
step 1: get the indexes of the rows for sorted unique array
row_indexes = np.unique(stacked, return_index=True, axis=0)[1]
Note: row_indexes is holding the indexes for sorted array
step 2 : now iterate through the stacked array with the sorted index
sorted_index=sorted(row_indexes)
new_arr=[]
for i in range(len(sorted_index)):
new_arr.append(stacked[sorted_index[i]]
thats it!!!!!
I have a huge (N*20) matrix where every 5 rows is a valid sample, ie. every (5*20) matrix. I'm trying to reshape it into a (N/5,1,20,5) matrix where the dimension 20 is kept unchanged. I could do it in tensroflow using keep_dim, but how can I achieve this in numpy?
Thanks in advance.
Reshape and then swap the axes around:
arr1 = arr.reshape(N/5,5,1,20)
arr2 = arr1.transpose(0,2,3,1)
for example
In [476]: arr = np.arange(24).reshape(6,4)
In [477]: arr
Out[477]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]])
In [478]: arr1 = arr.reshape(2,3,1,4)
In [479]: arr2 = arr1.transpose(0,2,3,1)
In [480]: arr2.shape
Out[480]: (2, 1, 4, 3)
In [482]: arr2
Out[482]:
array([[[[ 0, 4, 8],
[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11]]],
[[[12, 16, 20],
[13, 17, 21],
[14, 18, 22],
[15, 19, 23]]]])
Short version
I want to manipulate a numpy array (test, see first code snippet), so that it becomes rearranged (evenodd_single_column, see second code snippet). I wrote a for loop, but since I'm working with semi-big data I would be glad if there is a better way to achieve this.
Long version
I am writing a script where at one point I should be doing the following manipulation on a numpy array:
test = np.arange(24).reshape(8,3)
test
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23]])
needs to be converted to an array which takes the first x (timepoints, in this example 2) from all the columns (experiments, here 3) and puts it in an array. Then it goes to the next two values of all the columns and appends the array, until all iterations (y) are over. In the end it should look like that:
>>> evenodd_single_column
array([[ 0, 3],
[ 1, 4],
[ 2, 5],
[12, 15],
[13, 16],
[14, 17],
[ 6, 9],
[ 7, 10],
[ 8, 11],
[18, 21],
[19, 22],
[20, 23]])
To achieve this, I had to write a for loop:
all_odd = []
all_even = []
x = 2
y = 4
test = np.arange(24).reshape(8,3)
counter = 0
for i in range(1, int(test.shape[0]/2)+1):
time_window = i * x
if math.modf(counter / 2)[0] == 0:
for j in range(0, test.shape[1]):
all_even.extend(test[time_window - x:time_window, j])
else:
for j in range(0,test.shape[1]):
all_odd.extend(test[time_window - x:time_window, j])
counter = counter + 1
even_single_column_test = np.asarray(all_even).reshape((int(y / 2 * test.shape[1]), x))
odd_single_column_test = np.asarray(all_odd).reshape((int(y / 2 * test.shape[1]), x))
evenodd_single_column = even_single_column_test
evenodd_single_column = np.append(evenodd_single_column, odd_single_column_test).reshape(int(odd_single_column_test.shape[0]*2), x)
My question: Can this be done with one of the elegant (and more importantly - faster) numpy matrix manipulations? I don't want to go around loops, making lists to then transform them to numpy arrays again.
I am not a programmer by training, I apologize in advance if the solution is an obvious one!
Thanks!
You can use a combination of np.reshape and np.transpose -
test.reshape(2,2,2,3).transpose(1,0,3,2).reshape(-1,2)
Sample run -
In [42]: test
Out[42]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23]])
In [43]: test.reshape(2,2,2,3).transpose(1,0,3,2).reshape(-1,2)
Out[43]:
array([[ 0, 3],
[ 1, 4],
[ 2, 5],
[12, 15],
[13, 16],
[14, 17],
[ 6, 9],
[ 7, 10],
[ 8, 11],
[18, 21],
[19, 22],
[20, 23]])