Short version
I want to manipulate a numpy array (test, see first code snippet), so that it becomes rearranged (evenodd_single_column, see second code snippet). I wrote a for loop, but since I'm working with semi-big data I would be glad if there is a better way to achieve this.
Long version
I am writing a script where at one point I should be doing the following manipulation on a numpy array:
test = np.arange(24).reshape(8,3)
test
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23]])
needs to be converted to an array which takes the first x (timepoints, in this example 2) from all the columns (experiments, here 3) and puts it in an array. Then it goes to the next two values of all the columns and appends the array, until all iterations (y) are over. In the end it should look like that:
>>> evenodd_single_column
array([[ 0, 3],
[ 1, 4],
[ 2, 5],
[12, 15],
[13, 16],
[14, 17],
[ 6, 9],
[ 7, 10],
[ 8, 11],
[18, 21],
[19, 22],
[20, 23]])
To achieve this, I had to write a for loop:
all_odd = []
all_even = []
x = 2
y = 4
test = np.arange(24).reshape(8,3)
counter = 0
for i in range(1, int(test.shape[0]/2)+1):
time_window = i * x
if math.modf(counter / 2)[0] == 0:
for j in range(0, test.shape[1]):
all_even.extend(test[time_window - x:time_window, j])
else:
for j in range(0,test.shape[1]):
all_odd.extend(test[time_window - x:time_window, j])
counter = counter + 1
even_single_column_test = np.asarray(all_even).reshape((int(y / 2 * test.shape[1]), x))
odd_single_column_test = np.asarray(all_odd).reshape((int(y / 2 * test.shape[1]), x))
evenodd_single_column = even_single_column_test
evenodd_single_column = np.append(evenodd_single_column, odd_single_column_test).reshape(int(odd_single_column_test.shape[0]*2), x)
My question: Can this be done with one of the elegant (and more importantly - faster) numpy matrix manipulations? I don't want to go around loops, making lists to then transform them to numpy arrays again.
I am not a programmer by training, I apologize in advance if the solution is an obvious one!
Thanks!
You can use a combination of np.reshape and np.transpose -
test.reshape(2,2,2,3).transpose(1,0,3,2).reshape(-1,2)
Sample run -
In [42]: test
Out[42]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23]])
In [43]: test.reshape(2,2,2,3).transpose(1,0,3,2).reshape(-1,2)
Out[43]:
array([[ 0, 3],
[ 1, 4],
[ 2, 5],
[12, 15],
[13, 16],
[14, 17],
[ 6, 9],
[ 7, 10],
[ 8, 11],
[18, 21],
[19, 22],
[20, 23]])
Related
I have a multidimensional numpy array like this:
[
[
[1,2,3,4,5],
[6,7,8,9,10],
[11,12,13,14,15]
],
[
[16,17,18,19,20],
[21,22,23,24,25],
[26,27,28,29,30]
]
]
and would like to create a function that cuts it in half alongside a specified axis, without including the middle element in case the size is uneven. So if I say my_function(my_ndarray, 0), I want to get
[
[
[1,2,3,4,5],
[6,7,8,9,10],
[11,12,13,14,15]
]
]
for my_function(my_ndarray, 1) I want to get
[
[
[1,2,3,4,5]
],
[
[16,17,18,19,20]
]
]
and for my_function(my_ndarray, 2) I want to get
[
[
[1,2],
[6,7],
[11,12]
],
[
[16,17],
[21,22],
[26,27]
]
]
My first attempt involved the np.split() method, but it unfortunately runs into problems when the length of the axis is an uneven number and doesn't allow me to specify that I would like to omit. In theory I could make an if statement and cut away the last slice of the selected axis if this is the case, but I would like to know if there is a more efficient way to solve this problem.
Given an axis axis and an array a, I think you can can do
def my_function(a, axis):
l = a.shape[axis]//2
return a.take(range(l), axis=axis)
Examples:
>>> my_function(a, 0)
array([[[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15]]])
>>> my_function(a, 1)
array([[[ 1, 2, 3, 4, 5]],
[[16, 17, 18, 19, 20]]])
>>> my_function(a, 2)
array([[[ 1, 2],
[ 6, 7],
[11, 12]],
[[16, 17],
[21, 22],
[26, 27]]])
What about:
def slice_n(a, n):
slices = [slice(None)]*a.ndim
slices[n] = slice(0, a.shape[n]//2)
return a[tuple(slices)]
slice_n(a, 0)
array([[[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15]]])
slice_n(a, 1)
array([[[ 1, 2, 3, 4, 5]],
[[16, 17, 18, 19, 20]]])
slice_n(a, 2)
array([[[ 1, 2],
[ 6, 7],
[11, 12]],
[[16, 17],
[21, 22],
[26, 27]]])
used input (a):
array([[[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15]],
[[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30]]])
I am generating multidimensional array of different sizes, though they'll all have an even number of columns.
>> import numpy as np
>> x = np.arange(24).reshape((3,8))
Which results in:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
I am able to slice with numpy and get the first column in an array:
>> newarr = x[0:,0:2]
array([[ 0, 1],
[ 8, 9],
[16, 17]])
However, I want to have one array that is just a list of the columns where column 1 and 2 are together, 3 and 4 are together, and so on.. For example:
array([[[ 0, 1],
[ 8, 9],
[16, 17]],
[[ 2, 3],
[10, 11],
[18, 19]],
etc....]
)
This code below works but it's clunky and my arrays are not all the same. Some arrays have 16 columns, some have 34, some have 50, etc.
>> newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]])]
There's got to be a better way to do this than
newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]+...+[x[0:,n:n+2]]
Help!
My idea is adding a for loop:
slice_len = 2
x_list = [x[0:, slice_len*i:slice_len*(i+1)] for i in range(x.shape[1] // slice_len)]
Output:
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]]), array([[ 4, 5],
[12, 13],
[20, 21]]), array([[ 6, 7],
[14, 15],
[22, 23]])]
I have a huge (N*20) matrix where every 5 rows is a valid sample, ie. every (5*20) matrix. I'm trying to reshape it into a (N/5,1,20,5) matrix where the dimension 20 is kept unchanged. I could do it in tensroflow using keep_dim, but how can I achieve this in numpy?
Thanks in advance.
Reshape and then swap the axes around:
arr1 = arr.reshape(N/5,5,1,20)
arr2 = arr1.transpose(0,2,3,1)
for example
In [476]: arr = np.arange(24).reshape(6,4)
In [477]: arr
Out[477]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]])
In [478]: arr1 = arr.reshape(2,3,1,4)
In [479]: arr2 = arr1.transpose(0,2,3,1)
In [480]: arr2.shape
Out[480]: (2, 1, 4, 3)
In [482]: arr2
Out[482]:
array([[[[ 0, 4, 8],
[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11]]],
[[[12, 16, 20],
[13, 17, 21],
[14, 18, 22],
[15, 19, 23]]]])
I have a 3D array a of data and a 2D array b of indices. I need to take a sub-array of a along the 3rd axis, using the indices from b. I can do it with take like this:
a = np.arange(24).reshape((2,3,4))
b = np.array([0,2,1,3]).reshape((2,2))
np.array([np.take(a_,b_,axis=1) for (a_,b_) in zip(a,b)])
Can I do it without list comprehension, using some fancy indexing? I am worried about efficiency, so if fancy indexing is not more efficient in this case, I would like to know it.
EDIT The 1st thing I've tried is a[[0,1],:,b] but it doesn't give the sub-array I need
In [317]: a
Out[317]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
In [318]: a = np.arange(24).reshape((2,3,4))
...: b = np.array([0,2,1,3]).reshape((2,2))
...: np.array([np.take(a_,b_,axis=1) for (a_,b_) in zip(a,b)])
...:
Out[318]:
array([[[ 0, 2],
[ 4, 6],
[ 8, 10]],
[[13, 15],
[17, 19],
[21, 23]]])
So you want the 0 & 2 columns from the 1st block, and 1 & 3 from the second.
Make a c that matches b in shape, and embodies this observation
In [319]: c=np.array([[0,0],[1,1]])
In [320]: c
Out[320]:
array([[0, 0],
[1, 1]])
In [321]: b
Out[321]:
array([[0, 2],
[1, 3]])
In [322]: a[c,:,b]
Out[322]:
array([[[ 0, 4, 8],
[ 2, 6, 10]],
[[13, 17, 21],
[15, 19, 23]]])
That's the right numbers, but not the right shape.
A column vector can be used instead of c.
In [323]: a[np.arange(2)[:,None],:,b] # or a[[[0],[1]],:,b]
Out[323]:
array([[[ 0, 4, 8],
[ 2, 6, 10]],
[[13, 17, 21],
[15, 19, 23]]])
As for the shape, we can transpose the last two axes
In [324]: a[np.arange(2)[:,None],:,b].transpose(0,2,1)
Out[324]:
array([[[ 0, 2],
[ 4, 6],
[ 8, 10]],
[[13, 15],
[17, 19],
[21, 23]]])
This transpose is required because we have a slice between two index arrays, a mix of basic and advanced indexing. It's documented, but never the less often puzzling. It put the slice dimension (3) last, and we have to transpose it back.
Nice little indexing puzzle!
The latest question and explanation of this advanced/basic transpose:
Indexing numpy multidimensional arrays depends on a slicing method
This is my first try. I will see if I can do better.
#using numpy broadcasting.
np.r_[a[0][:,b[0]],a[1][:,b[1]]].reshape(2,3,2)
Out[300]: In [301]:
array([[[ 0, 2],
[ 4, 6],
[ 8, 10]],
[[13, 15],
[17, 19],
[21, 23]]])
Second try:
#convert both a and b to a 2d array and then slice all rows and only columns determined by b.
a.reshape(6,4)[np.arange(6)[:,None],b.repeat(3,0)].reshape(2,3,2)
Out[429]:
array([[[ 0, 2],
[ 4, 6],
[ 8, 10]],
[[13, 15],
[17, 19],
[21, 23]]])
I have a problem with printing a column in a numpy 3D Matrix.
Here is a simplified version of the problem:
import numpy as np
Matrix = np.zeros(((10,9,3))) # Creates a 10 x 9 x 3 3D matrix
Matrix[2][2][6] = 578
# I want to print Matrix[2][x][6] for x in range(9)
# the purpose of this is that I want to get all the Values in Matrix[2][x][6]
Much appreciated if you guys can help me out.
Thanks in advance.
Slicing would work:
a = np.zeros((10, 9, 3))
a[6, 2, 2] = 578
for x in a[6, :, 2]:
print(x)
Output:
0.0
0.0
578.0
0.0
0.0
0.0
0.0
0.0
0.0
Not sure if Numpy supports this, but you can do it with normal lists this way:
If you have three lists a = [1,2,3], b = [4,5,6], and c = [7,8,9], you can get the second dimension [2,5,8] for example by doing
list(zip(a,b,c))[1]
EDIT:
Turns out this is pretty simple in Numpy. According to this thread you can just do:
Matrix[:,1]
No sure if this is what you want. Here is demo:
In [1]: x = np.random.randint(0, 20, size=(4, 5, 3))
In [2]: x
Out[2]:
array([[[ 5, 13, 9],
[ 8, 16, 5],
[15, 17, 1],
[ 6, 14, 5],
[11, 13, 9]],
[[ 5, 8, 0],
[ 8, 15, 5],
[ 9, 2, 13],
[18, 4, 14],
[ 8, 3, 13]],
[[ 3, 7, 4],
[15, 11, 6],
[ 7, 8, 14],
[12, 8, 18],
[ 4, 2, 8]],
[[10, 1, 16],
[ 5, 2, 1],
[11, 12, 13],
[11, 9, 1],
[14, 5, 1]]])
In [4]: x[:, 2, :]
Out[4]:
array([[15, 17, 1],
[ 9, 2, 13],
[ 7, 8, 14],
[11, 12, 13]])