I have the following slicing problem in numpy.
a = np.arange(36).reshape(-1,4)
a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23],
[24, 25, 26, 27],
[28, 29, 30, 31],
[32, 33, 34, 35]])
In my problem always three rows represent one sample, in my case coordinates.
I want to access this matrix in a way that if I use a[0:2] to get the following:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]
These are the first two coordinate samples.
I have to extract a large amount of these coordinate sets from an array.
Thanks
Based on How do you split a list into evenly sized chunks?, I found the following solution, which gives me the desired result.
def chunks(l, n, indices):
return np.vstack([l[idx*n:idx*n+n] for idx in indices])
chunks(a,3,[0,2])
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[24, 25, 26, 27],
[28, 29, 30, 31],
[32, 33, 34, 35]])
Probably this solution could be improved and somebody won't need the stacking.
If three rows are a sample, you can reshape your array to reflect that, use fancy indexing to retrieve your samples, then undo the shape change:
>>> a = a.reshape(-1, 3, 4)
>>> a[[0, 2]].reshape(-1, 4)
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[24, 25, 26, 27],
[28, 29, 30, 31],
[32, 33, 34, 35]])
Related
Is it possible to apply few conditions to numpy.array while indexing them? In my case I want to show first 10 elements and then 2 neighbour elements with step 5:
numpy.arange(40)
#Output is:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35, 36, 37, 38, 39])
Applying my conditions to this array I want to get this:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 15, 20, 21, 26, 27, 32,
33, 38, 39])
I haven't found any solution. I thought it should look something like this:
np.arange(40)[0:10, 10:len(np.arange(40)):5]
But it's not working for me.
You can try custom indexing on reshaped array:
n = 40
idx = np.zeros(n//2, dtype=bool)
idx[:5] = True
idx[4:None:3] = True
>>> np.arange(n).reshape(-1,2)[idx]
array([[ 0, 1],
[ 2, 3],
[ 4, 5],
[ 6, 7],
[ 8, 9],
[14, 15],
[20, 21],
[26, 27],
[32, 33],
[38, 39]])
>>> np.arange(n).reshape(-1,2)[idx].ravel()
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 15, 20, 21, 26, 27, 32,
33, 38, 39])
I am building a neural network. where I have to flatten my training dataset.
I have two options.
1 is:
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T
and 2nd one is:
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[1]*train_x_orig.shape[2]*train_x_orig.shape[3], 209)
both gave the same shape but I found difference while computing cost?
why is that? thank you
Your original tensor is of at least rank 4 based on the second example. The first example pulls each element, ordered by increasing the right-most index, and inserts the elements into rows the length of the zeroth shape. Then transposes.
The second example again pull elements from by incrementing from the right-most index, i.e.:
element = train_x_orig[0, 0, 0, 0]
new_row.append(element)
element = train_x_orig[0, 0, 0, 1]
new_row.append(element)
but the size of the row is different. It is now the dimension of everything else in the tensor.
Here is an example to illustrate.
First we create an ordered array and reshape it to rank 4.
import numpy as np
x = np.arange(36).reshape(3,2,3,2)
x
# returns:
array([[[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[ 6, 7],
[ 8, 9],
[10, 11]]],
[[[12, 13],
[14, 15],
[16, 17]],
[[18, 19],
[20, 21],
[22, 23]]],
[[[24, 25],
[26, 27],
[28, 29]],
[[30, 31],
[32, 33],
[34, 35]]]])
Here is the output of the first example
x.reshape(x.shape[0], -1).T
# returns:
array([[ 0, 12, 24],
[ 1, 13, 25],
[ 2, 14, 26],
[ 3, 15, 27],
[ 4, 16, 28],
[ 5, 17, 29],
[ 6, 18, 30],
[ 7, 19, 31],
[ 8, 20, 32],
[ 9, 21, 33],
[10, 22, 34],
[11, 23, 35]])
And here is the second example
x.reshape(x.shape[1]*x.shape[2]*x.shape[3], -1)
# returns:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23],
[24, 25, 26],
[27, 28, 29],
[30, 31, 32],
[33, 34, 35]])
How the elements get reordered is fundamentally different.
If I have a list:
lst = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
I would like to cast the above list into an array with the following arrangements of the elements:
array([[ 1, 2, 3, 7, 8, 9]
[ 4, 5, 6, 10, 11, 12]
[13, 14, 15, 19, 20, 21]
[16, 17, 18, 22, 23, 24]])
How do I do this or what is the best way to do this? Many thanks.
I have done this in a crude way below where I will just get all the sub-matrix and then concatenate all of them at the end:
np.array(results[arr.shape[0]*arr.shape[1]*0:arr.shape[0]*arr.shape[1]*1]).reshape(arr.shape[0], arr.shape[1])
array([[1, 2, 3],
[4, 5, 6]])
np.array(results[arr.shape[0]*arr.shape[1]*1:arr.shape[0]*arr.shape[1]*2]).reshape(arr.shape[0], arr.shape[1])
array([[ 7, 8, 9],
[ 10, 11, 12]])
etc,
But I will need a more generalized way of doing this (if there is one) as I will need to do this for an array of any size.
You could use the reshape function from numpy, with a bit of indexing :
a = np.arange(24)
>>> a
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23])
Using reshape and a bit of indexing :
a = a.reshape((8,3))
idx = np.arange(2)
idx = np.concatenate((idx,idx+4))
idx = np.ravel([idx,idx+2],'F')
b = a[idx,:].reshape((4,6))
Ouptut :
>>> b
array([[ 0, 1, 2, 6, 7, 8],
[ 3, 4, 5, 9, 10, 11],
[12, 13, 14, 18, 19, 20],
[15, 16, 17, 21, 22, 23]])
Here the tuple (4,6) passed to reshape indicates that you want your array to be 2 dimensional, and have 4 arrays of 6 elements. Those values can be computed.
Then we compute the index to set the correct order of the data. Obvisouly, this a complicated bit here. As I'm not sure what you mean by "any size of data", its difficult for me to give you a agnostic way to compute that index.
Obviously, if you are using a list and not an np.array, you might have to convert the list first, for example by using np.array(your_list).
Edit :
I'm not sure if this exactly what you are after, but this should work for any array evenly divisible by 6 :
def custom_order(size):
a = np.arange(size)
a = a.reshape((size//3,3))
idx = np.arange(2)
idx = np.concatenate([idx+4*i for i in range(0,size//(6*2))])
idx = np.ravel([idx,idx+2],'F')
b = a[idx,:].reshape((size//6,6))
return b
>>> custom_order(48)
array([[ 0, 1, 2, 6, 7, 8],
[ 3, 4, 5, 9, 10, 11],
[12, 13, 14, 18, 19, 20],
[15, 16, 17, 21, 22, 23],
[24, 25, 26, 30, 31, 32],
[27, 28, 29, 33, 34, 35],
[36, 37, 38, 42, 43, 44],
[39, 40, 41, 45, 46, 47]])
Hi I have an array that I want to sum the elements vertically. Just wonder are there any functions can do this easily ?
a = [[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]]
I want to print the answers of 1+6+11+16+21 , 2+7+12+17, 3+8+13, 4+9, 5
As you can see, in each iteration, there is one element less.
This is one approach using zip and a simple iteration.
Ex:
a = [[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]]
print([sum(v[:-i]) if i else sum(v) for i, v in enumerate(zip(*a))])
Output:
[55, 38, 24, 13, 5]
Converting to a numpy array, and then using the following list comprehension
a = np.array(a)
[a[:5-i,i].sum() for i in range(5)]
yields the following:
[55, 38, 24, 13, 5]
I have a 2d array like z and a 1d array denoting the "start column position" like starts. In addition I have a fixed row_length = 2
z = np.arange(35).reshape(5, -1)
# --> array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34]])
starts = np.array([1,5,3,3,2])
What I want is the outcome of this slow for-loop, just quicker if possible.
result = np.zeros(
(z.shape[0], row_length),
dtype=z.dtype
)
for i in range(z.shape[0]):
s = starts[i]
result[i] = z[i, s:s+row_length]
So result in this example should look like this in the end:
array([[ 1, 2],
[12, 13],
[17, 18],
[24, 25],
[30, 31]])
I can't seem to find a way using either fancy indexing or np.take to deliver this result.
One approach would be to get those indices using broadcasted additions with those starts and row_length and then use NumPy's advanced-indexing to extract out all of those elements off the data array, like so -
idx = starts[:,None] + np.arange(row_length)
out = z[np.arange(idx.shape[0])[:,None], idx]
Sample run -
In [197]: z
Out[197]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34]])
In [198]: starts = np.array([1,5,3,3,2])
In [199]: row_length = 2
In [200]: idx = starts[:,None] + np.arange(row_length)
In [202]: z[np.arange(idx.shape[0])[:,None], idx]
Out[202]:
array([[ 1, 2],
[12, 13],
[17, 18],
[24, 25],
[30, 31]])