Lst's say there is a matrix A
A = [[ 34 61 29 74(17)32 72 92 93 57 ]
[(46)10 23 84 74 57 56 88 90 36 ]
[ 23(83)58 42 93 54 82 48 63 73 ]]
and a vector b of size 3
b = [4, 0, 1]
Does numpy has any function that would do the following job?
>>>A.choose_row_wise(b)
would output:
[17, 46, 83]
With numpy.take_along_axis:
b = np.array([4, 0, 1])
res = np.take_along_axis(A, b[:, None], axis=1).flatten()
print(res)
[17 46 83]
This can be done using indexing.
import numpy as np
A = np.array([[34, 61, 29, 74, 17, 32, 72, 92, 93, 57],[46, 10, 23, 84, 74, 57, 56, 88, 90, 36],[23, 83, 58, 42, 93, 54, 82, 48, 63, 73]])
b = np.array([4, 0, 1])
result = A[np.arange(len(b)), b]
[17 46 83]
Maybe you can use the following code:
import numpy as np
A = np.array([[34,61,29,74,(17),32,72,92,93,57],[(46),10,23,84,74,57,56,88,90,36],[23,(83),58,42,93,54,82,48,63,73]])
b = [4, 0, 1]
print([A[b.index(i), i] for i in b])
I made some numpy array np3
np1 = np.array(range(2*3*5))
np3 = np1.reshape(2,3,5)
and np3 has shape like this:
[[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]]
[[15 16 17 18 19]
[20 21 22 23 24]
[25 26 27 28 29]]]
then, I made new numpy array np_55
np_55 = np.full((3,1),55)
and np_55 has shape like this:
[[55]
[55]
[55]]
I want make numpy array like below using both numpy arrays np3 and np_55 (I'll call that 'ANSWER'):
[[[ 0 1 2 3 4 55]
[ 5 6 7 8 9 55]
[10 11 12 13 14 55]]
[[15 16 17 18 19 55]
[20 21 22 23 24 55]
[25 26 27 28 29 55]]]
but I can't make it using both numpy arrays np3 and np_55. Of course I can make hard code like this:
a = np.append((np3[0]), np3_55, axis=1)
b = np.append((np3[1]), np3_55, axis=1)
a = a.reshape(1,3,6)
b = b.reshape(1,3,6)
np.append(a, b, axis=0)
but I don't know how can I solve ANSWER simply.
You can try the following:
import numpy as np
a = np.arange(2*3*5).reshape(2, 3, 5)
b = np.full((3,1),55)
np.c_[a, np.broadcast_to(b, (a.shape[0], *b.shape))]
It gives:
array([[[ 0, 1, 2, 3, 4, 55],
[ 5, 6, 7, 8, 9, 55],
[10, 11, 12, 13, 14, 55]],
[[15, 16, 17, 18, 19, 55],
[20, 21, 22, 23, 24, 55],
[25, 26, 27, 28, 29, 55]]])
I have an nxn numpy array, and I would like to divide it evenly into nxn tiles and randomly shuffle these, while retaining the pattern inside the tiles.
For example, if I have an array that's size (200,200), I want to be able to divide this into say 16 arrays of size (50,50), or even 64 arrays of size (25,25), and randomly shuffle these, while retaining the same shape of the original array (200,200) and retaining the order of numbers inside of the smaller arrays.
I have looked up specific numpy functions, and I found the numpy.random.shuffle(x) function, but this will randomly shuffle the individual elements of an array. I would only like to shuffle these smaller arrays within the larger array.
Is there any numpy function or quick way that will do this? I'm not sure where to begin.
EDIT: To further clarify exactly what I want:
Let's say I have an input 2D array of shape (10,10) of values:
0 1 2 3 4 5 6 7 8 9
10 11 12 13 14 15 16 17 18 19
20 21 22 23 24 25 26 27 28 29
30 31 32 33 34 35 36 37 38 39
40 41 42 43 44 45 46 47 48 49
50 51 52 53 54 55 56 57 58 59
60 61 62 63 64 65 66 67 68 69
70 71 72 73 74 75 76 77 78 79
80 81 82 83 84 85 86 87 88 89
90 91 92 93 94 95 96 97 98 99
I choose a tile size such that it fits evenly into this array, so since this array has shape (10,10), I can either choose to split this into 4 (5,5) tiles, or 25 (2,2) tiles. So if I choose 4 (5,5) tiles, I want to randomly shuffle these tiles that results in an output array that could look like this:
50 51 52 53 54 0 1 2 3 4
60 61 62 63 64 10 11 12 13 14
70 71 72 73 74 20 21 22 23 24
80 81 82 83 84 30 31 32 33 34
90 91 92 93 94 40 41 42 43 44
55 56 57 58 59 5 6 7 8 9
65 66 67 68 69 15 16 17 18 19
75 76 77 78 79 25 26 27 28 29
85 86 87 88 89 35 36 37 38 39
95 96 97 98 99 45 46 47 48 49
Every array (both the input array, the output array, and the separate tiles) would be squares, so that when randomly shuffled the size and dimension of the main array stays the same (10,10).
here is my solution using loop
import numpy as np
arr = np.arange(36).reshape(6,6)
def suffle_section(arr, n_sections):
assert arr.shape[0]==arr.shape[1], "arr must be square"
assert arr.shape[0]%n_sections == 0, "arr size must divideable into equal n_sections"
size = arr.shape[0]//n_sections
new_arr = np.empty_like(arr)
## randomize section's row index
rand_indxes = np.random.permutation(n_sections*n_sections)
for i in range(n_sections):
## randomize section's column index
for j in range(n_sections):
rand_i = rand_indxes[i*n_sections + j]//n_sections
rand_j = rand_indxes[i*n_sections + j]%n_sections
new_arr[i*size:(i+1)*size, j*size:(j+1)*size] = \
arr[rand_i*size:(rand_i+1)*size, rand_j*size:(rand_j+1)*size]
return new_arr
result = suffle_section(arr, 3)
display(arr)
display(result)
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]])
array([[ 4, 5, 16, 17, 24, 25],
[10, 11, 22, 23, 30, 31],
[14, 15, 2, 3, 0, 1],
[20, 21, 8, 9, 6, 7],
[26, 27, 12, 13, 28, 29],
[32, 33, 18, 19, 34, 35]])
If you have access to skimage (it comes with Spyder) you could use view_as_blocks:
from skimage.util import view_as_blocks
def shuffle_tiles(arr, m, n):
a_= view_as_blocks(arr,(m,n)).reshape(-1,m,n)
# shuffle works along 1st dimension and in-place
np.random.shuffle(a_)
return a_
We will use np.random.shuffle alongwith axes permutations to achieve the desired results. There are two interpretations to it. Hence, two solutions.
Shuffle randomly within each block
Elements in each block are randomized and that same randomized order is maintaiined in all blocks.
def randomize_tiles_shuffle_within(a, M, N):
# M,N are the height and width of the blocks
m,n = a.shape
b = a.reshape(m//M,M,n//N,N).swapaxes(1,2).reshape(-1,M*N)
np.random.shuffle(b.T)
return b.reshape(m//M,n//N,M,N).swapaxes(1,2).reshape(a.shape)
Shuffle randomly blocks w.r.t each other
Blocks are randomized w.r.t each other, while keeping the order within each block same as in the original array.
def randomize_tiles_shuffle_blocks(a, M, N):
m,n = a.shape
b = a.reshape(m//M,M,n//N,N).swapaxes(1,2).reshape(-1,M*N)
np.random.shuffle(b)
return b.reshape(m//M,n//N,M,N).swapaxes(1,2).reshape(a.shape)
Sample runs -
In [47]: a
Out[47]:
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]])
In [48]: randomize_tiles_shuffle_within(a, 3, 3)
Out[48]:
array([[ 1, 7, 13, 4, 10, 16],
[14, 8, 12, 17, 11, 15],
[ 0, 6, 2, 3, 9, 5],
[19, 25, 31, 22, 28, 34],
[32, 26, 30, 35, 29, 33],
[18, 24, 20, 21, 27, 23]])
In [49]: randomize_tiles_shuffle_blocks(a, 3, 3)
Out[49]:
array([[ 3, 4, 5, 18, 19, 20],
[ 9, 10, 11, 24, 25, 26],
[15, 16, 17, 30, 31, 32],
[ 0, 1, 2, 21, 22, 23],
[ 6, 7, 8, 27, 28, 29],
[12, 13, 14, 33, 34, 35]])
Here is an approach that tries hard to avoid unnecessary copies:
import numpy as np
def f_pp(a,bs):
i,j = a.shape
k,l = bs
esh = i//k,k,j//l,l
bc = esh[::2]
sh1,sh2 = np.unravel_index(np.random.permutation(bc[0]*bc[1]),bc)
ns1,ns2 = np.unravel_index(np.arange(bc[0]*bc[1]),bc)
out = np.empty_like(a)
out.reshape(esh)[ns1,:,ns2] = a.reshape(esh)[sh1,:,sh2]
return out
Timings:
pp 0.41529153706505895
dv 1.3133141631260514
br 1.6034217830747366
Test script (continued)
# Divakar
def f_dv(a,bs):
M,N = bs
m,n = a.shape
b = a.reshape(m//M,M,n//N,N).swapaxes(1,2).reshape(-1,M*N)
np.random.shuffle(b)
return b.reshape(m//M,n//N,M,N).swapaxes(1,2).reshape(a.shape)
from skimage.util import view_as_blocks
# Brenlla shape fixed by pp
def f_br(arr,bs):
m,n = bs
a_= view_as_blocks(arr,(m,n))
sh = a_.shape
a_ = a_.reshape(-1,m,n)
# shuffle works along 1st dimension and in-place
np.random.shuffle(a_)
return a_.reshape(sh).swapaxes(1,2).reshape(arr.shape)
ex = np.arange(100000).reshape(1000,100)
bs = 10,10
tst = np.tile(np.arange(np.prod(bs)).reshape(bs),np.floor_divide(ex.shape,bs))
from timeit import timeit
for n,f in list(globals().items()):
if n.startswith('f_'):
assert (tst==f(tst,bs)).all()
print(n[2:],timeit(lambda:f(ex,bs),number=1000))
Here's code to shuffle row order but keep row items exactly as is:
import numpy as np
np.random.seed(0)
#creates a 6x6 array
a = np.random.randint(0,100,(6,6))
a
array([[44, 47, 64, 67, 67, 9],
[83, 21, 36, 87, 70, 88],
[88, 12, 58, 65, 39, 87],
[46, 88, 81, 37, 25, 77],
[72, 9, 20, 80, 69, 79],
[47, 64, 82, 99, 88, 49]])
#creates a number for each row index, 0,1,2,3,4,5
order = np.arange(6)
#shuffle index array
np.random.shuffle(order)
#make new array in shuffled order
shuffled = np.array([a[y] for y in order])
shuffled
array([[46, 88, 81, 37, 25, 77],
[88, 12, 58, 65, 39, 87],
[83, 21, 36, 87, 70, 88],
[47, 64, 82, 99, 88, 49],
[44, 47, 64, 67, 67, 9],
[72, 9, 20, 80, 69, 79]])
As a disclaimer I'm very new to python and numpy arrays. Reading some of the answers to similar questions and trying their solutions for my own data hasn't been very helpful so I thought I'd just post my own question. For example, Reshaping 3D Numpy Array to a 2D array. Its entirely believable though that I've just implemented those other solutions wrong.
I have a 3D numpy array "C"
C = np.reshape(np.arange(3*3*4),(3,3,4))
print(C)
[[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
[[12 13 14 15]
[16 17 18 19]
[20 21 22 23]]
[[24 25 26 27]
[28 29 30 31]
[32 33 34 35]]]
I would like to reshape it into something like:
[0 12 14], [1,13,25], [2,24,26] ..... etc
where the first elements of each of the 3 arrays gets put into its own array, then the second elements of each array get put into a new array, and so on.
It seems trivial, but I'm stumped. I've tried different types combinations of .reshape, just for example,
output=C.reshape(12,3)
I've tried changing the order from "C" to "F", playing around with different .reshape() parameters, but can't seem to actually get the final result in the desired structure
Any tips would be much appreciated.
I think this is what you want:
C = np.reshape(np.arange(3*3*4),(3,3,4))
C.reshape(3,12).T
array([[ 0, 12, 24],
[ 1, 13, 25],
[ 2, 14, 26],
[ 3, 15, 27],
[ 4, 16, 28],
[ 5, 17, 29],
[ 6, 18, 30],
[ 7, 19, 31],
[ 8, 20, 32],
[ 9, 21, 33],
[10, 22, 34],
[11, 23, 35]])
Hi there I have the following matrix
[[ 47 43 51 81 54 81 52 54 31 46]
[ 35 21 30 16 37 11 35 30 39 37]
[ 8 17 11 2 5 4 11 9 17 10]
[ 5 9 4 0 1 1 0 3 9 3]
[ 2 7 2 0 0 0 0 1 2 1]
[215 149 299 199 159 325 179 249 249 199]
[ 27 49 24 4 21 8 35 15 45 25]
[100 100 100 100 100 100 100 100 100 100]]
I need to iterate over the matrix summing all elements in rows 0,1,2,3,4 only
example: I need
row_0_sum = 47+43+51+81....46
Furthermore I need to store each rows sum in an array like this
[row0_sum, row1_sum, row2_sum, row3_sum, row4_sum]
So far I have tried this code but its not doing the job:
mu = np.zeros(shape=(1,6))
#get an average
def standardize_ratings(matrix):
sum = 0
for i, eli in enumerate(matrix):
for j, elj in enumerate(eli):
if(i<5):
sum = sum + matrix[i][j]
if(j==elj.len -1):
mu[i] = sum
sum = 0
print "mu[i]="
print mu[i]
This just gives me an Error: numpy.int32 object has no attribute 'len'
So can someone help me. What's the best way to do this and which type of array in Python should I use to store this. Im new to Python but have done programming....
Thannks
Make your data, matrix, a numpy.ndarray object, instead of a list of lists, and then just do matrix.sum(axis=1).
>>> matrix = np.asarray([[ 47, 43, 51, 81, 54, 81, 52, 54, 31, 46],
[ 35, 21, 30, 16, 37, 11, 35, 30, 39, 37],
[ 8, 17, 11, 2, 5, 4, 11, 9, 17, 10],
[ 5, 9, 4, 0, 1, 1, 0, 3, 9, 3],
[ 2, 7, 2, 0, 0, 0, 0, 1, 2, 1],
[215, 149, 299, 199, 159, 325, 179, 249, 249, 199],
[ 27, 49, 24, 4, 21, 8, 35, 15, 45, 25],
[100, 100, 100, 100, 100, 100, 100, 100, 100, 100]])
>>> print matrix.sum(axis=1)
[ 540 291 94 35 15 2222 253 1000]
To get the first five rows from the result, you can just do:
>>> row_sums = matrix.sum(axis=1)
>>> rows_0_through_4_sums = row_sums[:5]
>>> print rows_0_through_4_sums
[540 291 94 35 15]
Or, you can alternatively sub-select only those rows to begin with and only apply the summation to them:
>>> rows_0_through_4 = matrix[:5,:]
>>> print rows_0_through_4.sum(axis=1)
[540 291 94 35 15]
Some helpful links will be:
NumPy for Matlab Users, if you are familiar with these things in Matlab/Octave
Slicing/Indexing in NumPy