I got a numpy array X which have shape (2 , 2 , 3) like this:
X = [[[1 , 2 , 3 ]
[4 , 5 , 6]] ,
[[7 , 8 , 9 ]
[10, 11, 12 ]],
I want to flatten all subarrays and turn X to the shape of (2 , 6) which is represented like this:
X = [[ 1 , 2 , 3 , 4, 5 , 6 ] ,
[ 7 , 8 , 9 , 10, 11 , 12 ] ]
But when I used X.flatten(), it just turned out to be like this:
X = [ 1 , 2, 3, 4 , 5, ... , 12]
Is there any function to help me transform the array like what I mean.
Just reshape....
import numpy as np
arr = np.array([[[1 , 2 , 3 ],
[4 , 5 , 6]],
[[7 , 8 , 9 ],
[10, 11, 12 ]]])
arr.reshape(2, 6)
result:
array([[ 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12]])
Iterate over the array and flatten the sublists
arr = np.array([x.flatten() for x in X])
Or for numpy solution you can also use np.hstack()
arr = np.hstack(X)
Output
print(arr)
#[[ 1 2 3 4 5 6]
# [ 7 8 9 10 11 12]]
Loop through the array and flatten the subcomponents:
x = np.array([[[1,2],[3,4]], [[5,6],[7,8]]])
y = np.array([i.flatten() for i in x])
print(x)
print(y)
[[[1 2]
[3 4]]
[[5 6]
[7 8]]]
[[1 2 3 4]
[5 6 7 8]]
If its an numpy array you can use reshape
x.reshape(2,6)
Input:
x = np.array([[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]]]
x.reshape(2,6)
Output:
array([[ 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12]])
Related
I would like to know if there was a way to generate a matrix with values based from a larger matrix. For example, if I have
larger_matrix = np.random.randint(10, size=(10,5))
Out[1]:
array([[0, 9, 0, 0, 3],
[9, 4, 7, 7, 0],
[9, 4, 5, 6, 9],
[6, 3, 1, 7, 3],
[8, 4, 6, 9, 7],
[8, 1, 5, 8, 8],
[9, 9, 6, 0, 9],
[9, 9, 6, 8, 7],
[5, 5, 6, 6, 4],
[4, 4, 7, 0, 7]])
and I want to create smaller_matrix of size (4, 5), with values randomly sampled from larger_matrix, how should I go about this? I'm aware that the function np.random.choice() exists, but I'm quite unsure if it would be helpful for my problem because I'm dealing with matrices instead of lists. Thank you.
Use flatten to convert 2d larger_matrix to 1d.
Then you can use random.choice to get random sample from larger_matrix
Finally, use reshape to convert 1d list to 2d matrix
code:
import numpy as np
larger_matrix = np.random.randint(10, size=(10,5))
print(larger_matrix)
n = 4
m = 5
print(np.reshape(np.random.choice(larger_matrix.flatten(),size = n*m),(n,m)))
result:
[[7 4 4 6 0]
[5 7 0 6 8]
[9 9 0 0 5]
[9 8 0 6 7]
[0 9 8 8 1]
[3 7 1 0 0]
[8 9 2 3 8]
[6 3 7 2 9]
[9 7 5 9 3]
[8 8 3 5 8]]
[[0 0 8 0 9]
[6 9 2 7 0]
[8 7 6 0 7]
[7 4 9 3 7]]
You can run a for loop inside a for loop and use it to fill the smaller matrix with random indexes from the matrix.
For i in range(len(larger_matrix)): For j in range(len(larger_matrix[0])): smaller_matrix[i][j] = larger_matrix[rand1][rand2]
That should cover it. Just make sure you generate 2 new numbers each time.
You could do it like this but bear in mind that the choices taken from the large array may be duplicated:-
import numpy as np
import random
R1 = 10
R2 = 4
C = 5
m = np.random.randint(R1, size=(R1, C))
print(m)
print()
n = []
for _ in range(R2):
n.append(random.choice(m))
print(np.array(n))
How can we get a new matrix containing the average value of A row for each column if B[i,j] == 1 ?
Suppose we have a matrix A(3,4) and a matrix B(3,3)
A = [1 2 3 4
15 20 7 10
0 5 18 12]
And an adjacency matrix
B = [1 0 1
0 0 1
1 1 1 ]
Expected output matrix C which takes the average value of the connected pixels in B :
for example [(1+0)/2 (2+5)/2 (3+18)/2 (4+12)/2] so we get [0.5 , 3.5 10.5 8] in the first row.
C =[0.5 3.5 10.5 8
0 5 18 12
5.33 9 9.33 8.66]
To find the neighborhood of each i, I implemented the following code :
for i in range(A.shape[0]):
for j in range(A.shape[0]):
if (B[i,j] == 1):
print(j)
You can form the sums you need by matrix multiplying:
>>> A = np.array([[1, 2, 3, 4], [15, 20, 7, 10], [0, 5, 18, 12]])
>>> B = np.array([[1, 0, 1], [0, 0, 1], [1, 1, 1]])
>>> summed_groups = B#A
>>> summed_groups
array([[ 1, 7, 21, 16],
[ 0, 5, 18, 12],
[16, 27, 28, 26]])
To get the means normalize by the number of terms per group:
>>> group_sizes = B.sum(axis=1,keepdims=True)
>>> group_sizes
array([[2],
[1],
[3]])
>>> summed_groups / group_sizes
array([[ 0.5 , 3.5 , 10.5 , 8. ],
[ 0. , 5. , 18. , 12. ],
[ 5.33333333, 9. , 9.33333333, 8.66666667]])
Side note: you could also get the group sizes by matrix multiplication:
>>> group_sizes_alt = B#np.ones((len(A),1))
>>> group_sizes_alt
array([[2.],
[1.],
[3.]])
It is convenient to use boolean indexing. For example,
>>> A[[True, False, True], :]
array([[ 1, 2, 3, 4],
[ 0, 5, 18, 12]])
this selects rows 0 and 2 of the A matrix. You can loop over the columns of B and construct the C matrix:
A = np.array([[1, 2, 3, 4], [15, 20, 7, 10], [0, 5, 18, 12]])
B = np.array([[1, 0, 1], [0, 0, 1], [1, 1, 1]]).astype(bool)
C = np.array([A[B[:, i], :].mean(axis=0) for i in range(A.shape[0])])
print(np.around(C, 2))
Result:
[[ 0.5 3.5 10.5 8. ]
[ 0. 5. 18. 12. ]
[ 5.33 9. 9.33 8.67]]
I have 2-dimensional data with shape m by n that I want to window with size w along the first axis into a dataset of m-w many two-dimensional arrays each of size w by n. For instance if the data is:
[[0, 1, 2 ],
[3, 4, 5 ],
[6, 7, 8 ],
[9, 10, 11]]
then I want to window it into
[[[0, 1 , 2 ],
[3, 4 , 5 ],
[6, 7 , 8 ]],
[[3, 4 , 5 ],
[6, 7 , 8 ],
[9, 10, 11]]]
I can window the data together into the right sets:
dataset = tf.data.Dataset.from_tensor_slices(np.arange(5*3).reshape(5,3))
dataset = dataset.window(size=3,shift=1,drop_remainder=True)
for window in dataset : print(list(window.as_numpy_iterator()))
>>>[array([0, 1, 2]), array([3, 4, 5]), array([6, 7, 8])]
>>>[array([3, 4, 5]), array([6, 7, 8]), array([ 9, 10, 11])]
>>>[array([6, 7, 8]), array([ 9, 10, 11]), array([12, 13, 14])]
but I can't figure out how to get the data back into the stacked shape again. I thought maybe tf.stack, but no dice on that. Does anybody know how to finish this?
I found the answer here actually. I don't know why it works, but it does:
dataset = tf.data.Dataset.from_tensor_slices(np.arange(5*3).reshape(5,3))
dataset = dataset.window(size=3,shift=1)
dataset = dataset.flat_map(lambda x : x.batch(3))
for d in dataset : print(d)
which makes
tf.Tensor(
[[0 1 2]
[3 4 5]
[6 7 8]], shape=(3, 3), dtype=int64)
tf.Tensor(
[[ 3 4 5]
[ 6 7 8]
[ 9 10 11]], shape=(3, 3), dtype=int64)
tf.Tensor(
[[ 6 7 8]
[ 9 10 11]
[12 13 14]], shape=(3, 3), dtype=int64)
tf.Tensor(
[[ 9 10 11]
[12 13 14]], shape=(2, 3), dtype=int64)
tf.Tensor([[12 13 14]], shape=(1, 3), dtype=int64)
I am a rookie in the python language and have a question regarding the shape of arrays.
So far as I understand, if a 3 dimensional numpy array is created like this
temp = numpy.asarray([[[0, 0, 0], [1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4], [5, 5, 5]], [[6, 6, 6], [7, 7, 7], [8, 8, 8]]]),
the shape is created like in the following figure:
shape of 3 dimensional array
To calculate the sum, median etc. an axis can be defined to calculate the values e.g.
>>> print(numpy.median(temp, axis=0))
[[3. 3. 3.] [4. 4. 4.] [5. 5. 5.]]
>>> print(numpy.median(temp, axis=1))
[[1. 1. 1.] [4. 4. 4.] [7. 7. 7.]]
>>> print(numpy.median(temp, axis=2))
[[0. 1. 2.] [3. 4. 5.] [6. 7. 8.]]
which implies to me a shape like this shape of 3 dimensional array using axis parameter
Why is the shape handled differently when calculateing the sum, median etc.with the axis parameter?
Your numpy array temp = numpy.asarray([[[0, 0, 0], [1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4], [5, 5, 5]], [[6, 6, 6], [7, 7, 7], [8, 8, 8]]]) looks actually like this:
axis=2
|
v
[[[0 0 0] <-axis=1
[1 1 1]
[2 2 2]] <- axis=0
[[3 3 3]
[4 4 4]
[5 5 5]]
[[6 6 6]
[7 7 7]
[8 8 8]]]
Therefore, when you take the median over specific axis, numpy keeps the rest of the axis as is and finds the median along the specified axis. To have a better understanding, I am going to use the suggested array in comments by #hpaulj:
temp:
axis=2
|
v
[[[ 0 1 2 3] <-axis=1
[ 4 5 6 7]
[ 8 9 10 11]] <- axis=0
[[12 13 14 15]
[16 17 18 19]
[20 21 22 23]]]
We then have:
numpy.median(temp, axis=0):
#The first element is median of [0,12], second one median of [1,13] and so on.
[[ 6. 7. 8. 9.]
[10. 11. 12. 13.]
[14. 15. 16. 17.]]
np.median(temp, axis=1)
#The first element is median of [0,4,8], second one median of [1,5,9] and so on.
[[ 4. 5. 6. 7.]
[16. 17. 18. 19.]]
np.median(temp, axis=2)
#The first element is median of [0,1,2,3], second one median of [4,5,6,7] and so on.
[[ 1.5 5.5 9.5]
[13.5 17.5 21.5]]
Given a 2d tensor (matrix), I would like to partition it into several small ones with equal size. You can regard it as the preprocessing of the max pooling. For instance,
1 2 3 4 5 6 7 8
2 3 4 5 6 7 8 9
3 4 5 6 7 8 9 10
4 5 6 7 8 9 10 11
Given the a dynamic desired_size of 2 * 4, the outputs should be:
1 2 3 4
2 3 4 5
5 6 7 8
6 7 8 9
3 4 5 6
4 5 6 7
7 8 9 10
8 9 10 11
I have studied slice and gather for a while. But I still don't have idea how to do it. Could you tell me how to get that? Thanks in advance!
You could use tf.extract_image_patches, even though it turns out somewhat verbose:
import numpy as np
import tensorflow as tf
x = tf.constant(np.arange(8) + np.arange(1,5)[:,np.newaxis])
e = tf.extract_image_patches(x[tf.newaxis,:,:,tf.newaxis],
[1, 2, 4, 1], [1, 2, 4, 1], [1, 1, 1, 1], padding='VALID')
e = tf.reshape(e, [-1, 2, 4])
sess = tf.InteractiveSession()
e.eval()
# returns
# array([[[ 1, 2, 3, 4],
# [ 2, 3, 4, 5]],
# [[ 5, 6, 7, 8],
# [ 6, 7, 8, 9]],
# [[ 3, 4, 5, 6],
# [ 4, 5, 6, 7]],
# [[ 7, 8, 9, 10],
# [ 8, 9, 10, 11]]])
I tied with tf.split():
num_splits = 2
desired_size = (2, 4)
A = tf.constant(a)
C = tf.concat(tf.split(A, desired_size[0], 0),1)
D = tf.reshape(tf.concat(tf.split(C, num_splits*desired_size[0], 1), 0), (-1, desired_size[0], desired_size[1]))
#The result
[[[ 1 2 3 4]
[ 2 3 4 5]]
[[ 5 6 7 8]
[ 6 7 8 9]]
[[ 3 4 5 6]
[ 4 5 6 7]]
[[ 7 8 9 10]
[ 8 9 10 11]]]
# For num_splits = 4, desired_size = (2, 2) you get
[[[ 1 2]
[ 2 3]]
[[ 3 4]
[ 4 5]]
[[ 5 6]
[ 6 7]]
[[ 7 8]
[ 8 9]]
[[ 3 4]
[ 4 5]]
[[ 5 6]
[ 6 7]]
[[ 7 8]
[ 8 9]]
[[ 9 10]
[10 11]]]