I have a numpy 2D-array:
c = np.arange(36).reshape(6, 6)
[[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]]
I want to split it to multiple 2D-arrays by grid 3x3. (It's like a split big image to 9 small images by grid 3x3):
[[ 0, 1,| 2, 3,| 4, 5],
[ 6, 7,| 8, 9,| 10, 11],
---------+--------+---------
[12, 13,| 14, 15,| 16, 17],
[18, 19,| 20, 21,| 22, 23],
---------+--------+---------
[24, 25,| 26, 27,| 28, 29],
[30, 31,| 32, 33,| 34, 35]]
At final i need array with 9 2D-arrays. Like this:
[[[0, 1], [6, 7]],
[[2, 3], [8, 9]],
[[4, 5], [10, 11]],
[[12, 13], [18, 19]],
[[14, 15], [20, 21]],
[[16, 17], [22, 23]],
[[24, 25], [30, 31]],
[[26, 27], [32, 33]],
[[28, 29], [34, 35]]]
It's just a sample what i need. I want to know how to make small 2D arrays from big 2D array by grid (N,M)
You can use something like:
from numpy.lib.stride_tricks import sliding_window_view
out = np.vstack(sliding_window_view(c, (2, 2))[::2, ::2])
Output:
>>> out.tolist()
[[[0, 1], [6, 7]],
[[2, 3], [8, 9]],
[[4, 5], [10, 11]],
[[12, 13], [18, 19]],
[[14, 15], [20, 21]],
[[16, 17], [22, 23]],
[[24, 25], [30, 31]],
[[26, 27], [32, 33]],
[[28, 29], [34, 35]]]
Related
I'm trying to get the rows of a 3D tensor in a specific order of indices. Here are the inputs:
import tensorflow as tf
matrix = tf.constant([
[[0, 1], [2, 3], [4, 5], [6, 7]],
[[8, 9], [10, 11], [12, 13], [14, 15]],
[[16, 17], [18, 19], [20, 21], [22, 23]],
[[24, 25], [26, 27], [28, 29], [30, 31]],
[[32, 33], [34, 35], [36, 37], [38, 39]]
])
indx = tf.constant([[3,2,1,0], [0,1,2,3], [1,0,3,2], [0,3,1,2], [1,2,3,0]])
# required output tensor:
[[[6, 7], [4, 5], [2, 3], [0, 1]],
[[8, 9], [10, 11], [12, 13], [14, 15]],
[[18, 19], [16, 17], [22, 23], [20, 21]],
[[24, 25], [30, 31], [26, 27], [28, 29]],
[[34, 35], [36, 37], [38, 39], [32, 33]]]
I'm struggling with tf.gather_nd(). Any suggestion? I can see it's happening here but I'm not sure how to apply on entire matrix without using for loop or tf.map_fn
print(tf.gather_nd(matrix[0], tf.expand_dims(indx, -1)[0]).numpy().tolist())
print(tf.gather_nd(matrix[1], tf.expand_dims(indx, -1)[1]).numpy().tolist())
print(tf.gather_nd(matrix[2], tf.expand_dims(indx, -1)[2]).numpy().tolist())
print(tf.gather_nd(matrix[3], tf.expand_dims(indx, -1)[3]).numpy().tolist())
print(tf.gather_nd(matrix[4], tf.expand_dims(indx, -1)[4]).numpy().tolist())
"""
[[6, 7], [4, 5], [2, 3], [0, 1]]
[[8, 9], [10, 11], [12, 13], [14, 15]]
[[18, 19], [16, 17], [22, 23], [20, 21]]
[[24, 25], [30, 31], [26, 27], [28, 29]]
[[34, 35], [36, 37], [38, 39], [32, 33]]
"""
EDIT: I asked a similar question with respect to numpy. A clever indexing answer does solves the numpy version, but it's hard to apply it on Tensors. Feel free to take a look at the accepted answer here: How can I get elements from 3D matrix using specified indices in numpy?
Duh, that was stupid! There is already a very great function available that works on multi-dimensional array in tensorflow; tf.gather() Check out the batch_dims argument for more information.
>> tf.gather(matrix, indx, batch_dims=1).numpy().tolist()
[[[6, 7], [4, 5], [2, 3], [0, 1]],
[[8, 9], [10, 11], [12, 13], [14, 15]],
[[18, 19], [16, 17], [22, 23], [20, 21]],
[[24, 25], [30, 31], [26, 27], [28, 29]],
[[34, 35], [36, 37], [38, 39], [32, 33]]]
I am building a neural network. where I have to flatten my training dataset.
I have two options.
1 is:
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T
and 2nd one is:
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[1]*train_x_orig.shape[2]*train_x_orig.shape[3], 209)
both gave the same shape but I found difference while computing cost?
why is that? thank you
Your original tensor is of at least rank 4 based on the second example. The first example pulls each element, ordered by increasing the right-most index, and inserts the elements into rows the length of the zeroth shape. Then transposes.
The second example again pull elements from by incrementing from the right-most index, i.e.:
element = train_x_orig[0, 0, 0, 0]
new_row.append(element)
element = train_x_orig[0, 0, 0, 1]
new_row.append(element)
but the size of the row is different. It is now the dimension of everything else in the tensor.
Here is an example to illustrate.
First we create an ordered array and reshape it to rank 4.
import numpy as np
x = np.arange(36).reshape(3,2,3,2)
x
# returns:
array([[[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[ 6, 7],
[ 8, 9],
[10, 11]]],
[[[12, 13],
[14, 15],
[16, 17]],
[[18, 19],
[20, 21],
[22, 23]]],
[[[24, 25],
[26, 27],
[28, 29]],
[[30, 31],
[32, 33],
[34, 35]]]])
Here is the output of the first example
x.reshape(x.shape[0], -1).T
# returns:
array([[ 0, 12, 24],
[ 1, 13, 25],
[ 2, 14, 26],
[ 3, 15, 27],
[ 4, 16, 28],
[ 5, 17, 29],
[ 6, 18, 30],
[ 7, 19, 31],
[ 8, 20, 32],
[ 9, 21, 33],
[10, 22, 34],
[11, 23, 35]])
And here is the second example
x.reshape(x.shape[1]*x.shape[2]*x.shape[3], -1)
# returns:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23],
[24, 25, 26],
[27, 28, 29],
[30, 31, 32],
[33, 34, 35]])
How the elements get reordered is fundamentally different.
this is my actual code:
saptamani = []
for months in range(1, 12):
luna = calendar.monthcalendar(2020, months)
for i in luna:
prepare = [m for m in i if m > 0]
if [prepare[0] != prepare[-1]]:
new_w = [prepare[0], prepare[-1]]
saptamani.append(new_w)
Now my code print this:
[[1, 5], [6, 12], [13, 19], [20, 26], [27, 31], [1, 2], [3, 9], [10, 16], [17, 23], [24, 29], [1, 1], [2, 8], [9, 15], [16, 22], [23, 29], [30, 31], [1, 5], [6, 12], [13, 19], [20, 26], [27, 30], [1, 3], [4, 10],
[11, 17], [18, 24], [25, 31], [1, 7], [8, 14], [15, 21], [22, 28], [29, 30], [1, 5], [6, 12], [13, 19], [20, 26], [27, 31], [1, 2], [3, 9], [10, 16], [17, 23], [24, 30], [31, 31], [1, 6], [7, 13], [14, 20], [21, 2
7], [28, 30], [1, 4], [5, 11], [12, 18], [19, 25], [26, 31], [1, 1], [2, 8], [9, 15], [16, 22], [23, 29], [30, 30]]
What I wanted to do is to add the month number too, to be like this
[01.01, 05.01], [05.01, 12.01]....
You think something like this perhaps?
saptamani = []
for months in range(1, 12):
luna = calendar.monthcalendar(2020, months)
for i in luna:
prepare = [m for m in i if m > 0]
if [prepare[0] != prepare[-1]]:
new_w = ["{:02d}.{:02d}".format(prepare[0],months), "{:02d}.{:02d}".format(prepare[-1],months)]
saptamani.append(new_w)
This writes saptamani variable like this:
[['01.01', '05.01'], ['06.01', '12.01'], ['13.01', '19.01'], ['20.01', '26.01'], ['27.01', '31.01'], ['01.02', '02.02'], ['03.02', '09.02'], ['10.02', '16.02'], ['17.02', '23.02'], ['24.02', '29.02'], ['01.03', '01.03'], ['02.03', '08.03'], ['09.03', '15.03'], ['16.03', '22.03'], ['23.03', '29.03'], ['30.03', '31.03'], ['01.04', '05.04'], ['06.04', '12.04'], ['13.04', '19.04'], ['20.04', '26.04'], ['27.04', '30.04'], ['01.05', '03.05'], ['04.05', '10.05'], ['11.05', '17.05'], ['18.05', '24.05'], ['25.05', '31.05'], ['01.06', '07.06'], ['08.06', '14.06'], ['15.06', '21.06'], ['22.06', '28.06'], ['29.06', '30.06'], ['01.07', '05.07'], ['06.07', '12.07'], ['13.07', '19.07'], ['20.07', '26.07'], ['27.07', '31.07'], ['01.08', '02.08'], ['03.08', '09.08'], ['10.08', '16.08'], ['17.08', '23.08'], ['24.08', '30.08'], ['31.08', '31.08'], ['01.09', '06.09'], ['07.09', '13.09'], ['14.09', '20.09'], ['21.09', '27.09'], ['28.09', '30.09'], ['01.10', '04.10'], ['05.10', '11.10'], ['12.10', '18.10'], ['19.10', '25.10'], ['26.10', '31.10'], ['01.11', '01.11'], ['02.11', '08.11'], ['09.11', '15.11'], ['16.11', '22.11'], ['23.11', '29.11'], ['30.11', '30.11']]
Your script wrote integer values but for getting something like "01.01" you need string instead.
I have a three dimensional array in this format:
x = [
[[1,2,3,4,5],[6,7,8,9,10]],
[[11,12,13,14,15],[16,17,18,19,20]],
[[21,22,23,24,25],[26,27,28,29,30]],
[[21,22,23,24,25]]
]
I'd like to split it into two, three dimensional arrays in this format:
y = [
[[1,2,3],[6,7,8]],
[[11,12,13],[16,17,18]],
[[21,22,23],[26,27,28]],
[[21,22,23]]
]
z = [
[[4,5],[9,10]],
[[14,15],[19,20]],
[[24,25],[29,30]],
[[24,25]]
]
I came up with this list comprehension for creating y:
[j[:3] for i in x for j in i]
Which returns this:
[[1, 2, 3], [6, 7, 8], [11, 12, 13], [16, 17, 18], [21, 22, 23], [26, 27, 28], [31, 32, 33]]
But as you'll see, it doesn't maintain the same multi-dimensional shape. Does anyone have any ideas?
You need to iterate one level deeper:
x = [[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [[11, 12, 13, 14, 15], [16, 17, 18, 19, 20]], [[21, 22, 23, 24, 25], [26, 27, 28, 29, 30]], [[21, 22, 23, 24, 25]]]
y = [[i[:3] for i in b] for b in x]
z = [[i[-2:] for i in b] for b in x]
Output:
[[[1, 2, 3], [6, 7, 8]], [[11, 12, 13], [16, 17, 18]], [[21, 22, 23], [26, 27, 28]], [[21, 22, 23]]]
[[[4, 5], [9, 10]], [[14, 15], [19, 20]], [[24, 25], [29, 30]], [[24, 25]]]
Move your inner most loop into a nested comprehension so that the inner lists are preserved:
y = [[j[:3] for j in i] for i in x]
I have a list that holds several sublist, each one of them with a given number of elements inside. I need to move all elements inside all sublists into another list, ie: remove the separation among elements imposed by the sublists.
This is a MWE if what I mean:
a = [[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]], [[17, 18, 19, 20], [21, 22, 23, 24]], [[25, 26, 27, 28], [26, 30, 31, 32], [33, 34, 35, 36]]]
b = []
for elem in a:
for item in elem:
b.append(item)
which results in:
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24], [25, 26, 27, 28], [26, 30, 31, 32], [33, 34, 35, 36]]
I'm sure there's a more elegant and simpler way to do this in python.
Use itertools.chain.from_iterable:
>>> from itertools import chain
>>> list(chain.from_iterable(a))
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24], [25, 26, 27, 28], [26, 30, 31, 32], [33, 34, 35, 36]]
Timing comparison:
Try this:
[item for sublist in a for item in sublist]