I want to create a torch tensor sequence out of a multi-dimensional numpy array. I've achieved it with 1-d arrays, but can't find the proper way with more dimensions...
This is a 1-d vector example:
import numpy as np
import torch
n = np.arange(10)
t = torch.tensor([n[i: i + 3] for i in range(7)])
Being the output:
tensor([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8]])
Let's say that instead of a 1-d vector I have a 2-d one.
import numpy as np
import torch
n = np.array([np.arange(10), np.arange(10, 20)])
t = torch.tensor([n[..., i: i + 3] for i in range(7)]).view(2, -1, 3)
The output is:
tensor([[[ 0, 1, 2],
[10, 11, 12],
[ 1, 2, 3],
[11, 12, 13],
[ 2, 3, 4],
[12, 13, 14],
[ 3, 4, 5]],
[[13, 14, 15],
[ 4, 5, 6],
[14, 15, 16],
[ 5, 6, 7],
[15, 16, 17],
[ 6, 7, 8],
[16, 17, 18]]])
And what I’m looking for is:
tensor([[[ 0, 1, 2],
[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8]],
[[10, 11, 12],
[11, 12, 13],
[12, 13, 14],
[13, 14, 15],
[14, 15, 16],
[15, 16, 17],
[16, 17, 18]]])
As you can see the rows are alternated... Is there any way of achieving it?
P.D: In case there is a more elegant solution to solve the problem, I will be very grateful too! I've tried with methods as torch.repeat_interleave but couldn't achieve anything...
Thanks a lot!
It is not very clear what formula you want to implement, but looks like it should be t[i, j, k] = i*10 + j + k. This formula is nothing but the outer sum of the three index ranges. The most straightforward way to obtain it is
t = torch.tensor(np.add.outer(np.arange(2)*10, np.add.outer(np.arange(7), np.arange(3))))
Which gives
tensor([[[ 0, 1, 2],
[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8]],
[[10, 11, 12],
[11, 12, 13],
[12, 13, 14],
[13, 14, 15],
[14, 15, 16],
[15, 16, 17],
[16, 17, 18]]], dtype=torch.int32)
You can achieve this by concatenating the sub-tensors along the appropriate axis:
n = np.array([np.arange(10), np.arange(10, 20)])
t = torch.tensor([n[..., i: i + 3] for i in range(7)])
t = torch.cat([_ for _ in t.unsqueeze(-2)], dim=1)
Related
Inputs
I have list as follow
r1 = [([[[1, 2, 3], [1, 2, 3]],
[[4, 5, 6], [4, 5, 6]]],
[[[7, 8], [7, 8]],
[[9, 10], [9, 10]]]),
([[[11, 12, 13], [11, 12, 13]],
[[14, 15, 16], [14, 15, 16]]],
[[[17, 18], [17, 18]],
[[19, 20], [19, 20]]])]
I'm going to make 2 torch tensors from the input above.
my desired output is as follow
output
output =
[tensor([[[ 1, 2, 3],
[ 1, 2, 3]],
[[ 4, 5, 6],
[ 4, 5, 6]],
[[11, 12, 13],
[11, 12, 13]],
[[14, 15, 16],
[14, 15, 16]]]),
tensor([[[ 7, 8],
[ 7, 8]],
[[ 9, 10],
[ 9, 10]],
[[17, 18],
[17, 18]],
[[19, 20],
[19, 20]]])]
My code is as follows.
output = []
for i in range(len(r1[0])):
templates = []
for j in range(len(r1)):
templates.append(torch.tensor(r1[j][i]))
template = torch.cat(templates)
output.append(template)
Is there a simpler or easier way to get the result I want?
This will do:
output = [torch.Tensor([*a, *b]) for a, b in zip(*r1)]
It concatenates the corresponding items of the two list first then create the Tensor
I am generating multidimensional array of different sizes, though they'll all have an even number of columns.
>> import numpy as np
>> x = np.arange(24).reshape((3,8))
Which results in:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
I am able to slice with numpy and get the first column in an array:
>> newarr = x[0:,0:2]
array([[ 0, 1],
[ 8, 9],
[16, 17]])
However, I want to have one array that is just a list of the columns where column 1 and 2 are together, 3 and 4 are together, and so on.. For example:
array([[[ 0, 1],
[ 8, 9],
[16, 17]],
[[ 2, 3],
[10, 11],
[18, 19]],
etc....]
)
This code below works but it's clunky and my arrays are not all the same. Some arrays have 16 columns, some have 34, some have 50, etc.
>> newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]])]
There's got to be a better way to do this than
newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]+...+[x[0:,n:n+2]]
Help!
My idea is adding a for loop:
slice_len = 2
x_list = [x[0:, slice_len*i:slice_len*(i+1)] for i in range(x.shape[1] // slice_len)]
Output:
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]]), array([[ 4, 5],
[12, 13],
[20, 21]]), array([[ 6, 7],
[14, 15],
[22, 23]])]
I have three 2D arrays a1, a2, and a3
In [165]: a1
Out[165]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
In [166]: a2
Out[166]:
array([[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20]])
In [167]: a3
Out[167]:
array([[6, 7, 8],
[4, 5, 5]])
And I stacked these arrays into a single array:
In [168]: stacked = np.vstack((a1, a2, a3))
In [170]: stacked
Out[170]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 6, 7, 8],
[ 4, 5, 5]])
Now, I want to get rid of the duplicate rows. So, numpy.unique does the job.
In [169]: np.unique(stacked, axis=0)
Out[169]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 4, 5, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20]])
However, there is one issue here. The original order is lost when taking the unique rows. How can I retain the original ordering and still take the unique rows?
So, the expected output should be:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 4, 5, 5]])
Using return_index
_,idx=np.unique(stacked, axis=0,return_index=True)
stacked[np.sort(idx)]
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[15, 16, 17],
[18, 19, 20],
[ 4, 5, 5]])
after get the stacked array
step 1: get the indexes of the rows for sorted unique array
row_indexes = np.unique(stacked, return_index=True, axis=0)[1]
Note: row_indexes is holding the indexes for sorted array
step 2 : now iterate through the stacked array with the sorted index
sorted_index=sorted(row_indexes)
new_arr=[]
for i in range(len(sorted_index)):
new_arr.append(stacked[sorted_index[i]]
thats it!!!!!
I know that numpy.roll can shift an array along one or more existing axes. How would I create a new axis on array x along which I want to have views or copies of itself rolled by array shift?
Example:
x = np.arange(10)
shift = np.array([2, 4])
#input
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
#output
array(
[[8, 6],
[9, 7],
[0, 8],
[1, 9],
[2, 0],
[3, 1],
[4, 2],
[5, 3],
[6, 4],
[7, 5]])
Edit: I'm looking for a general solution (ideally without looping) that can also be applied on higher dimensional arrays. Another example:
x = np.arange(20).reshape(2, 10)
shift = np.array([2, 4])
#input
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
#output
array(
[[[ 8, 6],
[ 9, 7],
[ 0, 8],
[ 1, 9],
[ 2, 0],
[ 3, 1],
[ 4, 2],
[ 5, 3],
[ 6, 4],
[ 7, 5]],
[[18, 16],
[19, 17],
[10, 18],
[11, 19],
[12, 10],
[13, 11],
[14, 12],
[15, 13],
[16, 14],
[17, 15]]])
Here's a vectorized solution leveraging broadcasting that covers generic n-dim array cases -
np.take(x,(-shift + np.arange(x.shape[-1])[:,None]),axis=-1)
Sample runs
1) x as 1D -
In [114]: x = np.arange(10)
...: shift = np.array([2, 4])
In [115]: np.take(x,(-shift + np.arange(x.shape[-1])[:,None]),axis=-1)
Out[115]:
array([[8, 6],
[9, 7],
[0, 8],
[1, 9],
[2, 0],
[3, 1],
[4, 2],
[5, 3],
[6, 4],
[7, 5]])
2) x as 2D -
In [116]: x = np.arange(20).reshape(2, 10)
...: shift = np.array([2, 4])
In [117]: np.take(x,(-shift + np.arange(x.shape[-1])[:,None]),axis=-1)
Out[117]:
array([[[ 8, 6],
[ 9, 7],
[ 0, 8],
[ 1, 9],
[ 2, 0],
[ 3, 1],
[ 4, 2],
[ 5, 3],
[ 6, 4],
[ 7, 5]],
[[18, 16],
[19, 17],
[10, 18],
[11, 19],
[12, 10],
[13, 11],
[14, 12],
[15, 13],
[16, 14],
[17, 15]]])
I almost hate to provide this alternative because I think #BenT's answer is simple and logical
np.array([np.roll(x,sh) for sh in shift]).T
np.stack([np.roll(x,sh) for sh in shift], axis=1) # may be easier to generalize
but I can do the original x=np.arange(10) case with as_strided:
Perform all shifts:
In [352]: arr = np.lib.stride_tricks.as_strided(np.hstack((x,x)),shape=(10,10), strides=(8,8))
In [353]: arr
Out[353]:
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0],
[2, 3, 4, 5, 6, 7, 8, 9, 0, 1],
[3, 4, 5, 6, 7, 8, 9, 0, 1, 2],
[4, 5, 6, 7, 8, 9, 0, 1, 2, 3],
[5, 6, 7, 8, 9, 0, 1, 2, 3, 4],
[6, 7, 8, 9, 0, 1, 2, 3, 4, 5],
[7, 8, 9, 0, 1, 2, 3, 4, 5, 6],
[8, 9, 0, 1, 2, 3, 4, 5, 6, 7],
[9, 0, 1, 2, 3, 4, 5, 6, 7, 8]])
Then select the ones you want:
In [358]: arr[::-1][shift-1]
Out[358]:
array([[8, 9, 0, 1, 2, 3, 4, 5, 6, 7],
[6, 7, 8, 9, 0, 1, 2, 3, 4, 5]])
I wrote and tested the stack version with one try, but had to try several things to get the as_strided right.
I'd also prefer generalizing the list comprehension to higher dimensions.
For your 2d x:
np.stack([np.roll(x,sh, axis=1) for sh in shift],2)
I'm struggling to get each of the four numbers of a python array contained within a 2x2 into their separate arrays much like a Sudoku grid. Order doesn't matter. I would have tried writing a code or something but my mind has gone blank.
example grid
[
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]
]
I want to be able to get it in the form
[
[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]
]
Here is a pure Python solution. If you are after something more readable consider using NumPy (see below).
>>> from itertools import count, chain
>>>
# create 2x2 blocks of 2x2
>>> c = count(1)
>>> L4D = [[[[next(c) for i in range(2)] for j in range(2)] for k in range(2)] for l in range(2)]
>>> L4D
[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]
# swap middle dimensions
>>> L4D = [zip(*i) for i in L4D]
# next line is not necessary, only here so we can see what's going on
>>> L4D = [list(i) for i in L4D]
>>> L4D
[[([1, 2], [5, 6]), ([3, 4], [7, 8])], [([9, 10], [13, 14]), ([11, 12], [15, 16])]]
# join first two and last two dimensions
>>> result = [list(chain.from_iterable(j)) for j in chain.from_iterable(L4D)]
>>> result
[[1, 2, 5, 6], [3, 4, 7, 8], [9, 10, 13, 14], [11, 12, 15, 16]]
If using NumPy is an option this can be simplified. Here are three different possibilities. The first is a direct translation of the pure Python solution:
>>> import numpy as np
>>>
>>> np.arange(1, 17).reshape(2, 2, 2, 2).swapaxes(1, 2).reshape(4, 4)
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
>>> np.block(list(map(list, np.arange(1, 17).reshape(2, 2, 2, 2))))
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
>>> a = np.arange(4).reshape(2, 2)
>>> b = np.ones((2, 2), dtype = int)
>>> 4 * np.kron(a, b) + np.kron(b, a) + 1
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
I finally found a solution to my question with a little modification to this answer
cells = [[] * 4 for x in range(4)]
for row_index, row in enumerate(grid):
for col_index, col in enumerate(row):
i = col_index//2 + 2*(row_index//2)
cells[i].append(col)