Related
I have a matrix that I want to convert to 3D so that I can be able to print the element of list[i][j][k]
a = [[[0, 1, 2, 3, 4, 5], [1, 2, 3, 4, 5, 6], [2, 3, 2, 3, 4,5], [3, 2, 3, 4, 3, 4], [4, 3, 4, 5, 4, 3], [5, 4, 5, 6, 5, 4]]]
print(a[5][5][0]) # I want to be able to print in 3D`
I get right output if I do a[5][5] but wrong when I add the [0]. Is there anyway of converting my matrix such that this will be solved?
I tried to just wrap the list up with brackets [list], but it did not work. I also did:
b = [[i] for i in a]
which gave me [[[0,1,2,3,4,5]],[[1,2,3,4,5,6]],...
and it still did not work!
NOTE: I want the i to be the row, j to be the column and k to be 0 or 1, so k = 0 (in which case the value is the row index of the cell is pointing to), or the k = 1 (the value is the column index).
Tried to reproduce your issue. To me, it works if you use the right index. Here, it perfectly works if you do for instance
print(a[0][0][5]) # I want to be able to print in 3D`
for list a = [[[0, 1, 2, 3, 4, 5], [1, 2, 3, 4, 5, 6], [2, 3, 2, 3, 4,5], [3, 2, 3, 4, 3, 4], [4, 3, 4, 5, 4, 3], [5, 4, 5, 6, 5, 4]]] you have just a[0][n][n]. You can try a[0][5][5]
You have index like below:
a = [
#0 element i
[
#0 element j
[0, 1, 2, 3, 4, 5],
#1 element j
[1, 2, 3, 4, 5, 6],
#2 element j
[2, 3, 2, 3, 4,5],
#3 element j
[3, 2, 3, 4, 3, 4],
#4 element j
[4, 3, 4, 5, 4, 3],
#5 element j
[5, 4, 5, 6, 5, 4]
]
]
print(a[0][5][5]) # a[i][j][k]
import torch
import numpy as np
a = torch.tensor([[1, 4], [2, 5],[3, 6]])
bb=a.detach().numpy()
b = a.view(6).detach().numpy()
Element b is like:
[1 4 2 5 3 6]
How do I reshape back to the following:
[1 2 3 4 5 6]
This is just an example, want some generic answers, even 3D.
In Pytorch you can use reshape and permute as in this example:
Import torch
a = torch.randn((3,3,2))
b = a.permute(2,0,1).reshape(-1)
a
tensor([[[ 0.2372, 0.5550],
[ 0.7700, -0.3693],
[-0.4151, 0.6247]],
[[ 1.2179, 0.6992],
[ 0.5033, 1.6290],
[-1.2165, -0.4180]],
[[ 0.3189, 0.3208],
[ 0.3894, 2.5544],
[-1.3069, -0.6905]]])
b
tensor([ 0.2372, 0.7700, -0.4151, 1.2179, 0.5033, -1.2165, 0.3189, 0.3894,
-1.3069, 0.5550, -0.3693, 0.6247, 0.6992, 1.6290, -0.4180, 0.3208,
2.5544, -0.6905])
I think this solves the problem.
If you want to remain in PyTorch, you can view b in a's shape, then apply a transpose and flatten:
>>> b.view(-1,2).T.flatten()
tensor([1, 2, 3, 4, 5, 6])
In the 3D case, you can perform similar manipulations using torch.transpose which enables you to swap two axes. You get the desired result by combining it with torch.view:
First case (extra dimension last):
>>> b = a.view(-1, 1).expand(-1,3).flatten()
tensor([1, 1, 1, 4, 4, 4, 2, 2, 2, 5, 5, 5, 3, 3, 3, 6, 6, 6])
>>> b.view(-1,2,3).transpose(0,1).flatten()
tensor([1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 6])
Second case (extra dimension first):
>>> b = a.view(1,-1).expand(3,-1).flatten()
tensor([1, 4, 2, 5, 3, 6, 1, 4, 2, 5, 3, 6, 1, 4, 2, 5, 3, 6])
>>> b.view(3,-1).T.view(-1,2,3).transpose(0,1).flatten()
tensor([1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 6])
I can't help with the torch step, but starting with a numpy array:
In [70]: a=np.array([[1, 4], [2, 5],[3, 6]])
In [71]: a
Out[71]:
array([[1, 4],
[2, 5],
[3, 6]])
In [72]: a.ravel() # can also use reshape
Out[72]: array([1, 4, 2, 5, 3, 6])
To get a column major copy:
In [73]: a.ravel(order='F')
Out[73]: array([1, 2, 3, 4, 5, 6])
In [74]: a.T.ravel()
Out[74]: array([1, 2, 3, 4, 5, 6])
the transpose:
In [79]: a.T
Out[79]:
array([[1, 2, 3],
[4, 5, 6]])
For 3d arrays, you can use transpose with an order parameter.
Consider an array, M, made up of pairs of elements. (I've used spaces to emphasize that we will be dealing with element PAIRS). The actual arrays will have a large number of rows, and 4,6,8 or 10 columns.
import numpy as np
M = np.array([[1,3, 2,1, 4,2, 3,3],
[3,5, 6,9, 5,1, 3,4],
[1,3, 2,4, 3,4, 7,2],
[4,5, 1,2, 2,1, 2,3],
[6,4, 4,1, 6,1, 4,7],
[6,7, 7,6, 9,7, 6,2],
[5,3, 1,5, 3,3, 3,3]])
PROBLEM: I want to eliminate rows from M having an element pair that has no common elements with any of the other pairs in that row.
In array M, the 2nd row and the 4th row should be eliminated. Here's why:
2nd row: the pair (6,9) has no common element with (3,5), (5,1), or (3,4)
4th row: the pair (4,5) has no common element with (1,2), (2,1), or (2,3)
I'm sure there's a nice broadcasting solution, but I can't see it.
This is a broadcasting solution. Hope it's self-explained:
a = M.reshape(M.shape[0],-1,2)
mask = ~np.eye(a.shape[1], dtype=bool)[...,None]
is_valid = (((a[...,None,:]==a[:,None,...])&mask).any(axis=(-1,-2))
|((a[...,None,:]==a[:,None,:,::-1])&mask).any(axis=(-1,-2))
).all(-1)
M[is_valid]
Output:
array([[1, 3, 2, 1, 4, 2, 3, 3],
[1, 3, 2, 4, 3, 4, 7, 2],
[6, 4, 4, 1, 6, 1, 4, 7],
[6, 7, 7, 6, 9, 7, 6, 2],
[5, 3, 1, 5, 3, 3, 3, 3]])
Another way of solving this would be the following -
M = np.array([[1,3, 2,1, 4,2, 3,3],
[3,5, 6,9, 5,1, 3,4],
[1,3, 2,4, 3,4, 7,2],
[4,5, 1,2, 2,1, 2,3],
[6,4, 4,1, 6,1, 4,7],
[6,7, 7,6, 9,7, 6,2],
[5,3, 1,5, 3,3, 3,3]])
MM = M.reshape(M.shape[0],-1,2)
matches_M = np.any(MM[:,:,None,:,None] == MM[:,None,:,None,:], axis=(-1,-2))
mask = ~np.eye(MM.shape[1], dtype=bool)[None,:]
is_valid = np.all(np.any(matches_M&mask, axis=-1), axis=-1)
M[is_valid]
array([[1, 3, 2, 1, 4, 2, 3, 3],
[1, 3, 2, 4, 3, 4, 7, 2],
[6, 4, 4, 1, 6, 1, 4, 7],
[6, 7, 7, 6, 9, 7, 6, 2],
[5, 3, 1, 5, 3, 3, 3, 3]])
Consider a variable setSize (it can take value 2 or 3), and a numpy array v.
The number of columns in v is divisible by setSize. Here's a small sample:
import numpy as np
setSize = 2
# the array spaces are shown to emphasize that the rows
# are made up of sets having, in this case, 2 elements each.
v = np.array([[2,5, 3,5, 1,8],
[4,6, 2,7, 5,9],
[1,8, 2,3, 1,4],
[2,8, 1,4, 3,5],
[5,7, 2,3, 7,8],
[1,2, 4,6, 3,5],
[3,5, 2,8, 1,4]])
PROBLEM: For the rows that have all elements unique, I need to ALPHABETIZE the sets.
For example: set 1,14 would precede set 3,5, which would precede set 5,1.
As a final step, I need to eliminate any duplicated rows that may result.
In this example above, the array rows having indices 1,3,5,and 6 have unique elements,
so these rows must be alphabetized. The other rows are not changed.
Further, the rows v[3] and v[6], after alphabetization, are now identical. One of them may be dropped.
The final output looks like:
v = [[2,5, 3,5, 1,8],
[2,7, 4,6, 5,9],
[1,8, 2,3, 1,4],
[1,4, 2,8, 3,5],
[5,7, 2,3, 7,8],
[1,2, 3,5, 4,6]]
I can identify the rows having unique elements with code like below, but I stuck with the alphabetization code.
s = np.sort(v,axis=1)
v[(s[:,:-1] != s[:,1:]).all(1)]
Assuming you have unsuitable rows dropped with:
s = np.sort(v, axis=1)
idx = (s[:,:-1] != s[:,1:]).all(1)
w = v[idx]
Then you can get orders of each row with np.lexsort on a reshaped array:
w = w.reshape(-1,3,2)
s = np.lexsort((w[:,:,1], w[:,:,0]))
Then you can apply fancy indexing and reshape it back:
rows, orders = np.repeat(np.arange(len(s)), 3), s.flatten()
v[idx] = w[rows, orders].reshape((-1,6))
If you need to drop duplicated rows, you can do it like so:
u, idx = np.unique(v, return_index=True, axis=0)
output = v[np.sort(idx)]
Visualization of process:
Sample run:
>>> s
array([[1, 0, 2],
[1, 0, 2],
[0, 2, 1],
[2, 1, 0]], dtype=int64)
>>> rows
array([0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3])
>>> orders
array([1, 0, 2, 1, 0, 2, 0, 2, 1, 2, 1, 0], dtype=int64)
>>> v[idx]
array([[2, 7, 4, 6, 5, 9],
[1, 4, 2, 8, 3, 5],
[1, 2, 3, 5, 4, 6],
[1, 4, 2, 8, 3, 5]])
>>> v
array([[2, 5, 3, 5, 1, 8],
[2, 7, 4, 6, 5, 9],
[1, 8, 2, 3, 1, 4],
[1, 4, 2, 8, 3, 5],
[5, 7, 2, 3, 7, 8],
[1, 2, 3, 5, 4, 6],
[1, 4, 2, 8, 3, 5]])
>>> output
array([[2, 5, 3, 5, 1, 8],
[2, 7, 4, 6, 5, 9],
[1, 8, 2, 3, 1, 4],
[1, 4, 2, 8, 3, 5],
[5, 7, 2, 3, 7, 8],
[1, 2, 3, 5, 4, 6]])
Input: A 2D numpy array
Output: An array of indices that will sort the array row by row (or column by column)
E.g.: Say the function is get_sorted_indices(array, axis=0)
a = np.array([[1,2,3,4,5]
,[2,3,4,5,6]
,[1,2,3,4,5]
,[2,3,4,6,6]
,[2,3,4,5,6]])
ind = get_sorted_indices(a, axis=0)
Then we will get
>>> ind
[0, 2, 1, 4, 3]
>>> a[ind] # should be equals to a.sort(axis = 0)
array([[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[2, 3, 4, 5, 6],
[2, 3, 4, 5, 6],
[2, 3, 4, 6, 6]])
>>> a.sort(axis=0)
array([[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[2, 3, 4, 5, 6],
[2, 3, 4, 5, 6],
[2, 3, 4, 6, 6]])
I've looked at argsort but I don't understand its output and reading the documentation doesn't help:
>>> a.argsort()
array([[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]])
>>> a.argsort(axis=0)
array([[0, 0, 0, 0, 0],
[2, 2, 2, 2, 2],
[1, 1, 1, 1, 1],
[3, 3, 3, 4, 3],
[4, 4, 4, 3, 4]])
I can do this manually but I think I'm misunderstanding argsort or I'm missing something from numpy.
Is there a standard way to do this or I have no choice but to do this manually?