I am trying to process a 3D image in chunks (non-overlapping windows). Once this is done I want to put the chunks back together in the right order.
I have been chunking the image as below:
tens = torch.tensor(range(64))
tens = tens.view((4,4,4))
print(tens)
>>>tensor([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23],
[24, 25, 26, 27],
[28, 29, 30, 31]],
[[32, 33, 34, 35],
[36, 37, 38, 39],
[40, 41, 42, 43],
[44, 45, 46, 47]],
[[48, 49, 50, 51],
[52, 53, 54, 55],
[56, 57, 58, 59],
[60, 61, 62, 63]]])
tens = torch.chunk(tens,2, -1)
tens = torch.stack(tens)
tens = torch.chunk(tens,2, -2)
tens = torch.concat(tens)
tens = torch.chunk(tens,2, -3)
tens = torch.concat(tens)
print(torch.shape)
>>>torch.Size([8, 2, 2, 2])
Then I want to put it back together in the original order
tens = tens.view([4,4,2,2])
tens = tens.view([2,4,4,2])
tens = tens.view([4,4,4])
print(tens)
>>>tensor([[[ 0, 1, 4, 5],
[16, 17, 20, 21],
[ 2, 3, 6, 7],
[18, 19, 22, 23]],
[[ 8, 9, 12, 13],
[24, 25, 28, 29],
[10, 11, 14, 15],
[26, 27, 30, 31]],
[[32, 33, 36, 37],
[48, 49, 52, 53],
[34, 35, 38, 39],
[50, 51, 54, 55]],
[[40, 41, 44, 45],
[56, 57, 60, 61],
[42, 43, 46, 47],
[58, 59, 62, 63]]])
and I can't figure out how to get the elements in the right order. I realise I probably missed something in the docs or something else obvious but I can't find it. Any ideas?
Operator torch.chunk doesn't reduce dimensions so its inverse is torch.cat, not torch.stack.
Here are the transforms with the corresponding inverse operations:
Splitting the last dimension in two:
>>> chunks = tens.chunk(chunks=2, dim=-1)
>>> torch.cat(chunks, dim=-1)
Splitting the second dimension into two:
>>> chunks = tens.chunk(chunks=2, dim=-2)
>>> torch.cat(chunks, dim=-2)
Splitting the first dimension into two:
>>> chunks = tens.chunk(chunks=2, dim=-3)
>>> torch.cat(chunks, dim=-3)
If you want to invert the whole sequence, you just have to keep in mind that torch.cat is the reverse operation of torch.chunk:
>>> tens = torch.cat(tens.chunk(2), dim=-3)
>>> tens = torch.cat(tens.chunk(2), dim=-2)
>>> tens = torch.cat(tens.chunk(2), dim=-1)
Related
i have two list with the same length over 11 rows. I would like df[0] to find a match in any position in df2[0] and df[1] to find a match in any position df2[1] and etc.... Instead of me typing one by one is there a easier method?
df = [[[1, 5,7,9,12,13,17],
[2,17,18,23,32,34,45],
[3,5,11,33,34,36,45]],
[[6,21,22,50,56,58,72],
[7,5,12,13,55,56,74],
[8,23,24,32,56,58,64]]]
df2 = [[[100,5,12,15,27,32,54],
[120,10,17,18,19,43,55],
[99,21,32,33,34,36,54]],
[[41,16,32,45,66,67,76],
[56,10,11,43,54,55,56],
[77,12,16,18,19,21,23]]]
i would like my output to be like this.
output = [[[[5,12,],[17]],
[[17,18],[32,34,36]]],
[[[55,56],[32]],[[56]]]
As of your reworked question, it is still not quite clear to me what exactly you want to accomplish. I assume you want element based matching. By using this approach we can find the matching sequences of two lists.
For the presented case, we just need to iterate over all the elements of your array.
The matches function will find all matching sequences. Using it in the nested for loop allows for element wise comparison. The matching sequences are the ten written to matched_sequences which will hold all identified matches.
import difflib
df = [
[[1, 5, 7, 9, 12, 13, 17], [2, 17, 18, 23, 32, 34, 45], [3, 5, 11, 33, 34, 36, 45]],
[[6, 21, 22, 50, 56, 58, 72], [7, 5, 12, 13, 55, 56, 74], [8, 23, 24, 32, 56, 58, 64]],
]
df2 = [
[[100, 5, 12, 15, 27, 32, 54], [120, 10, 17, 18, 19, 43, 55], [99, 21, 32, 33, 34, 36, 54]],
[[41, 16, 32, 45, 66, 67, 76], [56, 10, 11, 43, 54, 55, 56], [77, 12, 16, 18, 19, 21, 23]],
]
def matches(list1, list2):
while True:
mbs = difflib.SequenceMatcher(None, list1, list2).get_matching_blocks()
if len(mbs) == 1:
break
for i, j, n in mbs[::-1]:
if n > 0:
yield list1[i : i + n]
del list1[i : i + n]
del list2[j : j + n]
matched_sequences = []
for row_df, row_df2 in zip(df, df2):
for el1, el2 in zip(row_df, row_df2):
matched_sequences.extend(list(matches(el1, el2)))
print(matched_sequences)
This will produce as identified matches:
[[12], [5], [17, 18], [33, 34, 36], [55, 56], [23]]
given a 3D array:
a = np.arange(3*4*5).reshape(3,4,5)
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]]])
I would like to create the following matrix:
result =
array([[20, 21, 22, 23, 24],
[ 5, 6, 7, 8, 9],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]])
Using the indices idx = [1,0,2,2]
I.e, I would like to "take" per matrix, the row specified in idx, where len(idx)==a.shape[1] and np.max(idx)<a.shape[0] as idx choose from dimension 1.
Given that your array has three dimensions (x,y,z), since you want to take one value for each row in the yth direction, you can do this:
a[idx, range(a.shape[1])]
Output:
array([[20, 21, 22, 23, 24],
[ 5, 6, 7, 8, 9],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]])
I want to create a 6x6 numpy matrix, with the first row filled with: 0, 1, ..., 5, the second row filled with 10, 11, ... , 15, and the last row filled with 50, 51, ... , 55.
I thought about using (1) nested (two layer) list comprehensions, and then converting list-of-list into a numpy.matrix object, or (2) using variables inside of range function, i.e. - range(x) and vary x from 1 to 6. I was not able to get either of these two ideas to work.
Below is my non-vectorized / looping code to construct this matrix. Is there a more Pythonic way of doing this?
a = np.zeros((6,6))
for i in range(6):
for j in range(6):
a[i,j] = 10*i + j
print(a)
(This is one of the examples given at 39:00 in the intro video to NumPy on Youtube:
Intro to Numerical Computing with NumPy
How about np.ogrid?
np.add(*np.ogrid[:60:10, :6])
# array([[ 0, 1, 2, 3, 4, 5],
# [10, 11, 12, 13, 14, 15],
# [20, 21, 22, 23, 24, 25],
# [30, 31, 32, 33, 34, 35],
# [40, 41, 42, 43, 44, 45],
# [50, 51, 52, 53, 54, 55]])
Details
ogrid returns an open meshgrid:
a, b = np.ogrid[:60:10, :6]
a
# array([[ 0],
# [10],
# [20],
# [30],
# [40],
# [50]])
b
# array([[0, 1, 2, 3, 4, 5]])
You can then perform broadcasted addition:
# a + b
np.add(a, b)
# array([[ 0, 1, 2, 3, 4, 5],
# [10, 11, 12, 13, 14, 15],
# [20, 21, 22, 23, 24, 25],
# [30, 31, 32, 33, 34, 35],
# [40, 41, 42, 43, 44, 45],
# [50, 51, 52, 53, 54, 55]])
Similarly, you can also generate two ranges using np.arange and add them:
np.arange(0, 60, 10)[:,None] + np.arange(6)
# array([[ 0, 1, 2, 3, 4, 5],
# [10, 11, 12, 13, 14, 15],
# [20, 21, 22, 23, 24, 25],
# [30, 31, 32, 33, 34, 35],
# [40, 41, 42, 43, 44, 45],
# [50, 51, 52, 53, 54, 55]])
This can be accomplished with broadcasting,
arange(0, 6) + 10*arange(0, 6)[:, None]
array([[ 0, 1, 2, 3, 4, 5],
[10, 11, 12, 13, 14, 15],
[20, 21, 22, 23, 24, 25],
[30, 31, 32, 33, 34, 35],
[40, 41, 42, 43, 44, 45],
[50, 51, 52, 53, 54, 55]])
I'd recommend reading https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html and https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html. "Pythonic" doesn't really matter when working with numpy. Some times iterating, list comprehensions, and other pythonic approaches work well with arrays, other times they are terribly inefficient. However, the links given cover some high level concepts that are very powerfull with numpy.
I'm having problems with the following task.
Assume we have a matrix, looking like this:
Mat = np.array([
[11, 12, 13, 14, 15], \
[21, 22, 23, 24, 25], \
[31, 32, 33, 34, 35], \
[41, 42, 43, 44, 45], \
[51, 52, 53, 54, 55]])
What I want to do is to replace the entries 22, 33 and 44 with something different that I calculated before. I know I could do this with for loops but I think there has to be a more elegant way.
I have something like this in mind:
Subselect the main diagonal from [1,1] to [-2,-2] and save it as an array.
Modify this array in the desired manner.
Save the modified array as part of the main diagonal of the matrix.
I found the np.diagonal() to get the diagonal and got so far:
Mat = np.array([
[11, 12, 13, 14, 15], \
[21, 22, 23, 24, 25], \
[31, 32, 33, 34, 35], \
[41, 42, 43, 44, 45], \
[51, 52, 53, 54, 55]])
print(Mat)
snipA = Mat.diagonal()
snipB = snipA[1:len(snipA)-1]
print(snipA)
print(snipB)
There are two problems now. First, I cannot modify snipB in any way. I get the error: "output array is read-only". Second, how can I save a modified snipB into the matrix again?
Any help is appreciated.
You can index and modify a part of the diagonal like so:
>>> subdiag = np.arange(1, len(mat)-1)
>>> mat[subdiag, subdiag]
array([22, 33, 44])
>>> mat[subdiag, subdiag] = 0
>>> mat
array([[11, 12, 13, 14, 15],
[21, 0, 23, 24, 25],
[31, 32, 0, 34, 35],
[41, 42, 43, 0, 45],
[51, 52, 53, 54, 55]])
>>>
>>> mat[subdiag, subdiag] = [22, 33, 44]
>>> mat
array([[11, 12, 13, 14, 15],
[21, 22, 23, 24, 25],
[31, 32, 33, 34, 35],
[41, 42, 43, 44, 45],
[51, 52, 53, 54, 55]])
You can also do this with einsum since numpy 1.10
np.einsum('ii->i', mat)[1:-1] = 0
mat
array([[11, 12, 13, 14, 15],
[21, 0, 23, 24, 25],
[31, 32, 0, 34, 35],
[41, 42, 43, 0, 45],
[51, 52, 53, 54, 55]])
import random
orig=list(range(1,65))
temp_orig= [[0, 1, 2, 3, 4, 5, 6, 7],
[8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29, 30, 31],
[32, 33, 34, 35, 36, 37, 38, 39],
[40, 41, 42, 43, 44, 45, 46, 47],
[48, 49, 50, 51, 52, 53, 54, 55],
[56, 57, 58, 59, 60, 61, 62, 63]]
ip= [[40, 24, 9, 41, 42, 56, 43, 45],
[4, 23, 21, 60, 35, 6, 59, 0],
[36, 53, 32, 16, 7, 37, 17, 18],
[10, 62, 61, 38, 29, 34, 31, 25],
[54, 57, 51, 49, 39, 3, 50, 30],
[11, 46, 33, 27, 44, 15, 13, 48],
[12, 58, 1, 26, 47, 20, 28, 52],
[19, 55, 2, 63, 22, 8, 14, 5]]
ipinv=[[15, 50, 58, 37, 8, 63, 13, 20],
[61, 2, 24, 40, 48, 46, 62, 45],
[19, 22, 23, 56, 53, 10, 60, 9],
[1, 31, 51, 43, 54, 28, 39, 30],
[18, 42, 29, 12, 16, 21, 27, 36],
[0, 3, 4, 6, 44, 7, 41, 52],
[47, 35, 38, 34, 55, 17, 32, 57],
[5, 33, 49, 14, 11, 26, 25, 59]]
print "\n\nOriginal position of Bits:"
for i in range(len(ipinv)):
print "\t",temp_orig[i]
print "\n\nInitial Permutation Table"
for i in range(len(ipinv)):
print "\t",ip[i]
print "\n\nInverse Initail permutation Table"
for i in range(len(ipinv)):
print "\t",ipinv[i]
print "\nRound 1:Initial Permutation"
plaintext=list(raw_input("\n\tEnter not more than 8 char:"))
l=len(plaintext)
for i in range(8,len(plaintext)):
del plaintext[l-1+8-i]
print "\n\tPlain text:",plaintext,"\n"
for i in range(8):
plaintext[i]=list(bin(ord(plaintext[i])).zfill(8))
print "\t",plaintext[i]
ip1=temp_orig
for i in range(8):
for j in range(8):
ip1[i][j]=plaintext[(ip[i][j])/8][(ip[i][j])%8]
print "\nEnciphered list:"
for i in range(8):
print "\t",ip1[i]
print "\n73\n",ip1[7][3],"\n73"
ip1_d=temp_orig
for i in range(8):
for j in range(8):
ip1_d[i][j]=ip1[(ipinv[i][j])/8][(ipinv[i][j])%8]
print ip1[7][3],"ip1_d",[i],[j],"=ip1[",(ipinv[i][j])/8,"][",(ipinv[i][j])%8,"]"
#print "ip1_d",[i],[j],"=",ip1[(ipinv[i][j])/8][(ipinv[i][j])%8]
print "\nDeciphered list:"
for i in range(8):
print "\t",ip1_d[i]
The de sunstitution becomes wrong, just for example i tried printing ip1[7][3] and all of a sudden it changes value when i=7 and j=3 without any assignment to ip1
You do ip1=temp_orig a few times. That tells me that you probably think this copies the temp_orig list - it does not. You have to explicitly copy the list here:
ip1=[list(row) for row in temp_orig]
and so on. Your code just gives new names to temp_orig but still modifies the temp_orig list.
I think this is the most chaotic code I've seen in a while. You should start thinking more in terms of transformations of your data instead of juggling indices like this. I still have no idea what your code does or how it's supposed to work.