Remove and append middle column from a 3D numpy array - python

Suppose I have a 3D numpy array A, say given below:
A = np.array( [[[1,2,3], [4,5,6]] , [[7,8,9] , [10,11,12], [13,14,15]] ] , ndmin = 3 )
The only thing given about A is that it is a 3D arrays which is an array of arbitrary number of 2D arrays, where each 2D array is an array of arbitrary number of 1D arrays, and each 1D array has exactly 3 elements.
I want to remove the middle element from each 1D array from this 3D array, basically get the new array A1, and the removed column as X given below:
A1 = np.array( [[[1,3], [4,6]] , [[7,9] , [10,12], [13,15]] ] , ndmin = 3 )
X = np.array( [ [[2],[5]], [[8],[11],[14]] ], ndmin = 3 )
I want to write a function that given A it outputs (A1, X) and another function which given (A1, X) outputs A. I believe it should be possible to write the first function via array slicing, but I am not able to do so. Also how do I write the second function.

For you ragged array, it is better to store in a list of np.arrays with shape n by 3:
A = [np.array([[1,2,3],
[4,5,6]]) ,
np.array([[7,8,9],
[10,11,12],
[13,14,15]])]
Now you could:
def remove_middle(arr):
x = [a[:, 1] for a in arr]
arr_new = [np.delete(a, 1, axis = 1) for a in arr]
return arr_new, x
def insert_middle(arr, x):
return [np.concatenate([a[:, :1], xx.reshape(-1, 1), a[:, 1:]], axis = 1) for a, xx in zip(arr, x)]
remove_middle(A)
([array([[1, 3],
[4, 6]]),
array([[ 7, 9],
[10, 12],
[13, 15]])],
[array([2, 5]), array([ 8, 11, 14])])
insert_middle(*remove_middle(A))
# gets back the original A
[array([[1, 2, 3],
[4, 5, 6]]),
array([[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15]])]

Without the ndmin=3 argument I can solve your answer using two nested list comprehensions in which the first indexes the middle argument and the second one deletes the middle argument of the inner arrays.
import numpy
A = np.array([[[1,2,3], [4,5,6]] , [[7,8,9] , [10,11,12], [13,14,15]]])
middle = [[array1d[1] for array1d in array2d] for array2d in A]
without_middle = [[np.delete(array1d, 1) for array1d in array2d] for array2d in A]

With your strange data, this absurd-looking quintuple-nested list-comprehension is the best I could come up with :P
A1 = [[[[[e for i, e in enumerate(d) if i != 1] for d in c] for c in b] for b in a] for a in A]
X = [[[[[e for i, e in enumerate(d) if i == 1] for d in c] for c in b] for b in a] for a in A]
Output:
>>> A1
[[[[[1, 3], [4, 6]], [[7, 9], [10, 12], [13, 15]]]]]
>>> X
[[[[[2], [5]], [[8], [11], [14]]]]]

Here is a solution. Note that lists are returned as raw python list which you can use as you want.
I changed your definition of A to a more suitable object.
import numpy as np
def f(A):
A1 = A.tolist()
X = []
for i in range(len(A1)):
temp = []
for j in range(len(A1[i])):
temp.append([A1[i][j].pop(1)])
X.append(temp)
return (A1, X)
def g(A1, X):
A = A1
for i in range(len(A1)):
for j in range(len(A1[i])):
A[i][j].insert(1, X[i][j][0])
return A
def main():
#A = np.array( [ [ [1,2,3], [4,5,6] ] , [ [7,8,9] , [10,11,12], [13,14,15] ] ] , ndmin = 3 )
A = np.asarray([ [ [1,2,3], [4,5,6] ] , [ [7,8,9] , [10,11,12], [13,14,15] ] ])
B, X = f(A)
print(g(B,X))
if __name__ == '__main__':
main()
Finally please note that this is one solution among many possible alternatives.

Related

In python numpy, how to replace some rows in array A with array B if we know the index

In python numpy, how to replace some rows in array A with array B if we know the index.
For example
we have
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
I want to change a to
a = np.array([[10,10],[3,4],[1000,1000]])
I have considered the funtion np.where but it need to create the bool condition, not very convenient,
I would do it following way
import numpy as np
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
a[index] = b
print(a)
gives output
[[ 10 10]
[ 3 4]
[1000 1000]]
You can use :
a[index] = b
For example :
import numpy as np
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
a[index] = b
print(a)
Result :
[[ 10 10]
[ 3 4]
[1000 1000]]
In Python's NumPy library, you can use the numpy.put() method to replace some rows in array A with array B if you know the index. Here's an example:
import numpy as np
# Initialize array A
A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
# Initialize array B
B = np.array([[10, 20, 30], [40, 50, 60]])
# Indices of the rows to be replaced in array A
indices = [0, 1]
# Replace rows in array A with rows in array B
np.put(A, indices, B)
print(A)
In this example, the first two rows in array A are replaced with the first two rows in array B, so the output will be
[[10 20 30]
[40 50 60]
[ 7 8 9]]
Simply a[indices] = b or if you want to be more fancy np.put(a, indices, b)

Numpy python - calculating sum of columns from irregular dimension

I have a multi-dimensional array for scores, and for which, I need to get sum of each columns at 3rd level in Python. I am using Numpy to achieve this.
import numpy as np
Data is something like:
score_list = [
[[1,1,3], [1,2,5]],
[[2,7,5], [4,1,3]]
]
This should return:
[[3 8 8] [5 3 8]]
Which is happening correctly using this:
sum_array = np_array.sum(axis=0)
print(sum_array)
However, if I have irregular shape like this:
score_list = [
[[1,1], [1,2,5]],
[[2,7], [4,1,3]]
]
I expect it to return:
[[3 8] [5 3 8]]
However, it comes up with warning and the return value is:
[list([1, 1, 2, 7]) list([1, 2, 5, 4, 1, 3])]
How can I get expected result?
numpy will try to cast it into an nd array which will fail, instead consider passing each sublist individually using zip.
score_list = [
[[1,1], [1,2,5]],
[[2,7], [4,1,3]]
]
import numpy as np
res = [np.sum(x,axis=0) for x in zip(*score_list)]
print(res)
[array([3, 8]), array([5, 3, 8])]
Here is one solution for doing this, keep in mind that it doesn't use numpy and will be very inefficient for larger matrices (but for smaller matrices runs just fine).
# Create matrix
score_list = [
[[1,1,3], [1,2,5]],
[[2,7,5], [4,1,3]]
]
# Get each row
for i in range(1, len(score_list)):
# Get each list within the row
for j in range(len(score_list[i])):
# Get each value in each list
for k in range(len(score_list[i][j])):
# Add current value to the same index
# on the first row
score_list[0][j][k] += score_list[i][j][k]
print(score_list[0])
There is bound to be a better solution but this is a temporary fix for you :)
Edit. Made more efficient
A possible solution:
a = np.vstack([np.array(score_list[x], dtype='object')
for x in range(len(score_list))])
[np.add(*[x for x in a[:, i]]) for i in range(a.shape[1])]
Another possible solution:
a = sum(score_list, [])
b = [a[x] for x in range(0,len(a),2)]
c = [a[x] for x in range(1,len(a),2)]
[np.add(x[0], x[1]) for x in [b, c]]
Output:
[array([3, 8]), array([5, 3, 8])]

Select specific indexes of 3D Pytorch Tensor using a 1D long tensor that represents indexes

So I have a tensor that is M x B x C, where M is the number of models, B is the batch and C is the classes and each cell is the probability of a class for a given model and batch. Then I have a tensor of the correct answers which is just a 1D of size B we'll call "t". How do I use the 1D of size B to just return a M x B x 1, where the returned tensor is just the value at the correct class? Say the M x B x C tensor is called "blah" I've tried
blah[:, :, C]
for i in range(M):
blah[i, :, C]
blah[:, C, :]
The top 2 just return the values of indexes t in the 3rd dimension of every slice. The last one returns the values at t indexes in the 2nd dimension. How do I do this?
We can get the desired result by combining advanced and basic indexing
import torch
# shape [2, 3, 4]
blah = torch.tensor([
[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
# shape [3]
t = torch.tensor([2, 1, 0])
b = torch.arange(blah.shape[1]).type_as(t)
# shape [2, 3, 1]
result = blah[:, b, t].unsqueeze(-1)
which results in
>>> result
tensor([[[ 2],
[ 5],
[ 8]],
[[14],
[17],
[20]]])
Here is one way to do it:
Suppose a is your M x B x C shaped tensor. I am taking some representative values below,
>>> M = 3
>>> B = 5
>>> C = 4
>>> a = torch.rand(M, B, C)
>>> a
tensor([[[0.6222, 0.6703, 0.0057, 0.3210],
[0.6251, 0.3286, 0.8451, 0.5978],
[0.0808, 0.8408, 0.3795, 0.4872],
[0.8589, 0.8891, 0.8033, 0.8906],
[0.5620, 0.5275, 0.4272, 0.2286]],
[[0.2419, 0.0179, 0.2052, 0.6859],
[0.1868, 0.7766, 0.3648, 0.9697],
[0.6750, 0.4715, 0.9377, 0.3220],
[0.0537, 0.1719, 0.0013, 0.0537],
[0.2681, 0.7514, 0.6523, 0.7703]],
[[0.5285, 0.5360, 0.7949, 0.6210],
[0.3066, 0.1138, 0.6412, 0.4724],
[0.3599, 0.9624, 0.0266, 0.1455],
[0.7474, 0.2999, 0.7476, 0.2889],
[0.1779, 0.3515, 0.8900, 0.2301]]])
Let's say the 1D class tensor is t, which gives the true class of each example in the batch. So it is a 1D tensor of shape (B, ) having class labels in the range {0, 1, 2, ..., C-1}.
>>> t = torch.randint(C, size = (B, ))
>>> t
tensor([3, 2, 1, 1, 0])
So basically you want to select the indices corresponding to t from the innermost dimension of a. This can be achieved using fancy indexing and broadcasting combined as follows:
>>> i = torch.arange(M).reshape(M, 1, 1)
>>> j = torch.arange(B).reshape(1, B, 1)
>>> k = t.reshape(1, B, 1)
Note that once you index anything by (i, j, k), they are going to expand and take the shape (M, B, 1) which is the desired output shape.
Now just indexing a by i, j and k gives:
>>> a[i, j, k]
tensor([[[0.3210],
[0.8451],
[0.8408],
[0.8891],
[0.5620]],
[[0.6859],
[0.3648],
[0.4715],
[0.1719],
[0.2681]],
[[0.6210],
[0.6412],
[0.9624],
[0.2999],
[0.1779]]])
So essentially, if you generate the index arrays conveying your access pattern beforehand, you can directly use them to extract some slice of the tensor.
You simply need to pass:
your index as the third slice
range(B) as the second slice
(i.e. which element in the 2nd dim each 3rd dim index corresponds to)
blah[:,range(B),t]

How to sum up (W * H) of 3D matrix and store it in 1D matrix with length=depth(third dimension of input matrix)

I want to sum up all elements (W * H) of 3D matrix and store it in 1D matrix with length=depth(third dimension of input matrix)
To make myself clear:
Input dimension = 1D in the form of (W * H * D).
Required output = 1D again with length=D
let's consider below 3D Matrix : 2 x 3 x 2.
Layer 1 Layer 2
[1, 2, 3 [7, 8, 9
4, 5, 6] 10, 11, 12]
output is 1D : [21, 57]
I am new to python and wrote like this:
def test(w, h, c, image_inp):
output = [image_inp[j * w + k] for i in enumerate(image_inp)
for j in range(0,h)
for k in range(0,w)
#image_inp[j * w + k] comment
]
printout(output)
I know this will copy the input array as it is to output array.
also output array length is not equal to Depth.
Some one please help me in getting this right
def test(w, h, c, image_inp):
output = [hwsum for i in enumerate(image_inp)
hwsum += wsum for j in range(0,h)
wsum += image_inp[j*w + k] for k in range(0,w)
#image_inp[j * w + k]
]
print "Calling outprint"
printout(output)
Note: I do not want to use numpy(with this it is working) or any math libraries.
reason being I am writing test code in python to evaluate a working on language.
EDIT:
input matrix will be entering the test function as 1D with w, h, c as arguments,
so it takes the form as:
[1,2,3,4,5,6,7,8,9,10,12],
with w, h, c have to compute considering input1D as 3D matrix.
thanks
Numpy is very suitable for slicing and manipulating single and multiple dimensional data. It is fast, easy to use and very "pythonic".
Following your example, you can just do:
>>> import numpy
>>> img3d=numpy.array([[[1,2,3],[4,5,6]],[[7,8,9],[10,12,12]]])
>>> img3d.shape
(2, 2, 3)
You can see here that img3d has 2 layers, 2 rows and 3 columns. You can just slice using indexing like this:
>>> img3d[0,:,:]
array([[1, 2, 3],
[4, 5, 6]])
To go from 3D to 1D, just use numpy.flatten():
>>> f=img3d.flatten()
>>> f
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 12])
And reversed, use numpy.reshape():
>>> f.reshape((2,2,3))
array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 12, 12]]])
Now add just jusing numpy.sum, giving the dimensions you want to add (in your case, dimensions 1 and 2 (dimensions being 0-indexed):
>>> numpy.sum(img3d,(1,2))
array([21, 58])
Just to summarize in a oneliner, you can do (variable names from your question):
>>> numpy.sum(numpy.array(image_inp).reshape(w,h,c),(1,2))
From the numpy manual on numpy.sum:
numpy.sum
numpy.sum(a, axis=None, dtype=None, out=None, keepdims=numpy._globals._NoValue>)
Sum of array elements over a given axis.
Parameters:
a : array_like Elements to sum.
axis : None or int or
tuple of ints, optional Axis or axes along which a sum is performed.
The default, axis=None, will sum all of the elements of the input
array. If axis is negative it counts from the last to the first axis.
New in version 1.7.0.: If axis is a tuple of ints, a sum is performed
on all of the axes specified in the tuple instead of a single axis or
all the axes as before.
If your matrix is set as your post implies with your "3D" matrix as an array of arrays:
M = [ [1, 2, 3,
4, 5, 6],
[ 7, 8, 9,
10,11,12],
]
array_of_sums = []
for pseudo_2D_matrix in M:
array_of_sums.append(sum(pseudo_2D_matrix))
If your 3D matrix, as a real three dimensional object, is set up as:
M = [
[ [ 1, 2, 3],
[ 4, 5, 6]
],
[ [ 7, 8, 9],
[10,11,12],
]
You could create a 1D array of sums by doing the following:
array_of_sums = []
for 2D_matrix in M:
s = 0
for row in 2D_matrix:
s += sum(row)
array_of_sums.append(s)
It's a bit unclear how your data are formatted, but hopefully you get the idea from these two examples.
EDIT:
In light of clarification on input you could easily accomplish this:
If dimensions w,h,c are given as dimensional breakout of the array [1,2,3,4,5,6,7,8,9,10,12], then you simply need to boundary off those regions and sum based on that:
input_array = [1,2,3,4,5,6,7,8,9,10,11,12]
w,h,c = 2,3,2
array_of_sums = []
i = 0
while i < w:
array_of_sums.append(sum(input_array[i*h*c:(i+1)*h*c]))
i += 1
as a simplified method:
def sum_2D_slices(w,h,c,matrix_3D):
return [sum(matrix_3D[i*h*c:(i+1)*h*c]) for i in range(w)]

Extracting coordinates from two numpy arrays

Say you have two numpy arrays one, call it A = [x1,x2,x3,x4,x5] which has all the x coordinates, then I have another array, call it B = [y1,y2,y3,y4,y5].. How would one "extract" a set of coordinates e.g (x1,y1) so that i could actually do something with it? Could I use a forloop or something similar? I can't seem to find any good examples, so if you could direct me or show me some I would be grateful.
Not sure if that's what you're looking for. But you can use numpy.concatenate. You just have to add a fake dimension before with [:,None] :
import numpy as np
a = np.array([1,2,3,4,5])
b = np.array([6,7,8,9,10])
arr_2d = np.concatenate([a[:,None],b[:,None]], axis=1)
print arr_2d
# [[ 1 6] [ 2 7] [ 3 8] [ 4 9] [ 5 10]]
Once you have generated a 2D array you can just use arr_2d[i] to get the i-th set of coordinates.
import numpy as np
a = np.array([1, 2, 3, 4, 5])
b = np.array([6, 7, 8, 9, 10])
print(np.hstack([a[:, np.newaxis], b[:, np.newaxis]]))
[[ 1 6]
[ 2 7]
[ 3 8]
[ 4 9]
[ 5 10]]
As #user2314737 said in a comment, you could manually do it by simply grabbing the same element from each array like so:
a = np.array([1,2,3])
b = np.array([4,5,6])
index = 2 #completely arbitrary index choice
#as individual values
pointA = a[index]
pointB = b[index]
#or in tuple form
point = (a[index], b[index])
If you need all of them converted to coordinate form, then #Nuageux's answer is probably better
Let's say you have x = np.array([ 0.48, 0.51, -0.43, 2.46, -0.91]) and y = np.array([ 0.97, -1.07, 0.62, -0.92, -1.25])
Then you can use the zip function
zip(x,y)
This will create a generator. Turn this generator into a list and turn the result into a numpy array
np.array(list(zip(x,y)))
the result will look like this
array([[ 0.48, 0.97],
[ 0.51, -1.07],
[-0.43, 0.62],
[ 2.46, -0.92],
[-0.91, -1.25]])

Categories

Resources