Using numpy, how can I subtract the elements of a numpy array that are matrices by each other?
a = np.array([ [ [1., 2., 3.], [4., 5., 6.], [7., 8., 9.] ],
[ [20., 21., 22.], [23., 24., 25.], [26., 27., 28.] ],
[ [30., 31., 32.], [33., 34., 35.], [36., 37., 38.] ]
])
or in plain English:
a = ([matrix1], [matrix2], [matrix3])
I want to get a np.array that calculates as follows:
[matrix1-matrix1][matrix1-matrix2][matrix1-matrix3]
[matrix2-matrix1][matrix2-matrix2][matrix2-matrix3]
[matrix3-matrix1][matrix3-matrix2][matrix3-matrix3]
Diagonal will be a matrix with zero values in the matrix.
How can I accomplish this?
Your a is a 3d array
In [258]: a = np.array([ [ [1., 2., 3.], [4., 5., 6.], [7., 8., 9.] ],
.....: [ [20., 21., 22.], [23., 24., 25.], [26., 27., 28.] ],
.....: [ [30., 31., 32.], [33., 34., 35.], [36., 37., 38.] ]
.....: ])
In [259]: a.shape
Out[259]: (3, 3, 3)
2d arrays can be accessed with indexing
In [260]: a[0,:,:]
Out[260]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]])
and a tuple of 3 2d arrays can be obtained with:
In [261]: (a[0,:,:],a[1,:,:],a[2,:,:])
Out[261]:
(array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]]),
array([[ 20., 21., 22.],
[ 23., 24., 25.],
[ 26., 27., 28.]]),
array([[ 30., 31., 32.],
[ 33., 34., 35.],
[ 36., 37., 38.]]))
This is equivalent to your (matrix1, matrix2, matrix3) expression. I'm using 2d array instead of matrix because numpy has an array subclass called matrix, so the term can be confusing.
A way to generate your subarray cross-difference is with broadcasting. I use the None syntax to expand the dimension of a.
In [262]: a[:,None,:,:]-a[None,:,:,:]
Out[262]:
array([[[[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]],
[[-19., -19., -19.],
[-19., -19., -19.],
[-19., -19., -19.]],
...
[[ 10., 10., 10.],
[ 10., 10., 10.],
[ 10., 10., 10.]],
[[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]]]])
In [263]: _.shape
Out[263]: (3, 3, 3, 3)
The shape of this new array is 4d, which can be thought of as a 3x3 array of 3x3 arrays. And yes, the diagonal elements A[i,i,:,:] are all 0s.
In [264]: A=a[:,None,:,:]-a[None,:,:,:]
In [265]: A[2,2,:,:]
Out[265]:
array([[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]])
Related
This is the standard batch matrix multiplication:
import torch
a = torch.arange(12, dtype=torch.float).view(2,3,2)
b = torch.arange(12, dtype=torch.float).view(2,3,2) - 1
c = a.matmul(b.transpose(-1,-2))
a,b,c
>>
(tensor([[[ 0., 1.],
[ 2., 3.],
[ 4., 5.]],
[[ 6., 7.],
[ 8., 9.],
[10., 11.]]]),
tensor([[[-1., 0.],
[ 1., 2.],
[ 3., 4.]],
[[ 5., 6.],
[ 7., 8.],
[ 9., 10.]]]),
tensor([[[ 0., 2., 4.],
[ -2., 8., 18.],
[ -4., 14., 32.]],
[[ 72., 98., 124.],
[ 94., 128., 162.],
[116., 158., 200.]]]))
This is the one that I have:
e = a.view(6,2)
f = b.view(6,2)
g = e.matmul(f.transpose(-1,-2))
e,f,g
>>
(tensor([[ 0., 1.],
[ 2., 3.],
[ 4., 5.],
[ 6., 7.],
[ 8., 9.],
[10., 11.]]),
tensor([[-1., 0.],
[ 1., 2.],
[ 3., 4.],
[ 5., 6.],
[ 7., 8.],
[ 9., 10.]]),
tensor([[ 0., 2., 4., 6., 8., 10.],
[ -2., 8., 18., 28., 38., 48.],
[ -4., 14., 32., 50., 68., 86.],
[ -6., 20., 46., 72., 98., 124.],
[ -8., 26., 60., 94., 128., 162.],
[-10., 32., 74., 116., 158., 200.]]))
It's obvious that g covers c. I want to know if there is an efficient way to retrieve/slice c from g. Note that such retrieving/slicing method should generalize well to any shape of a and b.
Got it. We can just slice g with fancy indexing. We just extract the matrix multiplication result within the same batch:
g = g.view(2,3,2,3)
res = g[range(2),:,range(2),:]
res
I'm trying to put a small 8x7 2D array, inside an 8x8 2D array.
Here's what I'm working with:
--> Array called 'a' with shape 8x7
a = [[ 16., 11., 10., 16., 24., 40., 51.],
[ 12., 12., 14., 19., 26., 58., 60.],
[ 14., 13., 16., 24., 40., 57., 69.],
[ 14., 17., 22., 29., 51., 87., 80.],
[ 18., 22., 37., 56., 68., 109., 103.],
[ 24., 35., 55., 64., 81., 104., 113.],
[ 49., 64., 78., 87., 103., 121., 120.],
[ 72., 92., 95., 98., 112., 100., 103.]]
--> Array called 'b' with shape 8x8
b = [[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0.]]
So basically, what I want is:
--> Array called 'c' with shape 8x8
c = [[ 16., 11., 10., 16., 24., 40., 51., 0],
[ 12., 12., 14., 19., 26., 58., 60., 0],
[ 14., 13., 16., 24., 40., 57., 69., 0],
[ 14., 17., 22., 29., 51., 87., 80., 0],
[ 18., 22., 37., 56., 68., 109., 103., 0],
[ 24., 35., 55., 64., 81., 104., 113., 0],
[ 49., 64., 78., 87., 103., 121., 120., 0],
[ 72., 92., 95., 98., 112., 100., 103., 0]]
Is there an easy way to do this, preferably without using loops, like 'for' , 'while', 'map'
or list comprehension?
Thank you in advance!
You can just slice assign to b up to the dimensions of a:
x, y = a.shape
b[:x, :y] = a
print(b)
array([[ 16., 11., 10., 16., 24., 40., 51., 0.],
[ 12., 12., 14., 19., 26., 58., 60., 0.],
[ 14., 13., 16., 24., 40., 57., 69., 0.],
[ 14., 17., 22., 29., 51., 87., 80., 0.],
[ 18., 22., 37., 56., 68., 109., 103., 0.],
[ 24., 35., 55., 64., 81., 104., 113., 0.],
[ 49., 64., 78., 87., 103., 121., 120., 0.],
[ 72., 92., 95., 98., 112., 100., 103., 0.]])
I would like to replace values in a NumpyArray, in only one column, on several selected rows only, using putmask. I wish to use indexing on the array to be modified as well as the mask used. Therefor I create a nd.array, a mask and and array of desired replacements. as follows:
import numpy as np
a = np.linspace(1,30,30)
a.shape(10,3)
mask = np.random.randint(2, size=8)
replacements = a[[2,4,5,6,7,8],0]*a[[2,4,5,6,7,8],1]
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
mask
array([0, 1, 0, 0, 1, 0, 1, 1])
replacements
array([ 56., 182., 272., 380., 506., 650.])
np.putmask(a[[2,4,5,6,7,8],2], mask[2::], replacements)
My expected result would look like this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 272.],
[19., 20., 21.],
[22., 23., 506.],
[25., 26., 650.],
[28., 29., 30.]])
But instead I get this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
Anybody has an idea maybe?
Note that you are using fancy indexing, so when using np.putmask you are modifying a copy rather than a sliced view, and thus the original array remains unchanged. You can check this by trying to index using slice notation, np.putmask(a[2:8,2], mask[2::], replacements) for instance, which would in this case modify the values in a.
What you could do is use np.where and reassign the values to the corresponding indices in a:
a[[2,4,5,6,7,8],2] = np.where(mask[2::], replacements, a[[2,4,5,6,7,8],2])
Output
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 56.],
[ 10., 11., 12.],
[ 13., 14., 182.],
[ 16., 17., 272.],
[ 19., 20., 380.],
[ 22., 23., 506.],
[ 25., 26., 650.],
[ 28., 29., 30.]])
For a numpy array of dimension n, I'd like to apply np.nanmax() to n-1 dimensions producing a 1 dimensional array of maxima, ignoring all values set to np.nan.
q = np.arange(5*4*3.).reshape(3,4,5) % (42+1)
q[q%5==0] = np.nan
producing:
array([[[ nan, 1., 2., 3., 4.],
[ nan, 6., 7., 8., 9.],
[ nan, 11., 12., 13., 14.],
[ nan, 16., 17., 18., 19.]],
[[ nan, 21., 22., 23., 24.],
[ nan, 26., 27., 28., 29.],
[ nan, 31., 32., 33., 34.],
[ nan, 36., 37., 38., 39.]],
[[ nan, 41., 42., nan, 1.],
[ 2., 3., 4., nan, 6.],
[ 7., 8., 9., nan, 11.],
[ 12., 13., 14., nan, 16.]]])
If I know ahead of time that I want to use the last axis as the remaining dimension, I can use the -1 feature in .reshape() and do this:
np.nanmax(q.reshape(-1, q.shape[-1]), axis=0)
which produces the result I want:
array([ 12., 41., 42., 38., 39.])
However, suppose I don't know ahead of time to which one of the axes that I don't want to apply the maximum? Suppose I started with n=4 dimensions, and wanted it to apply to all axes except the mth axis, which could be 0, 1, 2, or 3? Would have to actually use a conditional if-elif-else ?
Is there something that would work like a hypothetical exeptaxis=m?
The axis argument of nanmax can be a tuple of axes over which the maximum is computed. In your case, you want that tuple to contain all the axes except m. Here's one way you could do that:
In [62]: x
Out[62]:
array([[[[ 4., 3., nan, nan],
[ 0., 2., 2., nan],
[ 4., 5., nan, 3.],
[ 2., 0., 3., 1.]],
[[ 2., 0., 0., 1.],
[ nan, 3., 0., nan],
[ 0., 1., nan, 2.],
[ 5., 4., 0., 1.]],
[[ 4., 0., 2., 0.],
[ 4., 0., 4., 5.],
[ 3., 4., 1., 0.],
[ 5., 3., 4., 3.]]],
[[[ 2., nan, 6., 4.],
[ 3., 1., 2., nan],
[ 5., 4., 1., 0.],
[ 2., 6., 0., nan]],
[[ 4., 1., 4., 2.],
[ nan, 1., 5., 5.],
[ 2., 0., 1., 1.],
[ 6., 3., 6., 5.]],
[[ 1., 0., 0., 1.],
[ 1., nan, 2., nan],
[ 3., 4., 0., 5.],
[ 1., 6., 2., 3.]]]])
In [63]: m = 0
In [64]: np.nanmax(x, axis=tuple(i for i in range(x.ndim) if i != m))
Out[64]: array([ 5., 6.])
I want to reshape the numpy array as it is depicted, from 3D to 2D. Unfortunately, the order is not correct.
A assume to have a numpy array (1024, 64, 100) and want to convert it to (1024*100, 64).
Does anybody has an idea how to maintain the order?
I have a sample data
data[0,0,0]=1
data[0,1,0]=2
data[0,2,0]=3
data[0,3,0]=4
data[1,0,0]=5
data[1,1,0]=6
data[1,2,0]=7
data[1,3,0]=8
data[2,0,0]=9
data[2,1,0]=10
data[2,2,0]=11
data[2,3,0]=12
data[0,0,1]=20
data[0,1,1]=21
data[0,2,1]=22
data[0,3,1]=23
data[1,0,1]=24
data[1,1,1]=25
data[1,2,1]=26
data[1,3,1]=27
data[2,0,1]=28
data[2,1,1]=29
data[2,2,1]=30
data[2,3,1]=31
and I would like to have an outcome like this:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
Moreover, I would also like to have the reshaping in the other way, i.e. from:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
to the desired output:
[[[ 1. 20.]
[ 2. 21.]
[ 3. 22.]
[ 4. 23.]]
[[ 5. 24.]
[ 6. 25.]
[ 7. 26.]
[ 8. 27.]]
[[ 9. 28.]
[ 10. 29.]
[ 11. 30.]
[ 12. 31.]]]
It looks like you can use numpy.transpose and then reshape, like so -
data.transpose(2,0,1).reshape(-1,data.shape[1])
Sample run -
In [63]: data
Out[63]:
array([[[ 1., 20.],
[ 2., 21.],
[ 3., 22.],
[ 4., 23.]],
[[ 5., 24.],
[ 6., 25.],
[ 7., 26.],
[ 8., 27.]],
[[ 9., 28.],
[ 10., 29.],
[ 11., 30.],
[ 12., 31.]]])
In [64]: data.shape
Out[64]: (3, 4, 2)
In [65]: data.transpose(2,0,1).reshape(-1,data.shape[1])
Out[65]:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
In [66]: data.transpose(2,0,1).reshape(-1,data.shape[1]).shape
Out[66]: (6, 4)
To get back original 3D array, use reshape and then numpy.transpose, like so -
In [70]: data2D.reshape(np.roll(data.shape,1)).transpose(1,2,0)
Out[70]:
array([[[ 1., 20.],
[ 2., 21.],
[ 3., 22.],
[ 4., 23.]],
[[ 5., 24.],
[ 6., 25.],
[ 7., 26.],
[ 8., 27.]],
[[ 9., 28.],
[ 10., 29.],
[ 11., 30.],
[ 12., 31.]]])
Using einops:
# start with (1024, 64, 100) to (1024*100, 64):
einops.rearrange('h w i -> (i h) w')
# or we could concatenate along horizontal axis to get (1024, 64 * 100):
einops.rearrange('h w i -> h (i w)')
See docs for more examples