I would like to replace values in a NumpyArray, in only one column, on several selected rows only, using putmask. I wish to use indexing on the array to be modified as well as the mask used. Therefor I create a nd.array, a mask and and array of desired replacements. as follows:
import numpy as np
a = np.linspace(1,30,30)
a.shape(10,3)
mask = np.random.randint(2, size=8)
replacements = a[[2,4,5,6,7,8],0]*a[[2,4,5,6,7,8],1]
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
mask
array([0, 1, 0, 0, 1, 0, 1, 1])
replacements
array([ 56., 182., 272., 380., 506., 650.])
np.putmask(a[[2,4,5,6,7,8],2], mask[2::], replacements)
My expected result would look like this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 272.],
[19., 20., 21.],
[22., 23., 506.],
[25., 26., 650.],
[28., 29., 30.]])
But instead I get this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
Anybody has an idea maybe?
Note that you are using fancy indexing, so when using np.putmask you are modifying a copy rather than a sliced view, and thus the original array remains unchanged. You can check this by trying to index using slice notation, np.putmask(a[2:8,2], mask[2::], replacements) for instance, which would in this case modify the values in a.
What you could do is use np.where and reassign the values to the corresponding indices in a:
a[[2,4,5,6,7,8],2] = np.where(mask[2::], replacements, a[[2,4,5,6,7,8],2])
Output
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 56.],
[ 10., 11., 12.],
[ 13., 14., 182.],
[ 16., 17., 272.],
[ 19., 20., 380.],
[ 22., 23., 506.],
[ 25., 26., 650.],
[ 28., 29., 30.]])
Related
This is the standard batch matrix multiplication:
import torch
a = torch.arange(12, dtype=torch.float).view(2,3,2)
b = torch.arange(12, dtype=torch.float).view(2,3,2) - 1
c = a.matmul(b.transpose(-1,-2))
a,b,c
>>
(tensor([[[ 0., 1.],
[ 2., 3.],
[ 4., 5.]],
[[ 6., 7.],
[ 8., 9.],
[10., 11.]]]),
tensor([[[-1., 0.],
[ 1., 2.],
[ 3., 4.]],
[[ 5., 6.],
[ 7., 8.],
[ 9., 10.]]]),
tensor([[[ 0., 2., 4.],
[ -2., 8., 18.],
[ -4., 14., 32.]],
[[ 72., 98., 124.],
[ 94., 128., 162.],
[116., 158., 200.]]]))
This is the one that I have:
e = a.view(6,2)
f = b.view(6,2)
g = e.matmul(f.transpose(-1,-2))
e,f,g
>>
(tensor([[ 0., 1.],
[ 2., 3.],
[ 4., 5.],
[ 6., 7.],
[ 8., 9.],
[10., 11.]]),
tensor([[-1., 0.],
[ 1., 2.],
[ 3., 4.],
[ 5., 6.],
[ 7., 8.],
[ 9., 10.]]),
tensor([[ 0., 2., 4., 6., 8., 10.],
[ -2., 8., 18., 28., 38., 48.],
[ -4., 14., 32., 50., 68., 86.],
[ -6., 20., 46., 72., 98., 124.],
[ -8., 26., 60., 94., 128., 162.],
[-10., 32., 74., 116., 158., 200.]]))
It's obvious that g covers c. I want to know if there is an efficient way to retrieve/slice c from g. Note that such retrieving/slicing method should generalize well to any shape of a and b.
Got it. We can just slice g with fancy indexing. We just extract the matrix multiplication result within the same batch:
g = g.view(2,3,2,3)
res = g[range(2),:,range(2),:]
res
I want to change an array such as
array([[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.]])
To
array([[[7., 3.],
[7., 3.],
[7., 3.],
[7., 3.]],
[[14., 1.],
[14., 1.],
[14., 1.],
[14., 1.]],
[[9., 17.],
[9., 17.],
[9., 17.],
[9., 17.]]])
I thought I would manage with reshape, but none of the order statements work in this case. Currently, I am doing it this way
np.vstack([mat[:, i-2:i] for i in range(2, mat.shape[1]+1, 2)]).reshape(3,-1,2)
And I was wondering if there is a better way to do it
reshape and then swapaxes:
import numpy as np
a = np.array(
[[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.],
[ 7., 3., 14., 1., 9., 17.]])
a.reshape((a.shape[0], -1, 2)).swapaxes(0, 1)
I have the array
A = array([[ 1., 2., 3., 10., 11., 12.],
[ 4., 5., 6., 13., 14., 15.],
[ 7., 8., 9., 16., 17., 18.],
[ 19., 20., 21., 28., 29., 30.],
[ 22., 23., 24., 31., 32., 33.],
[ 25., 26., 27., 34., 35., 36.]])
I would like to reshape it in order to obtain
B = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]
I have tried
>>> B = A.reshape(1,36)
array([[ 1., 2., 3., 10., 11., 12., 4., 5., 6., 13., 14.,
15., 7., 8., 9., 16., 17., 18., 19., 20., 21., 28.,
29., 30., 22., 23., 24., 31., 32., 33., 25., 26., 27.,
34., 35., 36.]])
But, obviously, I didn't reach the result. My real data differs from the example, so I can't sort the array A to obtain B.
I suppose I need more reshapes...
Split each of those two axes such that the remaining ones are of lengths 2 each with a reshape giving us a 4D array and then swap the middle two axes with np.swapaxes() and finally flatten with np.ravel() -
A.reshape(2,3,2,3).swapaxes(1,2).ravel()
Generically put -
m,n = A.shape
A.reshape(2,m//2,2,n//2).swapaxes(1,2).ravel()
Sample run -
In [15]: A
Out[15]:
array([[ 1., 2., 3., 10., 11., 12.],
[ 4., 5., 6., 13., 14., 15.],
[ 7., 8., 9., 16., 17., 18.],
[ 19., 20., 21., 28., 29., 30.],
[ 22., 23., 24., 31., 32., 33.],
[ 25., 26., 27., 34., 35., 36.]])
In [16]: A.reshape(2,3,2,3).swapaxes(1,2).ravel()
Out[16]:
array([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11.,
12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.,
23., 24., 25., 26., 27., 28., 29., 30., 31., 32., 33.,
34., 35., 36.])
I have a 3D np arrays like this :
x= array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]])
I would like to repeat my array n times ( eg 3 times) like this :
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]])
I have tried like this :
xx=np.vstack([x]*3)
print xx.reshape(6,4,3)
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]])
How can I get in the order what I want, there should be the easy way to do this. Thanks in advance for your suggestions.
After a bit of trial and error I have found a way to do it:
np.tile(x.reshape(2,12), [1,3]).reshape(6,4,3)
You can use np.repeat with axis = 0:
np.repeat(x, [3, 3], axis = 0) # or more generally np.repeat(x, [n] * len(x), axis = 0)
# here n is the repeat times
Out[514]:
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]])
Another option would be to index it as:
x[[0,0,0,1,1,1]]
Or programmatically:
x[[i for i in range(len(x)) for j in range(3)]]
Out[518]:
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[ 10., 11., 12.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]],
[[ 13., 14., 15.],
[ 16., 17., 18.],
[ 19., 20., 21.],
[ 22., 23., 24.]]])
I want to reshape the numpy array as it is depicted, from 3D to 2D. Unfortunately, the order is not correct.
A assume to have a numpy array (1024, 64, 100) and want to convert it to (1024*100, 64).
Does anybody has an idea how to maintain the order?
I have a sample data
data[0,0,0]=1
data[0,1,0]=2
data[0,2,0]=3
data[0,3,0]=4
data[1,0,0]=5
data[1,1,0]=6
data[1,2,0]=7
data[1,3,0]=8
data[2,0,0]=9
data[2,1,0]=10
data[2,2,0]=11
data[2,3,0]=12
data[0,0,1]=20
data[0,1,1]=21
data[0,2,1]=22
data[0,3,1]=23
data[1,0,1]=24
data[1,1,1]=25
data[1,2,1]=26
data[1,3,1]=27
data[2,0,1]=28
data[2,1,1]=29
data[2,2,1]=30
data[2,3,1]=31
and I would like to have an outcome like this:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
Moreover, I would also like to have the reshaping in the other way, i.e. from:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
to the desired output:
[[[ 1. 20.]
[ 2. 21.]
[ 3. 22.]
[ 4. 23.]]
[[ 5. 24.]
[ 6. 25.]
[ 7. 26.]
[ 8. 27.]]
[[ 9. 28.]
[ 10. 29.]
[ 11. 30.]
[ 12. 31.]]]
It looks like you can use numpy.transpose and then reshape, like so -
data.transpose(2,0,1).reshape(-1,data.shape[1])
Sample run -
In [63]: data
Out[63]:
array([[[ 1., 20.],
[ 2., 21.],
[ 3., 22.],
[ 4., 23.]],
[[ 5., 24.],
[ 6., 25.],
[ 7., 26.],
[ 8., 27.]],
[[ 9., 28.],
[ 10., 29.],
[ 11., 30.],
[ 12., 31.]]])
In [64]: data.shape
Out[64]: (3, 4, 2)
In [65]: data.transpose(2,0,1).reshape(-1,data.shape[1])
Out[65]:
array([[ 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.],
[ 20., 21., 22., 23.],
[ 24., 25., 26., 27.],
[ 28., 29., 30., 31.]])
In [66]: data.transpose(2,0,1).reshape(-1,data.shape[1]).shape
Out[66]: (6, 4)
To get back original 3D array, use reshape and then numpy.transpose, like so -
In [70]: data2D.reshape(np.roll(data.shape,1)).transpose(1,2,0)
Out[70]:
array([[[ 1., 20.],
[ 2., 21.],
[ 3., 22.],
[ 4., 23.]],
[[ 5., 24.],
[ 6., 25.],
[ 7., 26.],
[ 8., 27.]],
[[ 9., 28.],
[ 10., 29.],
[ 11., 30.],
[ 12., 31.]]])
Using einops:
# start with (1024, 64, 100) to (1024*100, 64):
einops.rearrange('h w i -> (i h) w')
# or we could concatenate along horizontal axis to get (1024, 64 * 100):
einops.rearrange('h w i -> h (i w)')
See docs for more examples